2024-02-22T18:27:30.984 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-22T18:27:30.984 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} --name yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947 --verbose -- /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/orig.config.yaml 2024-02-22T18:27:31.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-22T18:27:31.107 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947 branch: pacific-release description: rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} email: null first_in_suite: false job_id: '7570947' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: pacific-release ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - flag\(s\) set - \(CACHE_POOL_NO_HIT_SET\) - \(PG_ - \(OSD_ - mons down sha1: eb66ed921e744301e6863be6353618d63967ea59 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: eb66ed921e744301e6863be6353618d63967ea59 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-ksirivad-pacific-release-whitelist sha1: 391ea41edcaa85aa674d6e816fb2469974fc7d8e owner: scheduled_yuriw@teuthology priority: 61 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 264 sha1: eb66ed921e744301e6863be6353618d63967ea59 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-ksirivad-pacific-release-whitelist suite_path: /home/teuthworker/src/github.com_kamoltat_ceph_391ea41edcaa85aa674d6e816fb2469974fc7d8e/qa suite_relpath: qa suite_repo: https://github.com/kamoltat/ceph.git suite_sha1: 391ea41edcaa85aa674d6e816fb2469974fc7d8e targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOI4rojnoI6Cj/wmccWQ7COp7RKjGmWG+oDLCAf7XNh9IoWB7NRijeFjK3Zn9Nm+oOitTc+lG4M9onRpH9JesZY= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOz9ZXprhz5PIDAlX4d6TWO3ZT/stu4KLsWka1WkYZTZ27SMa7XUj21w5PcsJCNV052L3cwe5/nt+GpQVXCjVu4= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-22_18:15:46 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-22T18:27:31.107 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_kamoltat_ceph_391ea41edcaa85aa674d6e816fb2469974fc7d8e/qa; will attempt to use it 2024-02-22T18:27:31.108 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_kamoltat_ceph_391ea41edcaa85aa674d6e816fb2469974fc7d8e/qa/tasks 2024-02-22T18:27:31.108 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-22T18:27:31.109 INFO:teuthology.task.internal:Checking packages... 2024-02-22T18:27:31.133 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'eb66ed921e744301e6863be6353618d63967ea59' 2024-02-22T18:27:31.133 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T18:27:31.133 INFO:teuthology.packaging:ref: None 2024-02-22T18:27:31.133 INFO:teuthology.packaging:tag: None 2024-02-22T18:27:31.133 INFO:teuthology.packaging:branch: pacific-release 2024-02-22T18:27:31.133 INFO:teuthology.packaging:sha1: eb66ed921e744301e6863be6353618d63967ea59 2024-02-22T18:27:31.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific-release 2024-02-22T18:27:31.367 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-550.geb66ed92 2024-02-22T18:27:31.368 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-22T18:27:31.375 INFO:teuthology.task.internal:no buildpackages task found 2024-02-22T18:27:31.375 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-22T18:27:31.383 INFO:teuthology.task.internal:Saving configuration 2024-02-22T18:27:31.396 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-22T18:27:31.404 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T18:27:31.428 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 18:18:37.333090', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOI4rojnoI6Cj/wmccWQ7COp7RKjGmWG+oDLCAf7XNh9IoWB7NRijeFjK3Zn9Nm+oOitTc+lG4M9onRpH9JesZY='} 2024-02-22T18:27:31.448 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 18:18:37.334614', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOz9ZXprhz5PIDAlX4d6TWO3ZT/stu4KLsWka1WkYZTZ27SMa7XUj21w5PcsJCNV052L3cwe5/nt+GpQVXCjVu4='} 2024-02-22T18:27:31.448 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-22T18:27:31.454 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-22T18:27:31.454 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-22T18:27:31.454 INFO:teuthology.run_tasks:Running task console_log... 2024-02-22T18:27:31.532 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f712bd760d0>, signals=[15]) 2024-02-22T18:27:31.532 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-22T18:27:31.541 INFO:teuthology.task.internal:Opening connections... 2024-02-22T18:27:31.541 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-02-22T18:27:31.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:27:31.617 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-02-22T18:27:31.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:27:31.689 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-22T18:27:31.696 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-02-22T18:27:31.719 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-02-22T18:27:31.719 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8" 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8" 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2024-02-22T18:27:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-22T18:27:31.782 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-02-22T18:27:31.782 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-22T18:27:31.782 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-02-22T18:27:31.782 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-22T18:27:31.782 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-22T18:27:31.782 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-22T18:27:31.783 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-02-22T18:27:31.802 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-02-22T18:27:31.825 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-02-22T18:27:31.825 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-02-22T18:27:31.886 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-22T18:27:31.887 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-22T18:27:31.887 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-02-22T18:27:31.911 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-22T18:27:31.920 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-22T18:27:31.927 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-22T18:27:31.928 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-02-22T18:27:31.930 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-02-22T18:27:31.950 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-22T18:27:31.957 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-22T18:27:31.957 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-02-22T18:27:31.992 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-02-22T18:27:32.019 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-22T18:27:32.077 INFO:teuthology.run_tasks:Running task kernel... 2024-02-22T18:27:32.093 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-22T18:27:32.093 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-22T18:27:32.093 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-22T18:27:32.094 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-22T18:27:32.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T18:27:32.114 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-02-22T18:27:32.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T18:27:32.115 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-02-22T18:27:32.175 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-489.el8.x86_64 2024-02-22T18:27:32.175 INFO:teuthology.task.kernel:Running kernel on smithi064: 4.18.0-489.el8.x86_64 2024-02-22T18:27:32.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-02-22T18:27:32.177 INFO:teuthology.orchestra.run.smithi053.stdout:4.18.0-489.el8.x86_64 2024-02-22T18:27:32.177 INFO:teuthology.task.kernel:Running kernel on smithi053: 4.18.0-489.el8.x86_64 2024-02-22T18:27:32.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-02-22T18:27:39.085 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - AppStream 6.3 MB/s | 27 MB 00:04 2024-02-22T18:27:40.900 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - AppStream 4.5 MB/s | 27 MB 00:06 2024-02-22T18:36:21.276 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - BaseOS 20 kB/s | 10 MB 08:37 2024-02-22T18:36:23.239 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - BaseOS 20 kB/s | 10 MB 08:37 2024-02-22T18:36:23.451 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - Extras 38 kB/s | 18 kB 00:00 2024-02-22T18:36:23.943 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 6.9 kB 00:00 2024-02-22T18:36:24.219 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-02-22T18:36:25.372 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras 42 kB/s | 18 kB 00:00 2024-02-22T18:36:25.856 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 6.9 kB 00:00 2024-02-22T18:36:25.997 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 16 MB 00:01 2024-02-22T18:36:26.166 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2024-02-22T18:36:28.478 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2024-02-22T18:36:29.970 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 297 kB/s | 24 kB 00:00 2024-02-22T18:36:32.425 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 551 kB/s | 24 kB 00:00 2024-02-22T18:36:33.241 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-22T18:36:33.242 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-22T18:36:33.345 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout: kernel x86_64 4.18.0-540.el8 baseos 10 M 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core x86_64 4.18.0-540.el8 baseos 44 M 2024-02-22T18:36:33.346 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules x86_64 4.18.0-540.el8 baseos 36 M 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout:Install 3 Packages 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 90 M 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 97 M 2024-02-22T18:36:33.347 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T18:36:35.755 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-22T18:36:35.756 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-22T18:36:35.870 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 4.18.0-540.el8 baseos 10 M 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 4.18.0-540.el8 baseos 44 M 2024-02-22T18:36:35.871 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 4.18.0-540.el8 baseos 36 M 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout:Install 3 Packages 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 90 M 2024-02-22T18:36:35.872 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 97 M 2024-02-22T18:36:35.873 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T18:40:42.956 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] kernel-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T18:40:42.957 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] kernel-core-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-core-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T18:40:42.957 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] kernel-modules-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-modules-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T18:40:44.027 INFO:teuthology.orchestra.run.smithi053.stdout:(1/3): kernel-4.18.0-540.el8.x86_64.rpm 42 kB/s | 10 MB 04:10 2024-02-22T18:40:44.358 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] kernel-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T18:40:44.358 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] kernel-core-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-core-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T18:40:44.359 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] kernel-modules-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-modules-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T18:40:45.888 INFO:teuthology.orchestra.run.smithi064.stdout:(1/3): kernel-4.18.0-540.el8.x86_64.rpm 42 kB/s | 10 MB 04:09 2024-02-22T18:40:45.985 INFO:teuthology.orchestra.run.smithi053.stdout:(2/3): kernel-modules-4.18.0-540.el8.x86_64.rpm 147 kB/s | 36 MB 04:12 2024-02-22T18:40:46.560 INFO:teuthology.orchestra.run.smithi053.stdout:(3/3): kernel-core-4.18.0-540.el8.x86_64.rpm 177 kB/s | 44 MB 04:13 2024-02-22T18:40:46.561 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T18:40:46.561 INFO:teuthology.orchestra.run.smithi053.stdout:Total 365 kB/s | 90 MB 04:13 2024-02-22T18:40:47.241 INFO:teuthology.orchestra.run.smithi064.stdout:(2/3): kernel-core-4.18.0-540.el8.x86_64.rpm 178 kB/s | 44 MB 04:11 2024-02-22T18:40:47.270 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T18:40:47.729 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T18:40:47.729 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T18:40:47.872 INFO:teuthology.orchestra.run.smithi064.stdout:(3/3): kernel-modules-4.18.0-540.el8.x86_64.rpm 148 kB/s | 36 MB 04:11 2024-02-22T18:40:47.872 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T18:40:47.872 INFO:teuthology.orchestra.run.smithi064.stdout:Total 367 kB/s | 90 MB 04:12 2024-02-22T18:40:48.586 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T18:40:49.037 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T18:40:49.037 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T18:40:49.050 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T18:40:49.075 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T18:40:50.338 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T18:40:50.375 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T18:40:52.073 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T18:40:52.616 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T18:40:53.365 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T18:40:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T18:40:56.414 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T18:40:56.832 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T18:40:57.670 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T18:40:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T18:41:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T18:41:00.858 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:01.770 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T18:41:02.079 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:24.253 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:24.276 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:24.651 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:24.651 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-4.18.0-540.el8.x86_64 1/3 2024-02-22T18:41:24.651 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-core-4.18.0-540.el8.x86_64 2/3 2024-02-22T18:41:25.167 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:25.167 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T18:41:25.167 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T18:41:25.167 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-4.18.0-540.el8.x86_64 kernel-core-4.18.0-540.el8.x86_64 2024-02-22T18:41:25.167 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-4.18.0-540.el8.x86_64 2024-02-22T18:41:25.168 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T18:41:25.168 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T18:41:25.317 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T18:41:25.546 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T18:41:25.547 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T18:41:25.547 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-540.el8.x86_64 2024-02-22T18:41:25.547 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-22T18:41:25.547 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-22T18:41:25.547 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-22T18:41:25.547 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-02-22T18:41:25.547 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T18:41:25.769 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T18:41:25.770 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T18:41:25.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-02-22T18:41:25.804 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-02-22T18:41:25.804 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-02-22T18:41:25.804 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.02 2024-02-22T18:41:25.804 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 129.el8 2024-02-22T18:41:25.804 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Group : System Environment/Base 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 9265586 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Relocations : (not relocatable) 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : CentOS BuildSystem 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T18:41:25.805 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-22T18:41:25.806 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-22T18:41:25.806 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-22T18:41:25.806 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-02-22T18:41:25.806 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T18:41:25.806 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-02-22T18:41:25.807 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-22T18:41:25.807 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 4.18.0-540.el8.x86_64 2024-02-22T18:41:25.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-22T18:41:25.836 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:25.861 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:26.233 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:26.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-4.18.0-540.el8.x86_64 1/3 2024-02-22T18:41:26.234 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-4.18.0-540.el8.x86_64 2/3 2024-02-22T18:41:26.791 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T18:41:26.791 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T18:41:26.791 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T18:41:26.791 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-4.18.0-540.el8.x86_64 kernel-core-4.18.0-540.el8.x86_64 2024-02-22T18:41:26.791 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-4.18.0-540.el8.x86_64 2024-02-22T18:41:26.792 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T18:41:26.792 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T18:41:26.943 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T18:41:27.170 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T18:41:27.170 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T18:41:27.170 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-540.el8.x86_64 2024-02-22T18:41:27.170 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-22T18:41:27.170 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-22T18:41:27.171 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-22T18:41:27.171 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-02-22T18:41:27.171 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T18:41:27.356 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-02-22T18:41:27.391 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T18:41:27.391 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T18:41:27.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-02-22T18:41:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.02 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 129.el8 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Group : System Environment/Base 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 9265586 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-22T18:41:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Relocations : (not relocatable) 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : CentOS BuildSystem 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T18:41:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-02-22T18:41:27.428 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-22T18:41:27.428 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 4.18.0-540.el8.x86_64 2024-02-22T18:41:27.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-22T18:41:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-02-22T18:41:29.114 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-02-22T18:41:29.116 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-02-22T18:41:29.144 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.YyfBajc1MG 2024-02-22T18:41:29.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YyfBajc1MG 2024-02-22T18:41:29.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.YyfBajc1MG 2024-02-22T18:41:29.363 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.YyfBajc1MG is 6KB 2024-02-22T18:41:29.411 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.YyfBajc1MG 2024-02-22T18:41:29.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-02-22T18:41:29.506 INFO:teuthology.orchestra.run.smithi053.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-22T18:41:29.506 INFO:teuthology.orchestra.run.smithi053.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-22T18:41:29.506 INFO:teuthology.orchestra.run.smithi053.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-22T18:41:29.506 INFO:teuthology.orchestra.run.smithi053.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64.conf 2024-02-22T18:41:29.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64 2024-02-22T18:41:29.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-02-22T18:41:30.972 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-02-22T18:41:30.973 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-02-22T18:41:30.995 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.Qh45TTFaR2 2024-02-22T18:41:30.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Qh45TTFaR2 2024-02-22T18:41:31.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.Qh45TTFaR2 2024-02-22T18:41:31.216 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.Qh45TTFaR2 is 6KB 2024-02-22T18:41:31.263 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.Qh45TTFaR2 2024-02-22T18:41:31.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-02-22T18:41:31.347 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-22T18:41:31.347 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-22T18:41:31.347 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-22T18:41:31.347 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64.conf 2024-02-22T18:41:31.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64 2024-02-22T18:41:31.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-02-22T18:41:59.716 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-22T18:41:59.716 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-02-22T18:41:59.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:42:01.560 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-22T18:42:01.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T18:42:01.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:42:18.048 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-02-22T18:42:20.092 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-22T18:42:27.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-02-22T18:42:27.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:42:29.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T18:42:29.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:42:32.156 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-22T18:42:44.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T18:42:44.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:42:58.877 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-02-22T18:42:59.244 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-02-22T18:42:59.244 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-540.el8.x86_64"... 2024-02-22T18:42:59.244 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-02-22T18:42:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:4.18.0-540.el8.x86_64 2024-02-22T18:42:59.263 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-540.el8.x86_64 vs 4.18.0-540.el8.x86_64 2024-02-22T18:42:59.263 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-22T18:42:59.263 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-22T18:43:00.264 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-22T18:43:00.264 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-22T18:43:00.346 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-02-22T18:43:00.361 DEBUG:teuthology.parallel:result is None 2024-02-22T18:43:02.588 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-22T18:43:17.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T18:43:17.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T18:43:17.949 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-02-22T18:43:18.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T18:43:18.201 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-540.el8.x86_64"... 2024-02-22T18:43:18.202 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-02-22T18:43:18.263 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-540.el8.x86_64 2024-02-22T18:43:18.263 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-540.el8.x86_64 vs 4.18.0-540.el8.x86_64 2024-02-22T18:43:18.263 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-22T18:43:18.263 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-22T18:43:19.264 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-22T18:43:19.265 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-22T18:43:19.861 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-02-22T18:43:19.900 DEBUG:teuthology.parallel:result is None 2024-02-22T18:43:19.900 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-22T18:43:19.911 INFO:teuthology.task.internal:Creating test directory... 2024-02-22T18:43:19.911 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-22T18:43:19.916 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-22T18:43:20.372 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-22T18:43:20.379 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-22T18:43:20.387 INFO:teuthology.task.internal:Creating archive directory... 2024-02-22T18:43:20.387 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-22T18:43:20.390 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-22T18:43:20.575 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-22T18:43:20.581 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-22T18:43:20.582 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-22T18:43:20.584 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-22T18:43:20.630 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T18:43:20.636 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T18:43:20.642 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T18:43:20.647 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T18:43:20.648 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-22T18:43:20.655 INFO:teuthology.task.internal:Configuring sudo... 2024-02-22T18:43:20.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-22T18:43:20.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-22T18:43:20.687 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-22T18:43:20.696 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-22T18:43:20.697 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-22T18:43:20.730 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-22T18:43:20.748 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T18:43:20.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T18:43:20.888 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T18:43:20.974 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T18:43:21.019 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T18:43:21.019 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-22T18:43:21.089 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T18:43:21.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T18:43:21.218 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T18:43:21.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T18:43:21.308 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T18:43:21.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-22T18:43:21.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-02-22T18:43:21.378 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-02-22T18:43:21.445 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T18:43:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T18:43:21.918 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-22T18:43:21.926 INFO:teuthology.task.internal:Starting timer... 2024-02-22T18:43:21.926 INFO:teuthology.run_tasks:Running task pcp... 2024-02-22T18:43:21.936 INFO:teuthology.run_tasks:Running task selinux... 2024-02-22T18:43:21.949 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-22T18:43:21.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-02-22T18:43:22.068 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-02-22T18:43:22.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-02-22T18:43:22.147 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-02-22T18:43:22.151 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-22T18:43:22.151 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-02-22T18:43:22.199 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-02-22T18:43:22.199 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-02-22T18:43:22.223 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-02-22T18:43:22.223 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive'} 2024-02-22T18:43:22.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T18:43:22.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T18:43:22.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T18:43:22.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T18:43:22.293 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-22T18:43:22.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-02-22T18:43:22.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-02-22T18:43:22.362 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-22T18:43:22.372 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-22T18:43:22.390 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-22T18:43:22.392 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi064.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-22T19:02:07.281 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com')] 2024-02-22T19:02:07.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-02-22T19:02:07.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T19:02:07.360 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-02-22T19:02:07.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-02-22T19:02:07.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T19:02:07.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T19:02:07.457 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-02-22T19:02:07.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-22T19:02:07.528 INFO:teuthology.run_tasks:Running task clock... 2024-02-22T19:02:07.539 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-22T19:02:07.540 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-22T19:02:07.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T19:02:07.543 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-22T19:02:07.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T19:02:07.573 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-22T19:02:07.588 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-22T19:02:07.601 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-22T19:02:07.615 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-22T19:02:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-02-22T19:02:07.662 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-02-22T19:02:07.677 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-22T19:02:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-02-22T19:02:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-22T19:02:07.693 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-02-22T19:02:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-22T19:02:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-22T19:02:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2024-02-22T19:02:07.767 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T19:02:07.767 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-02-22T19:02:07.767 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.767 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.767 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.767 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.793 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-02-22T19:02:07.796 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T19:02:07.797 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-02-22T19:02:07.797 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.797 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.797 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.797 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T19:02:07.797 INFO:teuthology.run_tasks:Running task pexec... 2024-02-22T19:02:07.806 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-22T19:02:07.807 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi053.front.sepia.ceph.com 2024-02-22T19:02:07.807 DEBUG:teuthology.orchestra.run.smithi053:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-22T19:02:07.807 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi064.front.sepia.ceph.com 2024-02-22T19:02:07.807 DEBUG:teuthology.orchestra.run.smithi064:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-22T19:02:08.627 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:05:47 ago on Thu 22 Feb 2024 06:56:21 PM UTC. 2024-02-22T19:02:08.662 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:05:45 ago on Thu 22 Feb 2024 06:56:23 PM UTC. 2024-02-22T19:02:10.808 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout:Resetting modules: 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:02:10.809 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:10.885 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:02:10.886 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:02:10.886 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout:Resetting modules: 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:02:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:11.090 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:02:11.153 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:02:11.979 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:05:50 ago on Thu 22 Feb 2024 06:56:21 PM UTC. 2024-02-22T19:02:12.044 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:05:48 ago on Thu 22 Feb 2024 06:56:23 PM UTC. 2024-02-22T19:02:14.184 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout:Installing group/module packages: 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-22T19:02:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: crun x86_64 1.14.1-1.module_el8+951+32019cde appstream 257 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout:Installing module profiles: 2024-02-22T19:02:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools/common 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools rhel8 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-02-22T19:02:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:14.226 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 27 M 2024-02-22T19:02:14.226 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 91 M 2024-02-22T19:02:14.226 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:02:14.236 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout:Installing group/module packages: 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: crun x86_64 1.14.1-1.module_el8+951+32019cde appstream 257 k 2024-02-22T19:02:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout:Installing module profiles: 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools/common 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:14.274 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools rhel8 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:14.275 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 27 M 2024-02-22T19:02:14.276 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 91 M 2024-02-22T19:02:14.276 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:02:14.724 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] cockpit-podman-83-1.module_el8+847+7863d4e6.noarch.rpm: Status code: 503 for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/cockpit-podman-83-1.module_el8%2b847%2b7863d4e6.noarch.rpm (IP: 199.165.82.202) 2024-02-22T19:02:14.724 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] buildah-1.33.5-1.module_el8+885+7da147f3.x86_64.rpm: Status code: 503 for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/buildah-1.33.5-1.module_el8%2b885%2b7da147f3.x86_64.rpm (IP: 199.165.82.202) 2024-02-22T19:02:15.139 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 1.0 MB/s | 688 kB 00:00 2024-02-22T19:02:15.383 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64.rpm: Status code: 503 for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/criu-libs-3.15-3.module_el8.7.0%2b1216%2bb022c01d.x86_64.rpm (IP: 199.165.82.202) 2024-02-22T19:02:15.456 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 120 kB/s | 38 kB 00:00 2024-02-22T19:02:15.534 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 543 kB/s | 688 kB 00:01 2024-02-22T19:02:15.565 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): crun-1.14.1-1.module_el8+951+32019cde.x 2.3 MB/s | 257 kB 00:00 2024-02-22T19:02:15.834 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 658 kB/s | 1.0 MB 00:01 2024-02-22T19:02:15.834 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64.rpm: Status code: 503 for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/criu-libs-3.15-3.module_el8.7.0%2b1216%2bb022c01d.x86_64.rpm (IP: 199.165.82.202) 2024-02-22T19:02:15.993 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 83 kB/s | 38 kB 00:00 2024-02-22T19:02:16.176 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): buildah-1.33.5-1.module_el8+885+7da147f 5.7 MB/s | 9.7 MB 00:01 2024-02-22T19:02:16.282 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.7 MB/s | 180 kB 00:00 2024-02-22T19:02:16.340 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 425 kB/s | 25 kB 00:00 2024-02-22T19:02:16.399 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): netavark-1.10.3-1.module_el8+951+32019c 4.9 MB/s | 4.1 MB 00:00 2024-02-22T19:02:16.424 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.1 MB/s | 94 kB 00:00 2024-02-22T19:02:16.688 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): netavark-1.10.3-1.module_el8+951+32019c 5.9 MB/s | 4.1 MB 00:00 2024-02-22T19:02:16.752 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 2.6 MB/s | 180 kB 00:00 2024-02-22T19:02:16.810 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 425 kB/s | 25 kB 00:00 2024-02-22T19:02:16.835 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): crun-1.14.1-1.module_el8+951+32019cde.x 257 kB/s | 257 kB 00:01 2024-02-22T19:02:16.904 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.0 MB/s | 94 kB 00:00 2024-02-22T19:02:17.100 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 3.7 MB/s | 2.5 MB 00:00 2024-02-22T19:02:17.175 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 704 kB/s | 53 kB 00:00 2024-02-22T19:02:17.211 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 8.2 MB/s | 2.5 MB 00:00 2024-02-22T19:02:17.233 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 704 kB/s | 41 kB 00:00 2024-02-22T19:02:17.303 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 574 kB/s | 53 kB 00:00 2024-02-22T19:02:17.362 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 703 kB/s | 41 kB 00:00 2024-02-22T19:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): skopeo-1.14.3-0.1.module_el8+885+7da14 11 MB/s | 8.8 MB 00:00 2024-02-22T19:02:18.625 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): skopeo-1.14.3-0.1.module_el8+885+7da14 4.0 MB/s | 8.8 MB 00:02 2024-02-22T19:02:18.696 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): buildah-1.33.5-1.module_el8+885+7da147 2.2 MB/s | 9.7 MB 00:04 2024-02-22T19:02:18.696 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:02:18.696 INFO:teuthology.orchestra.run.smithi053.stdout:Total 6.1 MB/s | 27 MB 00:04 2024-02-22T19:02:18.979 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:02:19.001 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:02:19.001 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:02:19.026 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): aardvark-dns-1.10.0-1.module_el8+871+5 227 kB/s | 1.0 MB 00:04 2024-02-22T19:02:19.026 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:02:19.026 INFO:teuthology.orchestra.run.smithi064.stdout:Total 5.8 MB/s | 27 MB 00:04 2024-02-22T19:02:19.282 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:02:19.285 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:02:19.308 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:02:19.331 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:02:19.331 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:02:19.625 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:02:19.628 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:02:20.029 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:02:20.328 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:02:21.020 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-22T19:02:21.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T19:02:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-22T19:02:21.403 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-22T19:02:21.495 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T19:02:21.576 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-22T19:02:21.648 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-22T19:02:21.836 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-22T19:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-22T19:02:22.198 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-22T19:02:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-22T19:02:22.783 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-22T19:02:23.461 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-02-22T19:02:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-22T19:02:23.915 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-02-22T19:02:23.919 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : crun-1.14.1-1.module_el8+951+32019cde.x86_64 9/13 2024-02-22T19:02:24.157 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-22T19:02:24.330 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : crun-1.14.1-1.module_el8+951+32019cde.x86_64 9/13 2024-02-22T19:02:24.593 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-22T19:02:24.819 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T19:02:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-22T19:02:25.034 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T19:02:25.156 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T19:02:25.227 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T19:02:25.396 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T19:02:25.563 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : crun-1.14.1-1.module_el8+951+32019cde.x86_64 5/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-22T19:02:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-22T19:02:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-22T19:02:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T19:02:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T19:02:26.009 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T19:02:26.009 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-22T19:02:26.009 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : crun-1.14.1-1.module_el8+951+32019cde.x86_64 5/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T19:02:26.010 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-22T19:02:26.273 INFO:teuthology.orchestra.run.smithi053.stdout: crun-1.14.1-1.module_el8+951+32019cde.x86_64 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:02:26.274 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:02:26.549 DEBUG:teuthology.parallel:result is None 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: crun-1.14.1-1.module_el8+951+32019cde.x86_64 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-22T19:02:26.819 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:02:26.820 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:02:27.082 DEBUG:teuthology.parallel:result is None 2024-02-22T19:02:27.082 INFO:teuthology.run_tasks:Running task install... 2024-02-22T19:02:27.092 DEBUG:teuthology.task.install:project ceph 2024-02-22T19:02:27.092 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'}} 2024-02-22T19:02:27.092 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'} 2024-02-22T19:02:27.092 INFO:teuthology.task.install:Using flavor: default 2024-02-22T19:02:27.098 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-22T19:02:27.098 INFO:teuthology.task.install:extra packages: [] 2024-02-22T19:02:27.098 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59', 'tag': None, 'wait_for_package': False} 2024-02-22T19:02:27.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-22T19:02:27.101 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59', 'tag': None, 'wait_for_package': False} 2024-02-22T19:02:27.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-22T19:02:27.252 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/ 2024-02-22T19:02:27.252 INFO:teuthology.task.install.rpm:Package version is 16.2.14-550.geb66ed92 2024-02-22T19:02:27.256 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/ 2024-02-22T19:02:27.257 INFO:teuthology.task.install.rpm:Package version is 16.2.14-550.geb66ed92 2024-02-22T19:02:27.362 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-22T19:02:27.362 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:02:27.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-22T19:02:27.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-22T19:02:27.372 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:02:27.372 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-22T19:02:27.393 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-22T19:02:27.393 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/eb66ed921e744301e6863be6353618d63967ea59/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-22T19:02:27.407 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-22T19:02:27.407 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/eb66ed921e744301e6863be6353618d63967ea59/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-22T19:02:27.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-22T19:02:27.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-22T19:02:27.542 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-22T19:02:27.560 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-22T19:02:27.609 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-02-22T19:02:27.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-02-22T19:02:27.629 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-02-22T19:02:27.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-02-22T19:02:27.898 INFO:teuthology.orchestra.run.smithi064.stdout:76 files removed 2024-02-22T19:02:27.911 INFO:teuthology.orchestra.run.smithi053.stdout:76 files removed 2024-02-22T19:02:27.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw 2024-02-22T19:02:27.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw 2024-02-22T19:02:28.701 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 220 kB/s | 76 kB 00:00 2024-02-22T19:02:28.873 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 151 kB/s | 76 kB 00:00 2024-02-22T19:02:28.977 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 65 kB/s | 15 kB 00:00 2024-02-22T19:02:29.361 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 4.1 kB/s | 1.4 kB 00:00 2024-02-22T19:02:29.365 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 34 kB/s | 15 kB 00:00 2024-02-22T19:02:29.850 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 3.2 kB/s | 1.4 kB 00:00 2024-02-22T19:02:34.986 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - AppStream 4.9 MB/s | 27 MB 00:05 2024-02-22T19:02:35.325 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - AppStream 5.0 MB/s | 27 MB 00:05 2024-02-22T19:11:00.353 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - BaseOS 21 kB/s | 10 MB 08:20 2024-02-22T19:11:02.519 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras 36 kB/s | 18 kB 00:00 2024-02-22T19:11:02.578 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - BaseOS 21 kB/s | 10 MB 08:22 2024-02-22T19:11:03.203 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.9 kB 00:00 2024-02-22T19:11:04.570 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - Extras 58 kB/s | 18 kB 00:00 2024-02-22T19:11:04.929 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - Extras common packages 29 kB/s | 6.9 kB 00:00 2024-02-22T19:11:05.613 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - PowerTools 2.1 MB/s | 4.9 MB 00:02 2024-02-22T19:11:06.740 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-02-22T19:11:07.563 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 8 - PowerTools 1.9 MB/s | 4.9 MB 00:02 2024-02-22T19:11:07.900 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-02-22T19:11:08.673 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-02-22T19:11:10.135 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-02-22T19:11:11.881 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 390 kB/s | 24 kB 00:00 2024-02-22T19:11:14.074 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 355 kB/s | 24 kB 00:00 2024-02-22T19:11:15.691 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:11:15.695 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:11:15.695 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:11:15.695 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:11:15.695 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 12 M 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 4.2 M 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.1 M 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 24 M 2024-02-22T19:11:15.696 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:16.2.14-550.geb66ed92.el8 ceph 26 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 825 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 494 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.8 M 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-22T19:11:15.697 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 45 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 99 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:16.2.14-550.geb66ed92.el8 ceph 215 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:16.2.14-550.geb66ed92.el8 ceph 389 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 369 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 114 k 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:11:15.698 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:11:15.699 INFO:teuthology.orchestra.run.smithi064.stdout:Install 21 Packages 2024-02-22T19:11:15.699 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 2 Packages 2024-02-22T19:11:15.699 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:11:15.699 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 58 M 2024-02-22T19:11:15.699 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:11:16.611 INFO:teuthology.orchestra.run.smithi064.stdout:(1/23): ceph-base-16.2.14-550.geb66ed92.el8.x86 11 MB/s | 6.1 MB 00:00 2024-02-22T19:11:16.666 INFO:teuthology.orchestra.run.smithi064.stdout:(2/23): ceph-selinux-16.2.14-550.geb66ed92.el8. 466 kB/s | 26 kB 00:00 2024-02-22T19:11:16.758 INFO:teuthology.orchestra.run.smithi064.stdout:(3/23): libcephfs2-16.2.14-550.geb66ed92.el8.x8 8.8 MB/s | 825 kB 00:00 2024-02-22T19:11:16.925 INFO:teuthology.orchestra.run.smithi064.stdout:(4/23): libradosstriper1-16.2.14-550.geb66ed92. 2.9 MB/s | 494 kB 00:00 2024-02-22T19:11:17.184 INFO:teuthology.orchestra.run.smithi064.stdout:(5/23): librgw2-16.2.14-550.geb66ed92.el8.x86_6 15 MB/s | 3.8 MB 00:00 2024-02-22T19:11:17.250 INFO:teuthology.orchestra.run.smithi064.stdout:(6/23): python3-ceph-argparse-16.2.14-550.geb66 678 kB/s | 45 kB 00:00 2024-02-22T19:11:17.326 INFO:teuthology.orchestra.run.smithi064.stdout:(7/23): python3-ceph-common-16.2.14-550.geb66ed 1.3 MB/s | 99 kB 00:00 2024-02-22T19:11:17.401 INFO:teuthology.orchestra.run.smithi064.stdout:(8/23): python3-cephfs-16.2.14-550.geb66ed92.el 2.8 MB/s | 215 kB 00:00 2024-02-22T19:11:17.476 INFO:teuthology.orchestra.run.smithi064.stdout:(9/23): python3-rados-16.2.14-550.geb66ed92.el8 5.1 MB/s | 389 kB 00:00 2024-02-22T19:11:17.543 INFO:teuthology.orchestra.run.smithi064.stdout:(10/23): python3-rbd-16.2.14-550.geb66ed92.el8. 5.4 MB/s | 369 kB 00:00 2024-02-22T19:11:17.593 INFO:teuthology.orchestra.run.smithi064.stdout:(11/23): python3-rgw-16.2.14-550.geb66ed92.el8. 2.2 MB/s | 114 kB 00:00 2024-02-22T19:11:17.890 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 12 M 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 4.2 M 2024-02-22T19:11:17.893 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.1 M 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 24 M 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:16.2.14-550.geb66ed92.el8 ceph 26 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 825 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 494 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-22T19:11:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.8 M 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 45 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 99 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:16.2.14-550.geb66ed92.el8 ceph 215 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:16.2.14-550.geb66ed92.el8 ceph 389 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 369 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 114 k 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:11:17.895 INFO:teuthology.orchestra.run.smithi053.stdout:Install 21 Packages 2024-02-22T19:11:17.896 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 2 Packages 2024-02-22T19:11:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:11:17.896 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 58 M 2024-02-22T19:11:17.896 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:11:18.110 INFO:teuthology.orchestra.run.smithi064.stdout:(12/23): ceph-radosgw-16.2.14-550.geb66ed92.el8 5.7 MB/s | 12 MB 00:02 2024-02-22T19:11:18.594 INFO:teuthology.orchestra.run.smithi064.stdout:(13/23): ceph-common-16.2.14-550.geb66ed92.el8. 9.6 MB/s | 24 MB 00:02 2024-02-22T19:11:18.752 INFO:teuthology.orchestra.run.smithi064.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 584 kB/s | 677 kB 00:01 2024-02-22T19:11:19.028 INFO:teuthology.orchestra.run.smithi064.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 283 kB/s | 259 kB 00:00 2024-02-22T19:11:19.103 INFO:teuthology.orchestra.run.smithi064.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 86 kB/s | 44 kB 00:00 2024-02-22T19:11:19.153 INFO:teuthology.orchestra.run.smithi064.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 2.4 MB/s | 306 kB 00:00 2024-02-22T19:11:19.229 INFO:teuthology.orchestra.run.smithi064.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 786 kB/s | 59 kB 00:00 2024-02-22T19:11:19.254 INFO:teuthology.orchestra.run.smithi064.stdout:(19/23): leveldb-1.22-1.el8.x86_64.rpm 1.2 MB/s | 181 kB 00:00 2024-02-22T19:11:19.279 INFO:teuthology.orchestra.run.smithi064.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2024-02-22T19:11:19.892 INFO:teuthology.orchestra.run.smithi053.stdout:(1/23): ceph-radosgw-16.2.14-550.geb66ed92.el8. 8.2 MB/s | 12 MB 00:01 2024-02-22T19:11:19.975 INFO:teuthology.orchestra.run.smithi053.stdout:(2/23): ceph-selinux-16.2.14-550.geb66ed92.el8. 309 kB/s | 26 kB 00:00 2024-02-22T19:11:20.047 INFO:teuthology.orchestra.run.smithi064.stdout:(21/23): librbd1-16.2.14-550.geb66ed92.el8.x86_ 5.4 MB/s | 4.2 MB 00:00 2024-02-22T19:11:20.125 INFO:teuthology.orchestra.run.smithi053.stdout:(3/23): libcephfs2-16.2.14-550.geb66ed92.el8.x8 5.4 MB/s | 825 kB 00:00 2024-02-22T19:11:20.251 INFO:teuthology.orchestra.run.smithi053.stdout:(4/23): libradosstriper1-16.2.14-550.geb66ed92. 3.9 MB/s | 494 kB 00:00 2024-02-22T19:11:20.438 INFO:teuthology.orchestra.run.smithi064.stdout:(22/23): librados2-16.2.14-550.geb66ed92.el8.x8 3.2 MB/s | 3.7 MB 00:01 2024-02-22T19:11:20.484 INFO:teuthology.orchestra.run.smithi053.stdout:(5/23): ceph-common-16.2.14-550.geb66ed92.el8.x 12 MB/s | 24 MB 00:02 2024-02-22T19:11:20.535 INFO:teuthology.orchestra.run.smithi053.stdout:(6/23): python3-ceph-argparse-16.2.14-550.geb66 906 kB/s | 45 kB 00:00 2024-02-22T19:11:20.619 INFO:teuthology.orchestra.run.smithi053.stdout:(7/23): python3-ceph-common-16.2.14-550.geb66ed 1.2 MB/s | 99 kB 00:00 2024-02-22T19:11:20.677 INFO:teuthology.orchestra.run.smithi053.stdout:(8/23): librgw2-16.2.14-550.geb66ed92.el8.x86_6 8.9 MB/s | 3.8 MB 00:00 2024-02-22T19:11:20.702 INFO:teuthology.orchestra.run.smithi053.stdout:(9/23): python3-cephfs-16.2.14-550.geb66ed92.el 2.5 MB/s | 215 kB 00:00 2024-02-22T19:11:20.736 INFO:teuthology.orchestra.run.smithi053.stdout:(10/23): python3-rados-16.2.14-550.geb66ed92.el 6.5 MB/s | 389 kB 00:00 2024-02-22T19:11:20.761 INFO:teuthology.orchestra.run.smithi053.stdout:(11/23): python3-rbd-16.2.14-550.geb66ed92.el8. 6.2 MB/s | 369 kB 00:00 2024-02-22T19:11:20.786 INFO:teuthology.orchestra.run.smithi053.stdout:(12/23): python3-rgw-16.2.14-550.geb66ed92.el8. 2.2 MB/s | 114 kB 00:00 2024-02-22T19:11:21.195 INFO:teuthology.orchestra.run.smithi053.stdout:(13/23): ceph-base-16.2.14-550.geb66ed92.el8.x8 2.3 MB/s | 6.1 MB 00:02 2024-02-22T19:11:21.737 INFO:teuthology.orchestra.run.smithi053.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 273 kB/s | 259 kB 00:00 2024-02-22T19:11:21.762 INFO:teuthology.orchestra.run.smithi053.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 77 kB/s | 44 kB 00:00 2024-02-22T19:11:22.112 INFO:teuthology.orchestra.run.smithi053.stdout:(16/23): librdkafka-1.6.1-1.el8.x86_64.rpm 501 kB/s | 677 kB 00:01 2024-02-22T19:11:22.171 INFO:teuthology.orchestra.run.smithi053.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 749 kB/s | 306 kB 00:00 2024-02-22T19:11:22.263 INFO:teuthology.orchestra.run.smithi053.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 676 kB/s | 59 kB 00:00 2024-02-22T19:11:22.347 INFO:teuthology.orchestra.run.smithi053.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 902 kB/s | 75 kB 00:00 2024-02-22T19:11:22.480 INFO:teuthology.orchestra.run.smithi053.stdout:(20/23): leveldb-1.22-1.el8.x86_64.rpm 492 kB/s | 181 kB 00:00 2024-02-22T19:11:22.806 INFO:teuthology.orchestra.run.smithi053.stdout:(21/23): librbd1-16.2.14-550.geb66ed92.el8.x86_ 13 MB/s | 4.2 MB 00:00 2024-02-22T19:11:23.431 INFO:teuthology.orchestra.run.smithi053.stdout:(22/23): librados2-16.2.14-550.geb66ed92.el8.x8 3.5 MB/s | 3.7 MB 00:01 2024-02-22T19:15:25.414 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] librabbitmq-0.9.0-5.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/librabbitmq-0.9.0-5.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:15:25.751 INFO:teuthology.orchestra.run.smithi064.stdout:(23/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 195 B/s | 47 kB 04:06 2024-02-22T19:15:25.751 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:15:25.751 INFO:teuthology.orchestra.run.smithi064.stdout:Total 237 kB/s | 58 MB 04:10 2024-02-22T19:15:25.794 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:15:25.846 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:15:25.846 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:15:26.199 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:15:26.202 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:15:27.281 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:15:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-22T19:15:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T19:15:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T19:15:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-22T19:15:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-22T19:15:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-22T19:15:28.948 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-22T19:15:29.386 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-22T19:15:29.590 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-22T19:15:29.664 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-22T19:15:29.862 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-22T19:15:30.365 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-22T19:15:30.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-22T19:15:30.562 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-22T19:15:30.771 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-22T19:15:30.938 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 11/25 2024-02-22T19:15:31.094 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-22T19:15:31.277 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-22T19:15:31.609 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 14/25 2024-02-22T19:15:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-22T19:15:31.817 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-22T19:15:32.007 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-22T19:15:32.154 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] librabbitmq-0.9.0-5.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/librabbitmq-0.9.0-5.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:15:32.190 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-22T19:15:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-22T19:15:32.491 INFO:teuthology.orchestra.run.smithi053.stdout:(23/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 192 B/s | 47 kB 04:10 2024-02-22T19:15:32.491 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:15:32.491 INFO:teuthology.orchestra.run.smithi053.stdout:Total 233 kB/s | 58 MB 04:14 2024-02-22T19:15:32.533 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:15:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 19/25 2024-02-22T19:15:32.585 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:15:32.585 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:15:32.951 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:15:32.953 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:15:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:15:34.155 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-22T19:15:34.314 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T19:15:34.705 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T19:15:34.889 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-22T19:15:34.983 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-22T19:15:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-22T19:15:35.243 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-22T19:15:35.372 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-22T19:15:35.456 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-22T19:15:35.809 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-22T19:15:35.964 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-22T19:15:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-22T19:15:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-22T19:15:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-22T19:15:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-22T19:15:36.730 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-22T19:15:36.749 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-22T19:15:36.864 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-22T19:15:36.872 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-22T19:15:36.928 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-22T19:15:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-22T19:15:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 11/25 2024-02-22T19:15:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-22T19:15:37.584 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-22T19:15:37.799 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 14/25 2024-02-22T19:15:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-22T19:15:37.991 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-22T19:15:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-22T19:15:38.364 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-22T19:15:38.646 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-22T19:15:38.840 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 19/25 2024-02-22T19:15:41.500 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-22T19:15:41.739 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-22T19:15:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-22T19:15:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-22T19:15:42.854 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-22T19:15:42.996 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-22T19:15:56.647 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /run 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-02-22T19:15:56.648 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:15:57.916 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-22T19:15:57.991 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-22T19:15:57.991 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T19:15:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T19:15:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T19:15:59.039 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T19:15:59.039 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 1/25 2024-02-22T19:15:59.039 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2/25 2024-02-22T19:15:59.039 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 6/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 7/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 8/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 9/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 10/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 11/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-22T19:15:59.040 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-22T19:15:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 24/25 2024-02-22T19:15:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: leveldb-1.22-1.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-22T19:15:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:15:59.921 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:16:00.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-test 2024-02-22T19:16:00.998 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:04:49 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:16:02.682 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:16:02.683 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:16.2.14-550.geb66ed92.el8 ceph 52 M 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:16.2.14-550.geb66ed92.el8 ceph 174 k 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:16:02.684 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:16:02.685 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-22T19:16:02.685 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:16:02.685 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 53 M 2024-02-22T19:16:02.685 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 197 M 2024-02-22T19:16:02.685 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:16:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-22T19:16:02.797 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-02-22T19:16:02.797 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-02-22T19:16:02.797 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2024-02-22T19:16:02.797 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /run 2024-02-22T19:16:02.798 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-02-22T19:16:02.798 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-02-22T19:16:02.798 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-02-22T19:16:02.798 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-02-22T19:16:02.798 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2024-02-22T19:16:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:03.049 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): libcephsqlite-16.2.14-550.geb66ed92.el8. 962 kB/s | 174 kB 00:00 2024-02-22T19:16:03.782 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 222 kB/s | 203 kB 00:00 2024-02-22T19:16:03.941 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 211 kB/s | 188 kB 00:00 2024-02-22T19:16:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-22T19:16:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-22T19:16:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T19:16:04.479 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T19:16:04.480 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T19:16:04.583 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 110 kB/s | 71 kB 00:00 2024-02-22T19:16:05.025 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 260 kB/s | 323 kB 00:01 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 1/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 6/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 7/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 8/25 2024-02-22T19:16:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 9/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 10/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 11/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-22T19:16:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-22T19:16:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 24/25 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: leveldb-1.22-1.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:05.962 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:16:06.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-test 2024-02-22T19:16:07.058 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:04:53 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:16:08.742 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:16.2.14-550.geb66ed92.el8 ceph 52 M 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:16.2.14-550.geb66ed92.el8 ceph 174 k 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-22T19:16:08.743 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 53 M 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 197 M 2024-02-22T19:16:08.744 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:16:09.126 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): libcephsqlite-16.2.14-550.geb66ed92.el8. 894 kB/s | 174 kB 00:00 2024-02-22T19:16:09.491 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): ceph-test-16.2.14-550.geb66ed92.el8.x86_ 7.9 MB/s | 52 MB 00:06 2024-02-22T19:16:09.493 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:16:09.494 INFO:teuthology.orchestra.run.smithi064.stdout:Total 7.8 MB/s | 53 MB 00:06 2024-02-22T19:16:09.538 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:16:09.561 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:16:09.562 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:16:09.818 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 229 kB/s | 203 kB 00:00 2024-02-22T19:16:09.881 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:16:09.884 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:16:10.019 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 211 kB/s | 188 kB 00:00 2024-02-22T19:16:10.423 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:16:10.653 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 111 kB/s | 71 kB 00:00 2024-02-22T19:16:10.707 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-22T19:16:11.005 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-22T19:16:11.069 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 258 kB/s | 323 kB 00:01 2024-02-22T19:16:11.126 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T19:16:11.205 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T19:16:11.417 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-22T19:16:11.536 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): ceph-test-16.2.14-550.geb66ed92.el8.x86_ 20 MB/s | 52 MB 00:02 2024-02-22T19:16:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:16:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:Total 19 MB/s | 53 MB 00:02 2024-02-22T19:16:11.559 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-22T19:16:11.581 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:16:11.604 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:16:11.604 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:16:11.934 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:16:11.937 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:16:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:16:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-22T19:16:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-22T19:16:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T19:16:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T19:16:13.267 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-22T19:16:13.534 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-22T19:16:16.490 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-22T19:16:16.724 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-22T19:16:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-22T19:16:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 1/6 2024-02-22T19:16:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2/6 2024-02-22T19:16:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-22T19:16:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-22T19:16:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-8.el8.x86_64 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-22T19:16:17.457 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-22T19:16:17.458 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:16:17.458 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:16:17.847 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph 2024-02-22T19:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-22T19:16:18.633 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:05:07 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-22T19:16:18.967 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-22T19:16:18.967 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 1/6 2024-02-22T19:16:18.967 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2/6 2024-02-22T19:16:18.967 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-22T19:16:18.967 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-22T19:16:18.967 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-8.el8.x86_64 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-22T19:16:19.435 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-22T19:16:19.436 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:19.436 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:16:19.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph 2024-02-22T19:16:20.309 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:16:20.312 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:16:20.312 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:16:20.312 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:16:20.312 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:16:20.312 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.4 k 2024-02-22T19:16:20.312 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:16.2.14-550.geb66ed92.el8 ceph 2.4 M 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:16.2.14-550.geb66ed92.el8 ceph 1.6 M 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 245 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:16.2.14-550.geb66ed92.el8 ceph 5.2 M 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 19 M 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-22T19:16:20.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-22T19:16:20.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout:Install 32 Packages 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 32 M 2024-02-22T19:16:20.315 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 104 M 2024-02-22T19:16:20.316 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:16:20.469 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:05:06 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:16:20.919 INFO:teuthology.orchestra.run.smithi064.stdout:(1/32): ceph-16.2.14-550.geb66ed92.el8.x86_64.r 34 kB/s | 6.4 kB 00:00 2024-02-22T19:16:21.111 INFO:teuthology.orchestra.run.smithi064.stdout:(2/32): ceph-mgr-16.2.14-550.geb66ed92.el8.x86_ 4.2 MB/s | 1.6 MB 00:00 2024-02-22T19:16:21.152 INFO:teuthology.orchestra.run.smithi064.stdout:(3/32): ceph-mds-16.2.14-550.geb66ed92.el8.x86_ 5.7 MB/s | 2.4 MB 00:00 2024-02-22T19:16:21.228 INFO:teuthology.orchestra.run.smithi064.stdout:(4/32): ceph-mgr-modules-core-16.2.14-550.geb66 3.2 MB/s | 245 kB 00:00 2024-02-22T19:16:21.779 INFO:teuthology.orchestra.run.smithi064.stdout:(5/32): ceph-mon-16.2.14-550.geb66ed92.el8.x86_ 6.0 MB/s | 5.2 MB 00:00 2024-02-22T19:16:22.135 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.4 k 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:16.2.14-550.geb66ed92.el8 ceph 2.4 M 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:16.2.14-550.geb66ed92.el8 ceph 1.6 M 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 245 k 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:16.2.14-550.geb66ed92.el8 ceph 5.2 M 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 19 M 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-22T19:16:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-22T19:16:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout:Install 32 Packages 2024-02-22T19:16:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:16:22.142 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 32 M 2024-02-22T19:16:22.142 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 104 M 2024-02-22T19:16:22.142 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:16:22.170 INFO:teuthology.orchestra.run.smithi064.stdout:(6/32): ceph-osd-16.2.14-550.geb66ed92.el8.x86_ 18 MB/s | 19 MB 00:01 2024-02-22T19:16:22.196 INFO:teuthology.orchestra.run.smithi064.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 162 kB/s | 157 kB 00:00 2024-02-22T19:16:22.304 INFO:teuthology.orchestra.run.smithi064.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 74 kB/s | 39 kB 00:00 2024-02-22T19:16:22.571 INFO:teuthology.orchestra.run.smithi053.stdout:(1/32): ceph-16.2.14-550.geb66ed92.el8.x86_64.r 20 kB/s | 6.4 kB 00:00 2024-02-22T19:16:22.863 INFO:teuthology.orchestra.run.smithi064.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 85 kB/s | 57 kB 00:00 2024-02-22T19:16:22.947 INFO:teuthology.orchestra.run.smithi064.stdout:(10/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 133 kB/s | 103 kB 00:00 2024-02-22T19:16:23.814 INFO:teuthology.orchestra.run.smithi064.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 303 kB/s | 457 kB 00:01 2024-02-22T19:16:24.263 INFO:teuthology.orchestra.run.smithi053.stdout:(2/32): ceph-mds-16.2.14-550.geb66ed92.el8.x86_ 1.2 MB/s | 2.4 MB 00:02 2024-02-22T19:16:24.421 INFO:teuthology.orchestra.run.smithi053.stdout:(3/32): ceph-mgr-16.2.14-550.geb66ed92.el8.x86_ 765 kB/s | 1.6 MB 00:02 2024-02-22T19:16:24.597 INFO:teuthology.orchestra.run.smithi053.stdout:(4/32): ceph-mgr-modules-core-16.2.14-550.geb66 1.4 MB/s | 245 kB 00:00 2024-02-22T19:16:25.497 INFO:teuthology.orchestra.run.smithi053.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 174 kB/s | 157 kB 00:00 2024-02-22T19:16:25.839 INFO:teuthology.orchestra.run.smithi053.stdout:(6/32): ceph-osd-16.2.14-550.geb66ed92.el8.x86_ 12 MB/s | 19 MB 00:01 2024-02-22T19:16:26.014 INFO:teuthology.orchestra.run.smithi053.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 75 kB/s | 39 kB 00:00 2024-02-22T19:16:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 159 kB/s | 103 kB 00:00 2024-02-22T19:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 89 kB/s | 57 kB 00:00 2024-02-22T19:16:27.673 INFO:teuthology.orchestra.run.smithi053.stdout:(10/32): ceph-mon-16.2.14-550.geb66ed92.el8.x86 1.0 MB/s | 5.2 MB 00:05 2024-02-22T19:16:27.990 INFO:teuthology.orchestra.run.smithi053.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 305 kB/s | 457 kB 00:01 2024-02-22T19:20:32.612 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] python3-cffi-1.11.5-6.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.11.5-6.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:20:32.615 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] python3-cryptography-3.2.1-7.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cryptography-3.2.1-7.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:20:32.615 INFO:teuthology.orchestra.run.smithi064.stdout:[MIRROR] python3-pycparser-2.14-14.el8.noarch.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.14-14.el8.noarch.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:20:33.034 INFO:teuthology.orchestra.run.smithi064.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 448 B/s | 109 kB 04:09 2024-02-22T19:20:33.160 INFO:teuthology.orchestra.run.smithi064.stdout:(13/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 kB/s | 249 kB 04:10 2024-02-22T19:20:33.185 INFO:teuthology.orchestra.run.smithi064.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 291 kB/s | 44 kB 00:00 2024-02-22T19:20:33.252 INFO:teuthology.orchestra.run.smithi064.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.7 MB/s | 173 kB 00:00 2024-02-22T19:20:33.294 INFO:teuthology.orchestra.run.smithi064.stdout:(16/32): python3-cryptography-3.2.1-7.el8.x86_6 2.6 kB/s | 641 kB 04:10 2024-02-22T19:20:33.327 INFO:teuthology.orchestra.run.smithi064.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 5.0 MB/s | 384 kB 00:00 2024-02-22T19:20:33.361 INFO:teuthology.orchestra.run.smithi064.stdout:(18/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 922 kB/s | 185 kB 00:00 2024-02-22T19:20:33.386 INFO:teuthology.orchestra.run.smithi064.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 328 kB/s | 18 kB 00:00 2024-02-22T19:20:33.411 INFO:teuthology.orchestra.run.smithi064.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 989 kB/s | 49 kB 00:00 2024-02-22T19:20:33.437 INFO:teuthology.orchestra.run.smithi064.stdout:(21/32): python3-jaraco-6.2-6.el8.noarch.rpm 78 kB/s | 11 kB 00:00 2024-02-22T19:20:33.462 INFO:teuthology.orchestra.run.smithi064.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 777 kB/s | 59 kB 00:00 2024-02-22T19:20:33.487 INFO:teuthology.orchestra.run.smithi064.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 309 kB/s | 16 kB 00:00 2024-02-22T19:20:33.513 INFO:teuthology.orchestra.run.smithi064.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 373 kB/s | 19 kB 00:00 2024-02-22T19:20:33.538 INFO:teuthology.orchestra.run.smithi064.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 477 kB/s | 24 kB 00:00 2024-02-22T19:20:33.572 INFO:teuthology.orchestra.run.smithi064.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 499 kB/s | 29 kB 00:00 2024-02-22T19:20:33.597 INFO:teuthology.orchestra.run.smithi064.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 469 kB/s | 27 kB 00:00 2024-02-22T19:20:33.672 INFO:teuthology.orchestra.run.smithi064.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2024-02-22T19:20:33.697 INFO:teuthology.orchestra.run.smithi064.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.0 MB/s | 255 kB 00:00 2024-02-22T19:20:33.723 INFO:teuthology.orchestra.run.smithi064.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2024-02-22T19:20:33.739 INFO:teuthology.orchestra.run.smithi064.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 549 kB/s | 23 kB 00:00 2024-02-22T19:20:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2024-02-22T19:20:33.768 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:20:33.768 INFO:teuthology.orchestra.run.smithi064.stdout:Total 129 kB/s | 32 MB 04:13 2024-02-22T19:20:33.846 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:20:33.889 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:20:33.889 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:20:34.217 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:20:34.221 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:20:35.098 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] python3-cffi-1.11.5-6.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.11.5-6.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:20:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] python3-cryptography-3.2.1-7.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cryptography-3.2.1-7.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:20:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:[MIRROR] python3-pycparser-2.14-14.el8.noarch.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.14-14.el8.noarch.rpm [Recv failure: Connection reset by peer] 2024-02-22T19:20:35.520 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:20:35.526 INFO:teuthology.orchestra.run.smithi053.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 451 B/s | 109 kB 04:07 2024-02-22T19:20:35.609 INFO:teuthology.orchestra.run.smithi053.stdout:(13/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 kB/s | 249 kB 04:08 2024-02-22T19:20:35.668 INFO:teuthology.orchestra.run.smithi053.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 2.6 kB/s | 641 kB 04:07 2024-02-22T19:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-22T19:20:35.760 INFO:teuthology.orchestra.run.smithi053.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 187 kB/s | 44 kB 00:00 2024-02-22T19:20:35.902 INFO:teuthology.orchestra.run.smithi053.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.7 MB/s | 384 kB 00:00 2024-02-22T19:20:35.927 INFO:teuthology.orchestra.run.smithi053.stdout:(17/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 585 kB/s | 185 kB 00:00 2024-02-22T19:20:35.953 INFO:teuthology.orchestra.run.smithi053.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 618 kB/s | 173 kB 00:00 2024-02-22T19:20:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-22T19:20:35.978 INFO:teuthology.orchestra.run.smithi053.stdout:(19/32): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2024-02-22T19:20:36.003 INFO:teuthology.orchestra.run.smithi053.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 233 kB/s | 18 kB 00:00 2024-02-22T19:20:36.028 INFO:teuthology.orchestra.run.smithi053.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 647 kB/s | 49 kB 00:00 2024-02-22T19:20:36.053 INFO:teuthology.orchestra.run.smithi053.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 778 kB/s | 59 kB 00:00 2024-02-22T19:20:36.087 INFO:teuthology.orchestra.run.smithi053.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2024-02-22T19:20:36.112 INFO:teuthology.orchestra.run.smithi053.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 320 kB/s | 19 kB 00:00 2024-02-22T19:20:36.138 INFO:teuthology.orchestra.run.smithi053.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2024-02-22T19:20:36.163 INFO:teuthology.orchestra.run.smithi053.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 317 kB/s | 24 kB 00:00 2024-02-22T19:20:36.188 INFO:teuthology.orchestra.run.smithi053.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 386 kB/s | 29 kB 00:00 2024-02-22T19:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-22T19:20:36.213 INFO:teuthology.orchestra.run.smithi053.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 365 kB/s | 27 kB 00:00 2024-02-22T19:20:36.239 INFO:teuthology.orchestra.run.smithi053.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.3 MB/s | 255 kB 00:00 2024-02-22T19:20:36.289 INFO:teuthology.orchestra.run.smithi053.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-02-22T19:20:36.306 INFO:teuthology.orchestra.run.smithi053.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2024-02-22T19:20:36.331 INFO:teuthology.orchestra.run.smithi053.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.7 MB/s | 251 kB 00:00 2024-02-22T19:20:36.334 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:20:36.334 INFO:teuthology.orchestra.run.smithi053.stdout:Total 128 kB/s | 32 MB 04:14 2024-02-22T19:20:36.413 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:20:36.421 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-22T19:20:36.455 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:20:36.455 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-22T19:20:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:20:36.780 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:20:36.784 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-22T19:20:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-22T19:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-22T19:20:37.409 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-22T19:20:37.687 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-22T19:20:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:20:37.934 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-22T19:20:38.100 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-22T19:20:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-22T19:20:38.307 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-22T19:20:38.528 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-22T19:20:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-22T19:20:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-22T19:20:38.928 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-22T19:20:38.960 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-22T19:20:39.118 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-22T19:20:39.131 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-22T19:20:39.402 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-22T19:20:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-22T19:20:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-22T19:20:39.608 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T19:20:39.808 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-22T19:20:39.906 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-22T19:20:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-22T19:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-22T19:20:40.249 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-22T19:20:40.350 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-22T19:20:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-22T19:20:40.520 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-22T19:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-22T19:20:40.852 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-22T19:20:41.094 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-22T19:20:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-22T19:20:41.290 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-22T19:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-22T19:20:41.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-22T19:20:41.680 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-22T19:20:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T19:20:41.982 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-22T19:20:42.195 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-22T19:20:42.338 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 27/32 2024-02-22T19:20:42.397 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-22T19:20:42.527 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-22T19:20:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-22T19:20:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-22T19:20:42.991 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-22T19:20:43.207 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-22T19:20:43.427 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-22T19:20:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-22T19:20:44.117 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-22T19:20:44.135 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-22T19:20:44.322 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-22T19:20:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 27/32 2024-02-22T19:20:44.619 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-22T19:20:44.792 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-22T19:20:44.989 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-22T19:20:45.239 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-22T19:20:45.414 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-22T19:20:45.432 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-22T19:20:45.572 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-22T19:20:45.994 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-22T19:20:45.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 1/32 2024-02-22T19:20:45.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2/32 2024-02-22T19:20:45.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 3/32 2024-02-22T19:20:45.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 4/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 5/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 6/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T19:20:45.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-22T19:20:45.996 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-22T19:20:45.997 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-22T19:20:45.997 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-22T19:20:45.997 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-22T19:20:45.997 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-22T19:20:46.236 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-22T19:20:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-22T19:20:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-22T19:20:46.966 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-22T19:20:46.966 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:46.966 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:20:46.966 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:46.966 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-22T19:20:46.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-2.6-1.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-22T19:20:46.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:46.969 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:20:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-22T19:20:47.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-base 2024-02-22T19:20:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-22T19:20:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-22T19:20:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-22T19:20:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-22T19:20:48.033 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:09:37 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:20:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-22T19:20:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 1/32 2024-02-22T19:20:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 3/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 4/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 5/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 6/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-22T19:20:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-22T19:20:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-22T19:20:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-22T19:20:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-22T19:20:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-22T19:20:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-22T19:20:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-22T19:20:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-22T19:20:48.864 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-22T19:20:48.865 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-2.6-1.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-22T19:20:48.866 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:48.867 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:20:49.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-base 2024-02-22T19:20:49.657 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:20:49.708 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:20:49.709 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:20:49.709 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:20:49.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install cephadm 2024-02-22T19:20:49.911 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:09:35 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:20:50.616 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:09:39 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:20:51.546 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:20:51.596 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:20:51.598 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:20:51.598 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:20:51.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install cephadm 2024-02-22T19:20:52.309 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 85 k 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 85 k 2024-02-22T19:20:52.310 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 352 k 2024-02-22T19:20:52.311 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:20:52.481 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm-16.2.14-550.geb66ed92.el8.noarch.rpm 501 kB/s | 85 kB 00:00 2024-02-22T19:20:52.481 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:20:52.482 INFO:teuthology.orchestra.run.smithi064.stdout:Total 498 kB/s | 85 kB 00:00 2024-02-22T19:20:52.482 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:20:52.489 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:20:52.489 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:20:52.491 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:09:38 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:20:52.492 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:20:52.495 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:20:52.542 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:20:53.075 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:53.239 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:53.477 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:53.859 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:20:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:20:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:53.859 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:20:54.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-immutable-object-cache 2024-02-22T19:20:54.170 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 85 k 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:54.171 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:20:54.172 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:20:54.172 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:20:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:54.172 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 85 k 2024-02-22T19:20:54.172 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 352 k 2024-02-22T19:20:54.172 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:20:54.428 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm-16.2.14-550.geb66ed92.el8.noarch.rpm 332 kB/s | 85 kB 00:00 2024-02-22T19:20:54.429 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:20:54.429 INFO:teuthology.orchestra.run.smithi053.stdout:Total 331 kB/s | 85 kB 00:00 2024-02-22T19:20:54.429 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:20:54.437 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:20:54.437 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:20:54.439 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:20:54.442 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:20:54.484 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:09:43 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:20:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:55.713 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-22T19:20:55.713 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:55.713 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:20:55.714 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:20:55.714 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:55.714 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:20:55.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-immutable-object-cache 2024-02-22T19:20:56.524 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-550.geb66ed92.el8 ceph 161 k 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:56.525 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:20:56.526 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:20:56.526 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:20:56.526 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:56.526 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 161 k 2024-02-22T19:20:56.526 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 470 k 2024-02-22T19:20:56.526 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:20:56.748 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:09:42 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:20:56.828 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-immutable-object-cache-16.2.14-550.geb66ed 533 kB/s | 161 kB 00:00 2024-02-22T19:20:56.829 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:20:56.829 INFO:teuthology.orchestra.run.smithi064.stdout:Total 531 kB/s | 161 kB 00:00 2024-02-22T19:20:56.829 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:20:56.841 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:20:56.841 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:20:56.891 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:20:56.894 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:20:57.009 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:20:57.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-22T19:20:57.594 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-22T19:20:57.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-22T19:20:57.992 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:57.992 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:20:57.992 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:57.992 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:20:57.992 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:20:58.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr 2024-02-22T19:20:58.423 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-550.geb66ed92.el8 ceph 161 k 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:58.424 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:20:58.425 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:20:58.425 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:20:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:58.425 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 161 k 2024-02-22T19:20:58.425 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 470 k 2024-02-22T19:20:58.425 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:20:58.724 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-immutable-object-cache-16.2.14-550.geb66ed 540 kB/s | 161 kB 00:00 2024-02-22T19:20:58.725 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:20:58.725 INFO:teuthology.orchestra.run.smithi053.stdout:Total 537 kB/s | 161 kB 00:00 2024-02-22T19:20:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:20:58.737 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:20:58.738 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:20:58.788 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:20:58.792 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:20:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:20:58.992 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:09:47 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:20:59.105 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-22T19:20:59.532 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-22T19:20:59.906 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-22T19:20:59.906 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:59.906 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:20:59.906 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:20:59.907 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:20:59.907 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:00.151 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr 2024-02-22T19:21:00.626 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:21:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:00.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-dashboard 2024-02-22T19:21:00.940 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:09:46 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:01.580 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:09:50 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:02.564 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:02.614 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:02.616 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:21:02.616 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:02.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-dashboard 2024-02-22T19:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 3.1 M 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 24 k 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 15 k 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-22T19:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 3.4 M 2024-02-22T19:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 22 M 2024-02-22T19:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:03.512 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:09:49 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:03.765 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): ceph-prometheus-alerts-16.2.14-550.geb66 78 kB/s | 15 kB 00:00 2024-02-22T19:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): ceph-grafana-dashboards-16.2.14-550.geb6 84 kB/s | 24 kB 00:00 2024-02-22T19:21:03.907 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 301 kB/s | 43 kB 00:00 2024-02-22T19:21:04.149 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): ceph-mgr-dashboard-16.2.14-550.geb66ed92 5.4 MB/s | 3.1 MB 00:00 2024-02-22T19:21:04.174 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 109 kB/s | 34 kB 00:00 2024-02-22T19:21:04.249 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 574 kB/s | 196 kB 00:00 2024-02-22T19:21:04.254 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:04.254 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.6 MB/s | 3.4 MB 00:00 2024-02-22T19:21:04.266 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:04.277 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:04.278 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:04.444 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:04.447 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:04.760 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-22T19:21:05.186 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:05.187 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:05.187 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:21:05.187 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 3.1 M 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 24 k 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 15 k 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-02-22T19:21:05.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:05.189 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.4 M 2024-02-22T19:21:05.189 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 22 M 2024-02-22T19:21:05.189 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:05.263 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-22T19:21:05.455 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-22T19:21:05.576 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 4/6 2024-02-22T19:21:05.577 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): ceph-prometheus-alerts-16.2.14-550.geb66 101 kB/s | 15 kB 00:00 2024-02-22T19:21:05.669 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): ceph-grafana-dashboards-16.2.14-550.geb6 102 kB/s | 24 kB 00:00 2024-02-22T19:21:05.719 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 301 kB/s | 43 kB 00:00 2024-02-22T19:21:05.995 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): ceph-mgr-dashboard-16.2.14-550.geb66ed92 5.5 MB/s | 3.1 MB 00:00 2024-02-22T19:21:06.178 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 66 kB/s | 34 kB 00:00 2024-02-22T19:21:06.429 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 277 kB/s | 196 kB 00:00 2024-02-22T19:21:06.433 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:06.433 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.8 MB/s | 3.4 MB 00:01 2024-02-22T19:21:06.445 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:06.457 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:06.457 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:06.627 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:06.631 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:06.747 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 5/6 2024-02-22T19:21:06.910 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-22T19:21:07.132 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-22T19:21:07.347 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-22T19:21:07.426 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-22T19:21:07.426 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 1/6 2024-02-22T19:21:07.426 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 2/6 2024-02-22T19:21:07.426 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 3/6 2024-02-22T19:21:07.426 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-22T19:21:07.426 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-22T19:21:07.556 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-22T19:21:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 4/6 2024-02-22T19:21:07.826 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-22T19:21:07.826 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:07.827 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:07.827 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:07.827 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:07.827 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:07.827 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-22T19:21:07.827 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-22T19:21:07.828 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-22T19:21:07.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:07.828 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:07.957 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-22T19:21:08.744 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:09:57 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:08.785 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 5/6 2024-02-22T19:21:09.065 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-22T19:21:09.367 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-22T19:21:09.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 1/6 2024-02-22T19:21:09.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 2/6 2024-02-22T19:21:09.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 3/6 2024-02-22T19:21:09.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-22T19:21:09.367 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-22T19:21:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-22T19:21:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:09.774 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:09.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-22T19:21:10.435 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 8.6 M 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-22T19:21:10.437 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout:Install 8 Packages 2024-02-22T19:21:10.438 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:10.439 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 37 M 2024-02-22T19:21:10.439 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 210 M 2024-02-22T19:21:10.439 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:10.697 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:09:56 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:11.271 INFO:teuthology.orchestra.run.smithi064.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.14-55 14 MB/s | 8.6 MB 00:00 2024-02-22T19:21:11.338 INFO:teuthology.orchestra.run.smithi064.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 7.0 MB/s | 4.7 MB 00:00 2024-02-22T19:21:11.447 INFO:teuthology.orchestra.run.smithi064.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 6.2 MB/s | 4.9 MB 00:00 2024-02-22T19:21:11.472 INFO:teuthology.orchestra.run.smithi064.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.6 MB/s | 225 kB 00:00 2024-02-22T19:21:11.739 INFO:teuthology.orchestra.run.smithi064.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 2.4 MB/s | 645 kB 00:00 2024-02-22T19:21:11.798 INFO:teuthology.orchestra.run.smithi064.stdout:(6/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 7.1 MB/s | 3.7 MB 00:00 2024-02-22T19:21:11.814 INFO:teuthology.orchestra.run.smithi064.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 2.2 MB/s | 172 kB 00:00 2024-02-22T19:21:12.282 INFO:teuthology.orchestra.run.smithi064.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 17 MB/s | 14 MB 00:00 2024-02-22T19:21:12.282 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:12.282 INFO:teuthology.orchestra.run.smithi064.stdout:Total 20 MB/s | 37 MB 00:01 2024-02-22T19:21:12.384 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 8.6 M 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-22T19:21:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout:Install 8 Packages 2024-02-22T19:21:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:12.388 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 37 M 2024-02-22T19:21:12.388 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 210 M 2024-02-22T19:21:12.388 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:12.520 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:12.539 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:12.539 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:12.871 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:12.874 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:13.358 INFO:teuthology.orchestra.run.smithi053.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.14-55 15 MB/s | 8.6 MB 00:00 2024-02-22T19:21:13.501 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:13.559 INFO:teuthology.orchestra.run.smithi053.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 6.0 MB/s | 4.7 MB 00:00 2024-02-22T19:21:13.634 INFO:teuthology.orchestra.run.smithi053.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 5.6 MB/s | 4.9 MB 00:00 2024-02-22T19:21:13.643 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T19:21:13.659 INFO:teuthology.orchestra.run.smithi053.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.2 MB/s | 225 kB 00:00 2024-02-22T19:21:13.760 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T19:21:13.884 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T19:21:13.943 INFO:teuthology.orchestra.run.smithi053.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 2.2 MB/s | 645 kB 00:00 2024-02-22T19:21:14.101 INFO:teuthology.orchestra.run.smithi053.stdout:(6/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 5.0 MB/s | 3.7 MB 00:00 2024-02-22T19:21:14.135 INFO:teuthology.orchestra.run.smithi053.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 900 kB/s | 172 kB 00:00 2024-02-22T19:21:14.519 INFO:teuthology.orchestra.run.smithi053.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 16 MB/s | 14 MB 00:00 2024-02-22T19:21:14.519 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:14.519 INFO:teuthology.orchestra.run.smithi053.stdout:Total 17 MB/s | 37 MB 00:02 2024-02-22T19:21:14.540 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T19:21:14.735 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T19:21:14.758 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:14.777 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:14.777 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:15.111 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:15.114 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:15.392 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T19:21:15.518 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T19:21:15.695 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:15.827 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T19:21:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T19:21:16.261 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T19:21:16.512 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T19:21:16.634 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T19:21:16.826 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T19:21:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T19:21:17.769 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T19:21:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T19:21:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T19:21:18.714 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T19:21:19.347 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T19:21:19.669 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T19:21:19.885 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T19:21:20.070 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-22T19:21:20.242 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 1/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-22T19:21:20.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-22T19:21:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:21.030 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:21.180 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-rook 2024-02-22T19:21:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T19:21:21.967 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:10 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:23.137 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-22T19:21:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 1/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-22T19:21:23.656 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-22T19:21:23.665 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 27 k 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-22T19:21:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-02-22T19:21:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:23.669 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 1.9 M 2024-02-22T19:21:23.669 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 24 M 2024-02-22T19:21:23.669 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-22T19:21:24.079 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-22T19:21:24.080 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-22T19:21:24.080 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-22T19:21:24.080 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-22T19:21:24.080 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-22T19:21:24.080 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:24.080 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:24.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-rook 2024-02-22T19:21:24.307 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 194 kB/s | 27 kB 00:00 2024-02-22T19:21:24.332 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 123 kB/s | 20 kB 00:00 2024-02-22T19:21:24.441 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): ceph-mgr-rook-16.2.14-550.geb66ed92.el8 101 kB/s | 27 kB 00:00 2024-02-22T19:21:24.466 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 825 kB/s | 110 kB 00:00 2024-02-22T19:21:24.491 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 796 kB/s | 126 kB 00:00 2024-02-22T19:21:24.641 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 243 kB/s | 43 kB 00:00 2024-02-22T19:21:24.667 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 686 kB/s | 155 kB 00:00 2024-02-22T19:21:24.759 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 123 kB/s | 33 kB 00:00 2024-02-22T19:21:24.943 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 54 kB/s | 16 kB 00:00 2024-02-22T19:21:24.993 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 249 kB/s | 81 kB 00:00 2024-02-22T19:21:25.000 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:10 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:25.160 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): python3-kubernetes-11.0.0-6.el8.noarch 2.9 MB/s | 1.2 MB 00:00 2024-02-22T19:21:25.260 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): python3-rsa-4.9-2.el8.noarch.rpm 202 kB/s | 64 kB 00:00 2024-02-22T19:21:25.310 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 193 kB/s | 61 kB 00:00 2024-02-22T19:21:25.310 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:25.311 INFO:teuthology.orchestra.run.smithi064.stdout:Total 1.2 MB/s | 1.9 MB 00:01 2024-02-22T19:21:25.370 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:25.387 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:25.387 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:25.482 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:25.485 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:26.066 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-22T19:21:26.502 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-22T19:21:26.676 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:26.678 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:21:26.678 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:26.678 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 27 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-22T19:21:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 1.9 M 2024-02-22T19:21:26.680 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 24 M 2024-02-22T19:21:26.681 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:26.712 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-22T19:21:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-22T19:21:27.175 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 153 kB/s | 20 kB 00:00 2024-02-22T19:21:27.189 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-22T19:21:27.200 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 171 kB/s | 27 kB 00:00 2024-02-22T19:21:27.293 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): ceph-mgr-rook-16.2.14-550.geb66ed92.el8 110 kB/s | 27 kB 00:00 2024-02-22T19:21:27.318 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 890 kB/s | 126 kB 00:00 2024-02-22T19:21:27.343 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 774 kB/s | 110 kB 00:00 2024-02-22T19:21:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-22T19:21:27.493 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 243 kB/s | 43 kB 00:00 2024-02-22T19:21:27.552 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 599 kB/s | 155 kB 00:00 2024-02-22T19:21:27.627 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 116 kB/s | 33 kB 00:00 2024-02-22T19:21:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-22T19:21:27.777 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 57 kB/s | 16 kB 00:00 2024-02-22T19:21:27.886 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 243 kB/s | 81 kB 00:00 2024-02-22T19:21:27.970 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 334 kB/s | 64 kB 00:00 2024-02-22T19:21:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-22T19:21:28.003 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 524 kB/s | 61 kB 00:00 2024-02-22T19:21:28.054 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.7 MB/s | 1.2 MB 00:00 2024-02-22T19:21:28.054 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:28.054 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.4 MB/s | 1.9 MB 00:01 2024-02-22T19:21:28.114 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:28.130 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:28.131 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:28.226 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:28.229 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-22T19:21:29.138 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-22T19:21:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-22T19:21:29.431 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-22T19:21:29.478 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-22T19:21:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-22T19:21:29.715 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-22T19:21:29.791 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-22T19:21:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-22T19:21:30.008 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-22T19:21:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-22T19:21:30.303 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-22T19:21:30.303 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 1/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-22T19:21:30.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-22T19:21:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-22T19:21:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-22T19:21:30.779 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-22T19:21:30.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:30.781 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:30.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-cephadm 2024-02-22T19:21:31.691 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:20 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:31.798 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-22T19:21:32.040 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-22T19:21:32.232 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-22T19:21:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-22T19:21:32.601 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 1/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-22T19:21:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-22T19:21:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-22T19:21:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-22T19:21:33.375 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-22T19:21:33.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:33.375 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:33.375 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:33.375 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-22T19:21:33.375 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:33.376 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 116 k 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-22T19:21:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 5.9 M 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 24 M 2024-02-22T19:21:33.378 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:33.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-cephadm 2024-02-22T19:21:33.910 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 2.0 MB/s | 578 kB 00:00 2024-02-22T19:21:33.935 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): ceph-mgr-cephadm-16.2.14-550.geb66ed92.e 369 kB/s | 116 kB 00:00 2024-02-22T19:21:34.060 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 141 kB/s | 21 kB 00:00 2024-02-22T19:21:34.136 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): python3-babel-2.5.1-7.el8.noarch.rpm 9.3 MB/s | 4.8 MB 00:00 2024-02-22T19:21:34.161 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 487 kB/s | 49 kB 00:00 2024-02-22T19:21:34.186 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.4 MB/s | 355 kB 00:00 2024-02-22T19:21:34.188 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:34.188 INFO:teuthology.orchestra.run.smithi064.stdout:Total 7.3 MB/s | 5.9 MB 00:00 2024-02-22T19:21:34.240 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:34.250 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:34.250 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:34.263 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:20 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:34.354 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:34.357 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:34.953 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-22T19:21:35.270 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-22T19:21:35.945 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:35.946 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 116 k 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-02-22T19:21:35.947 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:35.948 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 5.9 M 2024-02-22T19:21:35.948 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 24 M 2024-02-22T19:21:35.948 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-22T19:21:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-22T19:21:36.661 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): ceph-mgr-cephadm-16.2.14-550.geb66ed92.e 613 kB/s | 116 kB 00:00 2024-02-22T19:21:36.786 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.8 MB/s | 578 kB 00:00 2024-02-22T19:21:36.825 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-22T19:21:36.836 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 121 kB/s | 21 kB 00:00 2024-02-22T19:21:36.895 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): python3-remoto-1.2.1-1.el8.noarch.rpm 847 kB/s | 49 kB 00:00 2024-02-22T19:21:36.978 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): python3-babel-2.5.1-7.el8.noarch.rpm 9.4 MB/s | 4.8 MB 00:00 2024-02-22T19:21:37.029 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.4 MB/s | 355 kB 00:00 2024-02-22T19:21:37.031 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:37.031 INFO:teuthology.orchestra.run.smithi053.stdout:Total 5.4 MB/s | 5.9 MB 00:01 2024-02-22T19:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-22T19:21:37.085 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:37.095 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:37.095 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:37.202 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:37.205 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:37.353 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-22T19:21:37.353 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/6 2024-02-22T19:21:37.353 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-22T19:21:37.354 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-22T19:21:37.354 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-22T19:21:37.354 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-22T19:21:37.490 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:37.762 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:37.763 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:37.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-fuse 2024-02-22T19:21:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-22T19:21:38.457 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-22T19:21:38.693 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:27 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-22T19:21:39.746 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-22T19:21:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-22T19:21:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-22T19:21:40.387 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:40.388 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:40.388 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 911 k 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:Install 2 Packages 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 994 k 2024-02-22T19:21:40.390 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 2.8 M 2024-02-22T19:21:40.390 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-22T19:21:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/6 2024-02-22T19:21:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-22T19:21:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-22T19:21:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-22T19:21:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-22T19:21:40.614 INFO:teuthology.orchestra.run.smithi064.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 452 kB/s | 83 kB 00:00 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-22T19:21:40.849 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:40.850 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:40.856 INFO:teuthology.orchestra.run.smithi064.stdout:(2/2): ceph-fuse-16.2.14-550.geb66ed92.el8.x86_ 2.1 MB/s | 911 kB 00:00 2024-02-22T19:21:40.857 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:40.857 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.1 MB/s | 994 kB 00:00 2024-02-22T19:21:40.868 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:40.890 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:40.890 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:40.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-fuse 2024-02-22T19:21:40.985 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:40.989 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:41.401 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-22T19:21:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-22T19:21:41.768 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:27 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-22T19:21:42.145 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/2 2024-02-22T19:21:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-22T19:21:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:42.613 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-22T19:21:42.613 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:42.613 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:42.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados-devel 2024-02-22T19:21:43.458 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:43.458 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:43.458 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 911 k 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout:Install 2 Packages 2024-02-22T19:21:43.459 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:43.460 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 994 k 2024-02-22T19:21:43.460 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 2.8 M 2024-02-22T19:21:43.460 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:43.627 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:32 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:43.683 INFO:teuthology.orchestra.run.smithi053.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 457 kB/s | 83 kB 00:00 2024-02-22T19:21:43.909 INFO:teuthology.orchestra.run.smithi053.stdout:(2/2): ceph-fuse-16.2.14-550.geb66ed92.el8.x86_ 2.2 MB/s | 911 kB 00:00 2024-02-22T19:21:43.909 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:43.910 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.2 MB/s | 994 kB 00:00 2024-02-22T19:21:43.920 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:43.942 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:43.942 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:44.035 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:44.039 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:44.209 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:44.493 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-22T19:21:44.690 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-22T19:21:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-22T19:21:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/2 2024-02-22T19:21:45.306 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:45.307 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:45.307 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-22T19:21:45.307 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:45.307 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 131 k 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:45.308 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 131 k 2024-02-22T19:21:45.309 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 432 k 2024-02-22T19:21:45.309 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:45.518 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-22T19:21:45.518 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:45.518 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:45.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-22T19:21:45.518 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:45.518 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:45.602 INFO:teuthology.orchestra.run.smithi064.stdout:librados-devel-16.2.14-550.geb66ed92.el8.x86_64 446 kB/s | 131 kB 00:00 2024-02-22T19:21:45.602 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:45.603 INFO:teuthology.orchestra.run.smithi064.stdout:Total 444 kB/s | 131 kB 00:00 2024-02-22T19:21:45.603 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:45.609 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:45.609 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:45.664 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:45.667 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:45.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados-devel 2024-02-22T19:21:45.781 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:46.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:46.368 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:46.515 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:32 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:46.830 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:46.831 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:46.831 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:46.831 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:21:46.831 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:46.831 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:47.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs2 2024-02-22T19:21:47.826 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:36 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:48.214 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:48.214 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 131 k 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:21:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:48.216 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 131 k 2024-02-22T19:21:48.216 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 432 k 2024-02-22T19:21:48.216 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:48.837 INFO:teuthology.orchestra.run.smithi053.stdout:librados-devel-16.2.14-550.geb66ed92.el8.x86_64 211 kB/s | 131 kB 00:00 2024-02-22T19:21:48.838 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:48.838 INFO:teuthology.orchestra.run.smithi053.stdout:Total 210 kB/s | 131 kB 00:00 2024-02-22T19:21:48.839 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:48.846 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:48.846 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:48.907 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:48.911 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:49.347 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:49.473 INFO:teuthology.orchestra.run.smithi064.stdout:Package libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:49.524 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:49.525 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:21:49.525 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:49.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs-devel 2024-02-22T19:21:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:50.095 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:50.095 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:50.095 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:50.095 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:21:50.095 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:50.095 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:50.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs2 2024-02-22T19:21:50.420 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:39 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:51.113 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:37 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:52.115 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:52.116 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:52.116 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-22T19:21:52.116 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:52.116 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:21:52.116 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 25 k 2024-02-22T19:21:52.116 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 25 k 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 107 k 2024-02-22T19:21:52.117 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:21:52.334 INFO:teuthology.orchestra.run.smithi064.stdout:libcephfs-devel-16.2.14-550.geb66ed92.el8.x86_6 118 kB/s | 25 kB 00:00 2024-02-22T19:21:52.334 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:52.334 INFO:teuthology.orchestra.run.smithi064.stdout:Total 117 kB/s | 25 kB 00:00 2024-02-22T19:21:52.335 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:21:52.338 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:21:52.339 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:21:52.341 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:21:52.344 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:21:52.395 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:21:52.618 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:52.745 INFO:teuthology.orchestra.run.smithi053.stdout:Package libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:52.796 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:52.797 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:21:52.797 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:52.886 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:52.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs-devel 2024-02-22T19:21:53.260 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:53.260 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:53.260 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:21:53.261 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:21:53.261 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:21:53.261 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:53.378 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados2 2024-02-22T19:21:53.702 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:39 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:54.166 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:43 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:55.380 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 25 k 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:21:55.381 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:21:55.382 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:55.382 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 25 k 2024-02-22T19:21:55.382 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 107 k 2024-02-22T19:21:55.382 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:21:55.601 INFO:teuthology.orchestra.run.smithi053.stdout:libcephfs-devel-16.2.14-550.geb66ed92.el8.x86_6 116 kB/s | 25 kB 00:00 2024-02-22T19:21:55.602 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:21:55.602 INFO:teuthology.orchestra.run.smithi053.stdout:Total 116 kB/s | 25 kB 00:00 2024-02-22T19:21:55.603 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:21:55.606 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:21:55.606 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:21:55.609 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:21:55.612 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:21:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:21:55.796 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:55.845 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:55.847 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:21:55.847 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:55.860 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:55.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librbd1 2024-02-22T19:21:56.123 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:21:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:56.481 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:21:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:21:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:21:56.481 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:56.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados2 2024-02-22T19:21:56.731 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:45 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:57.387 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:43 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:21:58.353 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:58.403 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:21:58.405 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:21:58.405 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:21:58.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rados 2024-02-22T19:21:59.010 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:21:59.061 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:21:59.063 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:21:59.063 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:21:59.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librbd1 2024-02-22T19:21:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:48 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:21:59.947 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:45 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:00.920 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:00.970 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:00.972 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:22:00.972 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:01.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rgw 2024-02-22T19:22:01.572 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:01.622 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:01.623 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:22:01.623 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:01.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rados 2024-02-22T19:22:01.861 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:50 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:22:02.523 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:48 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:03.473 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:03.523 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:03.525 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:22:03.525 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:03.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-cephfs 2024-02-22T19:22:04.152 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:04.202 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:04.203 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:22:04.203 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:04.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rgw 2024-02-22T19:22:04.445 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:53 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:22:05.100 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:51 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:06.056 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:06.109 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:06.110 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:22:06.110 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:06.213 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rbd 2024-02-22T19:22:06.754 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:06.806 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:06.808 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:22:06.808 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:06.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-cephfs 2024-02-22T19:22:06.999 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:55 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:22:07.752 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:53 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:08.618 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:08.668 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:08.670 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-22T19:22:08.670 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:08.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-fuse 2024-02-22T19:22:09.391 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:09.443 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:09.444 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:22:09.444 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:09.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rbd 2024-02-22T19:22:09.569 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:10:58 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:22:10.349 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:56 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:11.248 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 88 k 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 88 k 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 240 k 2024-02-22T19:22:11.249 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:22:11.415 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-fuse-16.2.14-550.geb66ed92.el8.x86_64.rpm 534 kB/s | 88 kB 00:00 2024-02-22T19:22:11.416 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:22:11.416 INFO:teuthology.orchestra.run.smithi064.stdout:Total 530 kB/s | 88 kB 00:00 2024-02-22T19:22:11.416 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:22:11.427 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:22:11.427 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:22:11.481 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:22:11.484 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:22:11.644 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:22:11.826 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:11.972 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-22T19:22:12.023 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:12.024 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-02-22T19:22:12.024 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:12.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-fuse 2024-02-22T19:22:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:12.855 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:22:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:22:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:12.855 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:12.913 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:10:58 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:13.102 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-mirror 2024-02-22T19:22:13.895 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:11:02 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:22:14.603 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 88 k 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:22:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:14.605 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 88 k 2024-02-22T19:22:14.605 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 240 k 2024-02-22T19:22:14.605 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:22:14.875 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-fuse-16.2.14-550.geb66ed92.el8.x86_64.rpm 327 kB/s | 88 kB 00:00 2024-02-22T19:22:14.876 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:22:14.876 INFO:teuthology.orchestra.run.smithi053.stdout:Total 325 kB/s | 88 kB 00:00 2024-02-22T19:22:14.877 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:22:14.887 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:22:14.887 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:22:14.941 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:22:14.945 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:22:15.053 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:22:15.219 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:15.510 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:15.573 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:15.573 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:15.573 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-22T19:22:15.573 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:15.573 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:22:15.573 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 3.7 M 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 15 M 2024-02-22T19:22:15.574 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:22:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:15.905 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:22:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:22:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:15.905 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:16.125 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-mirror 2024-02-22T19:22:16.835 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-mirror-16.2.14-550.geb66ed92.el8.x86_64.rpm 2.9 MB/s | 3.7 MB 00:01 2024-02-22T19:22:16.836 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:22:16.836 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.9 MB/s | 3.7 MB 00:01 2024-02-22T19:22:16.836 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:22:16.850 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:22:16.850 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:22:16.918 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:11:02 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:16.920 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:22:16.924 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:22:17.385 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:22:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:18.012 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:18.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:18.416 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:18.416 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:22:18.417 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:22:18.417 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:18.417 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:18.588 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:18.588 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-02-22T19:22:18.588 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:18.588 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:22:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-22T19:22:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:18.589 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:22:18.589 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:18.589 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:22:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:18.590 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.7 M 2024-02-22T19:22:18.590 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 15 M 2024-02-22T19:22:18.590 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:22:18.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-nbd 2024-02-22T19:22:19.424 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:11:08 ago on Thu 22 Feb 2024 07:11:11 PM UTC. 2024-02-22T19:22:19.562 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-mirror-16.2.14-550.geb66ed92.el8.x86_64.rpm 3.8 MB/s | 3.7 MB 00:00 2024-02-22T19:22:19.562 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:22:19.563 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.8 MB/s | 3.7 MB 00:00 2024-02-22T19:22:19.563 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:22:19.577 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:22:19.578 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:22:19.649 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:22:19.653 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:22:20.110 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:22:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:20.750 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:21.111 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-22T19:22:21.111 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:21.111 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-22T19:22:21.111 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:21.111 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 175 k 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 175 k 2024-02-22T19:22:21.112 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 523 k 2024-02-22T19:22:21.113 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-22T19:22:21.146 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:21.146 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:21.146 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:22:21.146 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:22:21.146 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:21.146 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:21.342 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-nbd 2024-02-22T19:22:21.407 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-nbd-16.2.14-550.geb66ed92.el8.x86_64.rpm 595 kB/s | 175 kB 00:00 2024-02-22T19:22:21.407 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:22:21.408 INFO:teuthology.orchestra.run.smithi064.stdout:Total 592 kB/s | 175 kB 00:00 2024-02-22T19:22:21.408 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-22T19:22:21.420 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-22T19:22:21.420 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-22T19:22:21.473 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-22T19:22:21.477 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-22T19:22:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-22T19:22:21.791 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:22.133 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:11:08 ago on Thu 22 Feb 2024 07:11:14 PM UTC. 2024-02-22T19:22:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:22.493 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-22T19:22:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:22:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:22:22.493 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-22T19:22:22.675 DEBUG:teuthology.parallel:result is None 2024-02-22T19:22:23.817 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 175 k 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-02-22T19:22:23.818 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:23.819 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 175 k 2024-02-22T19:22:23.819 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 523 k 2024-02-22T19:22:23.819 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-02-22T19:22:24.114 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-nbd-16.2.14-550.geb66ed92.el8.x86_64.rpm 595 kB/s | 175 kB 00:00 2024-02-22T19:22:24.114 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-02-22T19:22:24.114 INFO:teuthology.orchestra.run.smithi053.stdout:Total 593 kB/s | 175 kB 00:00 2024-02-22T19:22:24.115 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-02-22T19:22:24.126 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-02-22T19:22:24.126 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-02-22T19:22:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-02-22T19:22:24.184 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-02-22T19:22:24.301 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-02-22T19:22:24.523 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:24.810 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:25.212 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-22T19:22:25.212 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:25.212 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-02-22T19:22:25.212 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-22T19:22:25.212 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:22:25.212 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-02-22T19:22:25.405 DEBUG:teuthology.parallel:result is None 2024-02-22T19:22:25.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-22T19:22:25.625 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-22T19:22:25.649 INFO:teuthology.orchestra.run.smithi053.stdout:16.2.14-550.geb66ed92.el8 2024-02-22T19:22:25.650 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-550.geb66ed92.el8 2024-02-22T19:22:25.650 INFO:teuthology.task.install:The correct ceph version 16.2.14-550.geb66ed92 is installed. 2024-02-22T19:22:25.652 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-22T19:22:25.794 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-22T19:22:25.821 INFO:teuthology.orchestra.run.smithi064.stdout:16.2.14-550.geb66ed92.el8 2024-02-22T19:22:25.821 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-550.geb66ed92.el8 2024-02-22T19:22:25.821 INFO:teuthology.task.install:The correct ceph version 16.2.14-550.geb66ed92 is installed. 2024-02-22T19:22:25.823 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-22T19:22:25.823 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:22:25.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-22T19:22:25.852 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:22:25.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-22T19:22:25.895 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-22T19:22:25.895 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:22:25.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-02-22T19:22:25.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-22T19:22:25.989 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:22:25.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-02-22T19:22:26.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-22T19:22:26.094 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-22T19:22:26.094 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:22:26.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-22T19:22:26.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-22T19:22:26.188 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:22:26.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-22T19:22:26.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-22T19:22:26.293 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-22T19:22:26.293 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:22:26.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-02-22T19:22:26.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-22T19:22:26.388 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:22:26.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-02-22T19:22:26.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-22T19:22:26.490 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-22T19:22:26.561 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', 'flag\\(s\\) set', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(PG_', '\\(OSD_', 'mons down'], 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59', 'cephadm_mode': 'root'} 2024-02-22T19:22:26.561 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 2024-02-22T19:22:26.561 INFO:tasks.cephadm:Cluster fsid is b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:22:26.561 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-22T19:22:26.562 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.53', 'mon.c': '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]', 'mon.b': '172.21.15.64'} 2024-02-22T19:22:26.562 INFO:tasks.cephadm:First mon is mon.a on smithi053 2024-02-22T19:22:26.562 INFO:tasks.cephadm:First mgr is y 2024-02-22T19:22:26.562 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-22T19:22:26.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-02-22T19:22:26.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-02-22T19:22:26.621 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph-ci.git ref eb66ed921e744301e6863be6353618d63967ea59)... 2024-02-22T19:22:26.621 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/eb66ed921e744301e6863be6353618d63967ea59/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-22T19:22:26.762 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 22 19:22 /home/ubuntu/cephtest/cephadm 2024-02-22T19:22:26.763 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/eb66ed921e744301e6863be6353618d63967ea59/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-22T19:22:26.905 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 22 19:22 /home/ubuntu/cephtest/cephadm 2024-02-22T19:22:26.906 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-22T19:22:26.925 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-22T19:22:26.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-02-22T19:22:26.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-02-22T19:22:27.025 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-02-22T19:22:27.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-02-22T19:22:27.097 INFO:tasks.cephadm:Writing seed config... 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-02-22T19:22:27.098 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-02-22T19:22:27.099 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-02-22T19:22:27.100 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-02-22T19:22:27.100 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-22T19:22:27.100 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-02-22T19:22:27.100 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:22:27.100 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-22T19:22:27.121 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b6783888-d1b7-11ee-95bf-87774f69a715 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-02-22T19:22:27.121 DEBUG:teuthology.orchestra.run.smithi053:mon.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service 2024-02-22T19:22:27.164 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.y.service 2024-02-22T19:22:27.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:22:27.191 INFO:tasks.cephadm:Bootstrapping... 2024-02-22T19:22:27.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 -v bootstrap --fsid b6783888-d1b7-11ee-95bf-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.53 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:22:27.219 INFO:journalctl@ceph.mgr.y.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:22:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:-------------------------------------------------------------------------------- 2024-02-22T19:22:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59', '-v', 'bootstrap', '--fsid', 'b6783888-d1b7-11ee-95bf-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.53', '--skip-admin-label'] 2024-02-22T19:22:28.407 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: 4.9.3-dev 2024-02-22T19:22:28.416 INFO:teuthology.orchestra.run.smithi053.stderr:Verifying podman|docker is present... 2024-02-22T19:22:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: 4.9.3-dev 2024-02-22T19:22:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:Verifying lvm2 is present... 2024-02-22T19:22:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:Verifying time synchronization is in place... 2024-02-22T19:22:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-22T19:22:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: inactive 2024-02-22T19:22:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: enabled 2024-02-22T19:22:28.487 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: active 2024-02-22T19:22:28.487 INFO:teuthology.orchestra.run.smithi053.stderr:Unit chronyd.service is enabled and running 2024-02-22T19:22:28.487 INFO:teuthology.orchestra.run.smithi053.stderr:Repeating the final host check... 2024-02-22T19:22:28.509 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: 4.9.3-dev 2024-02-22T19:22:28.519 INFO:teuthology.orchestra.run.smithi053.stderr:podman (/bin/podman) version 4.9.3 is present 2024-02-22T19:22:28.520 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl is present 2024-02-22T19:22:28.520 INFO:teuthology.orchestra.run.smithi053.stderr:lvcreate is present 2024-02-22T19:22:28.526 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-22T19:22:28.532 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: inactive 2024-02-22T19:22:28.538 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: enabled 2024-02-22T19:22:28.544 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: active 2024-02-22T19:22:28.545 INFO:teuthology.orchestra.run.smithi053.stderr:Unit chronyd.service is enabled and running 2024-02-22T19:22:28.545 INFO:teuthology.orchestra.run.smithi053.stderr:Host looks OK 2024-02-22T19:22:28.546 INFO:teuthology.orchestra.run.smithi053.stderr:Cluster fsid: b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:22:28.546 INFO:teuthology.orchestra.run.smithi053.stderr:Acquiring lock 140544011028968 on /run/cephadm/b6783888-d1b7-11ee-95bf-87774f69a715.lock 2024-02-22T19:22:28.546 INFO:teuthology.orchestra.run.smithi053.stderr:Lock 140544011028968 acquired on /run/cephadm/b6783888-d1b7-11ee-95bf-87774f69a715.lock 2024-02-22T19:22:28.546 INFO:teuthology.orchestra.run.smithi053.stderr:Verifying IP 172.21.15.53 port 3300 ... 2024-02-22T19:22:28.547 INFO:teuthology.orchestra.run.smithi053.stderr:Verifying IP 172.21.15.53 port 6789 ... 2024-02-22T19:22:28.547 INFO:teuthology.orchestra.run.smithi053.stderr:Base mon IP(s) is [172.21.15.53:3300, 172.21.15.53:6789], mon addrv is [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-02-22T19:22:28.550 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.53 metric 100 2024-02-22T19:22:28.550 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.53 metric 100 2024-02-22T19:22:28.553 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-22T19:22:28.553 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-22T19:22:28.553 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1559sec hoplimit 64 pref medium 2024-02-22T19:22:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-22T19:22:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-22T19:22:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-22T19:22:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-22T19:22:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2024-02-22T19:22:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-22T19:22:28.557 INFO:teuthology.orchestra.run.smithi053.stderr:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2024-02-22T19:22:28.557 INFO:teuthology.orchestra.run.smithi053.stderr:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2024-02-22T19:22:28.557 INFO:teuthology.orchestra.run.smithi053.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-22T19:22:28.558 INFO:teuthology.orchestra.run.smithi053.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-22T19:22:28.559 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59... 2024-02-22T19:22:28.657 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59... 2024-02-22T19:22:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: Getting image source signatures 2024-02-22T19:22:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: Copying blob sha256:6a725db67f9d8a3d1854323a2987a594424da05f10d8a333e41377f3b137fc12 2024-02-22T19:22:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-22T19:22:50.768 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: Copying config sha256:7403d18623076b313def53d9a4ae24b05fb9f62358c436a81c22d7e96d9d9550 2024-02-22T19:22:50.935 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: Writing manifest to image destination 2024-02-22T19:22:51.196 INFO:teuthology.orchestra.run.smithi053.stderr:/bin/podman: 7403d18623076b313def53d9a4ae24b05fb9f62358c436a81c22d7e96d9d9550 2024-02-22T19:22:51.827 INFO:teuthology.orchestra.run.smithi053.stderr:ceph: ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable) 2024-02-22T19:22:52.407 INFO:teuthology.orchestra.run.smithi053.stderr:Ceph version: ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable) 2024-02-22T19:22:52.407 INFO:teuthology.orchestra.run.smithi053.stderr:Extracting ceph user uid/gid from container image... 2024-02-22T19:22:52.871 INFO:teuthology.orchestra.run.smithi053.stderr:stat: 167 167 2024-02-22T19:22:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:Creating initial keys... 2024-02-22T19:22:53.977 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-authtool: AQANn9dlDPh5NBAAsBLSgWmtA5A9AkXdLKAazw== 2024-02-22T19:22:55.007 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-authtool: AQAOn9dlJFK8NhAABUsvhhh6k+LBE++PS3Fzzg== 2024-02-22T19:22:56.096 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-authtool: AQAPn9dlFCPLOhAAPN3AeKJMGGYgLgqZyj1IkQ== 2024-02-22T19:22:56.619 INFO:teuthology.orchestra.run.smithi053.stderr:Creating initial monmap... 2024-02-22T19:22:57.113 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-22T19:22:57.113 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:22:57.113 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-22T19:22:57.641 INFO:teuthology.orchestra.run.smithi053.stderr:monmaptool for a [v2:172.21.15.53:3300,v1:172.21.15.53:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-22T19:22:57.641 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/monmaptool: set fsid to b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:22:57.641 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-22T19:22:57.641 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:22:57.641 INFO:teuthology.orchestra.run.smithi053.stderr:Creating mon... 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.048+0000 7f9979261880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 1 imported monmap: 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: fsid b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: last_changed 2024-02-22T19:22:57.014624+0000 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: created 2024-02-22T19:22:57.014624+0000 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 0 /usr/bin/ceph-mon: set fsid to b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: RocksDB version: 6.8.1 2024-02-22T19:22:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Compile date Feb 20 2024 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: DB SUMMARY 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.error_if_exists: 0 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.create_if_missing: 1 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.paranoid_checks: 1 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.env: 0x559b413bf080 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.fs: Posix File System 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.info_log: 0x559b422a1a00 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-22T19:22:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.statistics: (nil) 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.use_fsync: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_log_file_size: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.allow_fallocate: 1 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.use_direct_reads: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.db_log_dir: 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-22T19:22:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_subcompactions: 1 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_background_flushes: -1 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.write_buffer_manager: 0x559b422a7920 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-22T19:22:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.unordered_write: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.row_cache: None 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.wal_filter: None 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.preserve_deletes: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.two_write_queues: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.atomic_flush: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-22T19:22:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.log_readahead_size: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_background_jobs: 2 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_background_compactions: -1 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.max_open_files: -1 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-22T19:22:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Compression algorithms supported: 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kZSTD supported: 0 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kXpressCompression supported: 0 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kLZ4Compression supported: 1 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kBZip2Compression supported: 0 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kZlibCompression supported: 1 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: kSnappyCompression supported: 1 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.049+0000 7f9979261880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-22T19:22:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.merge_operator: 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_filter: None 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b421b7d18) 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-22T19:22:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_cache: 0x559b421ecd10 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-22T19:22:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression: NoCompression 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.num_levels: 7 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-22T19:22:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-22T19:22:58.180 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-22T19:22:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.table_properties_collectors: 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.inplace_update_support: 0 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.bloom_locality: 0 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.max_successive_merges: 0 2024-02-22T19:22:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.ttl: 2592000 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.050+0000 7f9979261880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.051+0000 7f9979261880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.051+0000 7f9979261880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.051+0000 7f9979261880 4 rocksdb: DB pointer 0x559b422b5800 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.051+0000 7f99620d4700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.052+0000 7f99620d4700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T19:22:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T19:22:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-22T19:22:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.053+0000 7f9979261880 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.053+0000 7f9979261880 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-22T19:22:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph-mon: debug 2024-02-22T19:22:58.053+0000 7f9979261880 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-02-22T19:22:58.701 INFO:teuthology.orchestra.run.smithi053.stderr:create mon.a on 2024-02-22T19:22:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-22T19:22:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-22T19:22:58.914 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b6783888-d1b7-11ee-95bf-87774f69a715.target → /etc/systemd/system/ceph-b6783888-d1b7-11ee-95bf-87774f69a715.target. 2024-02-22T19:22:58.915 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-b6783888-d1b7-11ee-95bf-87774f69a715.target → /etc/systemd/system/ceph-b6783888-d1b7-11ee-95bf-87774f69a715.target. 2024-02-22T19:22:59.127 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Failed to reset failed state of unit ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service: Unit ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service not loaded. 2024-02-22T19:22:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Created symlink /etc/systemd/system/ceph-b6783888-d1b7-11ee-95bf-87774f69a715.target.wants/ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service → /etc/systemd/system/ceph-b6783888-d1b7-11ee-95bf-87774f69a715@.service. 2024-02-22T19:22:59.262 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:22:59 smithi053 systemd[1]: Starting Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:22:59.799 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: disabled 2024-02-22T19:22:59.807 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: inactive 2024-02-22T19:22:59.807 INFO:teuthology.orchestra.run.smithi053.stderr:firewalld.service is not enabled 2024-02-22T19:22:59.807 INFO:teuthology.orchestra.run.smithi053.stderr:Not possible to enable service . firewalld.service is not available 2024-02-22T19:22:59.808 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for mon to start... 2024-02-22T19:22:59.808 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for mon... 2024-02-22T19:23:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:22:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.746180+0000 mon.a (mon.0) 0 : [INF] mkfs b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:22:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.731662+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-22T19:23:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:22:59 smithi053 bash[66788]: 31d709ed9dbb6ab0a38aeadf42cc07783f69f2390dd5633618dbcf0d672f7fe9 2024-02-22T19:23:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:22:59 smithi053 systemd[1]: Started Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: cluster: 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: id: b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: services: 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.670356s) 2024-02-22T19:23:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: data: 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: pgs: 2024-02-22T19:23:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:mon is available 2024-02-22T19:23:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:Assimilating anything we can from ceph.conf... 2024-02-22T19:23:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.745439+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-22T19:23:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.745559+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2024-02-22T19:23:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.747359+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-02-22T19:23:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.749227+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-22T19:23:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: cluster 2024-02-22T19:22:59.749600+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-22T19:23:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: audit 2024-02-22T19:23:00.415755+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.53:0/517360217' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: [global] 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: fsid = b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: [mgr] 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-22T19:23:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: [osd] 2024-02-22T19:23:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-22T19:23:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-22T19:23:01.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: audit 2024-02-22T19:23:01.604034+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.15.53:0/2835707206' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-22T19:23:01.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: audit 2024-02-22T19:23:01.606787+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.15.53:0/2835707206' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-22T19:23:02.144 INFO:teuthology.orchestra.run.smithi053.stderr:Generating new minimal ceph.conf... 2024-02-22T19:23:03.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: audit 2024-02-22T19:23:02.701308+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.53:0/1554515067' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:23:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:Restarting the monitor... 2024-02-22T19:23:03.262 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:03 smithi053 systemd[1]: Stopping Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:23:03.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:03 smithi053 bash[67124]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon.a" found: no such container 2024-02-22T19:23:03.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: debug 2024-02-22T19:23:03.489+0000 7fabbb9a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-22T19:23:03.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[66802]: debug 2024-02-22T19:23:03.489+0000 7fabbb9a0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-22T19:23:04.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:04 smithi053 bash[67162]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a 2024-02-22T19:23:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:05 smithi053 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service: Succeeded. 2024-02-22T19:23:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:05 smithi053 systemd[1]: Stopped Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:23:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:05 smithi053 systemd[1]: Starting Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:23:06.556 INFO:teuthology.orchestra.run.smithi053.stderr:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.508+0000 7fe2ee357880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.508+0000 7fe2ee357880 0 ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable), process ceph-mon, pid 7 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.508+0000 7fe2ee357880 0 pidfile_write: ignore empty --pid-file 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.512+0000 7fe2ee357880 0 load: jerasure load: lrc load: isa 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: RocksDB version: 6.8.1 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Compile date Feb 20 2024 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: DB SUMMARY 2024-02-22T19:23:06.807 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: CURRENT file: CURRENT 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 93729 ; 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.error_if_exists: 0 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.create_if_missing: 0 2024-02-22T19:23:06.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.paranoid_checks: 1 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.env: 0x56226ad09080 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.fs: Posix File System 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.info_log: 0x56226c182260 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.statistics: (nil) 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.use_fsync: 0 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_log_file_size: 0 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.allow_fallocate: 1 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-22T19:23:06.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.use_direct_reads: 0 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.db_log_dir: 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_subcompactions: 1 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_background_flushes: -1 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-22T19:23:06.810 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.write_buffer_manager: 0x56226ce743c0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.unordered_write: 0 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-22T19:23:06.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.row_cache: None 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.wal_filter: None 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.preserve_deletes: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.two_write_queues: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.atomic_flush: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.log_readahead_size: 0 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-22T19:23:06.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_background_jobs: 2 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_background_compactions: -1 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_open_files: -1 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Compression algorithms supported: 2024-02-22T19:23:06.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kZSTD supported: 0 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kXpressCompression supported: 0 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kLZ4Compression supported: 1 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kBZip2Compression supported: 0 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kZlibCompression supported: 1 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: kSnappyCompression supported: 1 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-22T19:23:06.815 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.merge_operator: 2024-02-22T19:23:06.816 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_filter: None 2024-02-22T19:23:06.816 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-22T19:23:06.816 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-22T19:23:06.816 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-22T19:23:06.816 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56226c127d10) 2024-02-22T19:23:06.816 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cache_index_and_filter_blocks: 1 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: pin_top_level_index_and_filter: 1 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: index_type: 0 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: data_block_index_type: 0 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: index_shortening: 1 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: data_block_hash_table_util_ratio: 0.750000 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: hash_index_allow_collision: 1 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: checksum: 1 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: no_block_cache: 0 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_cache: 0x56226c15cf10 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_cache_name: BinnedLRUCache 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_cache_options: 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: capacity : 536870912 2024-02-22T19:23:06.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: num_shard_bits : 4 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: strict_capacity_limit : 0 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: high_pri_pool_ratio: 0.000 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_cache_compressed: (nil) 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: persistent_cache: (nil) 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_size: 4096 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_size_deviation: 10 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_restart_interval: 16 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: index_block_restart_interval: 1 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: metadata_block_size: 4096 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: partition_filters: 0 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: use_delta_encoding: 1 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: whole_key_filtering: 1 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: verify_compression: 0 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: read_amp_bytes_per_bit: 0 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: format_version: 2 2024-02-22T19:23:06.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: enable_index_compression: 1 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: block_align: 0 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression: NoCompression 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.num_levels: 7 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-22T19:23:06.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-22T19:23:06.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-22T19:23:06.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-22T19:23:06.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.table_properties_collectors: 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.inplace_update_support: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.bloom_locality: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.max_successive_merges: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-22T19:23:06.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.ttl: 2592000 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.513+0000 7fe2ee357880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.514+0000 7fe2ee357880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.514+0000 7fe2ee357880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.514+0000 7fe2ee357880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629786515671, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.514+0000 7fe2ee357880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.515+0000 7fe2ee357880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.515+0000 7fe2ee357880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629786516648, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 89651, "table_properties": {"data_size": 87777, "index_size": 351, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 11380, "raw_average_key_size": 49, "raw_value_size": 81502, "raw_average_value_size": 351, "num_data_blocks": 11, "num_entries": 232, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708629786, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.515+0000 7fe2ee357880 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.516+0000 7fe2ee357880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629786517101, "job": 1, "event": "recovery_finished"} 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.516+0000 7fe2ee357880 4 rocksdb: DB pointer 0x56226c225800 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.516+0000 7fe2d5149700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.516+0000 7fe2d5149700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-22T19:23:06.824 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** DB Stats ** 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** Compaction Stats [default] ** 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: L0 2/0 89.20 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Sum 2/0 89.20 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.825 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** Compaction Stats [default] ** 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(Keys): cumulative 0, interval 0 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Cumulative compaction: 0.00 GB write, 26.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Interval compaction: 0.00 GB write, 26.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.826 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** File Read Latency Histogram By Level [default] ** 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** Compaction Stats [default] ** 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: L0 2/0 89.20 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Sum 2/0 89.20 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** Compaction Stats [default] ** 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.9 0.00 0.00 1 0.001 0 0 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T19:23:06.827 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: AddFile(Keys): cumulative 0, interval 0 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Cumulative compaction: 0.00 GB write, 26.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: ** File Read Latency Histogram By Level [default] ** 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.517+0000 7fe2ee357880 0 starting mon.a rank 0 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 1 mon.a@-1(???) e1 preinit fsid b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 0 mon.a@-1(???).mds e1 new map 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 0 mon.a@-1(???).mds e1 print_map 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: e1 2024-02-22T19:23:06.828 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: legacy client fscid: -1 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: No filesystems configured 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:02.701308+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.53:0/1554515067' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:23:06.518+0000 7fe2ee357880 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 bash[67283]: 1c8beaa1c2e01cfd4e11df37002384cca142ec836407bb1052629a1e0e2db661 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 systemd[1]: Started Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:23:06.829 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:06.534844+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-22T19:23:06.830 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:06.534887+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2024-02-22T19:23:06.830 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:06.535292+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-02-22T19:23:06.830 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:06.535321+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-22T19:23:06.830 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:06.535639+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-22T19:23:07.778 INFO:teuthology.orchestra.run.smithi053.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-22T19:23:07.778 INFO:teuthology.orchestra.run.smithi053.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:23:07.779 INFO:teuthology.orchestra.run.smithi053.stderr:Creating mgr... 2024-02-22T19:23:07.779 INFO:teuthology.orchestra.run.smithi053.stderr:Verifying port 9283 ... 2024-02-22T19:23:07.913 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Failed to reset failed state of unit ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.y.service: Unit ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.y.service not loaded. 2024-02-22T19:23:07.919 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: Created symlink /etc/systemd/system/ceph-b6783888-d1b7-11ee-95bf-87774f69a715.target.wants/ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-b6783888-d1b7-11ee-95bf-87774f69a715@.service. 2024-02-22T19:23:08.195 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:08 smithi053 systemd[1]: Starting Ceph mgr.y for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:23:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:07.194767+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.15.53:0/639520580' entity='client.admin' 2024-02-22T19:23:08.617 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: disabled 2024-02-22T19:23:08.625 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: inactive 2024-02-22T19:23:08.626 INFO:teuthology.orchestra.run.smithi053.stderr:firewalld.service is not enabled 2024-02-22T19:23:08.626 INFO:teuthology.orchestra.run.smithi053.stderr:Not possible to enable service . firewalld.service is not available 2024-02-22T19:23:08.633 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: disabled 2024-02-22T19:23:08.641 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: inactive 2024-02-22T19:23:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:firewalld.service is not enabled 2024-02-22T19:23:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-22T19:23:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for mgr to start... 2024-02-22T19:23:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for mgr... 2024-02-22T19:23:08.866 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:08 smithi053 bash[67557]: 107e8403ba0bd3eb524a98b6858ff7cce5acef977e83152c5f2c0452e0b82ed0 2024-02-22T19:23:08.866 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:08 smithi053 systemd[1]: Started Ceph mgr.y for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:23:08.866 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:08.648+0000 7f0eb4e44680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-22T19:23:08.866 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:08.740+0000 7f0eb4e44680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "health": { 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "checks": {}, 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mutes": [] 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-22T19:23:09.291 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum": [ 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 0 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "a" 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "monmap": { 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osdmap": { 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-22T19:23:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "pgmap": { 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-22T19:23:09.293 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "fsmap": { 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "available": false, 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "modules": [ 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "iostat", 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "nfs", 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "restful" 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "services": {} 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.294 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "servicemap": { 2024-02-22T19:23:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "modified": "2024-02-22T19:22:59.746536+0000", 2024-02-22T19:23:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "services": {} 2024-02-22T19:23:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-22T19:23:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:09.284807+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.53:0/1390051357' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-22T19:23:09.505 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:09.130+0000 7f0eb4e44680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-22T19:23:09.924 INFO:teuthology.orchestra.run.smithi053.stderr:mgr not available, waiting (1/15)... 2024-02-22T19:23:10.236 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:09.935+0000 7f0eb4e44680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-22T19:23:10.236 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:10.017+0000 7f0eb4e44680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-22T19:23:11.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:10.235+0000 7f0eb4e44680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-22T19:23:11.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:11.217+0000 7f0eb4e44680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-22T19:23:11.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:11.350+0000 7f0eb4e44680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:23:11.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:11.430+0000 7f0eb4e44680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-22T19:23:12.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:11.523+0000 7f0eb4e44680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-22T19:23:12.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:11.724+0000 7f0eb4e44680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-22T19:23:12.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:12.023+0000 7f0eb4e44680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-22T19:23:12.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:12.125+0000 7f0eb4e44680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-22T19:23:12.573 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:23:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "health": { 2024-02-22T19:23:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-22T19:23:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "checks": {}, 2024-02-22T19:23:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mutes": [] 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum": [ 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 0 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "a" 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "monmap": { 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osdmap": { 2024-02-22T19:23:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "pgmap": { 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-22T19:23:12.581 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "fsmap": { 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "available": false, 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "modules": [ 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "iostat", 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "nfs", 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "restful" 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "services": {} 2024-02-22T19:23:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "servicemap": { 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "modified": "2024-02-22T19:22:59.746536+0000", 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "services": {} 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-22T19:23:12.583 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:12.900 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:12.572938+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.15.53:0/24916106' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-22T19:23:13.104 INFO:teuthology.orchestra.run.smithi053.stderr:mgr not available, waiting (2/15)... 2024-02-22T19:23:13.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:12.899+0000 7f0eb4e44680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-22T19:23:13.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:12.983+0000 7f0eb4e44680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-22T19:23:13.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:13.078+0000 7f0eb4e44680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-22T19:23:13.526 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:13.259+0000 7f0eb4e44680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-22T19:23:13.800 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:13.525+0000 7f0eb4e44680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-22T19:23:13.800 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:13.661+0000 7f0eb4e44680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-22T19:23:14.065 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:13.799+0000 7f0eb4e44680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:23:14.065 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:13.978+0000 7f0eb4e44680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-22T19:23:14.065 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:14.064+0000 7f0eb4e44680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-22T19:23:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:14.078155+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-02-22T19:23:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:14.080646+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00260188s) 2024-02-22T19:23:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.081515+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T19:23:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.081713+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.081847+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.081991+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.082496+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:14.095714+0000 mon.a (mon.0) 16 : cluster [INF] Manager daemon y is now available 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.103739+0000 mon.a (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.110869+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.113125+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.115975+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' 2024-02-22T19:23:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:14.117610+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.53:0/2765946775' entity='mgr.y' 2024-02-22T19:23:15.875 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:15.875 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "health": { 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "checks": {}, 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mutes": [] 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum": [ 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 0 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-22T19:23:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "a" 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "quorum_age": 9, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "monmap": { 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osdmap": { 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-22T19:23:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-22T19:23:15.878 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-22T19:23:15.878 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-22T19:23:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "pgmap": { 2024-02-22T19:23:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "fsmap": { 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-22T19:23:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "available": true, 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "modules": [ 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "iostat", 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "nfs", 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "restful" 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ], 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "services": {} 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "servicemap": { 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-22T19:23:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "modified": "2024-02-22T19:22:59.746536+0000", 2024-02-22T19:23:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "services": {} 2024-02-22T19:23:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: }, 2024-02-22T19:23:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-22T19:23:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:16.172 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:15.083871+0000 mon.a (mon.0) 22 : cluster [DBG] mgrmap e3: y(active, since 1.00582s) 2024-02-22T19:23:16.173 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:15.871947+0000 mon.a (mon.0) 23 : audit [DBG] from='client.? 172.21.15.53:0/2842931680' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-22T19:23:17.455 INFO:teuthology.orchestra.run.smithi053.stderr:mgr is available 2024-02-22T19:23:17.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:16.084299+0000 mon.a (mon.0) 24 : cluster [DBG] mgrmap e4: y(active, since 2s) 2024-02-22T19:23:20.000 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:20.000 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: [global] 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: fsid = b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: [mgr] 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: [osd] 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-22T19:23:20.001 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-22T19:23:20.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:19.693029+0000 mon.a (mon.0) 25 : audit [INF] from='client.? 172.21.15.53:0/3283105118' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-22T19:23:25.144 INFO:teuthology.orchestra.run.smithi053.stderr:Enabling cephadm module... 2024-02-22T19:23:27.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:26.841684+0000 mon.a (mon.0) 26 : audit [INF] from='client.? 172.21.15.53:0/574588307' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-22T19:23:27.195 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ignoring --setuser ceph since I am not root 2024-02-22T19:23:27.195 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ignoring --setgroup ceph since I am not root 2024-02-22T19:23:27.195 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:27.051+0000 7f5d4cc4d680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-22T19:23:27.195 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:27.137+0000 7f5d4cc4d680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-22T19:23:27.721 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:27.526+0000 7f5d4cc4d680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-22T19:23:28.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:26.894815+0000 mon.a (mon.0) 27 : audit [INF] from='client.? 172.21.15.53:0/574588307' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-22T19:23:28.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:26.894862+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e5: y(active, since 12s) 2024-02-22T19:23:28.251 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:28.251 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-22T19:23:28.251 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "available": true, 2024-02-22T19:23:28.252 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-22T19:23:28.252 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-22T19:23:28.252 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:28.643 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:28.348+0000 7f5d4cc4d680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-22T19:23:28.643 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:28.430+0000 7f5d4cc4d680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-22T19:23:28.895 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:28.652+0000 7f5d4cc4d680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-22T19:23:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for the mgr to restart... 2024-02-22T19:23:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for mgr epoch 5... 2024-02-22T19:23:29.172 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:28.250799+0000 mon.a (mon.0) 29 : audit [DBG] from='client.? 172.21.15.53:0/2035172939' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-22T19:23:29.953 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:29.643+0000 7f5d4cc4d680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-22T19:23:29.953 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:29.777+0000 7f5d4cc4d680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:23:29.953 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:29.859+0000 7f5d4cc4d680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-22T19:23:30.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:29.952+0000 7f5d4cc4d680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-22T19:23:30.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:30.154+0000 7f5d4cc4d680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-22T19:23:30.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:30.446+0000 7f5d4cc4d680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-22T19:23:30.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:30.547+0000 7f5d4cc4d680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-22T19:23:31.662 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:31.302+0000 7f5d4cc4d680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-22T19:23:31.662 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:31.386+0000 7f5d4cc4d680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-22T19:23:31.663 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:31.482+0000 7f5d4cc4d680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-22T19:23:32.206 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:31.661+0000 7f5d4cc4d680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-22T19:23:32.206 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:31.930+0000 7f5d4cc4d680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-22T19:23:32.206 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:32.066+0000 7f5d4cc4d680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-22T19:23:32.471 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:32.205+0000 7f5d4cc4d680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:23:32.471 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:32.384+0000 7f5d4cc4d680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-22T19:23:32.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:32.470+0000 7f5d4cc4d680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-22T19:23:32.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:32.485100+0000 mon.a (mon.0) 30 : cluster [INF] Active manager daemon y restarted 2024-02-22T19:23:32.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:32.485303+0000 mon.a (mon.0) 31 : cluster [INF] Activating manager daemon y 2024-02-22T19:23:32.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:32.487916+0000 mon.a (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-22T19:23:32.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:32.487999+0000 mon.a (mon.0) 33 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00277192s) 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.488949+0000 mon.a (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.489611+0000 mon.a (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.490257+0000 mon.a (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.490411+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.490555+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:32.504397+0000 mon.a (mon.0) 39 : cluster [INF] Manager daemon y is now available 2024-02-22T19:23:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.516678+0000 mon.a (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-22T19:23:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "initialized": true 2024-02-22T19:23:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:33.808 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:23:32.514193+0000 mgr.y (mgr.14116) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.607468+0000 mon.a (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.608425+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.609878+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.611445+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.625668+0000 mon.a (mon.0) 45 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:32.628856+0000 mon.a (mon.0) 46 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-22T19:23:33.809 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:33.490897+0000 mon.a (mon.0) 47 : cluster [DBG] mgrmap e7: y(active, since 1.00567s) 2024-02-22T19:23:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:mgr epoch 5 is available 2024-02-22T19:23:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:Setting orchestrator backend to cephadm... 2024-02-22T19:23:35.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:33.490059+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-22T19:23:35.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:33.497161+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-22T19:23:35.713 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:34.609563+0000 mon.a (mon.0) 48 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-02-22T19:23:35.713 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:34.825595+0000 mgr.y (mgr.14116) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:35.713 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:34.829483+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:35.713 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:34.887123+0000 mon.a (mon.0) 50 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:36.158 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: value unchanged 2024-02-22T19:23:36.700 INFO:teuthology.orchestra.run.smithi053.stderr:Generating ssh key... 2024-02-22T19:23:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:36.157788+0000 mgr.y (mgr.14116) 5 : audit [DBG] from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: Generating public/private rsa key pair. 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: Your identification has been saved in /tmp/tmp4ly55ail/key. 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: Your public key has been saved in /tmp/tmp4ly55ail/key.pub. 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: The key fingerprint is: 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: SHA256:dPeILUAazmrVh+dA9dGy20YzXcKpYp9/sftyDbPpw4k ceph-b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: The key's randomart image is: 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: +---[RSA 3072]----+ 2024-02-22T19:23:37.745 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | . o.. .o . | 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | o * . ...= .| 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | = * + o+ o.| 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | o . Bo+oo+ .| 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | o S.+oo=.o | 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | . .+ =. | 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | = *+| 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | E O.+| 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: | ..*+| 2024-02-22T19:23:37.746 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: +----[SHA256]-----+ 2024-02-22T19:23:38.698 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7hvPVYC3CuiMlKFYdC9pa09dtJGmOhYjoGIpthDOJpbVJZhAsZfMmRgRjbOST76qdurijydE9OjMcK1D8+maaeYukgOoD28DqJBy7jcaQRBbey9KDwDx9UQjlHFmJag2IJ8h6hsOw2Bd3mykRkSJPIJMb1DQC6rAKQ24FExUj9JjgPEMsGfQN29Oovw0QsqlMoX5E3x8Oep6GwHN8TV+ChHOdn3MTBaVfMWzREG71rV+S5iO6/T+7K3gvI38Hba89DCmrmL538yXRTgzL/qk6sPWobdGM1c7mg2AivbA8W/mWZMOgX/PKj1PGXrj+Bkcm9hYoFFM2Z74FppY1IjZVC8dcGMQ3a8GjEpdZPVRqNQeAVeQ9b96FEkNvb0mVD8KGEH30M55wI92IB+mxE02lXoO4GXg64YGMyfAFupZ2KaVIw1o9t4QAdE+M2dlTaL+xx0BBRb2ZtdZ87GGGMz2K2fjY7e6ac6hJUbDHNA0PvyjRSSlbJq7FZX0b4Pm1PyU= ceph-b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:23:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:37.421814+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:23:37.422030+0000 mgr.y (mgr.14116) 7 : cephadm [INF] Generating ssh key... 2024-02-22T19:23:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:37.472258+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:37.473917+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:39.306 INFO:teuthology.orchestra.run.smithi053.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-22T19:23:39.306 INFO:teuthology.orchestra.run.smithi053.stderr:Adding key to root@localhost authorized_keys... 2024-02-22T19:23:39.306 INFO:teuthology.orchestra.run.smithi053.stderr:Adding host smithi053... 2024-02-22T19:23:39.568 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:38.697788+0000 mgr.y (mgr.14116) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:40.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [79B blob data] 2024-02-22T19:23:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: Added host 'smithi053' with addr '172.21.15.53' 2024-02-22T19:23:41.339 INFO:teuthology.orchestra.run.smithi053.stderr:Deploying unmanaged mon service... 2024-02-22T19:23:41.924 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:40.030708+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:41.924 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:40.664663+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:41.924 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:23:40.664976+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Added host smithi053 2024-02-22T19:23:41.924 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:40.668149+0000 mon.a (mon.0) 54 : audit [DBG] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:42.347 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-22T19:23:42.918 INFO:teuthology.orchestra.run.smithi053.stderr:Deploying unmanaged mgr service... 2024-02-22T19:23:43.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:42.343165+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:43.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:23:42.344425+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-22T19:23:43.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:42.346516+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:43.930 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-22T19:23:44.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:43.518742+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:44.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:43.774480+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:44.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:43.926640+0000 mgr.y (mgr.14116) 13 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:23:44.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:23:43.927553+0000 mgr.y (mgr.14116) 14 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-22T19:23:44.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:43.929671+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:46.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:45.407393+0000 mon.a (mon.0) 59 : audit [INF] from='client.? 172.21.15.53:0/3978594451' entity='client.admin' 2024-02-22T19:23:46.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:46.167772+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14116 172.21.15.53:0/1628023296' entity='mgr.y' 2024-02-22T19:23:48.137 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:46.841867+0000 mon.a (mon.0) 61 : audit [INF] from='client.? 172.21.15.53:0/2175843898' entity='client.admin' 2024-02-22T19:23:48.385 INFO:teuthology.orchestra.run.smithi053.stderr:Enabling the dashboard module... 2024-02-22T19:23:49.614 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:49.214592+0000 mon.a (mon.0) 62 : audit [INF] from='client.? 172.21.15.53:0/1834078708' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-22T19:23:49.614 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ignoring --setuser ceph since I am not root 2024-02-22T19:23:49.614 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ignoring --setgroup ceph since I am not root 2024-02-22T19:23:49.614 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:49.427+0000 7f984c4be680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-22T19:23:49.614 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:49.514+0000 7f984c4be680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-22T19:23:50.193 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:49.894+0000 7f984c4be680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-22T19:23:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:49.267936+0000 mon.a (mon.0) 63 : audit [INF] from='client.? 172.21.15.53:0/1834078708' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-22T19:23:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:49.267987+0000 mon.a (mon.0) 64 : cluster [DBG] mgrmap e9: y(active, since 16s) 2024-02-22T19:23:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-22T19:23:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "available": true, 2024-02-22T19:23:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-22T19:23:50.765 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-22T19:23:50.766 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:51.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:50.695+0000 7f984c4be680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-22T19:23:51.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:50.780+0000 7f984c4be680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-22T19:23:51.269 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:51.003+0000 7f984c4be680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-22T19:23:51.425 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for the mgr to restart... 2024-02-22T19:23:51.425 INFO:teuthology.orchestra.run.smithi053.stderr:Waiting for mgr epoch 9... 2024-02-22T19:23:51.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:50.764941+0000 mon.a (mon.0) 65 : audit [DBG] from='client.? 172.21.15.53:0/3883910255' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-22T19:23:52.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.001+0000 7f984c4be680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-22T19:23:52.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.138+0000 7f984c4be680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:23:52.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.220+0000 7f984c4be680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-22T19:23:52.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.313+0000 7f984c4be680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-22T19:23:52.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.514+0000 7f984c4be680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-22T19:23:53.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.806+0000 7f984c4be680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-22T19:23:53.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:52.908+0000 7f984c4be680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-22T19:23:54.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:53.665+0000 7f984c4be680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-22T19:23:54.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:53.750+0000 7f984c4be680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-22T19:23:54.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:53.847+0000 7f984c4be680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-22T19:23:54.292 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:54.029+0000 7f984c4be680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-22T19:23:54.577 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:54.299+0000 7f984c4be680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-22T19:23:54.577 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:54.436+0000 7f984c4be680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-22T19:23:54.844 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:54.575+0000 7f984c4be680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:23:54.844 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:54.755+0000 7f984c4be680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-22T19:23:55.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:23:54.843+0000 7f984c4be680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-22T19:23:55.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:23:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [79B blob data] 2024-02-22T19:23:55.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:54.856881+0000 mon.a (mon.0) 66 : cluster [INF] Active manager daemon y restarted 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:54.857116+0000 mon.a (mon.0) 67 : cluster [INF] Activating manager daemon y 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:54.859797+0000 mon.a (mon.0) 68 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:54.859910+0000 mon.a (mon.0) 69 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00289202s) 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.860855+0000 mon.a (mon.0) 70 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.861470+0000 mon.a (mon.0) 71 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.862286+0000 mon.a (mon.0) 72 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.862437+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.862552+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T19:23:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:54.864685+0000 mon.a (mon.0) 75 : cluster [INF] Manager daemon y is now available 2024-02-22T19:23:55.868 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: { 2024-02-22T19:23:55.868 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-22T19:23:55.871 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: "initialized": true 2024-02-22T19:23:55.871 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: } 2024-02-22T19:23:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.963550+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.964948+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:23:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.990008+0000 mon.a (mon.0) 78 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-22T19:23:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:54.993584+0000 mon.a (mon.0) 79 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-22T19:23:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:55.862229+0000 mon.a (mon.0) 80 : cluster [DBG] mgrmap e11: y(active, since 1.00521s) 2024-02-22T19:23:56.853 INFO:teuthology.orchestra.run.smithi053.stderr:mgr epoch 9 is available 2024-02-22T19:23:56.853 INFO:teuthology.orchestra.run.smithi053.stderr:Generating a dashboard self-signed certificate... 2024-02-22T19:23:57.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:55.861514+0000 mgr.y (mgr.14146) 1 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-22T19:23:57.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:55.867758+0000 mgr.y (mgr.14146) 2 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-22T19:23:57.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:56.251966+0000 mon.a (mon.0) 81 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:23:57.732 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-22T19:23:58.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:23:57.255777+0000 mon.a (mon.0) 82 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-02-22T19:23:58.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:57.729884+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:23:58.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:57.731633+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:23:58.507 INFO:teuthology.orchestra.run.smithi053.stderr:Creating initial admin user... 2024-02-22T19:23:59.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:23:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:57.594054+0000 mgr.y (mgr.14146) 3 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:24:00.172 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$fSCS7eRyQcoOzMt6EzT4lOFcFtnGahCXxAxYT11dgZWnFKsGMA/0e", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708629840, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-22T19:24:00.852 INFO:teuthology.orchestra.run.smithi053.stderr:Fetching dashboard port number... 2024-02-22T19:24:01.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:23:59.926222+0000 mgr.y (mgr.14146) 4 : audit [DBG] from='client.14158 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:24:01.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:00.171956+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:01.556 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: 8443 2024-02-22T19:24:02.194 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: disabled 2024-02-22T19:24:02.202 INFO:teuthology.orchestra.run.smithi053.stderr:systemctl: inactive 2024-02-22T19:24:02.203 INFO:teuthology.orchestra.run.smithi053.stderr:firewalld.service is not enabled 2024-02-22T19:24:02.203 INFO:teuthology.orchestra.run.smithi053.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr:Ceph Dashboard is now available at: 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr: URL: https://smithi053.front.sepia.ceph.com:8443/ 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr: User: admin 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr: Password: act9l43jc9 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:02.206 INFO:teuthology.orchestra.run.smithi053.stderr:Enabling autotune for osd_memory_target 2024-02-22T19:24:02.426 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:01.175685+0000 mon.a (mon.0) 86 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-02-22T19:24:02.426 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:01.555952+0000 mon.a (mon.0) 87 : audit [DBG] from='client.? 172.21.15.53:0/1021269589' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-22T19:24:04.419 INFO:teuthology.orchestra.run.smithi053.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-22T19:24:05.057 INFO:teuthology.orchestra.run.smithi053.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-22T19:24:05.057 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.057 INFO:teuthology.orchestra.run.smithi053.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:24:05.057 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.057 INFO:teuthology.orchestra.run.smithi053.stderr:Or, if you are only running a single cluster on this host: 2024-02-22T19:24:05.057 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: ceph telemetry on 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr:For more information see: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:24:05.058 INFO:teuthology.orchestra.run.smithi053.stderr:Bootstrap complete. 2024-02-22T19:24:05.102 INFO:tasks.cephadm:Fetching config... 2024-02-22T19:24:05.102 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:24:05.102 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-22T19:24:05.123 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-22T19:24:05.123 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:24:05.123 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-22T19:24:05.183 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-22T19:24:05.183 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:24:05.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-22T19:24:05.253 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-22T19:24:05.254 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:24:05.254 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-22T19:24:05.312 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-22T19:24:05.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7hvPVYC3CuiMlKFYdC9pa09dtJGmOhYjoGIpthDOJpbVJZhAsZfMmRgRjbOST76qdurijydE9OjMcK1D8+maaeYukgOoD28DqJBy7jcaQRBbey9KDwDx9UQjlHFmJag2IJ8h6hsOw2Bd3mykRkSJPIJMb1DQC6rAKQ24FExUj9JjgPEMsGfQN29Oovw0QsqlMoX5E3x8Oep6GwHN8TV+ChHOdn3MTBaVfMWzREG71rV+S5iO6/T+7K3gvI38Hba89DCmrmL538yXRTgzL/qk6sPWobdGM1c7mg2AivbA8W/mWZMOgX/PKj1PGXrj+Bkcm9hYoFFM2Z74FppY1IjZVC8dcGMQ3a8GjEpdZPVRqNQeAVeQ9b96FEkNvb0mVD8KGEH30M55wI92IB+mxE02lXoO4GXg64YGMyfAFupZ2KaVIw1o9t4QAdE+M2dlTaL+xx0BBRb2ZtdZ87GGGMz2K2fjY7e6ac6hJUbDHNA0PvyjRSSlbJq7FZX0b4Pm1PyU= ceph-b6783888-d1b7-11ee-95bf-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-22T19:24:05.394 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7hvPVYC3CuiMlKFYdC9pa09dtJGmOhYjoGIpthDOJpbVJZhAsZfMmRgRjbOST76qdurijydE9OjMcK1D8+maaeYukgOoD28DqJBy7jcaQRBbey9KDwDx9UQjlHFmJag2IJ8h6hsOw2Bd3mykRkSJPIJMb1DQC6rAKQ24FExUj9JjgPEMsGfQN29Oovw0QsqlMoX5E3x8Oep6GwHN8TV+ChHOdn3MTBaVfMWzREG71rV+S5iO6/T+7K3gvI38Hba89DCmrmL538yXRTgzL/qk6sPWobdGM1c7mg2AivbA8W/mWZMOgX/PKj1PGXrj+Bkcm9hYoFFM2Z74FppY1IjZVC8dcGMQ3a8GjEpdZPVRqNQeAVeQ9b96FEkNvb0mVD8KGEH30M55wI92IB+mxE02lXoO4GXg64YGMyfAFupZ2KaVIw1o9t4QAdE+M2dlTaL+xx0BBRb2ZtdZ87GGGMz2K2fjY7e6ac6hJUbDHNA0PvyjRSSlbJq7FZX0b4Pm1PyU= ceph-b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:24:05.405 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7hvPVYC3CuiMlKFYdC9pa09dtJGmOhYjoGIpthDOJpbVJZhAsZfMmRgRjbOST76qdurijydE9OjMcK1D8+maaeYukgOoD28DqJBy7jcaQRBbey9KDwDx9UQjlHFmJag2IJ8h6hsOw2Bd3mykRkSJPIJMb1DQC6rAKQ24FExUj9JjgPEMsGfQN29Oovw0QsqlMoX5E3x8Oep6GwHN8TV+ChHOdn3MTBaVfMWzREG71rV+S5iO6/T+7K3gvI38Hba89DCmrmL538yXRTgzL/qk6sPWobdGM1c7mg2AivbA8W/mWZMOgX/PKj1PGXrj+Bkcm9hYoFFM2Z74FppY1IjZVC8dcGMQ3a8GjEpdZPVRqNQeAVeQ9b96FEkNvb0mVD8KGEH30M55wI92IB+mxE02lXoO4GXg64YGMyfAFupZ2KaVIw1o9t4QAdE+M2dlTaL+xx0BBRb2ZtdZ87GGGMz2K2fjY7e6ac6hJUbDHNA0PvyjRSSlbJq7FZX0b4Pm1PyU= ceph-b6783888-d1b7-11ee-95bf-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-22T19:24:05.452 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7hvPVYC3CuiMlKFYdC9pa09dtJGmOhYjoGIpthDOJpbVJZhAsZfMmRgRjbOST76qdurijydE9OjMcK1D8+maaeYukgOoD28DqJBy7jcaQRBbey9KDwDx9UQjlHFmJag2IJ8h6hsOw2Bd3mykRkSJPIJMb1DQC6rAKQ24FExUj9JjgPEMsGfQN29Oovw0QsqlMoX5E3x8Oep6GwHN8TV+ChHOdn3MTBaVfMWzREG71rV+S5iO6/T+7K3gvI38Hba89DCmrmL538yXRTgzL/qk6sPWobdGM1c7mg2AivbA8W/mWZMOgX/PKj1PGXrj+Bkcm9hYoFFM2Z74FppY1IjZVC8dcGMQ3a8GjEpdZPVRqNQeAVeQ9b96FEkNvb0mVD8KGEH30M55wI92IB+mxE02lXoO4GXg64YGMyfAFupZ2KaVIw1o9t4QAdE+M2dlTaL+xx0BBRb2ZtdZ87GGGMz2K2fjY7e6ac6hJUbDHNA0PvyjRSSlbJq7FZX0b4Pm1PyU= ceph-b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:24:05.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-22T19:24:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:04.416378+0000 mon.a (mon.0) 88 : audit [INF] from='client.? 172.21.15.53:0/3135497960' entity='client.admin' 2024-02-22T19:24:08.951 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:07.708223+0000 mon.a (mon.0) 89 : audit [INF] from='client.? 172.21.15.53:0/2821420582' entity='client.admin' 2024-02-22T19:24:08.951 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:08.424583+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:08.972 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-22T19:24:08.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-22T19:24:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:08.781995+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:24:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:08.785278+0000 mon.a (mon.0) 92 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:08.786621+0000 mon.a (mon.0) 93 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:24:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:08.790078+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:12.014 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi064 2024-02-22T19:24:12.015 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:24:12.015 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.conf 2024-02-22T19:24:12.037 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:24:12.037 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:24:12.099 INFO:tasks.cephadm:Adding host smithi064 to orchestrator... 2024-02-22T19:24:12.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch host add smithi064 2024-02-22T19:24:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:11.389478+0000 mon.a (mon.0) 95 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:11.409288+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:24:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:11.410606+0000 mon.a (mon.0) 97 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:24:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:11.411204+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:24:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:24:11.412192+0000 mgr.y (mgr.14146) 6 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:24:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:24:11.426456+0000 mgr.y (mgr.14146) 7 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:24:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:11.479961+0000 mon.a (mon.0) 99 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:11.482105+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:16.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:14.863804+0000 mgr.y (mgr.14146) 8 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:18.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:16.864104+0000 mgr.y (mgr.14146) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:20.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:18.864405+0000 mgr.y (mgr.14146) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:22.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:20.864672+0000 mgr.y (mgr.14146) 11 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:24.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:22.865000+0000 mgr.y (mgr.14146) 12 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:26.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:24.865253+0000 mgr.y (mgr.14146) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:26.865482+0000 mgr.y (mgr.14146) 14 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:28.865739+0000 mgr.y (mgr.14146) 15 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:30.865985+0000 mgr.y (mgr.14146) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:32.866242+0000 mgr.y (mgr.14146) 17 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:34.866507+0000 mgr.y (mgr.14146) 18 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:38.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:36.866705+0000 mgr.y (mgr.14146) 19 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:40.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:38.867023+0000 mgr.y (mgr.14146) 20 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:42.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:40.867401+0000 mgr.y (mgr.14146) 21 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:44.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:42.867706+0000 mgr.y (mgr.14146) 22 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:46.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:44.867951+0000 mgr.y (mgr.14146) 23 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:48.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:46.868250+0000 mgr.y (mgr.14146) 24 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:50.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:48.868777+0000 mgr.y (mgr.14146) 25 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:52.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:50.869134+0000 mgr.y (mgr.14146) 26 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:54.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:52.869431+0000 mgr.y (mgr.14146) 27 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:54.869774+0000 mgr.y (mgr.14146) 28 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:57.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:24:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [89B blob data] 2024-02-22T19:24:58.109 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi064' with addr '172.21.15.64' 2024-02-22T19:24:58.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:56.870134+0000 mgr.y (mgr.14146) 29 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:24:58.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:24:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [79B blob data] 2024-02-22T19:24:58.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch host ls --format=json 2024-02-22T19:24:59.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:57.459059+0000 mgr.y (mgr.14146) 30 : audit [DBG] from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:24:59.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:58.107025+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:24:59.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:24:58.107468+0000 mgr.y (mgr.14146) 31 : cephadm [INF] Added host smithi064 2024-02-22T19:24:59.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:58.123276+0000 mon.a (mon.0) 102 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:24:59.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:24:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:58.676728+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:24:58.870571+0000 mgr.y (mgr.14146) 32 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:24:59.523233+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:00.739 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:25:00.739 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}] 2024-02-22T19:25:01.407 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-22T19:25:01.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd crush tunables default 2024-02-22T19:25:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:00.738827+0000 mgr.y (mgr.14146) 33 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:25:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:00.870967+0000 mgr.y (mgr.14146) 34 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:02.871199+0000 mgr.y (mgr.14146) 35 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:03.477439+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:03.516429+0000 mon.a (mon.0) 106 : audit [INF] from='client.? 172.21.15.53:0/2214360544' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.053444+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.054207+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.055180+0000 mon.a (mon.0) 109 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.055988+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:04.057297+0000 mgr.y (mgr.14146) 36 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:25:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:04.082680+0000 mgr.y (mgr.14146) 37 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:25:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.110204+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:05.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.114400+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:05.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:04.477442+0000 mon.a (mon.0) 113 : audit [INF] from='client.? 172.21.15.53:0/2214360544' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-22T19:25:05.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:04.477494+0000 mon.a (mon.0) 114 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-22T19:25:05.320 INFO:tasks.cephadm:Adding mon.a on smithi053 2024-02-22T19:25:05.321 INFO:tasks.cephadm:Adding mon.c on smithi053 2024-02-22T19:25:05.321 INFO:tasks.cephadm:Adding mon.b on smithi064 2024-02-22T19:25:05.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply mon '3;smithi053:172.21.15.53=a;smithi053:[v2:172.21.15.53:3301,v1:172.21.15.53:6790]=c;smithi064:172.21.15.64=b' 2024-02-22T19:25:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:04.871630+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:07.236 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mon update... 2024-02-22T19:25:07.925 DEBUG:teuthology.orchestra.run.smithi053:mon.c> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.c.service 2024-02-22T19:25:07.928 DEBUG:teuthology.orchestra.run.smithi064:mon.b> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.b.service 2024-02-22T19:25:07.931 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-22T19:25:07.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T19:25:07.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:25:07.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:25:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:06.872187+0000 mgr.y (mgr.14146) 39 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.230298+0000 mgr.y (mgr.14146) 40 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi053:[v2:172.21.15.53:3301,v1:172.21.15.53:6790]=c;smithi064:172.21.15.64=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:07.232566+0000 mgr.y (mgr.14146) 41 : cephadm [INF] Saving service mon spec with placement smithi053:172.21.15.53=a;smithi053:[v2:172.21.15.53:3301,v1:172.21.15.53:6790]=c;smithi064:172.21.15.64=b;count:3 2024-02-22T19:25:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.235691+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.286315+0000 mon.a (mon.0) 116 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.288000+0000 mon.a (mon.0) 117 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.288814+0000 mon.a (mon.0) 118 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.292400+0000 mon.a (mon.0) 119 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.294172+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:07.294958+0000 mon.a (mon.0) 121 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:07.295701+0000 mgr.y (mgr.14146) 42 : cephadm [INF] Deploying daemon mon.b on smithi064 2024-02-22T19:25:10.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:08.872489+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:11.562 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:25:11.562 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","modified":"2024-02-22T19:22:57.014624Z","created":"2024-02-22T19:22:57.014624Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T19:25:11.564 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 1 2024-02-22T19:25:12.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:10.872735+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:12.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:11.561581+0000 mon.a (mon.0) 122 : audit [DBG] from='client.? 172.21.15.64:0/3054730559' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:12 smithi064 bash[39813]: 23f5ee0442696e26fb183c3d890e3a241ebcbe97204e1eb2e00e4ddb83f8e60d 2024-02-22T19:25:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:12 smithi064 systemd[1]: Started Ceph mon.b for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:25:13.183 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-22T19:25:13.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T19:25:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:12.332530+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:25:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:12.332701+0000 mon.a (mon.0) 125 : cluster [INF] mon.a calling monitor election 2024-02-22T19:25:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:12.346161+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:12.873077+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:13.329652+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:14.329975+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:14.330774+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:14.873507+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:15.330147+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:16.329938+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:16.873852+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:17.330046+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:17.360311+0000 mon.a (mon.0) 132 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:17.363187+0000 mon.a (mon.0) 133 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:17.363679+0000 mon.a (mon.0) 134 : cluster [DBG] fsmap 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:17.363721+0000 mon.a (mon.0) 135 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:17.363930+0000 mon.a (mon.0) 136 : cluster [DBG] mgrmap e13: y(active, since 82s) 2024-02-22T19:25:17.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:17.364077+0000 mon.a (mon.0) 137 : cluster [INF] overall HEALTH_OK 2024-02-22T19:25:17.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:17.367152+0000 mon.a (mon.0) 138 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:17.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:17.368209+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:17.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:17.369022+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:12.332530+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:12.332701+0000 mon.a (mon.0) 125 : cluster [INF] mon.a calling monitor election 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:12.346161+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:12.873077+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:13.329652+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:14.329975+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:14.330774+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:14.873507+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:15.330147+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:16.329938+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:16.873852+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:17.330046+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:17.360311+0000 mon.a (mon.0) 132 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:17.363187+0000 mon.a (mon.0) 133 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:17.363679+0000 mon.a (mon.0) 134 : cluster [DBG] fsmap 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:17.363721+0000 mon.a (mon.0) 135 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:17.363930+0000 mon.a (mon.0) 136 : cluster [DBG] mgrmap e13: y(active, since 82s) 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:17.364077+0000 mon.a (mon.0) 137 : cluster [INF] overall HEALTH_OK 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:17.367152+0000 mon.a (mon.0) 138 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:17.368209+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:17.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:17.369022+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:25:18.500 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","modified":"2024-02-22T19:25:12.329926Z","created":"2024-02-22T19:22:57.014624Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-22T19:25:18.501 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 2 2024-02-22T19:25:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:17.369726+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Deploying daemon mon.c on smithi053 2024-02-22T19:25:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:18.330302+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:17.369726+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Deploying daemon mon.c on smithi053 2024-02-22T19:25:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:18.330302+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:19.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:18.499447+0000 mon.a (mon.0) 142 : audit [DBG] from='client.? 172.21.15.64:0/754054487' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:19.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:18.874240+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:18.499447+0000 mon.a (mon.0) 142 : audit [DBG] from='client.? 172.21.15.64:0/754054487' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:18.874240+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:20.146 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-22T19:25:20.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T19:25:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 systemd[1]: Starting Ceph mon.c for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:25:21.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.732+0000 7ff9fbf44880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.732+0000 7ff9fbf44880 0 ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable), process ceph-mon, pid 7 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.732+0000 7ff9fbf44880 0 pidfile_write: ignore empty --pid-file 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.736+0000 7ff9fbf44880 0 load: jerasure load: lrc load: isa 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: RocksDB version: 6.8.1 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Compile date Feb 20 2024 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: DB SUMMARY 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: CURRENT file: CURRENT 2024-02-22T19:25:21.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 636 ; 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.error_if_exists: 0 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.create_if_missing: 0 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.paranoid_checks: 1 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.env: 0x55a1a7bf1080 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.fs: Posix File System 2024-02-22T19:25:21.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.info_log: 0x55a1a9048260 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.statistics: (nil) 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.use_fsync: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_log_file_size: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.allow_fallocate: 1 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.use_direct_reads: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-22T19:25:21.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.db_log_dir: 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_subcompactions: 1 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_background_flushes: -1 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.write_buffer_manager: 0x55a1a9d3a390 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-22T19:25:21.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.unordered_write: 0 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-22T19:25:21.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.row_cache: None 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.wal_filter: None 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.preserve_deletes: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.two_write_queues: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.atomic_flush: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-22T19:25:21.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.log_readahead_size: 0 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_background_jobs: 2 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_background_compactions: -1 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_open_files: -1 2024-02-22T19:25:21.012 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Compression algorithms supported: 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kZSTD supported: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kXpressCompression supported: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kLZ4Compression supported: 1 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kBZip2Compression supported: 0 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kZlibCompression supported: 1 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: kSnappyCompression supported: 1 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.013 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.merge_operator: 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_filter: None 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a1a8fedd10) 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cache_index_and_filter_blocks: 1 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: pin_top_level_index_and_filter: 1 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: index_type: 0 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: data_block_index_type: 0 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: index_shortening: 1 2024-02-22T19:25:21.014 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: data_block_hash_table_util_ratio: 0.750000 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: hash_index_allow_collision: 1 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: checksum: 1 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: no_block_cache: 0 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_cache: 0x55a1a9022f10 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_cache_name: BinnedLRUCache 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_cache_options: 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: capacity : 536870912 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: num_shard_bits : 4 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: strict_capacity_limit : 0 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: high_pri_pool_ratio: 0.000 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_cache_compressed: (nil) 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: persistent_cache: (nil) 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_size: 4096 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_size_deviation: 10 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_restart_interval: 16 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: index_block_restart_interval: 1 2024-02-22T19:25:21.015 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: metadata_block_size: 4096 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: partition_filters: 0 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: use_delta_encoding: 1 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: whole_key_filtering: 1 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: verify_compression: 0 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: read_amp_bytes_per_bit: 0 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: format_version: 2 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: enable_index_compression: 1 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: block_align: 0 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression: NoCompression 2024-02-22T19:25:21.016 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.num_levels: 7 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-22T19:25:21.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-22T19:25:21.018 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-22T19:25:21.019 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-22T19:25:21.019 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-22T19:25:21.019 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-22T19:25:21.019 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-22T19:25:21.020 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.table_properties_collectors: 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.inplace_update_support: 0 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-22T19:25:21.021 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.bloom_locality: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.max_successive_merges: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.ttl: 2592000 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.737+0000 7ff9fbf44880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629920739076, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.738+0000 7ff9fbf44880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.738+0000 7ff9fbf44880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-22T19:25:21.022 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.738+0000 7ff9fbf44880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629920739580, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1562, "table_properties": {"data_size": 648, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708629920, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.738+0000 7ff9fbf44880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.738+0000 7ff9fbf44880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629920739980, "job": 1, "event": "recovery_finished"} 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.739+0000 7ff9fbf44880 4 rocksdb: DB pointer 0x55a1a90eb800 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.739+0000 7ff9e2d36700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.739+0000 7ff9e2d36700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** DB Stats ** 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.023 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** Compaction Stats [default] ** 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** Compaction Stats [default] ** 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T19:25:21.024 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(Keys): cumulative 0, interval 0 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** File Read Latency Histogram By Level [default] ** 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** Compaction Stats [default] ** 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** Compaction Stats [default] ** 2024-02-22T19:25:21.025 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: AddFile(Keys): cumulative 0, interval 0 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: ** File Read Latency Histogram By Level [default] ** 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.739+0000 7ff9fbf44880 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.740+0000 7ff9fbf44880 0 using public_addrv [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] 2024-02-22T19:25:21.026 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.740+0000 7ff9fbf44880 0 starting mon.c rank -1 at public addrs [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] at bind addrs [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.741+0000 7ff9fbf44880 1 mon.c@-1(???) e0 preinit fsid b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 0 mon.c@-1(synchronizing).mds e1 new map 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 0 mon.c@-1(synchronizing).mds e1 print_map 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: e1 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: legacy client fscid: -1 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: No filesystems configured 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 1 mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e0d32700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9ead46700 4 rocksdb: (Original Log Time 2024/02/22-19:25:20.760334) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9ead46700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2024-02-22T19:25:21.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9ead46700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708629920760396, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 1015, "num_deletes": 24, "total_data_size": 4218591, "memory_usage": 4243920, "flush_reason": "Manual Compaction"} 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9ead46700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 1 mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 1 mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 0 mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.759+0000 7ff9e553b700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:18.499447+0000 mon.a (mon.0) 142 : audit [DBG] from='client.? 172.21.15.64:0/754054487' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:18.874240+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:25:20.760+0000 7ff9e553b700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-22T19:25:21.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 bash[73824]: 2d407f33a63864cf785d691f54d1d8a5ab0ee266af9a1bf76739319807baa193 2024-02-22T19:25:21.029 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:20 smithi053 systemd[1]: Started Ceph mon.c for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:25:26.170 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:20.768817+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:20.768979+0000 mon.a (mon.0) 145 : cluster [INF] mon.a calling monitor election 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:20.774920+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:20.782080+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:20.782223+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:20.874572+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:21.765652+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:22.765669+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:22.767428+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:22.874914+0000 mgr.y (mgr.14146) 51 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:23.765770+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:24.765960+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:24.875254+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.171 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:25.766197+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:25.796759+0000 mon.a (mon.0) 153 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:25.799996+0000 mon.a (mon.0) 154 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0]} removed_ranks: {} 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:25.800541+0000 mon.a (mon.0) 155 : cluster [DBG] fsmap 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:25.800572+0000 mon.a (mon.0) 156 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:25.800799+0000 mon.a (mon.0) 157 : cluster [DBG] mgrmap e13: y(active, since 90s) 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:25.801129+0000 mon.a (mon.0) 158 : cluster [INF] overall HEALTH_OK 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:25.816050+0000 mon.a (mon.0) 159 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:26.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:25.817610+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:20.768817+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:25:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:20.768979+0000 mon.a (mon.0) 145 : cluster [INF] mon.a calling monitor election 2024-02-22T19:25:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:20.774920+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-22T19:25:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:20.782080+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:20.782223+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:20.874572+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:21.765652+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:22.765669+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:22.767428+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:22.874914+0000 mgr.y (mgr.14146) 51 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:23.765770+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:24.765960+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:24.875254+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:25.766197+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:25.796759+0000 mon.a (mon.0) 153 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:25.799996+0000 mon.a (mon.0) 154 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0]} removed_ranks: {} 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:25.800541+0000 mon.a (mon.0) 155 : cluster [DBG] fsmap 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:25.800572+0000 mon.a (mon.0) 156 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:25.800799+0000 mon.a (mon.0) 157 : cluster [DBG] mgrmap e13: y(active, since 90s) 2024-02-22T19:25:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:25.801129+0000 mon.a (mon.0) 158 : cluster [INF] overall HEALTH_OK 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:25.816050+0000 mon.a (mon.0) 159 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:25.817610+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:20.768817+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:20.768979+0000 mon.a (mon.0) 145 : cluster [INF] mon.a calling monitor election 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:20.774920+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:20.782080+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:20.782223+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:20.874572+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:21.765652+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:22.765669+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:22.767428+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:22.874914+0000 mgr.y (mgr.14146) 51 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:23.765770+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:24.765960+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:24.875254+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:25.766197+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:25.796759+0000 mon.a (mon.0) 153 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:25.799996+0000 mon.a (mon.0) 154 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0]} removed_ranks: {} 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:25.800541+0000 mon.a (mon.0) 155 : cluster [DBG] fsmap 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:25.800572+0000 mon.a (mon.0) 156 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:25.800799+0000 mon.a (mon.0) 157 : cluster [DBG] mgrmap e13: y(active, since 90s) 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:25.801129+0000 mon.a (mon.0) 158 : cluster [INF] overall HEALTH_OK 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:25.816050+0000 mon.a (mon.0) 159 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:26.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:25.817610+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:27.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:26.766189+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:26.766189+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:26.766189+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:25:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:26.875464+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:26.901717+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:26.875464+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:26.901717+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:26.875464+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:26.901717+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:29.951 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-22T19:25:29.952 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","modified":"2024-02-22T19:25:20.766243Z","created":"2024-02-22T19:22:57.014624Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3301","nonce":0},{"type":"v1","addr":"172.21.15.53:6790","nonce":0}]},"addr":"172.21.15.53:6790/0","public_addr":"172.21.15.53:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-22T19:25:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 3 2024-02-22T19:25:30.207 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:28.875707+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:28.875707+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:30.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:28.875707+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:30.619 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-22T19:25:30.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph config generate-minimal-conf 2024-02-22T19:25:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:29.951353+0000 mon.a (mon.0) 163 : audit [DBG] from='client.? 172.21.15.64:0/3923784172' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:31.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:29.998487+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:29.951353+0000 mon.a (mon.0) 163 : audit [DBG] from='client.? 172.21.15.64:0/3923784172' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:29.998487+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:29.951353+0000 mon.a (mon.0) 163 : audit [DBG] from='client.? 172.21.15.64:0/3923784172' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:25:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:29.998487+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.252 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:30.875932+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.068672+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.069255+0000 mon.a (mon.0) 166 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.069867+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:31.070707+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:31.070916+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.093033+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.129558+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.132311+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:31.133333+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.133525+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.134071+0000 mon.a (mon.0) 172 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:31.134622+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:31.135101+0000 mgr.y (mgr.14146) 59 : cephadm [INF] Reconfiguring daemon mon.a on smithi053 2024-02-22T19:25:32.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:30.875932+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.068672+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.069255+0000 mon.a (mon.0) 166 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.069867+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:31.070707+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:31.070916+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.093033+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.129558+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.132311+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:31.133333+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.133525+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.134071+0000 mon.a (mon.0) 172 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:31.134622+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:31.135101+0000 mgr.y (mgr.14146) 59 : cephadm [INF] Reconfiguring daemon mon.a on smithi053 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:30.875932+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.068672+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.069255+0000 mon.a (mon.0) 166 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.069867+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:31.070707+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:31.070916+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.093033+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.129558+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.132311+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:31.133333+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-22T19:25:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.133525+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.134071+0000 mon.a (mon.0) 172 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:31.134622+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:31.135101+0000 mgr.y (mgr.14146) 59 : cephadm [INF] Reconfiguring daemon mon.a on smithi053 2024-02-22T19:25:33.704 INFO:teuthology.orchestra.run.smithi053.stdout:# minimal ceph.conf for b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:25:33.704 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-02-22T19:25:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:25:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: mon_host = [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] 2024-02-22T19:25:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:32.876168+0000 mgr.y (mgr.14146) 60 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:33.361289+0000 mon.a (mon.0) 174 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:33.362109+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:33.362769+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:33.363512+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:33.704100+0000 mon.a (mon.0) 178 : audit [DBG] from='client.? 172.21.15.53:0/477240933' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:34.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:32.876168+0000 mgr.y (mgr.14146) 60 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:34.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:33.361289+0000 mon.a (mon.0) 174 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:34.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:33.362109+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:33.362769+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:33.363512+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:33.704100+0000 mon.a (mon.0) 178 : audit [DBG] from='client.? 172.21.15.53:0/477240933' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:32.876168+0000 mgr.y (mgr.14146) 60 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:33.361289+0000 mon.a (mon.0) 174 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:33.362109+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:33.362769+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:33.363512+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:33.704100+0000 mon.a (mon.0) 178 : audit [DBG] from='client.? 172.21.15.53:0/477240933' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:35.220 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-22T19:25:35.221 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:25:35.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-02-22T19:25:35.254 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:25:35.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:25:35.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:33.361850+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-22T19:25:35.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:33.364143+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Reconfiguring daemon mon.c on smithi053 2024-02-22T19:25:35.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:34.831768+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:35.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:34.832541+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:35.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:34.833175+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:35.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:34.833730+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:33.361850+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-22T19:25:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:33.364143+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Reconfiguring daemon mon.c on smithi053 2024-02-22T19:25:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:34.831768+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:34.832541+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:34.833175+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:34.833730+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:35.325 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:25:35.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-02-22T19:25:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:33.361850+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-22T19:25:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:33.364143+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Reconfiguring daemon mon.c on smithi053 2024-02-22T19:25:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:34.831768+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:34.832541+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T19:25:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:34.833175+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T19:25:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:34.833730+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:35.357 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:25:35.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:25:35.425 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-02-22T19:25:35.425 INFO:tasks.cephadm:Adding mgr.x on smithi064 2024-02-22T19:25:35.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply mgr '2;smithi053=y;smithi064=x' 2024-02-22T19:25:36.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:34.832304+0000 mgr.y (mgr.14146) 63 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-22T19:25:36.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:34.834310+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Reconfiguring daemon mon.b on smithi064 2024-02-22T19:25:36.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:34.876455+0000 mgr.y (mgr.14146) 65 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:36.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:34.832304+0000 mgr.y (mgr.14146) 63 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-22T19:25:36.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:34.834310+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Reconfiguring daemon mon.b on smithi064 2024-02-22T19:25:36.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:34.876455+0000 mgr.y (mgr.14146) 65 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:36.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:34.832304+0000 mgr.y (mgr.14146) 63 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-22T19:25:36.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:34.834310+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Reconfiguring daemon mon.b on smithi064 2024-02-22T19:25:36.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:34.876455+0000 mgr.y (mgr.14146) 65 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:37.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:36.293606+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:36.294534+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:36.296078+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:36.296883+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:36.301583+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:37.626 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mgr update... 2024-02-22T19:25:37.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:36.293606+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:37.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:36.294534+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:37.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:36.296078+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:37.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:36.296883+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:37.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:36.301583+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:37.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:36.293606+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:37.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:36.294534+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:37.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:36.296078+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:37.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:36.296883+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:37.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:36.301583+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.198 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.x.service 2024-02-22T19:25:38.200 INFO:tasks.cephadm:Deploying OSDs... 2024-02-22T19:25:38.201 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:25:38.201 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-02-22T19:25:38.223 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-22T19:25:38.224 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-02-22T19:25:38.237 INFO:journalctl@ceph.mgr.x.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:25:38.286 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-22T19:25:38.286 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:38.286 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 198959 Links: 1 2024-02-22T19:25:38.286 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:38.286 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:38.286 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-02-22 19:25:20.160675493 +0000 2024-02-22T19:25:38.287 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-02-22 19:01:25.735903661 +0000 2024-02-22T19:25:38.287 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-02-22 19:01:25.735903661 +0000 2024-02-22T19:25:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-02-22T19:25:38.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-22T19:25:38.358 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-02-22T19:25:38.359 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-02-22T19:25:38.359 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000120754 s, 4.2 MB/s 2024-02-22T19:25:38.360 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-22T19:25:38.422 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-02-22T19:25:38.475 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:36.876790+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:38.475 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.625968+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.670718+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.672406+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.673150+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.678114+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.679579+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.682941+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.683690+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T19:25:38.476 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.684584+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:38.483 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-22T19:25:38.483 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:38.483 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 199062 Links: 1 2024-02-22T19:25:38.483 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:38.484 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:38.484 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-02-22 19:25:20.161675473 +0000 2024-02-22T19:25:38.484 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-02-22 19:01:26.072896962 +0000 2024-02-22T19:25:38.484 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-02-22 19:01:26.072896962 +0000 2024-02-22T19:25:38.484 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-02-22T19:25:38.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:36.876790+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.625968+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.670718+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.672406+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.673150+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.678114+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.679579+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.682941+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.683690+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.684584+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:36.876790+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.625968+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.670718+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.672406+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.673150+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.678114+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.679579+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.682941+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.683690+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T19:25:38.553 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.684584+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:38.554 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-02-22T19:25:38.554 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-02-22T19:25:38.554 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.8756e-05 s, 5.2 MB/s 2024-02-22T19:25:38.555 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-22T19:25:38.617 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 199165 Links: 1 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-02-22 19:25:20.161675473 +0000 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-02-22 19:01:26.423889985 +0000 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-02-22 19:01:26.423889985 +0000 2024-02-22T19:25:38.678 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-02-22T19:25:38.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-22T19:25:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-02-22T19:25:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-02-22T19:25:38.748 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.8522e-05 s, 5.2 MB/s 2024-02-22T19:25:38.749 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-22T19:25:38.810 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-02-22T19:25:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-22T19:25:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:38.869 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 192371 Links: 1 2024-02-22T19:25:38.870 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:38.870 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:38.870 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-02-22 19:25:20.161675473 +0000 2024-02-22T19:25:38.870 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-02-22 19:01:26.757883346 +0000 2024-02-22T19:25:38.870 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-02-22 19:01:26.757883346 +0000 2024-02-22T19:25:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-02-22T19:25:38.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-22T19:25:38.937 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-02-22T19:25:38.937 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-02-22T19:25:38.937 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.6152e-05 s, 5.3 MB/s 2024-02-22T19:25:38.938 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-22T19:25:38.998 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:25:38.998 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-02-22T19:25:39.017 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-22T19:25:39.017 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 165860 Links: 1 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-22 19:25:11.267120208 +0000 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-22 19:01:29.959262010 +0000 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-22 19:01:29.959262010 +0000 2024-02-22T19:25:39.076 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-02-22T19:25:39.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-22T19:25:39.143 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-22T19:25:39.143 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-22T19:25:39.144 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000218976 s, 2.3 MB/s 2024-02-22T19:25:39.145 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-22T19:25:39.204 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-02-22T19:25:39.262 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 167698 Links: 1 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-22 19:25:11.267120208 +0000 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-22 19:01:30.302255229 +0000 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-22 19:01:30.302255229 +0000 2024-02-22T19:25:39.263 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-02-22T19:25:39.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-22T19:25:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:37.621390+0000 mgr.y (mgr.14146) 67 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi053=y;smithi064=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:37.622851+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Saving service mgr spec with placement smithi053=y;smithi064=x;count:2 2024-02-22T19:25:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:37.685303+0000 mgr.y (mgr.14146) 69 : cephadm [INF] Deploying daemon mgr.x on smithi064 2024-02-22T19:25:39.404 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-22T19:25:39.404 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-22T19:25:39.404 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000216934 s, 2.4 MB/s 2024-02-22T19:25:39.404 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-22T19:25:39.423 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-02-22T19:25:39.482 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-22T19:25:39.482 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:39.482 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 166640 Links: 1 2024-02-22T19:25:39.482 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:39.482 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:39.482 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-22 19:25:39.355555324 +0000 2024-02-22T19:25:39.483 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-22 19:01:30.654248271 +0000 2024-02-22T19:25:39.483 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-22 19:01:30.654248271 +0000 2024-02-22T19:25:39.483 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-02-22T19:25:39.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-22T19:25:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-22T19:25:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-22T19:25:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000193215 s, 2.6 MB/s 2024-02-22T19:25:39.552 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-22T19:25:39.612 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-02-22T19:25:39.671 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 169297 Links: 1 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-22 19:25:39.490552609 +0000 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-22 19:01:31.016241116 +0000 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-22 19:01:31.016241116 +0000 2024-02-22T19:25:39.672 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-02-22T19:25:39.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-22T19:25:39.736 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:39 smithi064 systemd[1]: Starting Ceph mgr.x for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:25:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-22T19:25:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-22T19:25:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00021987 s, 2.3 MB/s 2024-02-22T19:25:39.740 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-22T19:25:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:37.621390+0000 mgr.y (mgr.14146) 67 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi053=y;smithi064=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:37.622851+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Saving service mgr spec with placement smithi053=y;smithi064=x;count:2 2024-02-22T19:25:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:37.685303+0000 mgr.y (mgr.14146) 69 : cephadm [INF] Deploying daemon mgr.x on smithi064 2024-02-22T19:25:39.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:37.621390+0000 mgr.y (mgr.14146) 67 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi053=y;smithi064=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:39.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:37.622851+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Saving service mgr spec with placement smithi053=y;smithi064=x;count:2 2024-02-22T19:25:39.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:37.685303+0000 mgr.y (mgr.14146) 69 : cephadm [INF] Deploying daemon mgr.x on smithi064 2024-02-22T19:25:39.800 INFO:tasks.cephadm:Deploying osd.0 on smithi053 with /dev/vg_nvme/lv_4... 2024-02-22T19:25:39.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-22T19:25:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:38.877101+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:40.255988+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:40.257637+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:38.877101+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:40.255988+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:40.257637+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:38.877101+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:40.255988+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:40.257637+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:40.129+0000 7fe5d1157700 1 -- 172.21.15.64:0/3545765421 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561840f8a4e0 con 0x561840f5f000 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:40 smithi064 bash[41233]: d7ee98bbdfd530a171cb81c615f046f3cc03faa89c32bdfba0e8934425b2d746 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:40 smithi064 systemd[1]: Started Ceph mgr.x for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:40.258+0000 7fe5e1752680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-22T19:25:40.603 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:40.344+0000 7fe5e1752680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-22T19:25:41.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:40.751+0000 7fe5e1752680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-22T19:25:41.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:41.566+0000 7fe5e1752680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-22T19:25:41.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:41.648+0000 7fe5e1752680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-22T19:25:41.983 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-22T19:25:41.983 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-22T19:25:41.983 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-02-22T19:25:41.983 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-02-22T19:25:41.983 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133879 s, 783 MB/s 2024-02-22T19:25:41.983 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-02-22T19:25:42.267 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:41.866+0000 7fe5e1752680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-22T19:25:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:40.877427+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.267984+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.268637+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.269364+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.272843+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:41.274422+0000 mgr.y (mgr.14146) 72 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-22T19:25:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.274691+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T19:25:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.275587+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T19:25:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:41.276299+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:41.276994+0000 mgr.y (mgr.14146) 73 : cephadm [INF] Reconfiguring daemon mgr.y on smithi053 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:40.877427+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.267984+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.268637+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.269364+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.272843+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:41.274422+0000 mgr.y (mgr.14146) 72 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.274691+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.275587+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:41.276299+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:41.276994+0000 mgr.y (mgr.14146) 73 : cephadm [INF] Reconfiguring daemon mgr.y on smithi053 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:40.877427+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.267984+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:42.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.268637+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.269364+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.272843+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:41.274422+0000 mgr.y (mgr.14146) 72 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.274691+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.275587+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:41.276299+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:42.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:41.276994+0000 mgr.y (mgr.14146) 73 : cephadm [INF] Reconfiguring daemon mgr.y on smithi053 2024-02-22T19:25:43.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:42.849+0000 7fe5e1752680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-22T19:25:43.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:42.982+0000 7fe5e1752680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:25:43.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:43.063+0000 7fe5e1752680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-22T19:25:43.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-02-22T19:25:43.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:43.156+0000 7fe5e1752680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-22T19:25:43.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:43.357+0000 7fe5e1752680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-22T19:25:44.056 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:43.647+0000 7fe5e1752680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-22T19:25:44.056 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:43.747+0000 7fe5e1752680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-22T19:25:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:42.877827+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:43.057922+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:43.058958+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:43.060332+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:43.060964+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:43.064510+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:44.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:42.877827+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:44.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:43.057922+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:44.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:43.058958+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:44.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:43.060332+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:44.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:43.060964+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:44.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:43.064510+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:42.877827+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:43.057922+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:43.058958+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:43.060332+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:43.060964+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:43.064510+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:44.499+0000 7fe5e1752680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-22T19:25:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:44.583+0000 7fe5e1752680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-22T19:25:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:44.676+0000 7fe5e1752680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-22T19:25:45.122 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:44.854+0000 7fe5e1752680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-22T19:25:45.394 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:45.121+0000 7fe5e1752680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-22T19:25:45.394 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:45.256+0000 7fe5e1752680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-22T19:25:45.394 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:45.394+0000 7fe5e1752680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:25:45.657 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:45.571+0000 7fe5e1752680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:44.878118+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.004630+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.433213+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.435070+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.435581+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:45.671859+0000 mon.a (mon.0) 215 : cluster [DBG] Standby manager daemon x started 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.672864+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.673235+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.674133+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-22T19:25:46.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.674440+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:44.878118+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.004630+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.433213+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.435070+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.435581+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:45.671859+0000 mon.a (mon.0) 215 : cluster [DBG] Standby manager daemon x started 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.672864+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.673235+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.674133+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-22T19:25:46.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.674440+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T19:25:46.352 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:25:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:25:45.656+0000 7fe5e1752680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-22T19:25:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:44.878118+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.004630+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.433213+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.435070+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.435581+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:45.671859+0000 mon.a (mon.0) 215 : cluster [DBG] Standby manager daemon x started 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.672864+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.673235+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.674133+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-22T19:25:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.674440+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.64:0/2830104408' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T19:25:47.165 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:45.431036+0000 mgr.y (mgr.14146) 76 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:47.165 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:46.064163+0000 mon.a (mon.0) 216 : cluster [DBG] mgrmap e14: y(active, since 111s), standbys: x 2024-02-22T19:25:47.166 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:46.064271+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-22T19:25:47.166 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:45.431036+0000 mgr.y (mgr.14146) 76 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:47.166 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:46.064163+0000 mon.a (mon.0) 216 : cluster [DBG] mgrmap e14: y(active, since 111s), standbys: x 2024-02-22T19:25:47.166 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:46.064271+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-22T19:25:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:45.431036+0000 mgr.y (mgr.14146) 76 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:25:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:46.064163+0000 mon.a (mon.0) 216 : cluster [DBG] mgrmap e14: y(active, since 111s), standbys: x 2024-02-22T19:25:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:46.064271+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-22T19:25:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:46.878489+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:47.933471+0000 mon.a (mon.0) 218 : audit [INF] from='client.? 172.21.15.53:0/942489338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab"}]: dispatch 2024-02-22T19:25:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:47.937041+0000 mon.a (mon.0) 219 : audit [INF] from='client.? 172.21.15.53:0/942489338' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab"}]': finished 2024-02-22T19:25:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:47.937111+0000 mon.a (mon.0) 220 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-22T19:25:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:47.937214+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:46.878489+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:47.933471+0000 mon.a (mon.0) 218 : audit [INF] from='client.? 172.21.15.53:0/942489338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab"}]: dispatch 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:47.937041+0000 mon.a (mon.0) 219 : audit [INF] from='client.? 172.21.15.53:0/942489338' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab"}]': finished 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:47.937111+0000 mon.a (mon.0) 220 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:47.937214+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:46.878489+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:47.933471+0000 mon.a (mon.0) 218 : audit [INF] from='client.? 172.21.15.53:0/942489338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab"}]: dispatch 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:47.937041+0000 mon.a (mon.0) 219 : audit [INF] from='client.? 172.21.15.53:0/942489338' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab"}]': finished 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:47.937111+0000 mon.a (mon.0) 220 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-22T19:25:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:47.937214+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:25:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:48.512606+0000 mon.a (mon.0) 222 : audit [DBG] from='client.? 172.21.15.53:0/2441292976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:25:49.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:48.512606+0000 mon.a (mon.0) 222 : audit [DBG] from='client.? 172.21.15.53:0/2441292976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:25:49.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:48.512606+0000 mon.a (mon.0) 222 : audit [DBG] from='client.? 172.21.15.53:0/2441292976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:25:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:48.878890+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:48.878890+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:50.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:48.878890+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:52.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:50.879277+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:52.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:50.879277+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:50.879277+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:54.296 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:52.879650+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:54.296 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:52.879650+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:54.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:52.879650+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:55.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:54.316386+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T19:25:55.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:54.317073+0000 mon.a (mon.0) 224 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:55.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:25:54.317677+0000 mgr.y (mgr.14146) 81 : cephadm [INF] Deploying daemon osd.0 on smithi053 2024-02-22T19:25:55.347 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:54.316386+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T19:25:55.347 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:54.317073+0000 mon.a (mon.0) 224 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:55.347 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:25:54.317677+0000 mgr.y (mgr.14146) 81 : cephadm [INF] Deploying daemon osd.0 on smithi053 2024-02-22T19:25:55.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:54.316386+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T19:25:55.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:54.317073+0000 mon.a (mon.0) 224 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:25:55.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:25:54.317677+0000 mgr.y (mgr.14146) 81 : cephadm [INF] Deploying daemon osd.0 on smithi053 2024-02-22T19:25:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:54.879968+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:56.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:54.879968+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:54.879968+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:25:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:56.880296+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:25:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:56.880296+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:25:58.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:25:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:56.880296+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:25:58.880653+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:59.295212+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:25:59.316174+0000 mon.a (mon.0) 226 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:00.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:25:58.880653+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:00.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:59.295212+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:00.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:25:59.316174+0000 mon.a (mon.0) 226 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:25:58.880653+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:59.295212+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:25:59.316174+0000 mon.a (mon.0) 226 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:02.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:00.880999+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:02.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:01.704488+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:02.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:01.705115+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:02.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:01.705719+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:02.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:01.709051+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:02.422 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:00.880999+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:02.422 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:01.704488+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:02.422 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:01.705115+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:02.422 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:01.705719+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:02.422 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:01.709051+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:00.880999+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:01.704488+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:01.705115+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:01.705719+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:01.709051+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:02.719 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 0 on host 'smithi053' 2024-02-22T19:26:03.363 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:02.424335+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v2:172.21.15.53:6801/162271081' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T19:26:03.363 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:02.424811+0000 mon.a (mon.0) 231 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T19:26:03.363 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:02.718364+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:03.363 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:02.726709+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:03.364 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:02.424335+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v2:172.21.15.53:6801/162271081' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T19:26:03.364 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:02.424811+0000 mon.a (mon.0) 231 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T19:26:03.364 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:02.718364+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:03.364 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:02.726709+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:03.365 DEBUG:teuthology.orchestra.run.smithi053:osd.0> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.0.service 2024-02-22T19:26:03.367 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_3... 2024-02-22T19:26:03.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-22T19:26:03.399 INFO:journalctl@ceph.osd.0.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:26:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:02.424335+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v2:172.21.15.53:6801/162271081' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T19:26:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:02.424811+0000 mon.a (mon.0) 231 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T19:26:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:02.718364+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:02.726709+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:02.881229+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:03.300858+0000 mon.a (mon.0) 234 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T19:26:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:03.300950+0000 mon.a (mon.0) 235 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-22T19:26:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:03.301269+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:03.301653+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v2:172.21.15.53:6801/162271081' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:03.304402+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:04.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:02.881229+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:03.300858+0000 mon.a (mon.0) 234 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:03.300950+0000 mon.a (mon.0) 235 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:03.301269+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:03.301653+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v2:172.21.15.53:6801/162271081' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:03.304402+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:04.755 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-0[77321]: debug 2024-02-22T19:26:04.308+0000 7fc8439ea700 -1 osd.0 0 waiting for initial osdmap 2024-02-22T19:26:04.755 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-0[77321]: debug 2024-02-22T19:26:04.311+0000 7fc83e963700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:02.881229+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:04.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:03.300858+0000 mon.a (mon.0) 234 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T19:26:04.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:03.300950+0000 mon.a (mon.0) 235 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-22T19:26:04.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:03.301269+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:04.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:03.301653+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v2:172.21.15.53:6801/162271081' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:04.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:03.304402+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:04.305173+0000 mon.a (mon.0) 238 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:04.305264+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-22T19:26:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:04.305564+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:04.309574+0000 mon.a (mon.0) 241 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:04.305173+0000 mon.a (mon.0) 238 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:04.305264+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-22T19:26:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:04.305564+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:04.309574+0000 mon.a (mon.0) 241 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:04.305173+0000 mon.a (mon.0) 238 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:04.305264+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-22T19:26:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:04.305564+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:04.309574+0000 mon.a (mon.0) 241 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:03.380404+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:03.380454+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:04.881510+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:05.310528+0000 mon.a (mon.0) 242 : cluster [INF] osd.0 v2:172.21.15.53:6801/162271081 boot 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:05.310590+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:05.310979+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:03.380404+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:03.380454+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:04.881510+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:05.310528+0000 mon.a (mon.0) 242 : cluster [INF] osd.0 v2:172.21.15.53:6801/162271081 boot 2024-02-22T19:26:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:05.310590+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-22T19:26:06.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:05.310979+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:06.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:03.380404+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:06.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:03.380454+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:06.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:04.881510+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T19:26:06.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:05.310528+0000 mon.a (mon.0) 242 : cluster [INF] osd.0 v2:172.21.15.53:6801/162271081 boot 2024-02-22T19:26:06.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:05.310590+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-22T19:26:06.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:05.310979+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:26:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-22T19:26:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-22T19:26:06.687 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-02-22T19:26:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-02-22T19:26:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00827057 s, 1.3 GB/s 2024-02-22T19:26:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-02-22T19:26:07.324 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-02-22T19:26:07.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:06.320777+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-22T19:26:07.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:06.320777+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-22T19:26:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:06.320777+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-22T19:26:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:06.881915+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:08.004050+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:08.004908+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:08.009147+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:08.010036+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:08.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:08.010659+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:08.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:08.014778+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:06.881915+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:08.004050+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:08.004908+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:08.009147+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:08.010036+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:08.010659+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:08.014778+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:06.881915+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:08.004050+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:08.004908+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:08.009147+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:08.010036+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:08.010659+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:08.014778+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:07.999603+0000 mgr.y (mgr.14146) 89 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:08.005444+0000 mgr.y (mgr.14146) 90 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 16102M 2024-02-22T19:26:09.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:07.999603+0000 mgr.y (mgr.14146) 89 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:09.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:08.005444+0000 mgr.y (mgr.14146) 90 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 16102M 2024-02-22T19:26:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:07.999603+0000 mgr.y (mgr.14146) 89 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:08.005444+0000 mgr.y (mgr.14146) 90 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 16102M 2024-02-22T19:26:10.586 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:08.882259+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:09.532156+0000 mgr.y (mgr.14146) 92 : audit [DBG] from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:09.534131+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:09.535958+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:09.536530+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:08.882259+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:09.532156+0000 mgr.y (mgr.14146) 92 : audit [DBG] from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:09.534131+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:09.535958+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:26:10.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:09.536530+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:08.882259+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:09.532156+0000 mgr.y (mgr.14146) 92 : audit [DBG] from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:26:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:09.534131+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:26:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:09.535958+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:26:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:09.536530+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:11.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:10.882665+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:11.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:10.882665+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:11.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:10.882665+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:11.997453+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.53:0/4100110663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]: dispatch 2024-02-22T19:26:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:11.997766+0000 mon.a (mon.0) 255 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]: dispatch 2024-02-22T19:26:12.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:12.001886+0000 mon.a (mon.0) 256 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]': finished 2024-02-22T19:26:12.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:12.001956+0000 mon.a (mon.0) 257 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-22T19:26:12.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:12.002070+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:11.997453+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.53:0/4100110663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]: dispatch 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:11.997766+0000 mon.a (mon.0) 255 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]: dispatch 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:12.001886+0000 mon.a (mon.0) 256 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]': finished 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:12.001956+0000 mon.a (mon.0) 257 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:12.002070+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:11.997453+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.53:0/4100110663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]: dispatch 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:11.997766+0000 mon.a (mon.0) 255 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]: dispatch 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:12.001886+0000 mon.a (mon.0) 256 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2"}]': finished 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:12.001956+0000 mon.a (mon.0) 257 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-22T19:26:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:12.002070+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:12.620512+0000 mon.a (mon.0) 259 : audit [DBG] from='client.? 172.21.15.53:0/2429097115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:26:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:12.883219+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:13.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:12.620512+0000 mon.a (mon.0) 259 : audit [DBG] from='client.? 172.21.15.53:0/2429097115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:26:13.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:12.883219+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:13.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:12.620512+0000 mon.a (mon.0) 259 : audit [DBG] from='client.? 172.21.15.53:0/2429097115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:26:13.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:12.883219+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:16.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:14.883538+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:16.219 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:14.883538+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:14.883538+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:16.883912+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:17.958 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:16.883912+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:17.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:16.883912+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:18.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:18.243347+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T19:26:18.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:18.244021+0000 mon.a (mon.0) 261 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:18.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:18.244598+0000 mgr.y (mgr.14146) 97 : cephadm [INF] Deploying daemon osd.1 on smithi053 2024-02-22T19:26:18.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:18.243347+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T19:26:18.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:18.244021+0000 mon.a (mon.0) 261 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:18.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:18.244598+0000 mgr.y (mgr.14146) 97 : cephadm [INF] Deploying daemon osd.1 on smithi053 2024-02-22T19:26:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:18.243347+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T19:26:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:18.244021+0000 mon.a (mon.0) 261 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:18.244598+0000 mgr.y (mgr.14146) 97 : cephadm [INF] Deploying daemon osd.1 on smithi053 2024-02-22T19:26:19.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:18.884195+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:19.687 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:18.884195+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:18.884195+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:22.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:20.884489+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:22.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:20.884489+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:20.884489+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:22.884755+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:23.269435+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:23.312454+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:22.884755+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:23.269435+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:23.312454+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:22.884755+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:23.269435+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:23.312454+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:24.885139+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:25.759159+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:25.759892+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:25.760553+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:25.764392+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:26.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:24.885139+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:26.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:25.759159+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:26.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:25.759892+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:26.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:25.760553+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:26.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:25.764392+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:24.885139+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:25.759159+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:25.759892+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:25.760553+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:25.764392+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:26.861 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-02-22T19:26:27.461 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:26.361590+0000 mon.a (mon.0) 268 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T19:26:27.461 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:26.860231+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:27.461 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:26.882124+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:27.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:26.361590+0000 mon.a (mon.0) 268 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T19:26:27.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:26.860231+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:27.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:26.882124+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:27.483 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.1.service 2024-02-22T19:26:27.525 INFO:tasks.cephadm:Deploying osd.2 on smithi053 with /dev/vg_nvme/lv_2... 2024-02-22T19:26:27.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-22T19:26:27.558 INFO:journalctl@ceph.osd.1.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:26:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:26.361590+0000 mon.a (mon.0) 268 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T19:26:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:26.860231+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:26.882124+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:28.316 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-1[81742]: debug 2024-02-22T19:26:28.281+0000 7f2aa2aaf700 -1 osd.1 0 waiting for initial osdmap 2024-02-22T19:26:28.316 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-1[81742]: debug 2024-02-22T19:26:28.286+0000 7f2a9f22b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:26.885387+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:27.278074+0000 mon.a (mon.0) 271 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:27.278138+0000 mon.a (mon.0) 272 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:27.278286+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:27.278592+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:26.885387+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:27.278074+0000 mon.a (mon.0) 271 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T19:26:28.317 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:27.278138+0000 mon.a (mon.0) 272 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-22T19:26:28.318 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:27.278286+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:28.318 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:27.278592+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:26.885387+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:27.278074+0000 mon.a (mon.0) 271 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T19:26:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:27.278138+0000 mon.a (mon.0) 272 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-22T19:26:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:27.278286+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:27.278592+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:29.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:28.279421+0000 mon.a (mon.0) 275 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:28.279467+0000 mon.a (mon.0) 276 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:28.279757+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:28.285344+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:28.279421+0000 mon.a (mon.0) 275 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:28.279467+0000 mon.a (mon.0) 276 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:28.279757+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:29.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:28.285344+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:28.279421+0000 mon.a (mon.0) 275 : audit [INF] from='osd.1 v2:172.21.15.53:6805/2973403044' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:28.279467+0000 mon.a (mon.0) 276 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-22T19:26:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:28.279757+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:28.285344+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:30.388 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:27.347739+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:27.347784+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:28.885641+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:29.284468+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:29.288236+0000 mon.a (mon.0) 280 : cluster [INF] osd.1 v2:172.21.15.53:6805/2973403044 boot 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:29.288280+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:29.288593+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:27.347739+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:27.347784+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:28.885641+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:29.284468+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:29.288236+0000 mon.a (mon.0) 280 : cluster [INF] osd.1 v2:172.21.15.53:6805/2973403044 boot 2024-02-22T19:26:30.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:29.288280+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-22T19:26:30.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:29.288593+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:27.347739+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:27.347784+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:28.885641+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:29.284468+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:29.288236+0000 mon.a (mon.0) 280 : cluster [INF] osd.1 v2:172.21.15.53:6805/2973403044 boot 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:29.288280+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-22T19:26:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:29.288593+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:26:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-22T19:26:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-22T19:26:31.591 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-02-22T19:26:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-02-22T19:26:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00817794 s, 1.3 GB/s 2024-02-22T19:26:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-02-22T19:26:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:30.289124+0000 mon.a (mon.0) 283 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-22T19:26:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.252981+0000 mon.a (mon.0) 284 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:31.625 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:30.289124+0000 mon.a (mon.0) 283 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-22T19:26:31.625 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.252981+0000 mon.a (mon.0) 284 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:31.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:30.289124+0000 mon.a (mon.0) 283 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-22T19:26:31.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.252981+0000 mon.a (mon.0) 284 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:32.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-02-22T19:26:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:30.886029+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:31.248847+0000 mgr.y (mgr.14146) 105 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.559738+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.560678+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.565664+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.566576+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.567404+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:31.573147+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:30.886029+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:31.248847+0000 mgr.y (mgr.14146) 105 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.559738+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.560678+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.565664+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.566576+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.567404+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:31.573147+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:30.886029+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:31.248847+0000 mgr.y (mgr.14146) 105 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.559738+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.560678+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.565664+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.566576+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.567404+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:31.573147+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:31.561347+0000 mgr.y (mgr.14146) 106 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 8051M 2024-02-22T19:26:33.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:31.561347+0000 mgr.y (mgr.14146) 106 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 8051M 2024-02-22T19:26:33.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:31.561347+0000 mgr.y (mgr.14146) 106 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 8051M 2024-02-22T19:26:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:32.886468+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:34.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:32.886468+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:34.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:32.886468+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:34.348718+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:26:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:34.350550+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:26:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:34.351095+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:35.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:34.348718+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:26:35.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:34.350550+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:26:35.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:34.351095+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:35.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:34.348718+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:26:35.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:34.350550+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:26:35.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:34.351095+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:34.346736+0000 mgr.y (mgr.14146) 108 : audit [DBG] from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:26:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:34.886887+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:36.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:34.346736+0000 mgr.y (mgr.14146) 108 : audit [DBG] from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:26:36.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:34.886887+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:34.346736+0000 mgr.y (mgr.14146) 108 : audit [DBG] from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:26:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:34.886887+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:36.742587+0000 mon.c (mon.2) 5 : audit [INF] from='client.? 172.21.15.53:0/473742184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]: dispatch 2024-02-22T19:26:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:36.742913+0000 mon.a (mon.0) 294 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]: dispatch 2024-02-22T19:26:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:36.746818+0000 mon.a (mon.0) 295 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]': finished 2024-02-22T19:26:37.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:36.746890+0000 mon.a (mon.0) 296 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-22T19:26:37.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:36.747001+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:37.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:36.887247+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:37.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:37.344112+0000 mon.c (mon.2) 6 : audit [DBG] from='client.? 172.21.15.53:0/2633690855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:26:38.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:36.742587+0000 mon.c (mon.2) 5 : audit [INF] from='client.? 172.21.15.53:0/473742184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:36.742913+0000 mon.a (mon.0) 294 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:36.746818+0000 mon.a (mon.0) 295 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]': finished 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:36.746890+0000 mon.a (mon.0) 296 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:36.747001+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:36.887247+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:37.344112+0000 mon.c (mon.2) 6 : audit [DBG] from='client.? 172.21.15.53:0/2633690855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:36.742587+0000 mon.c (mon.2) 5 : audit [INF] from='client.? 172.21.15.53:0/473742184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:36.742913+0000 mon.a (mon.0) 294 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:36.746818+0000 mon.a (mon.0) 295 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83"}]': finished 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:36.746890+0000 mon.a (mon.0) 296 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:36.747001+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:36.887247+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:38.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:37.344112+0000 mon.c (mon.2) 6 : audit [DBG] from='client.? 172.21.15.53:0/2633690855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:26:40.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:38.887709+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:40.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:38.887709+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:38.887709+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:42.153 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:40.888079+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:42.154 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:40.888079+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:40.888079+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:42.888362+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:43.223929+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:43.224583+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:43.225133+0000 mgr.y (mgr.14146) 114 : cephadm [INF] Deploying daemon osd.2 on smithi053 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:42.888362+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:43.223929+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:43.224583+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:43.225133+0000 mgr.y (mgr.14146) 114 : cephadm [INF] Deploying daemon osd.2 on smithi053 2024-02-22T19:26:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:42.888362+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:43.223929+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T19:26:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:43.224583+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:43.225133+0000 mgr.y (mgr.14146) 114 : cephadm [INF] Deploying daemon osd.2 on smithi053 2024-02-22T19:26:46.217 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:44.888607+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:46.218 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:44.888607+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:44.888607+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:48.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:46.888881+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:48.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:46.888881+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:46.888881+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:49.328 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:48.280265+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:49.328 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:48.326297+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:49.328 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:48.280265+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:49.328 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:48.326297+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:48.280265+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:48.326297+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:50.549 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:48.889152+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:50.549 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:48.889152+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:48.889152+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:51.655 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 2 on host 'smithi053' 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:50.879289+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:50.879979+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:50.880712+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:50.884585+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:50.889362+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.406766+0000 mon.c (mon.2) 7 : audit [INF] from='osd.2 v2:172.21.15.53:6809/936980935' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.407044+0000 mon.a (mon.0) 306 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.651202+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:52.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.653645+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:50.879289+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:50.879979+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:50.880712+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:50.884585+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:50.889362+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.406766+0000 mon.c (mon.2) 7 : audit [INF] from='osd.2 v2:172.21.15.53:6809/936980935' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.407044+0000 mon.a (mon.0) 306 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.651202+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.653645+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.330 DEBUG:teuthology.orchestra.run.smithi053:osd.2> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.2.service 2024-02-22T19:26:52.332 INFO:tasks.cephadm:Deploying osd.3 on smithi053 with /dev/vg_nvme/lv_1... 2024-02-22T19:26:52.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-22T19:26:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:50.879289+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:50.879979+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:50.880712+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:50.884585+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:50.889362+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:52.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.406766+0000 mon.c (mon.2) 7 : audit [INF] from='osd.2 v2:172.21.15.53:6809/936980935' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T19:26:52.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.407044+0000 mon.a (mon.0) 306 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T19:26:52.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.651202+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:26:52.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.653645+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:52.367 INFO:journalctl@ceph.osd.2.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.887686+0000 mon.a (mon.0) 309 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:51.887739+0000 mon.a (mon.0) 310 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.887851+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.888264+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 v2:172.21.15.53:6809/936980935' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:51.888566+0000 mon.a (mon.0) 312 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.887686+0000 mon.a (mon.0) 309 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:51.887739+0000 mon.a (mon.0) 310 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-22T19:26:53.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.887851+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:53.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.888264+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 v2:172.21.15.53:6809/936980935' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:53.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:51.888566+0000 mon.a (mon.0) 312 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:53.256 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-2[86396]: debug 2024-02-22T19:26:52.892+0000 7f352a241700 -1 osd.2 0 waiting for initial osdmap 2024-02-22T19:26:53.256 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:26:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-2[86396]: debug 2024-02-22T19:26:52.897+0000 7f35229b5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:26:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.887686+0000 mon.a (mon.0) 309 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T19:26:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:51.887739+0000 mon.a (mon.0) 310 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-22T19:26:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.887851+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.888264+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 v2:172.21.15.53:6809/936980935' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:51.888566+0000 mon.a (mon.0) 312 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:26:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:52.889214+0000 mon.a (mon.0) 313 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:52.889318+0000 mon.a (mon.0) 314 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-22T19:26:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:52.889597+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:52.889650+0000 mon.a (mon.0) 315 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:52.896651+0000 mon.a (mon.0) 316 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:54.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:52.889214+0000 mon.a (mon.0) 313 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:52.889318+0000 mon.a (mon.0) 314 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:52.889597+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:52.889650+0000 mon.a (mon.0) 315 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:52.896651+0000 mon.a (mon.0) 316 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:52.889214+0000 mon.a (mon.0) 313 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:52.889318+0000 mon.a (mon.0) 314 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:52.889597+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:52.889650+0000 mon.a (mon.0) 315 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:54.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:52.896651+0000 mon.a (mon.0) 316 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:52.399156+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:52.399201+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:53.893545+0000 mon.a (mon.0) 317 : cluster [INF] osd.2 v2:172.21.15.53:6809/936980935 boot 2024-02-22T19:26:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:53.893590+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-22T19:26:55.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:53.894196+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:55.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:53.894618+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T19:26:55.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:54.889829+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:55.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:54.894865+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-22T19:26:55.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:54.894923+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-22T19:26:55.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:54.895636+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:52.399156+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:52.399201+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:53.893545+0000 mon.a (mon.0) 317 : cluster [INF] osd.2 v2:172.21.15.53:6809/936980935 boot 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:53.893590+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:53.894196+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:53.894618+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:54.889829+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:54.894865+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:54.894923+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-22T19:26:55.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:54.895636+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:52.399156+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:52.399201+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:53.893545+0000 mon.a (mon.0) 317 : cluster [INF] osd.2 v2:172.21.15.53:6809/936980935 boot 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:53.893590+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:53.894196+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:53.894618+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:54.889829+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:54.894865+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:54.894923+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-22T19:26:55.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:54.895636+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-22T19:26:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-22T19:26:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-22T19:26:56.530 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-02-22T19:26:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-02-22T19:26:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00828249 s, 1.3 GB/s 2024-02-22T19:26:56.530 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 2024-02-22T19:26:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:55.897361+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:55.897502+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:56.258217+0000 mgr.y (mgr.14146) 121 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.263215+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.264167+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.264837+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.265564+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:26:56.266051+0000 mgr.y (mgr.14146) 122 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 5367M 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.269812+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.271624+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.272243+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:56.276246+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:55.897361+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-22T19:26:57.213 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:55.897502+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:56.258217+0000 mgr.y (mgr.14146) 121 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.263215+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.264167+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.264837+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.265564+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:26:56.266051+0000 mgr.y (mgr.14146) 122 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 5367M 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.269812+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.271624+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.272243+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:57.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:56.276246+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-02-22T19:26:57.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:55.897361+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-22T19:26:57.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:55.897502+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:56.258217+0000 mgr.y (mgr.14146) 121 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.263215+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.264167+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.264837+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.265564+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:26:56.266051+0000 mgr.y (mgr.14146) 122 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 5367M 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.269812+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.271624+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.272243+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:26:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:56.276246+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:26:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:56.890277+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v101: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:26:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:26:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:57.277855+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-22T19:26:58.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:56.890277+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v101: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:26:58.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:26:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:57.277855+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-22T19:26:58.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:56.890277+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v101: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:26:58.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:26:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:57.277855+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-22T19:27:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:26:58.890740+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:59.357980+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:59.359853+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:59.360385+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:26:58.890740+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:59.357980+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:59.359853+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:59.360385+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:26:58.890740+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:59.357980+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:59.359853+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:00.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:59.360385+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:01.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:26:59.356045+0000 mgr.y (mgr.14146) 125 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:01.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:26:59.356045+0000 mgr.y (mgr.14146) 125 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:01.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:26:59.356045+0000 mgr.y (mgr.14146) 125 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:00.891315+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:01.795923+0000 mon.a (mon.0) 338 : audit [INF] from='client.? 172.21.15.53:0/432237176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993"}]: dispatch 2024-02-22T19:27:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:01.800311+0000 mon.a (mon.0) 339 : audit [INF] from='client.? 172.21.15.53:0/432237176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993"}]': finished 2024-02-22T19:27:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:01.800379+0000 mon.a (mon.0) 340 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-22T19:27:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:01.800515+0000 mon.a (mon.0) 341 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:02.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:00.891315+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:02.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:01.795923+0000 mon.a (mon.0) 338 : audit [INF] from='client.? 172.21.15.53:0/432237176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993"}]: dispatch 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:01.800311+0000 mon.a (mon.0) 339 : audit [INF] from='client.? 172.21.15.53:0/432237176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993"}]': finished 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:01.800379+0000 mon.a (mon.0) 340 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:01.800515+0000 mon.a (mon.0) 341 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:00.891315+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:01.795923+0000 mon.a (mon.0) 338 : audit [INF] from='client.? 172.21.15.53:0/432237176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993"}]: dispatch 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:01.800311+0000 mon.a (mon.0) 339 : audit [INF] from='client.? 172.21.15.53:0/432237176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993"}]': finished 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:01.800379+0000 mon.a (mon.0) 340 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-22T19:27:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:01.800515+0000 mon.a (mon.0) 341 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:02.421210+0000 mon.a (mon.0) 342 : audit [DBG] from='client.? 172.21.15.53:0/1209657111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:03.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:02.421210+0000 mon.a (mon.0) 342 : audit [DBG] from='client.? 172.21.15.53:0/1209657111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:03.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:02.421210+0000 mon.a (mon.0) 342 : audit [DBG] from='client.? 172.21.15.53:0/1209657111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:02.891610+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v106: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:04.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:02.891610+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v106: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:04.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:02.891610+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v106: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:04.892028+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:06.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:04.892028+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:06.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:04.892028+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:06.892426+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:08.231245+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T19:27:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:08.231884+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:08.729 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:06.892426+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:08.730 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:08.231245+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T19:27:08.730 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:08.231884+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:08.730 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:06.892426+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:08.730 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:08.231245+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T19:27:08.730 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:08.231884+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:08.232477+0000 mgr.y (mgr.14146) 130 : cephadm [INF] Deploying daemon osd.3 on smithi053 2024-02-22T19:27:09.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:08.232477+0000 mgr.y (mgr.14146) 130 : cephadm [INF] Deploying daemon osd.3 on smithi053 2024-02-22T19:27:09.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:08.232477+0000 mgr.y (mgr.14146) 130 : cephadm [INF] Deploying daemon osd.3 on smithi053 2024-02-22T19:27:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:08.892741+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:10.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:08.892741+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:10.616 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:08.892741+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:12.474 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:10.893237+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:12.475 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:10.893237+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:10.893237+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:12.893556+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:14.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:13.394701+0000 mon.a (mon.0) 345 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:14.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:13.424578+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:12.893556+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:13.394701+0000 mon.a (mon.0) 345 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:13.424578+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:12.893556+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:13.394701+0000 mon.a (mon.0) 345 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:13.424578+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:14.893855+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:15.233658+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:15.233733+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:15.315844+0000 mon.a (mon.0) 347 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:15.316005+0000 mon.a (mon.0) 348 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.149048+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.149879+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.150594+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:16.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.155741+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:14.893855+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:15.233658+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:15.233733+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:15.315844+0000 mon.a (mon.0) 347 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:15.316005+0000 mon.a (mon.0) 348 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.149048+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.149879+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.150594+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:16.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.155741+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:14.893855+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:15.233658+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:15.233733+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:15.315844+0000 mon.a (mon.0) 347 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-22T19:27:16.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:15.316005+0000 mon.a (mon.0) 348 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:16.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.149048+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:16.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.149879+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:16.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.150594+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:16.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.155741+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:16.832 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 3 on host 'smithi053' 2024-02-22T19:27:17.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.496677+0000 mon.c (mon.2) 9 : audit [INF] from='osd.3 v2:172.21.15.53:6813/3878043918' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.496992+0000 mon.a (mon.0) 353 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.830960+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:16.872794+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.496677+0000 mon.c (mon.2) 9 : audit [INF] from='osd.3 v2:172.21.15.53:6813/3878043918' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.496992+0000 mon.a (mon.0) 353 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.830960+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:17.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:16.872794+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:17.523 DEBUG:teuthology.orchestra.run.smithi053:osd.3> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.3.service 2024-02-22T19:27:17.526 INFO:tasks.cephadm:Deploying osd.4 on smithi064 with /dev/vg_nvme/lv_4... 2024-02-22T19:27:17.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-22T19:27:17.563 INFO:journalctl@ceph.osd.3.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:27:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.496677+0000 mon.c (mon.2) 9 : audit [INF] from='osd.3 v2:172.21.15.53:6813/3878043918' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T19:27:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.496992+0000 mon.a (mon.0) 353 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T19:27:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.830960+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:17.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:16.872794+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:16.894200+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:17.321980+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T19:27:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:17.322052+0000 mon.a (mon.0) 357 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-22T19:27:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:17.322215+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:17.322600+0000 mon.c (mon.2) 10 : audit [INF] from='osd.3 v2:172.21.15.53:6813/3878043918' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:27:18.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:17.322864+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:27:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:16.894200+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:17.321980+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:17.322052+0000 mon.a (mon.0) 357 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:17.322215+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:17.322600+0000 mon.c (mon.2) 10 : audit [INF] from='osd.3 v2:172.21.15.53:6813/3878043918' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:17.322864+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:27:18.755 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:27:18.327+0000 7f099f43b700 -1 osd.3 0 waiting for initial osdmap 2024-02-22T19:27:18.755 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:27:18.331+0000 7f0997baf700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:16.894200+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:17.321980+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T19:27:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:17.322052+0000 mon.a (mon.0) 357 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-22T19:27:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:17.322215+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:17.322600+0000 mon.c (mon.2) 10 : audit [INF] from='osd.3 v2:172.21.15.53:6813/3878043918' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:27:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:17.322864+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-02-22T19:27:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:18.323793+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:27:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:18.323865+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-22T19:27:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:18.324210+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:18.328219+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:18.323793+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:18.323865+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:18.324210+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:18.328219+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:18.323793+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:18.323865+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:18.324210+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:19.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:18.328219+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:19.767 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-22T19:27:19.767 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-22T19:27:19.767 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-22T19:27:19.768 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-22T19:27:19.768 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110946 s, 945 MB/s 2024-02-22T19:27:19.768 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-22T19:27:20.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-02-22T19:27:20.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:17.447982+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:20.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:17.448063+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:20.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:18.894608+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:20.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:19.329268+0000 mon.a (mon.0) 364 : cluster [INF] osd.3 v2:172.21.15.53:6813/3878043918 boot 2024-02-22T19:27:20.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:19.329321+0000 mon.a (mon.0) 365 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-22T19:27:20.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:19.329685+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:20.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:17.447982+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:20.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:17.448063+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:20.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:18.894608+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:20.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:19.329268+0000 mon.a (mon.0) 364 : cluster [INF] osd.3 v2:172.21.15.53:6813/3878043918 boot 2024-02-22T19:27:20.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:19.329321+0000 mon.a (mon.0) 365 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-22T19:27:20.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:19.329685+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:20.405 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:17.447982+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:20.405 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:17.448063+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:20.405 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:18.894608+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T19:27:20.405 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:19.329268+0000 mon.a (mon.0) 364 : cluster [INF] osd.3 v2:172.21.15.53:6813/3878043918 boot 2024-02-22T19:27:20.405 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:19.329321+0000 mon.a (mon.0) 365 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-22T19:27:20.405 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:19.329685+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:27:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:20.334659+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.652861+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.653714+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.654280+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.654775+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.655271+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.658714+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.660145+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.660686+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:21.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:20.664560+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:20.334659+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.652861+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.653714+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.654280+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.654775+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.655271+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.658714+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.660145+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.660686+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:20.664560+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:20.334659+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-22T19:27:21.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.652861+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.653714+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.654280+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.654775+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.655271+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.658714+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.660145+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.660686+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:21.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:20.664560+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:20.648017+0000 mgr.y (mgr.14146) 137 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:27:22.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:20.655666+0000 mgr.y (mgr.14146) 138 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 4025M 2024-02-22T19:27:22.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:20.895021+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:22.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:20.648017+0000 mgr.y (mgr.14146) 137 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:27:22.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:20.655666+0000 mgr.y (mgr.14146) 138 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 4025M 2024-02-22T19:27:22.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:20.895021+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:22.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:20.648017+0000 mgr.y (mgr.14146) 137 : cephadm [INF] Detected new or changed devices on smithi053 2024-02-22T19:27:22.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:20.655666+0000 mgr.y (mgr.14146) 138 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 4025M 2024-02-22T19:27:22.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:20.895021+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:22.375233+0000 mon.a (mon.0) 377 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:23.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:22.377650+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:23.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:22.378424+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:23.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:22.375233+0000 mon.a (mon.0) 377 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:23.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:22.377650+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:23.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:22.378424+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:23.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:22.375233+0000 mon.a (mon.0) 377 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:23.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:22.377650+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:23.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:22.378424+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:22.372710+0000 mgr.y (mgr.14146) 140 : audit [DBG] from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:22.895424+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:24.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:22.372710+0000 mgr.y (mgr.14146) 140 : audit [DBG] from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:24.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:22.895424+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:22.372710+0000 mgr.y (mgr.14146) 140 : audit [DBG] from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:22.895424+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:24.678796+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.64:0/1860210958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]: dispatch 2024-02-22T19:27:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:24.679452+0000 mon.a (mon.0) 380 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]: dispatch 2024-02-22T19:27:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:24.683996+0000 mon.a (mon.0) 381 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]': finished 2024-02-22T19:27:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:24.684110+0000 mon.a (mon.0) 382 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-22T19:27:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:24.684305+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:24.895778+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:25.242936+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.64:0/2531191539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:24.678796+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.64:0/1860210958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]: dispatch 2024-02-22T19:27:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:24.679452+0000 mon.a (mon.0) 380 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]: dispatch 2024-02-22T19:27:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:24.683996+0000 mon.a (mon.0) 381 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]': finished 2024-02-22T19:27:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:24.684110+0000 mon.a (mon.0) 382 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-22T19:27:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:24.684305+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:24.895778+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:25.242936+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.64:0/2531191539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:24.678796+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.64:0/1860210958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]: dispatch 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:24.679452+0000 mon.a (mon.0) 380 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]: dispatch 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:24.683996+0000 mon.a (mon.0) 381 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211"}]': finished 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:24.684110+0000 mon.a (mon.0) 382 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:24.684305+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:24.895778+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:25.242936+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.64:0/2531191539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:28.040 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:26.896148+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:26.896148+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:26.896148+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:28.896506+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:30.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:28.896506+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:30.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:28.896506+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:31.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:30.598186+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T19:27:31.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:30.599060+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:30.598186+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T19:27:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:30.599060+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:31.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:30.598186+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T19:27:31.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:30.599060+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:32.179 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:30.599838+0000 mgr.y (mgr.14146) 145 : cephadm [INF] Deploying daemon osd.4 on smithi064 2024-02-22T19:27:32.179 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:30.896923+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:30.599838+0000 mgr.y (mgr.14146) 145 : cephadm [INF] Deploying daemon osd.4 on smithi064 2024-02-22T19:27:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:30.896923+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:30.599838+0000 mgr.y (mgr.14146) 145 : cephadm [INF] Deploying daemon osd.4 on smithi064 2024-02-22T19:27:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:30.896923+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:33.149 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:31.990848+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:33.149 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:31.990933+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:31.990848+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:31.990933+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:33.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:31.990848+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:33.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:31.990933+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:32.897268+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:32.966215+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-22T19:27:34.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:32.966385+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:32.897268+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:32.966215+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-22T19:27:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:32.966385+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:34.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:32.897268+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:34.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:32.966215+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-22T19:27:34.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:32.966385+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:34.897568+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:35.449943+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:35.467703+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:34.897568+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:35.449943+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:35.467703+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:34.897568+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:35.449943+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:35.467703+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:36.897911+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:38.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:36.897911+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:38.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:36.897911+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:38.516 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 4 on host 'smithi064' 2024-02-22T19:27:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:37.680550+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:37.998226+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:37.999226+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:38.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:38.004500+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:38.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:38.514722+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:38.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:38.522096+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:38.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:38.562845+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v2:172.21.15.64:6800/2373040976' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T19:27:38.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:38.563391+0000 mon.a (mon.0) 396 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:37.680550+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:37.998226+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:37.999226+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:38.004500+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:38.514722+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:38.522096+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:38.562845+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v2:172.21.15.64:6800/2373040976' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:38.563391+0000 mon.a (mon.0) 396 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:37.680550+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:37.998226+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:37.999226+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:38.004500+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:38.514722+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:38.522096+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:38.562845+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v2:172.21.15.64:6800/2373040976' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T19:27:39.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:38.563391+0000 mon.a (mon.0) 396 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T19:27:39.259 DEBUG:teuthology.orchestra.run.smithi064:osd.4> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.4.service 2024-02-22T19:27:39.302 INFO:tasks.cephadm:Deploying osd.5 on smithi064 with /dev/vg_nvme/lv_3... 2024-02-22T19:27:39.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-22T19:27:39.333 INFO:journalctl@ceph.osd.4.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:27:40.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:38.898382+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:40.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:39.516628+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T19:27:40.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:39.516781+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:39.516954+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:39.517251+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v2:172.21.15.64:6800/2373040976' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:39.517642+0000 mon.a (mon.0) 400 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:38.898382+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:39.516628+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:39.516781+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:39.516954+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:39.517251+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v2:172.21.15.64:6800/2373040976' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:27:40.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:39.517642+0000 mon.a (mon.0) 400 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:27:40.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:38.898382+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:40.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:39.516628+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T19:27:40.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:39.516781+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-22T19:27:40.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:39.516954+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:40.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:39.517251+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v2:172.21.15.64:6800/2373040976' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:27:40.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:39.517642+0000 mon.a (mon.0) 400 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:27:40.740 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:27:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:27:40.521+0000 7f8bba0ac700 -1 osd.4 0 waiting for initial osdmap 2024-02-22T19:27:40.740 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:27:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:27:40.526+0000 7f8bb3822700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:27:41.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:40.518534+0000 mon.a (mon.0) 401 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:27:41.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:40.518689+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-22T19:27:41.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:40.518898+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:41.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:40.522752+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:41.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:40.898814+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:42.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:40.518534+0000 mon.a (mon.0) 401 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:27:42.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:40.518689+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:40.518898+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:40.522752+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:40.898814+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:40.518534+0000 mon.a (mon.0) 401 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:40.518689+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:40.518898+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:40.522752+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:40.898814+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T19:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:39.568722+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:39.568770+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:41.523987+0000 mon.a (mon.0) 405 : cluster [INF] osd.4 v2:172.21.15.64:6800/2373040976 boot 2024-02-22T19:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:41.524051+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-22T19:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:41.524449+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:41.569104+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:42.499442+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:42.500404+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:42.504819+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:42.505816+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:42.506660+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:42.511182+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:39.568722+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:39.568770+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:41.523987+0000 mon.a (mon.0) 405 : cluster [INF] osd.4 v2:172.21.15.64:6800/2373040976 boot 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:41.524051+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:41.524449+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:41.569104+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:42.499442+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:42.500404+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:42.504819+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:42.505816+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:42.506660+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:42.511182+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:39.568722+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:39.568770+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:41.523987+0000 mon.a (mon.0) 405 : cluster [INF] osd.4 v2:172.21.15.64:6800/2373040976 boot 2024-02-22T19:27:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:41.524051+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:41.524449+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:41.569104+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:42.499442+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:42.500404+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:42.504819+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:42.505816+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:42.506660+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:27:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:42.511182+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:27:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-22T19:27:43.043 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-22T19:27:43.043 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-22T19:27:43.043 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-22T19:27:43.043 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110763 s, 947 MB/s 2024-02-22T19:27:43.043 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-22T19:27:43.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-02-22T19:27:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:42.494763+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:27:43.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:42.501044+0000 mgr.y (mgr.14146) 153 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 17122M 2024-02-22T19:27:43.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:42.569515+0000 mon.a (mon.0) 415 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-22T19:27:43.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:42.899310+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:44.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:42.494763+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:27:44.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:42.501044+0000 mgr.y (mgr.14146) 153 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 17122M 2024-02-22T19:27:44.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:42.569515+0000 mon.a (mon.0) 415 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-22T19:27:44.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:42.899310+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:42.494763+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:27:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:42.501044+0000 mgr.y (mgr.14146) 153 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 17122M 2024-02-22T19:27:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:42.569515+0000 mon.a (mon.0) 415 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-22T19:27:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:42.899310+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:46.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:44.899781+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:46.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:45.494278+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:46.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:45.496743+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:45.497557+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:44.899781+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:45.494278+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:45.496743+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:45.497557+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:44.899781+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:45.494278+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:27:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:45.496743+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:27:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:45.497557+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:47.063 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:45.491965+0000 mgr.y (mgr.14146) 156 : audit [DBG] from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:47.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:45.491965+0000 mgr.y (mgr.14146) 156 : audit [DBG] from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:47.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:45.491965+0000 mgr.y (mgr.14146) 156 : audit [DBG] from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:27:48.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:46.900276+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:47.832746+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.64:0/261797543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]: dispatch 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:47.833438+0000 mon.a (mon.0) 419 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]: dispatch 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:47.838207+0000 mon.a (mon.0) 420 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]': finished 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:47.838263+0000 mon.a (mon.0) 421 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:47.838407+0000 mon.a (mon.0) 422 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:46.900276+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:47.832746+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.64:0/261797543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]: dispatch 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:47.833438+0000 mon.a (mon.0) 419 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]: dispatch 2024-02-22T19:27:48.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:47.838207+0000 mon.a (mon.0) 420 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]': finished 2024-02-22T19:27:48.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:47.838263+0000 mon.a (mon.0) 421 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-22T19:27:48.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:47.838407+0000 mon.a (mon.0) 422 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:27:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:46.900276+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:47.832746+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.64:0/261797543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]: dispatch 2024-02-22T19:27:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:47.833438+0000 mon.a (mon.0) 419 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]: dispatch 2024-02-22T19:27:48.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:47.838207+0000 mon.a (mon.0) 420 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07"}]': finished 2024-02-22T19:27:48.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:47.838263+0000 mon.a (mon.0) 421 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-22T19:27:48.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:47.838407+0000 mon.a (mon.0) 422 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:27:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:48.469981+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.64:0/2857400751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:48.469981+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.64:0/2857400751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:48.469981+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.64:0/2857400751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:27:50.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:48.900713+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:50.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:48.900713+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:48.900713+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:52.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:50.901156+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:52.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:50.901156+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:52.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:50.901156+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:54.226 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:52.901441+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:54.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:52.901441+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:52.901441+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:55.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:54.245974+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T19:27:55.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:54.246885+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:55.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:27:54.247675+0000 mgr.y (mgr.14146) 161 : cephadm [INF] Deploying daemon osd.5 on smithi064 2024-02-22T19:27:55.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:54.245974+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T19:27:55.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:54.246885+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:55.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:27:54.247675+0000 mgr.y (mgr.14146) 161 : cephadm [INF] Deploying daemon osd.5 on smithi064 2024-02-22T19:27:55.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:54.245974+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T19:27:55.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:54.246885+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:27:55.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:27:54.247675+0000 mgr.y (mgr.14146) 161 : cephadm [INF] Deploying daemon osd.5 on smithi064 2024-02-22T19:27:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:54.901862+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:54.901862+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:56.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:54.901862+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:57.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:27:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:56.902241+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:27:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:56.902241+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:27:58.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:27:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:56.902241+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:27:58.902575+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:59.166181+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:27:59.172445+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:27:58.902575+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:59.166181+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:00.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:27:59.172445+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:00.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:27:58.902575+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:00.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:59.166181+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:00.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:27:59.172445+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:00.139883+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:00.140003+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:00.171817+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-22T19:28:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:00.171996+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:01.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:00.139883+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:01.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:00.140003+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:01.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:00.171817+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-22T19:28:01.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:00.171996+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:01.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:00.139883+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:01.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:00.140003+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:01.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:00.171817+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-22T19:28:01.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:00.171996+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:02.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:00.902941+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:02.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:01.679489+0000 mon.a (mon.0) 429 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:02.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:01.680398+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:02.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:01.681284+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:02.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:01.685915+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:02.374 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 5 on host 'smithi064' 2024-02-22T19:28:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:00.902941+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:01.679489+0000 mon.a (mon.0) 429 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:01.680398+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:01.681284+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:01.685915+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:00.902941+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:01.679489+0000 mon.a (mon.0) 429 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:01.680398+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:01.681284+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:01.685915+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:02.986 DEBUG:teuthology.orchestra.run.smithi064:osd.5> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.5.service 2024-02-22T19:28:02.988 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2024-02-22T19:28:02.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-22T19:28:03.020 INFO:journalctl@ceph.osd.5.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:28:03.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:02.266286+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 v2:172.21.15.64:6804/2413553331' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T19:28:03.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:02.266799+0000 mon.a (mon.0) 433 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T19:28:03.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:02.372944+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:03.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:02.403624+0000 mon.a (mon.0) 435 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:02.266286+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 v2:172.21.15.64:6804/2413553331' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T19:28:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:02.266799+0000 mon.a (mon.0) 433 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T19:28:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:02.372944+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:02.403624+0000 mon.a (mon.0) 435 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:02.266286+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 v2:172.21.15.64:6804/2413553331' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T19:28:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:02.266799+0000 mon.a (mon.0) 433 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T19:28:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:02.372944+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:02.403624+0000 mon.a (mon.0) 435 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:04.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:02.903382+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:03.183316+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:03.183387+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:03.183556+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:03.183838+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 v2:172.21.15.64:6804/2413553331' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:03.184302+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:02.903382+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:03.183316+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:03.183387+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:03.183556+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:03.183838+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 v2:172.21.15.64:6804/2413553331' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:03.184302+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:04.596 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:28:04.190+0000 7ff71e169700 -1 osd.5 0 waiting for initial osdmap 2024-02-22T19:28:04.596 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:28:04.197+0000 7ff7188e1700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:28:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:02.903382+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:03.183316+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T19:28:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:03.183387+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-22T19:28:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:03.183556+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:03.183838+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 v2:172.21.15.64:6804/2413553331' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:03.184302+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:03.577573+0000 osd.1 (osd.1) 5 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:28:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:03.578536+0000 osd.1 (osd.1) 6 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:28:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:04.185929+0000 mon.a (mon.0) 440 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:04.186059+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-22T19:28:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:04.186533+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:04.191740+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:03.577573+0000 osd.1 (osd.1) 5 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:28:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:03.578536+0000 osd.1 (osd.1) 6 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:28:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:04.185929+0000 mon.a (mon.0) 440 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:04.186059+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-22T19:28:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:04.186533+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:04.191740+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:03.577573+0000 osd.1 (osd.1) 5 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:28:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:03.578536+0000 osd.1 (osd.1) 6 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:28:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:04.185929+0000 mon.a (mon.0) 440 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:04.186059+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-22T19:28:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:04.186533+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:04.191740+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:06.294 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-22T19:28:06.295 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-22T19:28:06.295 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-22T19:28:06.295 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-22T19:28:06.295 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110277 s, 951 MB/s 2024-02-22T19:28:06.295 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-22T19:28:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:03.246226+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:03.246275+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:04.903833+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:05.187832+0000 mon.a (mon.0) 444 : cluster [INF] osd.5 v2:172.21.15.64:6804/2413553331 boot 2024-02-22T19:28:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:05.187909+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-22T19:28:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:05.188130+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:06.188642+0000 mon.a (mon.0) 447 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:03.246226+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:03.246275+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:04.903833+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:05.187832+0000 mon.a (mon.0) 444 : cluster [INF] osd.5 v2:172.21.15.64:6804/2413553331 boot 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:05.187909+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:05.188130+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:06.188642+0000 mon.a (mon.0) 447 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-22T19:28:06.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:03.246226+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:06.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:03.246275+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:06.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:04.903833+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T19:28:06.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:05.187832+0000 mon.a (mon.0) 444 : cluster [INF] osd.5 v2:172.21.15.64:6804/2413553331 boot 2024-02-22T19:28:06.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:05.187909+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-22T19:28:06.567 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:05.188130+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:28:06.567 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:06.188642+0000 mon.a (mon.0) 447 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-22T19:28:07.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-02-22T19:28:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:06.804633+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.809782+0000 mon.a (mon.0) 448 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.810862+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.811694+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:06.812317+0000 mgr.y (mgr.14146) 169 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 8561M 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.816443+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.817511+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.818339+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:06.822935+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:06.904211+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:08.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:07.190140+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-22T19:28:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:06.804633+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.809782+0000 mon.a (mon.0) 448 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.810862+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.811694+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:06.812317+0000 mgr.y (mgr.14146) 169 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 8561M 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.816443+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.817511+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.818339+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:06.822935+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:06.904211+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:07.190140+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:06.804633+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.809782+0000 mon.a (mon.0) 448 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.810862+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.811694+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:06.812317+0000 mgr.y (mgr.14146) 169 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 8561M 2024-02-22T19:28:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.816443+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.817511+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:08.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.818339+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:08.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:06.822935+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:08.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:06.904211+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:08.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:07.190140+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-22T19:28:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:08.904558+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:09.186332+0000 mgr.y (mgr.14146) 172 : audit [DBG] from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:09.188760+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:28:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:09.191178+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:28:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:09.191963+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:10.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:08.904558+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:10.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:09.186332+0000 mgr.y (mgr.14146) 172 : audit [DBG] from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:10.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:09.188760+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:28:10.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:09.191178+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:28:10.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:09.191963+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:08.904558+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:09.186332+0000 mgr.y (mgr.14146) 172 : audit [DBG] from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:09.188760+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:28:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:09.191178+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:28:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:09.191963+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:10.904932+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:11.547843+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.64:0/2663992298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]: dispatch 2024-02-22T19:28:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:11.548508+0000 mon.a (mon.0) 459 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]: dispatch 2024-02-22T19:28:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:11.553187+0000 mon.a (mon.0) 460 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]': finished 2024-02-22T19:28:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:11.553264+0000 mon.a (mon.0) 461 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-22T19:28:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:11.553406+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:12.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:10.904932+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:12.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:11.547843+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.64:0/2663992298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]: dispatch 2024-02-22T19:28:12.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:11.548508+0000 mon.a (mon.0) 459 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]: dispatch 2024-02-22T19:28:12.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:11.553187+0000 mon.a (mon.0) 460 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]': finished 2024-02-22T19:28:12.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:11.553264+0000 mon.a (mon.0) 461 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-22T19:28:12.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:11.553406+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:12.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:10.904932+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:12.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:11.547843+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.64:0/2663992298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]: dispatch 2024-02-22T19:28:12.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:11.548508+0000 mon.a (mon.0) 459 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]: dispatch 2024-02-22T19:28:12.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:11.553187+0000 mon.a (mon.0) 460 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e"}]': finished 2024-02-22T19:28:12.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:11.553264+0000 mon.a (mon.0) 461 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-22T19:28:12.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:11.553406+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:13.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:12.144992+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.64:0/2356671688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:28:13.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:12.144992+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.64:0/2356671688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:28:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:12.144992+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.64:0/2356671688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:28:14.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:12.905246+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:14.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:12.905246+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:12.905246+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:16.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:14.905572+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:16.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:14.905572+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:16.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:14.905572+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:17.612 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:16.906028+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:18.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:16.906028+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:18.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:16.906028+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:17.632833+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-22T19:28:18.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:17.633658+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:18.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:17.634359+0000 mgr.y (mgr.14146) 177 : cephadm [INF] Deploying daemon osd.6 on smithi064 2024-02-22T19:28:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:17.632833+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-22T19:28:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:17.633658+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:17.634359+0000 mgr.y (mgr.14146) 177 : cephadm [INF] Deploying daemon osd.6 on smithi064 2024-02-22T19:28:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:17.632833+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-22T19:28:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:17.633658+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:17.634359+0000 mgr.y (mgr.14146) 177 : cephadm [INF] Deploying daemon osd.6 on smithi064 2024-02-22T19:28:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:18.906295+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:20.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:18.906295+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:18.906295+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:22.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:20.906773+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:22.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:20.906773+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:20.906773+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:22.901050+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:22.907277+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:22.932349+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:24.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:22.901050+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:24.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:22.907277+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:24.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:22.932349+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:24.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:22.901050+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:24.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:22.907277+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:24.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:22.932349+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:26.092 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:24.907685+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:26.115 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2024-02-22T19:28:26.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:24.907685+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:24.907685+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:26.746 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.6.service 2024-02-22T19:28:26.748 INFO:tasks.cephadm:Deploying osd.7 on smithi064 with /dev/vg_nvme/lv_1... 2024-02-22T19:28:26.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-22T19:28:26.784 INFO:journalctl@ceph.osd.6.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:28:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.001835+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 v2:172.21.15.64:6808/3043274732' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-22T19:28:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.002294+0000 mon.a (mon.0) 467 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-22T19:28:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.114559+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.806750+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:27.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.001835+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 v2:172.21.15.64:6808/3043274732' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-22T19:28:27.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.002294+0000 mon.a (mon.0) 467 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-22T19:28:27.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.114559+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:27.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.806750+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.001835+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 v2:172.21.15.64:6808/3043274732' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-22T19:28:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.002294+0000 mon.a (mon.0) 467 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-22T19:28:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.114559+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.806750+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:28.251 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:28:27.977+0000 7f6096876700 -1 osd.6 0 waiting for initial osdmap 2024-02-22T19:28:28.251 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:28:27.984+0000 7f6090fee700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:28:28.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:26.908184+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:28.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.968762+0000 mon.a (mon.0) 470 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-22T19:28:28.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:26.968865+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-22T19:28:28.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.969002+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:28.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.972481+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 v2:172.21.15.64:6808/3043274732' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:28.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:26.973309+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:26.908184+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.968762+0000 mon.a (mon.0) 470 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-22T19:28:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:26.968865+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-22T19:28:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.969002+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.972481+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 v2:172.21.15.64:6808/3043274732' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:26.973309+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:26.908184+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.968762+0000 mon.a (mon.0) 470 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:26.968865+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.969002+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.972481+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 v2:172.21.15.64:6808/3043274732' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:26.973309+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:29.188 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:27.970594+0000 mon.a (mon.0) 474 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:29.188 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:27.970706+0000 mon.a (mon.0) 475 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-22T19:28:29.188 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:27.971138+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:27.978296+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:28.972848+0000 mon.a (mon.0) 478 : cluster [INF] osd.6 v2:172.21.15.64:6808/3043274732 boot 2024-02-22T19:28:29.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:28.972900+0000 mon.a (mon.0) 479 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-02-22T19:28:29.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:28.973145+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:27.970594+0000 mon.a (mon.0) 474 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:29.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:27.970706+0000 mon.a (mon.0) 475 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-22T19:28:29.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:27.971138+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:27.978296+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:28.972848+0000 mon.a (mon.0) 478 : cluster [INF] osd.6 v2:172.21.15.64:6808/3043274732 boot 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:28.972900+0000 mon.a (mon.0) 479 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:28.973145+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:27.970594+0000 mon.a (mon.0) 474 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:27.970706+0000 mon.a (mon.0) 475 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:27.971138+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:27.978296+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:28.972848+0000 mon.a (mon.0) 478 : cluster [INF] osd.6 v2:172.21.15.64:6808/3043274732 boot 2024-02-22T19:28:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:28.972900+0000 mon.a (mon.0) 479 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-02-22T19:28:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:28.973145+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:28:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:27.035215+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:27.035269+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:28.908570+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:29.974242+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e46: 7 total, 7 up, 7 in 2024-02-22T19:28:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:27.035215+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:27.035269+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:28.908570+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:29.974242+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e46: 7 total, 7 up, 7 in 2024-02-22T19:28:30.336 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-22T19:28:30.337 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-22T19:28:30.337 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-22T19:28:30.337 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-22T19:28:30.337 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118542 s, 885 MB/s 2024-02-22T19:28:30.337 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-22T19:28:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:27.035215+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:27.035269+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:28.908570+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T19:28:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:29.974242+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e46: 7 total, 7 up, 7 in 2024-02-22T19:28:31.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-02-22T19:28:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:30.908933+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:30.974815+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-22T19:28:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.804209+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.805216+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.806102+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.806906+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.812640+0000 mon.a (mon.0) 487 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.813599+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.814441+0000 mon.a (mon.0) 489 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.818780+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.821108+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.822548+0000 mon.a (mon.0) 492 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.823298+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:31.827646+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:30.908933+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:32.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:30.974815+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.804209+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.805216+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.806102+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.806906+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.812640+0000 mon.a (mon.0) 487 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.813599+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.814441+0000 mon.a (mon.0) 489 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.818780+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.821108+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.822548+0000 mon.a (mon.0) 492 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.823298+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:31.827646+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:30.908933+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:30.974815+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-22T19:28:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.804209+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.805216+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.806102+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.806906+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.812640+0000 mon.a (mon.0) 487 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.813599+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.814441+0000 mon.a (mon.0) 489 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.818780+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.821108+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.822548+0000 mon.a (mon.0) 492 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.823298+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:31.827646+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:33.069 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:31.797826+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:33.070 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:31.807547+0000 mgr.y (mgr.14146) 186 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 5707M 2024-02-22T19:28:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:31.797826+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:31.807547+0000 mgr.y (mgr.14146) 186 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 5707M 2024-02-22T19:28:33.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:31.797826+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:33.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:31.807547+0000 mgr.y (mgr.14146) 186 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 5707M 2024-02-22T19:28:34.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:32.909389+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:34.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:33.741772+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:28:34.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:33.744239+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:28:34.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:33.745068+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:32.909389+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:33.741772+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:28:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:33.744239+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:28:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:33.745068+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:34.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:32.909389+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:34.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:33.741772+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T19:28:34.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:33.744239+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T19:28:34.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:33.745068+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:35.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:33.739357+0000 mgr.y (mgr.14146) 188 : audit [DBG] from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:33.739357+0000 mgr.y (mgr.14146) 188 : audit [DBG] from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:33.739357+0000 mgr.y (mgr.14146) 188 : audit [DBG] from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:34.909751+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:34.909751+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:34.909751+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:36.234606+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.64:0/493207369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]: dispatch 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:36.235322+0000 mon.a (mon.0) 498 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]: dispatch 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:36.239989+0000 mon.a (mon.0) 499 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]': finished 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:36.240067+0000 mon.a (mon.0) 500 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:36.240225+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:36.857550+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.64:0/2760668590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:36.234606+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.64:0/493207369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]: dispatch 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:36.235322+0000 mon.a (mon.0) 498 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]: dispatch 2024-02-22T19:28:37.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:36.239989+0000 mon.a (mon.0) 499 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]': finished 2024-02-22T19:28:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:36.240067+0000 mon.a (mon.0) 500 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-02-22T19:28:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:36.240225+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:36.857550+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.64:0/2760668590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:28:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:36.234606+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.64:0/493207369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]: dispatch 2024-02-22T19:28:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:36.235322+0000 mon.a (mon.0) 498 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]: dispatch 2024-02-22T19:28:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:36.239989+0000 mon.a (mon.0) 499 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a"}]': finished 2024-02-22T19:28:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:36.240067+0000 mon.a (mon.0) 500 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-02-22T19:28:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:36.240225+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:36.857550+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.64:0/2760668590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T19:28:38.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:36.480497+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:38.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:36.480552+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:38.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:36.910280+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:38.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:37.241389+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e49: 8 total, 7 up, 8 in 2024-02-22T19:28:38.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:37.241533+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:36.480497+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:38.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:36.480552+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:38.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:36.910280+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:38.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:37.241389+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e49: 8 total, 7 up, 8 in 2024-02-22T19:28:38.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:37.241533+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:36.480497+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:36.480552+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:36.910280+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:38.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:37.241389+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e49: 8 total, 7 up, 8 in 2024-02-22T19:28:38.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:37.241533+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:40.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:38.910665+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:38.910665+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:40.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:38.910665+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:42.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:40.911135+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:40.911135+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:40.911135+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:43.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:42.454596+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-22T19:28:43.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:42.456014+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:43.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:42.454596+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-22T19:28:43.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:42.456014+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:42.454596+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-22T19:28:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:42.456014+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:44.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:42.456784+0000 mgr.y (mgr.14146) 193 : cephadm [INF] Deploying daemon osd.7 on smithi064 2024-02-22T19:28:44.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:42.911644+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:44.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:42.456784+0000 mgr.y (mgr.14146) 193 : cephadm [INF] Deploying daemon osd.7 on smithi064 2024-02-22T19:28:44.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:42.911644+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:44.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:42.456784+0000 mgr.y (mgr.14146) 193 : cephadm [INF] Deploying daemon osd.7 on smithi064 2024-02-22T19:28:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:42.911644+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:46.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:44.912183+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:44.912183+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:44.912183+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:47.541 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:45.593556+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:47.541 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:45.593647+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:47.541 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:45.908309+0000 osd.1 (osd.1) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:47.541 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:45.908355+0000 osd.1 (osd.1) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:47.541 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:46.260043+0000 mon.a (mon.0) 506 : cluster [DBG] osdmap e50: 8 total, 7 up, 8 in 2024-02-22T19:28:47.541 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:46.260251+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:45.593556+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:45.593647+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:45.908309+0000 osd.1 (osd.1) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:45.908355+0000 osd.1 (osd.1) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:46.260043+0000 mon.a (mon.0) 506 : cluster [DBG] osdmap e50: 8 total, 7 up, 8 in 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:46.260251+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:45.593556+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:45.593647+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:45.908309+0000 osd.1 (osd.1) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:45.908355+0000 osd.1 (osd.1) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:46.260043+0000 mon.a (mon.0) 506 : cluster [DBG] osdmap e50: 8 total, 7 up, 8 in 2024-02-22T19:28:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:46.260251+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:48.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:46.912493+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:48.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:47.584809+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:48.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:47.588789+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:48.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:46.912493+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:48.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:47.584809+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:48.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:47.588789+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:46.912493+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:47.584809+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:48.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:47.588789+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:48.912958+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:50.649 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 7 on host 'smithi064' 2024-02-22T19:28:50.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:48.912958+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:50.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:48.912958+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:51.314 DEBUG:teuthology.orchestra.run.smithi064:osd.7> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.7.service 2024-02-22T19:28:51.316 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-22T19:28:51.317 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd stat -f json 2024-02-22T19:28:51.350 INFO:journalctl@ceph.osd.7.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:28:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:50.493952+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 v2:172.21.15.64:6812/2605082260' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-22T19:28:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:50.494534+0000 mon.a (mon.0) 510 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-22T19:28:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:50.648403+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:51.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:50.493952+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 v2:172.21.15.64:6812/2605082260' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-22T19:28:51.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:50.494534+0000 mon.a (mon.0) 510 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-22T19:28:51.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:50.648403+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:50.493952+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 v2:172.21.15.64:6812/2605082260' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-22T19:28:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:50.494534+0000 mon.a (mon.0) 510 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-22T19:28:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:50.648403+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:50.913423+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:51.275113+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:51.275177+0000 mon.a (mon.0) 513 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:51.275340+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:51.277230+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 v2:172.21.15.64:6812/2605082260' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:51.277824+0000 mon.a (mon.0) 515 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:51.402027+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:51.402084+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:52.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:51.410265+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:50.913423+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:51.275113+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:51.275177+0000 mon.a (mon.0) 513 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:51.275340+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:51.277230+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 v2:172.21.15.64:6812/2605082260' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:51.277824+0000 mon.a (mon.0) 515 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:51.402027+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:51.402084+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:52.540 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:51.410265+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:52.586 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:28:52.281+0000 7fd6d5d32700 -1 osd.7 0 waiting for initial osdmap 2024-02-22T19:28:52.586 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:28:52.288+0000 7fd6ceca7700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:50.913423+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:51.275113+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:51.275177+0000 mon.a (mon.0) 513 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:51.275340+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:51.277230+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 v2:172.21.15.64:6812/2605082260' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:51.277824+0000 mon.a (mon.0) 515 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:51.402027+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:51.402084+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:51.410265+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:53.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:52.276977+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:53.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:52.277070+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-22T19:28:53.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:52.277451+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:53.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:52.282535+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:53.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:52.276977+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:53.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:52.277070+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-22T19:28:53.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:52.277451+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:53.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:52.282535+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:53.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:52.276977+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-22T19:28:53.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:52.277070+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-22T19:28:53.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:52.277451+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:53.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:52.282535+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:28:54.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:51.540781+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:54.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:51.540835+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:54.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:52.913885+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:53.279187+0000 mon.a (mon.0) 521 : cluster [INF] osd.7 v2:172.21.15.64:6812/2605082260 boot 2024-02-22T19:28:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:53.279235+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-22T19:28:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:53.279426+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:53.734718+0000 mon.a (mon.0) 524 : audit [DBG] from='client.? 172.21.15.53:0/3680502363' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-22T19:28:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:54.280512+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:51.540781+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:51.540835+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:52.913885+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:53.279187+0000 mon.a (mon.0) 521 : cluster [INF] osd.7 v2:172.21.15.64:6812/2605082260 boot 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:53.279235+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:53.279426+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:53.734718+0000 mon.a (mon.0) 524 : audit [DBG] from='client.? 172.21.15.53:0/3680502363' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-22T19:28:54.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:54.280512+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:51.540781+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:51.540835+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:52.913885+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:53.279187+0000 mon.a (mon.0) 521 : cluster [INF] osd.7 v2:172.21.15.64:6812/2605082260 boot 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:53.279235+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:53.279426+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:53.734718+0000 mon.a (mon.0) 524 : audit [DBG] from='client.? 172.21.15.53:0/3680502363' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-22T19:28:54.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:54.280512+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-22T19:28:54.395 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"num_osds":8,"num_up_osds":8,"osd_up_since":1708630133,"num_in_osds":8,"osd_in_since":1708630116,"num_remapped_pgs":1} 2024-02-22T19:28:54.396 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi053 2024-02-22T19:28:54.396 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi053=foo.a' 2024-02-22T19:28:56.095 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:54.914307+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:28:56.095 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:54.947393+0000 mgr.y (mgr.14146) 201 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:56.095 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:54.953338+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:56.095 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:55.282281+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-22T19:28:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:54.914307+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:28:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:54.947393+0000 mgr.y (mgr.14146) 201 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:54.953338+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:55.282281+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-22T19:28:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:54.914307+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:28:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:54.947393+0000 mgr.y (mgr.14146) 201 : cephadm [INF] Detected new or changed devices on smithi064 2024-02-22T19:28:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:54.953338+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:55.282281+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-22T19:28:56.517 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled rgw.foo.a update... 2024-02-22T19:28:57.136 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.982171+0000 mon.a (mon.0) 528 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.136 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.983104+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.136 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.983858+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.136 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.984606+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.136 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:55.985187+0000 mgr.y (mgr.14146) 202 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 4280M 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.989301+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.990312+0000 mon.a (mon.0) 533 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.991137+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.995863+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.998362+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:55.999954+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.000758+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.005479+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.517193+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.522886+0000 mon.a (mon.0) 541 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.524580+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.525381+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.529873+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.545719+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.137 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.546411+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-22T19:28:57.138 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.551733+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-22T19:28:57.138 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.557793+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.138 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.558644+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.157 DEBUG:teuthology.orchestra.run.smithi053:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@rgw.foo.a.service 2024-02-22T19:28:57.160 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi064 2024-02-22T19:28:57.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool create iscsi 3 3 replicated 2024-02-22T19:28:57.191 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.982171+0000 mon.a (mon.0) 528 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.983104+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.983858+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.984606+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:55.985187+0000 mgr.y (mgr.14146) 202 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 4280M 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.989301+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.990312+0000 mon.a (mon.0) 533 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.991137+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.995863+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.998362+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:55.999954+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.000758+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.005479+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.517193+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.522886+0000 mon.a (mon.0) 541 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.524580+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.525381+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.529873+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.545719+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.546411+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.551733+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-22T19:28:57.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.557793+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.558644+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.982171+0000 mon.a (mon.0) 528 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.983104+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.983858+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.984606+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:55.985187+0000 mgr.y (mgr.14146) 202 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 4280M 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.989301+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.990312+0000 mon.a (mon.0) 533 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.991137+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.995863+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.998362+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:55.999954+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.000758+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.005479+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.517193+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.522886+0000 mon.a (mon.0) 541 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.524580+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.525381+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.529873+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.545719+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.546411+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.551733+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.557793+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:28:57.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.558644+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:28:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:56.511671+0000 mgr.y (mgr.14146) 203 : audit [DBG] from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi053=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:56.513220+0000 mgr.y (mgr.14146) 204 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi053=foo.a;count:1 2024-02-22T19:28:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:56.542626+0000 mgr.y (mgr.14146) 205 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi053=foo.a;count:1 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:28:56.559299+0000 mgr.y (mgr.14146) 206 : cephadm [INF] Deploying daemon rgw.foo.a on smithi053 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:56.914797+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:56.511671+0000 mgr.y (mgr.14146) 203 : audit [DBG] from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi053=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:56.513220+0000 mgr.y (mgr.14146) 204 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi053=foo.a;count:1 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:56.542626+0000 mgr.y (mgr.14146) 205 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi053=foo.a;count:1 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:28:56.559299+0000 mgr.y (mgr.14146) 206 : cephadm [INF] Deploying daemon rgw.foo.a on smithi053 2024-02-22T19:28:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:56.914797+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:28:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:56.511671+0000 mgr.y (mgr.14146) 203 : audit [DBG] from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi053=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:28:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:56.513220+0000 mgr.y (mgr.14146) 204 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi053=foo.a;count:1 2024-02-22T19:28:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:56.542626+0000 mgr.y (mgr.14146) 205 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi053=foo.a;count:1 2024-02-22T19:28:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:28:56.559299+0000 mgr.y (mgr.14146) 206 : cephadm [INF] Deploying daemon rgw.foo.a on smithi053 2024-02-22T19:28:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:56.914797+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:28:58.930 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:28:58 smithi053 systemd[1]: Starting Ceph rgw.foo.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:28:59.504 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:28:59 smithi053 bash[93489]: d51c8c8224dac4a07f9858992d1e8c8f8394079b0dc5edf2d1ecf3450664c243 2024-02-22T19:28:59.504 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:28:59 smithi053 systemd[1]: Started Ceph rgw.foo.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:29:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'iscsi' created 2024-02-22T19:29:00.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.118631+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.64:0/1383179209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-22T19:29:00.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:28:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.119075+0000 mon.a (mon.0) 550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-22T19:29:00.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.118631+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.64:0/1383179209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-22T19:29:00.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:28:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.119075+0000 mon.a (mon.0) 550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-22T19:29:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.118631+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.64:0/1383179209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-22T19:29:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:28:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.119075+0000 mon.a (mon.0) 550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-22T19:29:00.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool application enable iscsi rbd 2024-02-22T19:29:01.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:58.915147+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:01.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.994486+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:28:59.994623+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.994801+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.996256+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.996610+0000 mon.a (mon.0) 554 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:28:59.997046+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:00.037320+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:01.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:58.915147+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.994486+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:28:59.994623+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.994801+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.996256+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.996610+0000 mon.a (mon.0) 554 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:28:59.997046+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:00.037320+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:58.915147+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.994486+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:28:59.994623+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.994801+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.996256+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-22T19:29:01.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.996610+0000 mon.a (mon.0) 554 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-22T19:29:01.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:28:59.997046+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:01.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:00.037320+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:02.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:00.915621+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v199: 36 pgs: 15 creating+peering, 1 active+clean, 20 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:02.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:00.996464+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-22T19:29:02.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:00.996633+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-22T19:29:02.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:01.499695+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:02.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:01.500507+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:02.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:01.501133+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:00.915621+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v199: 36 pgs: 15 creating+peering, 1 active+clean, 20 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:00.996464+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-22T19:29:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:00.996633+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-22T19:29:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:01.499695+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:02.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:01.500507+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:01.501133+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:00.915621+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v199: 36 pgs: 15 creating+peering, 1 active+clean, 20 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:00.996464+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:00.996633+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:01.499695+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:01.500507+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:01.501133+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:enabled application 'rbd' on pool 'iscsi' 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:01.505068+0000 mgr.y (mgr.14146) 210 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:02.063686+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:02.065698+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:02.067435+0000 mon.a (mon.0) 563 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:02.068625+0000 mon.a (mon.0) 564 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:02.836290+0000 mon.b (mon.1) 24 : audit [INF] from='client.? 172.21.15.64:0/2142170005' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-22T19:29:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:02.836998+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:01.505068+0000 mgr.y (mgr.14146) 210 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:02.063686+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:02.065698+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:02.067435+0000 mon.a (mon.0) 563 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:02.068625+0000 mon.a (mon.0) 564 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:02.836290+0000 mon.b (mon.1) 24 : audit [INF] from='client.? 172.21.15.64:0/2142170005' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-22T19:29:03.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:02.836998+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:01.505068+0000 mgr.y (mgr.14146) 210 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:02.063686+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:02.065698+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:02.067435+0000 mon.a (mon.0) 563 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:02.068625+0000 mon.a (mon.0) 564 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:02.836290+0000 mon.b (mon.1) 24 : audit [INF] from='client.? 172.21.15.64:0/2142170005' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-22T19:29:03.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:02.836998+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-22T19:29:03.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply iscsi iscsi user password --placement '1;smithi064=iscsi.a' 2024-02-22T19:29:04.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:02.916271+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v202: 68 pgs: 24 creating+peering, 11 active+clean, 33 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:04.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:03.070552+0000 mon.a (mon.0) 566 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-22T19:29:04.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:03.070738+0000 mon.a (mon.0) 567 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-22T19:29:04.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:03.070855+0000 mon.a (mon.0) 568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-22T19:29:04.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:03.070929+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-22T19:29:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:02.916271+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v202: 68 pgs: 24 creating+peering, 11 active+clean, 33 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:03.070552+0000 mon.a (mon.0) 566 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-22T19:29:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:03.070738+0000 mon.a (mon.0) 567 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-22T19:29:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:03.070855+0000 mon.a (mon.0) 568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-22T19:29:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:03.070929+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-22T19:29:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:02.916271+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v202: 68 pgs: 24 creating+peering, 11 active+clean, 33 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:03.070552+0000 mon.a (mon.0) 566 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-22T19:29:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:03.070738+0000 mon.a (mon.0) 567 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-22T19:29:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:03.070855+0000 mon.a (mon.0) 568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-22T19:29:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:03.070929+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-22T19:29:05.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:04.074006+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-22T19:29:05.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:04.076430+0000 mon.a (mon.0) 571 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:04.077026+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:04.077768+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:04.074006+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-22T19:29:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:04.076430+0000 mon.a (mon.0) 571 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:04.077026+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:04.077768+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:04.074006+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-22T19:29:05.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:04.076430+0000 mon.a (mon.0) 571 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:04.077026+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.53:0/1058598254' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:05.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:04.077768+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-22T19:29:06.032 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled iscsi.iscsi update... 2024-02-22T19:29:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:04.916877+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v205: 100 pgs: 29 creating+peering, 25 active+clean, 46 unknown; 92 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:05.080074+0000 mon.a (mon.0) 573 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-22T19:29:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:05.080298+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-22T19:29:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:05.080418+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-22T19:29:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:06.031948+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:04.916877+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v205: 100 pgs: 29 creating+peering, 25 active+clean, 46 unknown; 92 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:05.080074+0000 mon.a (mon.0) 573 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-22T19:29:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:05.080298+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-22T19:29:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:05.080418+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-22T19:29:06.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:06.031948+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:04.916877+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v205: 100 pgs: 29 creating+peering, 25 active+clean, 46 unknown; 92 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:05.080074+0000 mon.a (mon.0) 573 : audit [INF] from='client.? 172.21.15.53:0/2672347046' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-22T19:29:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:05.080298+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-22T19:29:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:05.080418+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-22T19:29:06.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:06.031948+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:06.671 DEBUG:teuthology.orchestra.run.smithi064:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@iscsi.iscsi.a.service 2024-02-22T19:29:06.673 INFO:tasks.cephadm:Adding prometheus.a on smithi064 2024-02-22T19:29:06.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply prometheus '1;smithi064=a' 2024-02-22T19:29:06.706 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:29:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:05.808311+0000 mgr.y (mgr.14146) 213 : audit [DBG] from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi064=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:07.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:05.810287+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi064=iscsi.a;count:1 2024-02-22T19:29:07.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:06.083589+0000 mon.a (mon.0) 577 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-22T19:29:07.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:06.085409+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.53:0/65330745' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:06.085783+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:06.091590+0000 mon.b (mon.1) 25 : audit [INF] from='client.? 172.21.15.53:0/731255676' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:06.092046+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:05.808311+0000 mgr.y (mgr.14146) 213 : audit [DBG] from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi064=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:07.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:05.810287+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi064=iscsi.a;count:1 2024-02-22T19:29:07.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:06.083589+0000 mon.a (mon.0) 577 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-22T19:29:07.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:06.085409+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.53:0/65330745' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:06.085783+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:06.091590+0000 mon.b (mon.1) 25 : audit [INF] from='client.? 172.21.15.53:0/731255676' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:06.092046+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:05.808311+0000 mgr.y (mgr.14146) 213 : audit [DBG] from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi064=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:05.810287+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi064=iscsi.a;count:1 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:06.083589+0000 mon.a (mon.0) 577 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:06.085409+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.53:0/65330745' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:06.085783+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:06.091590+0000 mon.b (mon.1) 25 : audit [INF] from='client.? 172.21.15.53:0/731255676' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:07.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:06.092046+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-22T19:29:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:06.917497+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v208: 132 pgs: 19 creating+peering, 75 active+clean, 38 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-22T19:29:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:07.086241+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-22T19:29:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:07.086410+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-22T19:29:08.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:07.086569+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-22T19:29:08.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:07.095068+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.53:0/731255676' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:07.104105+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:07.104800+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.53:0/65330745' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:07.106010+0000 mon.a (mon.0) 584 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:06.917497+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v208: 132 pgs: 19 creating+peering, 75 active+clean, 38 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-22T19:29:08.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:07.086241+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:07.086410+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:07.086569+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:07.095068+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.53:0/731255676' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:07.104105+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:07.104800+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.53:0/65330745' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:07.106010+0000 mon.a (mon.0) 584 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:06.917497+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v208: 132 pgs: 19 creating+peering, 75 active+clean, 38 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:07.086241+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:07.086410+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:07.086569+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:07.095068+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.53:0/731255676' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:07.104105+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:07.104800+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.53:0/65330745' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:07.106010+0000 mon.a (mon.0) 584 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-22T19:29:08.845 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled prometheus update... 2024-02-22T19:29:09.216 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.100484+0000 mon.a (mon.0) 585 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-22T19:29:09.216 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.100609+0000 mon.a (mon.0) 586 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-22T19:29:09.216 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:08.100700+0000 mon.a (mon.0) 587 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-22T19:29:09.216 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.267978+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.216 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.274581+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.216 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.280005+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.217 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:08.286179+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-22T19:29:09.217 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.296667+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-22T19:29:09.217 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.300882+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-22T19:29:09.217 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.308918+0000 mon.a (mon.0) 593 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:09.217 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:08.309798+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi064 2024-02-22T19:29:09.217 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.845481+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.100484+0000 mon.a (mon.0) 585 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-22T19:29:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.100609+0000 mon.a (mon.0) 586 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-22T19:29:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:08.100700+0000 mon.a (mon.0) 587 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.267978+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.274581+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.280005+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:08.286179+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.296667+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.300882+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.308918+0000 mon.a (mon.0) 593 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:08.309798+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi064 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.845481+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.100484+0000 mon.a (mon.0) 585 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-22T19:29:09.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.100609+0000 mon.a (mon.0) 586 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:08.100700+0000 mon.a (mon.0) 587 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.267978+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.274581+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.280005+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:08.286179+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.296667+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.300882+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.308918+0000 mon.a (mon.0) 593 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:08.309798+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi064 2024-02-22T19:29:09.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.845481+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:10.303 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:10 smithi064 systemd[1]: Starting Ceph iscsi.iscsi.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:29:10.346 DEBUG:teuthology.orchestra.run.smithi064:prometheus.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@prometheus.a.service 2024-02-22T19:29:10.348 INFO:tasks.cephadm:Adding node-exporter.a on smithi053 2024-02-22T19:29:10.348 INFO:tasks.cephadm:Adding node-exporter.b on smithi064 2024-02-22T19:29:10.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply node-exporter '2;smithi053=a;smithi064=b' 2024-02-22T19:29:10.381 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:29:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:08.840625+0000 mgr.y (mgr.14146) 218 : audit [DBG] from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi064=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:08.842121+0000 mgr.y (mgr.14146) 219 : cephadm [INF] Saving service prometheus spec with placement smithi064=a;count:1 2024-02-22T19:29:10.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:08.918002+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v211: 132 pgs: 10 creating+peering, 103 active+clean, 19 unknown; 1.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-22T19:29:10.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:09.306603+0000 mon.a (mon.0) 595 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-22T19:29:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:08.840625+0000 mgr.y (mgr.14146) 218 : audit [DBG] from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi064=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:08.842121+0000 mgr.y (mgr.14146) 219 : cephadm [INF] Saving service prometheus spec with placement smithi064=a;count:1 2024-02-22T19:29:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:08.918002+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v211: 132 pgs: 10 creating+peering, 103 active+clean, 19 unknown; 1.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-22T19:29:10.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:09.306603+0000 mon.a (mon.0) 595 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-22T19:29:10.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:08.840625+0000 mgr.y (mgr.14146) 218 : audit [DBG] from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi064=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:10.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:08.842121+0000 mgr.y (mgr.14146) 219 : cephadm [INF] Saving service prometheus spec with placement smithi064=a;count:1 2024-02-22T19:29:10.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:08.918002+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v211: 132 pgs: 10 creating+peering, 103 active+clean, 19 unknown; 1.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-22T19:29:10.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:09.306603+0000 mon.a (mon.0) 595 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-22T19:29:11.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:11 smithi064 bash[61083]: log file path now is '/var/log/tcmu-runner.log' 2024-02-22T19:29:12.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 bash[61231]: b3da7fbf28a71616b71f7974644ffb2441a59124b354ac0e61ab2ab15f9f87ae 2024-02-22T19:29:12.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 systemd[1]: Started Ceph iscsi.iscsi.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:29:12.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug Started the configuration object watcher 2024-02-22T19:29:12.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug Checking for config object changes every 1s 2024-02-22T19:29:12.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug Processing osd blocklist entries for this node 2024-02-22T19:29:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:10.918652+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v212: 132 pgs: 5 creating+peering, 127 active+clean; 3.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.1 KiB/s wr, 117 op/s 2024-02-22T19:29:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:12.100004+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:12.101643+0000 mon.a (mon.0) 597 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:12.603 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug Reading the configuration object to update local LIO configuration 2024-02-22T19:29:12.603 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug Configuration does not have an entry for this host(smithi064.front.sepia.ceph.com) - nothing to define to LIO 2024-02-22T19:29:12.603 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-22T19:29:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:10.918652+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v212: 132 pgs: 5 creating+peering, 127 active+clean; 3.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.1 KiB/s wr, 117 op/s 2024-02-22T19:29:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:12.100004+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:12.101643+0000 mon.a (mon.0) 597 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:10.918652+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v212: 132 pgs: 5 creating+peering, 127 active+clean; 3.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.1 KiB/s wr, 117 op/s 2024-02-22T19:29:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:12.100004+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:12.101643+0000 mon.a (mon.0) 597 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:13.223 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled node-exporter update... 2024-02-22T19:29:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:12.509644+0000 mon.b (mon.1) 27 : audit [DBG] from='client.? 172.21.15.64:0/1783159365' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T19:29:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:13.223111+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:13.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:12.509644+0000 mon.b (mon.1) 27 : audit [DBG] from='client.? 172.21.15.64:0/1783159365' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T19:29:13.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:13.223111+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:13.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:12.509644+0000 mon.b (mon.1) 27 : audit [DBG] from='client.? 172.21.15.64:0/1783159365' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T19:29:13.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:13.223111+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:13.983 DEBUG:teuthology.orchestra.run.smithi053:node-exporter.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@node-exporter.a.service 2024-02-22T19:29:13.986 DEBUG:teuthology.orchestra.run.smithi064:node-exporter.b> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@node-exporter.b.service 2024-02-22T19:29:13.989 INFO:tasks.cephadm:Adding alertmanager.a on smithi053 2024-02-22T19:29:13.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply alertmanager '1;smithi053=a' 2024-02-22T19:29:14.019 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:29:14.023 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:29:14.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:12.919391+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v213: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.1 KiB/s wr, 164 op/s 2024-02-22T19:29:14.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:13.217696+0000 mgr.y (mgr.14146) 223 : audit [DBG] from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi053=a;smithi064=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:14.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:13.219113+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Saving service node-exporter spec with placement smithi053=a;smithi064=b;count:2 2024-02-22T19:29:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:12.919391+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v213: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.1 KiB/s wr, 164 op/s 2024-02-22T19:29:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:13.217696+0000 mgr.y (mgr.14146) 223 : audit [DBG] from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi053=a;smithi064=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:13.219113+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Saving service node-exporter spec with placement smithi053=a;smithi064=b;count:2 2024-02-22T19:29:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:12.919391+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v213: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.1 KiB/s wr, 164 op/s 2024-02-22T19:29:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:13.217696+0000 mgr.y (mgr.14146) 223 : audit [DBG] from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi053=a;smithi064=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:13.219113+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Saving service node-exporter spec with placement smithi053=a;smithi064=b;count:2 2024-02-22T19:29:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:14.919955+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v214: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-02-22T19:29:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:15.050664+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:15.489459+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:29:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:15.492601+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:29:16.386 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled alertmanager update... 2024-02-22T19:29:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:14.919955+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v214: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-02-22T19:29:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:15.050664+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:15.489459+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:29:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:15.492601+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:29:16.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:14.919955+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v214: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-02-22T19:29:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:15.050664+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:15.489459+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:29:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:15.492601+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:29:17.159 DEBUG:teuthology.orchestra.run.smithi053:alertmanager.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@alertmanager.a.service 2024-02-22T19:29:17.163 INFO:tasks.cephadm:Adding grafana.a on smithi064 2024-02-22T19:29:17.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph orch apply grafana '1;smithi064=a' 2024-02-22T19:29:17.196 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:-- Logs begin at Thu 2024-02-22 18:42:40 UTC. -- 2024-02-22T19:29:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.381436+0000 mgr.y (mgr.14146) 226 : audit [DBG] from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi053=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:16.382794+0000 mgr.y (mgr.14146) 227 : cephadm [INF] Saving service alertmanager spec with placement smithi053=a;count:1 2024-02-22T19:29:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.386612+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.594946+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.596150+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.597205+0000 mon.a (mon.0) 603 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.614570+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:16.620919+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:16.920766+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v215: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.7 KiB/s wr, 115 op/s 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.381436+0000 mgr.y (mgr.14146) 226 : audit [DBG] from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi053=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:16.382794+0000 mgr.y (mgr.14146) 227 : cephadm [INF] Saving service alertmanager spec with placement smithi053=a;count:1 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.386612+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.594946+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.596150+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.597205+0000 mon.a (mon.0) 603 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.614570+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:16.620919+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-22T19:29:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:16.920766+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v215: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.7 KiB/s wr, 115 op/s 2024-02-22T19:29:17.563 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.381436+0000 mgr.y (mgr.14146) 226 : audit [DBG] from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi053=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T19:29:17.563 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:16.382794+0000 mgr.y (mgr.14146) 227 : cephadm [INF] Saving service alertmanager spec with placement smithi053=a;count:1 2024-02-22T19:29:17.563 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.386612+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.563 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.594946+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.563 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.596150+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:17.563 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.597205+0000 mon.a (mon.0) 603 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:17.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.614570+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' 2024-02-22T19:29:17.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:16.620919+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-22T19:29:17.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:16.920766+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v215: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.7 KiB/s wr, 115 op/s 2024-02-22T19:29:17.825 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ignoring --setuser ceph since I am not root 2024-02-22T19:29:17.825 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ignoring --setgroup ceph since I am not root 2024-02-22T19:29:17.825 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:17.654+0000 7fcaea0a9700 1 -- 172.21.15.64:0/3078290302 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db6e3104e0 con 0x55db6e2e6c00 2024-02-22T19:29:17.825 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:17.780+0000 7fcafa286680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-22T19:29:18.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ignoring --setuser ceph since I am not root 2024-02-22T19:29:18.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ignoring --setgroup ceph since I am not root 2024-02-22T19:29:18.005 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:17.661+0000 7f53590d0700 1 -- 172.21.15.53:0/3373340591 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5622f331e4e0 con 0x5622f32f5400 2024-02-22T19:29:18.005 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:17.789+0000 7f53692ad680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-22T19:29:18.005 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:17.873+0000 7f53692ad680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-22T19:29:18.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:17.866+0000 7fcafa286680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-22T19:29:18.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:18.250+0000 7f53692ad680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-22T19:29:18.547 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:18.249+0000 7fcafa286680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-22T19:29:18.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:17.623090+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-22T19:29:18.824 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:17.623171+0000 mon.a (mon.0) 607 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-22T19:29:19.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:17.623090+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-22T19:29:19.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:17.623171+0000 mon.a (mon.0) 607 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-22T19:29:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:17.623090+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.53:0/4067013948' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-22T19:29:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:17.623171+0000 mon.a (mon.0) 607 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-22T19:29:19.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:19.050+0000 7fcafa286680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-22T19:29:19.340 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:19.035+0000 7f53692ad680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-22T19:29:19.340 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:19.120+0000 7f53692ad680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-22T19:29:19.340 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:19.339+0000 7f53692ad680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-22T19:29:19.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:19.138+0000 7fcafa286680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-22T19:29:19.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:19.357+0000 7fcafa286680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-22T19:29:20.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:20.331+0000 7fcafa286680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-22T19:29:20.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:20.464+0000 7fcafa286680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:29:20.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:20.546+0000 7fcafa286680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-22T19:29:20.628 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:20.316+0000 7f53692ad680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-22T19:29:20.628 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:20.449+0000 7f53692ad680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:29:20.628 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:20.532+0000 7f53692ad680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-22T19:29:21.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:20.627+0000 7f53692ad680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-22T19:29:21.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:20.829+0000 7f53692ad680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-22T19:29:21.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:20.638+0000 7fcafa286680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-22T19:29:21.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:20.840+0000 7fcafa286680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-22T19:29:21.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:21.123+0000 7f53692ad680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-22T19:29:21.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:21.225+0000 7f53692ad680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-22T19:29:21.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:21.132+0000 7fcafa286680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-22T19:29:21.602 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:21.234+0000 7fcafa286680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-22T19:29:22.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:21.980+0000 7f53692ad680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-22T19:29:22.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:22.065+0000 7f53692ad680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-22T19:29:22.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:22.161+0000 7f53692ad680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-22T19:29:22.343 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:21.985+0000 7fcafa286680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-22T19:29:22.343 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:22.069+0000 7fcafa286680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-22T19:29:22.343 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:22.163+0000 7fcafa286680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-22T19:29:22.343 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:22.342+0000 7fcafa286680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-22T19:29:22.611 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:22.342+0000 7f53692ad680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-22T19:29:22.884 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:22.608+0000 7fcafa286680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-22T19:29:22.884 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:22.744+0000 7fcafa286680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-22T19:29:22.884 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:22.883+0000 7fcafa286680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:29:22.888 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:22.610+0000 7f53692ad680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-22T19:29:22.888 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:22.747+0000 7f53692ad680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-22T19:29:22.888 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:22.887+0000 7f53692ad680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-22T19:29:23.149 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:23.061+0000 7fcafa286680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-22T19:29:23.155 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:23.066+0000 7f53692ad680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-22T19:29:23.410 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: debug 2024-02-22T19:29:23.148+0000 7fcafa286680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: [22/Feb/2024:19:29:23] ENGINE Bus STARTING 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: CherryPy Checker: 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: The Application mounted at '' has an empty config. 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: [22/Feb/2024:19:29:23] ENGINE Serving on http://:::9283 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: [22/Feb/2024:19:29:23] ENGINE Bus STARTED 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.163625+0000 mon.a (mon.0) 608 : cluster [DBG] Standby manager daemon x restarted 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.163722+0000 mon.a (mon.0) 609 : cluster [DBG] Standby manager daemon x started 2024-02-22T19:29:23.411 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.165089+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.165705+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.167689+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.167762+0000 mon.a (mon.0) 610 : cluster [INF] Active manager daemon y restarted 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.168113+0000 mon.a (mon.0) 611 : cluster [INF] Activating manager daemon y 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.168580+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.173756+0000 mon.a (mon.0) 612 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.174177+0000 mon.a (mon.0) 613 : cluster [DBG] mgrmap e17: y(active, starting, since 0.00616672s), standbys: x 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.177158+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.177325+0000 mon.a (mon.0) 615 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.177545+0000 mon.a (mon.0) 616 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.178370+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.178550+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-22T19:29:23.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.178797+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.179009+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.179230+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.179439+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.179663+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.179929+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.180200+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.180463+0000 mon.a (mon.0) 626 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.180743+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.180952+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.181473+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T19:29:23.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:23.183917+0000 mon.a (mon.0) 630 : cluster [INF] Manager daemon y is now available 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:29:23.154+0000 7f53692ad680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:29:23] ENGINE Bus STARTING 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [79B blob data] 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [79B blob data] 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.163625+0000 mon.a (mon.0) 608 : cluster [DBG] Standby manager daemon x restarted 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.163722+0000 mon.a (mon.0) 609 : cluster [DBG] Standby manager daemon x started 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.165089+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.165705+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T19:29:23.415 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.167689+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.167762+0000 mon.a (mon.0) 610 : cluster [INF] Active manager daemon y restarted 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.168113+0000 mon.a (mon.0) 611 : cluster [INF] Activating manager daemon y 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.168580+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.173756+0000 mon.a (mon.0) 612 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.174177+0000 mon.a (mon.0) 613 : cluster [DBG] mgrmap e17: y(active, starting, since 0.00616672s), standbys: x 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.177158+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.177325+0000 mon.a (mon.0) 615 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.177545+0000 mon.a (mon.0) 616 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.178370+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.178550+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-22T19:29:23.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.178797+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.163625+0000 mon.a (mon.0) 608 : cluster [DBG] Standby manager daemon x restarted 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.163722+0000 mon.a (mon.0) 609 : cluster [DBG] Standby manager daemon x started 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.165089+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.165705+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.167689+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.167762+0000 mon.a (mon.0) 610 : cluster [INF] Active manager daemon y restarted 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.168113+0000 mon.a (mon.0) 611 : cluster [INF] Activating manager daemon y 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.168580+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.? 172.21.15.64:0/2498208278' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.173756+0000 mon.a (mon.0) 612 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.174177+0000 mon.a (mon.0) 613 : cluster [DBG] mgrmap e17: y(active, starting, since 0.00616672s), standbys: x 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.177158+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.177325+0000 mon.a (mon.0) 615 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.177545+0000 mon.a (mon.0) 616 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-22T19:29:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.178370+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.178550+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.178797+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.179009+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.179230+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.179439+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.179663+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.179929+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.180200+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.180463+0000 mon.a (mon.0) 626 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.180743+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T19:29:23.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.180952+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.181473+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:23.183917+0000 mon.a (mon.0) 630 : cluster [INF] Manager daemon y is now available 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.179009+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.179230+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.179439+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.179663+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.179929+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.180200+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.180463+0000 mon.a (mon.0) 626 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.180743+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.180952+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T19:29:23.419 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.181473+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T19:29:23.420 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:23.183917+0000 mon.a (mon.0) 630 : cluster [INF] Manager daemon y is now available 2024-02-22T19:29:23.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: CherryPy Checker: 2024-02-22T19:29:23.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: The Application mounted at '' has an empty config. 2024-02-22T19:29:23.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: 2024-02-22T19:29:23.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:29:23] ENGINE Serving on http://:::9283 2024-02-22T19:29:23.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:29:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:29:23] ENGINE Bus STARTED 2024-02-22T19:29:24.189 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled grafana update... 2024-02-22T19:29:24.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.289503+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.291063+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.313377+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.332705+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:23.353026+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:24.178387+0000 mon.a (mon.0) 636 : cluster [DBG] mgrmap e18: y(active, since 1.01037s), standbys: x 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:24.188484+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.289503+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.291063+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.313377+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.332705+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:23.353026+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-22T19:29:24.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:24.178387+0000 mon.a (mon.0) 636 : cluster [DBG] mgrmap e18: y(active, since 1.01037s), standbys: x 2024-02-22T19:29:24.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:24.188484+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:24.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:29:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.289503+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.291063+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:29:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.313377+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.332705+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-22T19:29:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:23.353026+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-22T19:29:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:24.178387+0000 mon.a (mon.0) 636 : cluster [DBG] mgrmap e18: y(active, since 1.01037s), standbys: x 2024-02-22T19:29:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:24.188484+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:25.210 DEBUG:teuthology.orchestra.run.smithi064:grafana.a> sudo journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@grafana.a.service 2024-02-22T19:29:25.212 INFO:tasks.cephadm:Setting up client nodes... 2024-02-22T19:29:25.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-22T19:29:25.243 INFO:journalctl@ceph.grafana.a.smithi064.stdout:-- Logs begin at Thu 2024-02-22 18:42:58 UTC. -- 2024-02-22T19:29:25.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:24.184071+0000 mgr.y (mgr.14535) 3 : cephadm [INF] Saving service grafana spec with placement smithi064=a;count:1 2024-02-22T19:29:25.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:24.184113+0000 mgr.y (mgr.14535) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:25.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:24.940128+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:25.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:24.184071+0000 mgr.y (mgr.14535) 3 : cephadm [INF] Saving service grafana spec with placement smithi064=a;count:1 2024-02-22T19:29:25.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:24.184113+0000 mgr.y (mgr.14535) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:25.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:24.940128+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:24.184071+0000 mgr.y (mgr.14535) 3 : cephadm [INF] Saving service grafana spec with placement smithi064=a;count:1 2024-02-22T19:29:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:24.184113+0000 mgr.y (mgr.14535) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:24.940128+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:25.179953+0000 mgr.y (mgr.14535) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:25.240305+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:25.241064+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:25.241689+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:25.242369+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:25.242909+0000 mgr.y (mgr.14535) 6 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 3769M 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:25.248054+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:25.314518+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:25.314566+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.035455+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:26.184687+0000 mon.a (mon.0) 645 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:25.179953+0000 mgr.y (mgr.14535) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:25.240305+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:25.241064+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:25.241689+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:25.242369+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:25.242909+0000 mgr.y (mgr.14535) 6 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 3769M 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:25.248054+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:25.314518+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:25.314566+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.035455+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:26.184687+0000 mon.a (mon.0) 645 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-22T19:29:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:25.179953+0000 mgr.y (mgr.14535) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:25.240305+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:25.241064+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:25.241689+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:25.242369+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:25.242909+0000 mgr.y (mgr.14535) 6 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 3769M 2024-02-22T19:29:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:25.248054+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:25.314518+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:25.314566+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.035455+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:26.184687+0000 mon.a (mon.0) 645 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-22T19:29:27.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:26.247894+0000 mon.a (mon.0) 646 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-22T19:29:27.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.248691+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.249345+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.338545+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.339151+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.339671+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.340186+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:26.340587+0000 mgr.y (mgr.14535) 7 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 4024M 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.344558+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.345354+0000 mon.a (mon.0) 654 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.345959+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:26.347031+0000 mgr.y (mgr.14535) 8 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:26.347301+0000 mgr.y (mgr.14535) 9 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:26.365150+0000 mgr.y (mgr.14535) 10 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.394236+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:26.396258+0000 mgr.y (mgr.14535) 11 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:26.247894+0000 mon.a (mon.0) 646 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.248691+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.249345+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.338545+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.339151+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.339671+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.340186+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:26.340587+0000 mgr.y (mgr.14535) 7 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 4024M 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.344558+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.345354+0000 mon.a (mon.0) 654 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.345959+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:26.347031+0000 mgr.y (mgr.14535) 8 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:26.347301+0000 mgr.y (mgr.14535) 9 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:26.365150+0000 mgr.y (mgr.14535) 10 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.394236+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:26.396258+0000 mgr.y (mgr.14535) 11 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.442898+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.446266+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:26.456291+0000 mgr.y (mgr.14535) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi053 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:26.587547+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:26.587631+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-22T19:29:27.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.442898+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.446266+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:26.456291+0000 mgr.y (mgr.14535) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi053 2024-02-22T19:29:27.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:26.587547+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-22T19:29:27.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:26.587631+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:26.247894+0000 mon.a (mon.0) 646 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.248691+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.249345+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.338545+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.339151+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.339671+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.340186+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:26.340587+0000 mgr.y (mgr.14535) 7 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 4024M 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.344558+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.345354+0000 mon.a (mon.0) 654 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.345959+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:26.347031+0000 mgr.y (mgr.14535) 8 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.conf 2024-02-22T19:29:27.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:26.347301+0000 mgr.y (mgr.14535) 9 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:26.365150+0000 mgr.y (mgr.14535) 10 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.394236+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:26.396258+0000 mgr.y (mgr.14535) 11 : cephadm [INF] Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.442898+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.446266+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:26.456291+0000 mgr.y (mgr.14535) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi053 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:26.587547+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-22T19:29:27.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:26.587631+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-22T19:29:27.611 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-02-22T19:29:27.611 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCXoNdl79Y/JBAAaQUyGG0CdZPM0R3ODjCNtQ== 2024-02-22T19:29:28.298 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-02-22T19:29:28.298 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-22T19:29:28.298 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-22T19:29:28.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-22T19:29:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:27.180275+0000 mgr.y (mgr.14535) 13 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:27.607962+0000 mon.a (mon.0) 661 : audit [INF] from='client.? 172.21.15.53:0/886626092' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:28.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:27.611164+0000 mon.a (mon.0) 662 : audit [INF] from='client.? 172.21.15.53:0/886626092' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T19:29:29.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:27.180275+0000 mgr.y (mgr.14535) 13 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:29.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:27.607962+0000 mon.a (mon.0) 661 : audit [INF] from='client.? 172.21.15.53:0/886626092' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:29.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:27.611164+0000 mon.a (mon.0) 662 : audit [INF] from='client.? 172.21.15.53:0/886626092' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T19:29:29.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:27.180275+0000 mgr.y (mgr.14535) 13 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:29.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:27.607962+0000 mon.a (mon.0) 661 : audit [INF] from='client.? 172.21.15.53:0/886626092' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:29.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:27.611164+0000 mon.a (mon.0) 662 : audit [INF] from='client.? 172.21.15.53:0/886626092' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T19:29:29.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:29.212964+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:29.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:29.213020+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:29.212964+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:29.213020+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:30.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:29.212964+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:30.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:29.213020+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:30.582 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2024-02-22T19:29:30.582 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCaoNdltah5IhAAPfK69C35fl2uVEKbskLNYQ== 2024-02-22T19:29:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:29.180587+0000 mgr.y (mgr.14535) 14 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:30.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:29.599689+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-22T19:29:30.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:30.577710+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.64:0/1147597755' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:30.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:30.578220+0000 mon.a (mon.0) 664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:30.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:30.581646+0000 mon.a (mon.0) 665 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T19:29:31.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:29.180587+0000 mgr.y (mgr.14535) 14 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:31.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:29.599689+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-22T19:29:31.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:30.577710+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.64:0/1147597755' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:31.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:30.578220+0000 mon.a (mon.0) 664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:31.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:30.581646+0000 mon.a (mon.0) 665 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T19:29:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:29.180587+0000 mgr.y (mgr.14535) 14 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:29:31.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:29.599689+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-22T19:29:31.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:30.577710+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.64:0/1147597755' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:31.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:30.578220+0000 mon.a (mon.0) 664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T19:29:31.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:30.581646+0000 mon.a (mon.0) 665 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T19:29:31.244 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-22T19:29:31.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-22T19:29:31.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-22T19:29:31.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T19:29:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:31.180924+0000 mgr.y (mgr.14535) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:29:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:31.588609+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-22T19:29:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:31.180924+0000 mgr.y (mgr.14535) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:29:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:31.588609+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-22T19:29:33.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:31.180924+0000 mgr.y (mgr.14535) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:29:33.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:31.588609+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-22T19:29:34.589 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:29:34.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:33.181414+0000 mgr.y (mgr.14535) 16 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:29:35.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:33.181414+0000 mgr.y (mgr.14535) 16 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:29:35.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:33.181414+0000 mgr.y (mgr.14535) 16 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:29:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:34.191764+0000 mgr.y (mgr.14535) 17 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:35.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:34.191764+0000 mgr.y (mgr.14535) 17 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:35.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:34.191764+0000 mgr.y (mgr.14535) 17 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:29:36.106 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","created":"2024-02-22T19:22:59.746063+0000","modified":"2024-02-22T19:29:31.584347+0000","last_up_change":"2024-02-22T19:28:53.274045+0000","last_in_change":"2024-02-22T19:28:36.235896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-22T19:26:54.342689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-22T19:28:59.555543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-22T19:28:59.986190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-22T19:29:01.494051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-22T19:29:03.518927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-22T19:29:05.596341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"4326a76a-6144-4859-9b40-5ff69edbbcab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6801","nonce":162271081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":162271081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":162271081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":162271081}]},"public_addr":"172.21.15.53:6801/162271081","cluster_addr":"172.21.15.53:6802/162271081","heartbeat_back_addr":"172.21.15.53:6804/162271081","heartbeat_front_addr":"172.21.15.53:6803/162271081","state":["exists","up"]},{"osd":1,"uuid":"fc056bcd-1a3b-408c-b9d1-4992091df6c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6805","nonce":2973403044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2973403044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2973403044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2973403044}]},"public_addr":"172.21.15.53:6805/2973403044","cluster_addr":"172.21.15.53:6806/2973403044","heartbeat_back_addr":"172.21.15.53:6808/2973403044","heartbeat_front_addr":"172.21.15.53:6807/2973403044","state":["exists","up"]},{"osd":2,"uuid":"61f3f980-82cd-4e8e-9a3f-6d3f4559cc83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6809","nonce":936980935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":936980935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":936980935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":936980935}]},"public_addr":"172.21.15.53:6809/936980935","cluster_addr":"172.21.15.53:6810/936980935","heartbeat_back_addr":"172.21.15.53:6812/936980935","heartbeat_front_addr":"172.21.15.53:6811/936980935","state":["exists","up"]},{"osd":3,"uuid":"3f285320-aa28-4fc1-b1c7-456c9aecd993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":3878043918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3878043918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3878043918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6815","nonce":3878043918}]},"public_addr":"172.21.15.53:6813/3878043918","cluster_addr":"172.21.15.53:6814/3878043918","heartbeat_back_addr":"172.21.15.53:6816/3878043918","heartbeat_front_addr":"172.21.15.53:6815/3878043918","state":["exists","up"]},{"osd":4,"uuid":"773e8080-1028-48fe-8f6e-ed6beca60211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2373040976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6801","nonce":2373040976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":2373040976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2373040976}]},"public_addr":"172.21.15.64:6800/2373040976","cluster_addr":"172.21.15.64:6801/2373040976","heartbeat_back_addr":"172.21.15.64:6803/2373040976","heartbeat_front_addr":"172.21.15.64:6802/2373040976","state":["exists","up"]},{"osd":5,"uuid":"d79dbfb8-8f92-4d65-92ab-4b00aaccdb07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2413553331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6805","nonce":2413553331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":2413553331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2413553331}]},"public_addr":"172.21.15.64:6804/2413553331","cluster_addr":"172.21.15.64:6805/2413553331","heartbeat_back_addr":"172.21.15.64:6807/2413553331","heartbeat_front_addr":"172.21.15.64:6806/2413553331","state":["exists","up"]},{"osd":6,"uuid":"ebbb9499-df1e-42b7-89bb-df6c5935f39e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3043274732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6809","nonce":3043274732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":3043274732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3043274732}]},"public_addr":"172.21.15.64:6808/3043274732","cluster_addr":"172.21.15.64:6809/3043274732","heartbeat_back_addr":"172.21.15.64:6811/3043274732","heartbeat_front_addr":"172.21.15.64:6810/3043274732","state":["exists","up"]},{"osd":7,"uuid":"cde36201-cb77-4967-9402-6fdbe633b40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2605082260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6813","nonce":2605082260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6815","nonce":2605082260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2605082260}]},"public_addr":"172.21.15.64:6812/2605082260","cluster_addr":"172.21.15.64:6813/2605082260","heartbeat_back_addr":"172.21.15.64:6815/2605082260","heartbeat_front_addr":"172.21.15.64:6814/2605082260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:36.480557+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:45.908357+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:25.314573+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:45.593652+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:51.402086+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:29.213021+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:27.035272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:51.540837+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/4161439976":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/2578736337":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/762516580":"2024-02-23T19:23:54.856898+0000","172.21.15.53:6800/3676085495":"2024-02-23T19:29:23.167786+0000","172.21.15.53:6800/3516945073":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/828421682":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/4127851809":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/1906929179":"2024-02-23T19:23:32.485117+0000","172.21.15.53:0/2640420245":"2024-02-23T19:23:32.485117+0000","172.21.15.53:6800/2854906057":"2024-02-23T19:23:32.485117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T19:29:36.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:35.182220+0000 mgr.y (mgr.14535) 18 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:36.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:36.105789+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.53:0/2865779869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:29:36.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:36.589609+0000 mon.a (mon.0) 667 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-22T19:29:36.875 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:35.182220+0000 mgr.y (mgr.14535) 18 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:36.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:36.105789+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.53:0/2865779869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:29:36.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:36.589609+0000 mon.a (mon.0) 667 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-22T19:29:36.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:35.182220+0000 mgr.y (mgr.14535) 18 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:36.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:36.105789+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.53:0/2865779869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:29:36.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:36.589609+0000 mon.a (mon.0) 667 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-22T19:29:36.897 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}] 2024-02-22T19:29:36.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-22T19:29:37.618 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:37.182736+0000 mgr.y (mgr.14535) 19 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 674 B/s rd, 0 op/s 2024-02-22T19:29:38.858 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:37.182736+0000 mgr.y (mgr.14535) 19 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 674 B/s rd, 0 op/s 2024-02-22T19:29:38.858 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:37.182736+0000 mgr.y (mgr.14535) 19 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 674 B/s rd, 0 op/s 2024-02-22T19:29:39.254 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:29:38 smithi053 bash[96090]: 441bc77de7e0688f3d594af6abae595b7a46fc494e6c8b3cf4a92a1de1d06b80 2024-02-22T19:29:39.254 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:29:38 smithi053 systemd[1]: Started Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:29:40.233 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:39.002681+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:40.233 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:39.005286+0000 mgr.y (mgr.14535) 20 : cephadm [INF] Deploying daemon node-exporter.a on smithi053 2024-02-22T19:29:40.233 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:39.183236+0000 mgr.y (mgr.14535) 21 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:40.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:39.002681+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:40.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:39.005286+0000 mgr.y (mgr.14535) 20 : cephadm [INF] Deploying daemon node-exporter.a on smithi053 2024-02-22T19:29:40.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:39.183236+0000 mgr.y (mgr.14535) 21 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:40.268 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-02-22T19:29:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:39.002681+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:39.005286+0000 mgr.y (mgr.14535) 20 : cephadm [INF] Deploying daemon node-exporter.a on smithi053 2024-02-22T19:29:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:39.183236+0000 mgr.y (mgr.14535) 21 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:40.504 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:40 smithi053 systemd[1]: Starting Ceph node-exporter.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:29:40.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool get iscsi pg_num 2024-02-22T19:29:41.230 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:41.254 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:41 smithi053 bash[96449]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-02-22T19:29:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:39.375578+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:39.375641+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:40.010456+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-22T19:29:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:40.268674+0000 mon.a (mon.0) 670 : audit [DBG] from='client.? 172.21.15.53:0/661674922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-22T19:29:41.255 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:29:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[96125]: level=info ts=2024-02-22T19:29:40.861Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000418597s 2024-02-22T19:29:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:39.375578+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:39.375641+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:40.010456+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-22T19:29:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:40.268674+0000 mon.a (mon.0) 670 : audit [DBG] from='client.? 172.21.15.53:0/661674922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-22T19:29:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:39.375578+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:39.375641+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:40.010456+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-22T19:29:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:40.268674+0000 mon.a (mon.0) 670 : audit [DBG] from='client.? 172.21.15.53:0/661674922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-22T19:29:42.255 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:41 smithi053 bash[96449]: Getting image source signatures 2024-02-22T19:29:42.255 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:41 smithi053 bash[96449]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-02-22T19:29:42.256 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:41 smithi053 bash[96449]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-02-22T19:29:42.256 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:41 smithi053 bash[96449]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-02-22T19:29:42.742 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:41.183557+0000 mgr.y (mgr.14535) 22 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:42.742 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:41.590968+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-22T19:29:42.742 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:41.183557+0000 mgr.y (mgr.14535) 22 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:42.742 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:41.590968+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-22T19:29:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:41.183557+0000 mgr.y (mgr.14535) 22 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:41.590968+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-22T19:29:43.201 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 3 2024-02-22T19:29:43.504 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:43 smithi053 bash[96449]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-02-22T19:29:43.504 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:43 smithi053 bash[96449]: Writing manifest to image destination 2024-02-22T19:29:43.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:43.201543+0000 mon.a (mon.0) 672 : audit [DBG] from='client.? 172.21.15.53:0/3630946574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-22T19:29:43.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:43.319172+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:44.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:43.201543+0000 mon.a (mon.0) 672 : audit [DBG] from='client.? 172.21.15.53:0/3630946574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-22T19:29:44.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:43.319172+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:43.201543+0000 mon.a (mon.0) 672 : audit [DBG] from='client.? 172.21.15.53:0/3630946574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-22T19:29:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:43.319172+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:44.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool get .rgw.root pg_num 2024-02-22T19:29:44.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:29:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:44.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:43.183843+0000 mgr.y (mgr.14535) 23 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:29:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:43.183843+0000 mgr.y (mgr.14535) 23 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.490Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.490Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.490Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.490Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=arp 2024-02-22T19:29:44.755 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=edac 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-22T19:29:44.756 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-22T19:29:44.757 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=os 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=stat 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=time 2024-02-22T19:29:44.758 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=uname 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.491Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a[96695]: ts=2024-02-22T19:29:44.492Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 bash[96449]: 90d4dd653a2d9019c5a33ab29ca8cb6204557a11f2e8be1110debc4915dd016e 2024-02-22T19:29:44.759 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:29:44 smithi053 systemd[1]: Started Ceph node-exporter.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:29:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:43.183843+0000 mgr.y (mgr.14535) 23 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:29:45.185 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:45 smithi064 systemd[1]: Starting Ceph node-exporter.b for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:29:45.602 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:45 smithi064 bash[62982]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-02-22T19:29:45.777 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:44.202722+0000 mgr.y (mgr.14535) 24 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:45.777 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:44.638591+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:45.777 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:44.639583+0000 mgr.y (mgr.14535) 25 : cephadm [INF] Deploying daemon node-exporter.b on smithi064 2024-02-22T19:29:45.777 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:44.202722+0000 mgr.y (mgr.14535) 24 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:45.777 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:44.638591+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:45.778 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:44.639583+0000 mgr.y (mgr.14535) 25 : cephadm [INF] Deploying daemon node-exporter.b on smithi064 2024-02-22T19:29:45.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:44.202722+0000 mgr.y (mgr.14535) 24 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:45.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:44.638591+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:45.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:44.639583+0000 mgr.y (mgr.14535) 25 : cephadm [INF] Deploying daemon node-exporter.b on smithi064 2024-02-22T19:29:46.352 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:45 smithi064 bash[62982]: Getting image source signatures 2024-02-22T19:29:46.352 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:45 smithi064 bash[62982]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-02-22T19:29:46.352 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:45 smithi064 bash[62982]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-02-22T19:29:46.352 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:45 smithi064 bash[62982]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-02-22T19:29:46.444 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-02-22T19:29:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:45.184529+0000 mgr.y (mgr.14535) 26 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:46.029883+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:46.029938+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:46.444174+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.53:0/2207621485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-22T19:29:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:45.184529+0000 mgr.y (mgr.14535) 26 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:46.029883+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:46.029938+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:46.444174+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.53:0/2207621485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-22T19:29:47.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool get default.rgw.log pg_num 2024-02-22T19:29:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:45.184529+0000 mgr.y (mgr.14535) 26 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:46.029883+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:46.029938+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:46.444174+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.53:0/2207621485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-22T19:29:47.310 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:47.602 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:47 smithi064 bash[62982]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-02-22T19:29:47.602 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:47 smithi064 bash[62982]: Writing manifest to image destination 2024-02-22T19:29:47.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:46.647342+0000 mon.a (mon.0) 675 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-22T19:29:47.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:46.647342+0000 mon.a (mon.0) 675 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-22T19:29:48.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:46.647342+0000 mon.a (mon.0) 675 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-22T19:29:48.352 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.051Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.051Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=arp 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-22T19:29:48.353 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=edac 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-22T19:29:48.354 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=os 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=stat 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-22T19:29:48.355 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=time 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=uname 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.052Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b[63030]: ts=2024-02-22T19:29:48.053Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 bash[62982]: 0e2288f413a239a76480963f1dff9077ea5cb4043777c30de1033b217fc9bf7e 2024-02-22T19:29:48.356 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:29:48 smithi064 systemd[1]: Started Ceph node-exporter.b for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:29:48.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:47.184883+0000 mgr.y (mgr.14535) 27 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 713 B/s rd, 0 op/s 2024-02-22T19:29:48.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:48.211336+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:48.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:48.324216+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:48.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:47.184883+0000 mgr.y (mgr.14535) 27 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 713 B/s rd, 0 op/s 2024-02-22T19:29:48.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:48.211336+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:48.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:48.324216+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:49.100 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-02-22T19:29:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:47.184883+0000 mgr.y (mgr.14535) 27 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 713 B/s rd, 0 op/s 2024-02-22T19:29:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:48.211336+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:48.324216+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:29:49.130 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:29:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[96125]: level=info ts=2024-02-22T19:29:48.863Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.002788756s 2024-02-22T19:29:49.741 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:29:48.221272+0000 mgr.y (mgr.14535) 28 : cephadm [INF] Deploying daemon prometheus.a on smithi064 2024-02-22T19:29:49.741 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:49.100191+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.53:0/3903328370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-22T19:29:49.741 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:29:48.221272+0000 mgr.y (mgr.14535) 28 : cephadm [INF] Deploying daemon prometheus.a on smithi064 2024-02-22T19:29:49.742 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:49.100191+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.53:0/3903328370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-22T19:29:49.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool get default.rgw.control pg_num 2024-02-22T19:29:50.043 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:29:48.221272+0000 mgr.y (mgr.14535) 28 : cephadm [INF] Deploying daemon prometheus.a on smithi064 2024-02-22T19:29:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:49.100191+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.53:0/3903328370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-22T19:29:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:49.185422+0000 mgr.y (mgr.14535) 29 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:50.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:49.185422+0000 mgr.y (mgr.14535) 29 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:49.185422+0000 mgr.y (mgr.14535) 29 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:51.968 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-02-22T19:29:52.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool get default.rgw.meta pg_num 2024-02-22T19:29:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:51.185922+0000 mgr.y (mgr.14535) 30 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:29:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:51.594312+0000 mon.a (mon.0) 678 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-22T19:29:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:51.968660+0000 mon.a (mon.0) 679 : audit [DBG] from='client.? 172.21.15.53:0/3326791589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-22T19:29:52.928 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:51.185922+0000 mgr.y (mgr.14535) 30 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:29:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:51.594312+0000 mon.a (mon.0) 678 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-22T19:29:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:51.968660+0000 mon.a (mon.0) 679 : audit [DBG] from='client.? 172.21.15.53:0/3326791589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-22T19:29:53.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:51.185922+0000 mgr.y (mgr.14535) 30 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:29:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:51.594312+0000 mon.a (mon.0) 678 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-22T19:29:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:51.968660+0000 mon.a (mon.0) 679 : audit [DBG] from='client.? 172.21.15.53:0/3326791589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-22T19:29:54.595 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:29:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:29:54.753 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-02-22T19:29:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:53.186278+0000 mgr.y (mgr.14535) 31 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:29:55.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:53.186278+0000 mgr.y (mgr.14535) 31 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:29:55.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:53.186278+0000 mgr.y (mgr.14535) 31 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:29:55.400 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-22T19:29:55.400 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-22T19:29:55.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph mgr dump --format=json 2024-02-22T19:29:55.682 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:54.213369+0000 mgr.y (mgr.14535) 32 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:54.753024+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.53:0/844307662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-22T19:29:55.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:54.213369+0000 mgr.y (mgr.14535) 32 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:55.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:54.753024+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.53:0/844307662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-22T19:29:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:54.213369+0000 mgr.y (mgr.14535) 32 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:29:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:54.753024+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.53:0/844307662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-22T19:29:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:55.186942+0000 mgr.y (mgr.14535) 33 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:55.585597+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:55.585650+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:55.602355+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-22T19:29:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:56.594961+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-22T19:29:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:55.186942+0000 mgr.y (mgr.14535) 33 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:55.585597+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:55.585650+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:55.602355+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-22T19:29:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:56.594961+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-22T19:29:57.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:55.186942+0000 mgr.y (mgr.14535) 33 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:29:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:55.585597+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:55.585650+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:55.602355+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-22T19:29:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:56.594961+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-22T19:29:57.575 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:29:57.906 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:29:57.570182+0000 mon.a (mon.0) 683 : audit [DBG] from='client.? 172.21.15.53:0/3203923945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-22T19:29:57.920 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:29:57.570182+0000 mon.a (mon.0) 683 : audit [DBG] from='client.? 172.21.15.53:0/3203923945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-22T19:29:57.921 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:29:57.570182+0000 mon.a (mon.0) 683 : audit [DBG] from='client.? 172.21.15.53:0/3203923945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-22T19:29:58.206 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":19,"active_gid":14535,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3367299694}]},"active_addr":"172.21.15.53:6800/3367299694","active_change":"2024-02-22T19:29:23.168000+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":24443,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.53:8443/","prometheus":"http://172.21.15.53:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":65,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":3677601329}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":3907908915}]}]} 2024-02-22T19:29:58.210 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-22T19:29:58.210 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-22T19:29:58.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T19:29:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:29:58.964 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:57.187387+0000 mgr.y (mgr.14535) 34 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:29:58.964 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:57.555352+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:58.964 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:57.555396+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:58.964 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:57.607695+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-22T19:29:58.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:57.187387+0000 mgr.y (mgr.14535) 34 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:29:58.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:57.555352+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:58.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:57.555396+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:58.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:29:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:57.607695+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-22T19:29:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:57.187387+0000 mgr.y (mgr.14535) 34 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:29:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:57.555352+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:29:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:57.555396+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:29:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:29:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:57.607695+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-22T19:30:00.301 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:30:00.302 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":77,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","created":"2024-02-22T19:22:59.746063+0000","modified":"2024-02-22T19:29:57.600817+0000","last_up_change":"2024-02-22T19:28:53.274045+0000","last_in_change":"2024-02-22T19:28:36.235896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-22T19:26:54.342689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-22T19:28:59.555543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-22T19:28:59.986190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-22T19:29:01.494051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-22T19:29:03.518927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-22T19:29:05.596341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"4326a76a-6144-4859-9b40-5ff69edbbcab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6801","nonce":162271081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":162271081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":162271081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":162271081}]},"public_addr":"172.21.15.53:6801/162271081","cluster_addr":"172.21.15.53:6802/162271081","heartbeat_back_addr":"172.21.15.53:6804/162271081","heartbeat_front_addr":"172.21.15.53:6803/162271081","state":["exists","up"]},{"osd":1,"uuid":"fc056bcd-1a3b-408c-b9d1-4992091df6c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6805","nonce":2973403044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2973403044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2973403044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2973403044}]},"public_addr":"172.21.15.53:6805/2973403044","cluster_addr":"172.21.15.53:6806/2973403044","heartbeat_back_addr":"172.21.15.53:6808/2973403044","heartbeat_front_addr":"172.21.15.53:6807/2973403044","state":["exists","up"]},{"osd":2,"uuid":"61f3f980-82cd-4e8e-9a3f-6d3f4559cc83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6809","nonce":936980935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":936980935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":936980935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":936980935}]},"public_addr":"172.21.15.53:6809/936980935","cluster_addr":"172.21.15.53:6810/936980935","heartbeat_back_addr":"172.21.15.53:6812/936980935","heartbeat_front_addr":"172.21.15.53:6811/936980935","state":["exists","up"]},{"osd":3,"uuid":"3f285320-aa28-4fc1-b1c7-456c9aecd993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":3878043918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3878043918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3878043918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6815","nonce":3878043918}]},"public_addr":"172.21.15.53:6813/3878043918","cluster_addr":"172.21.15.53:6814/3878043918","heartbeat_back_addr":"172.21.15.53:6816/3878043918","heartbeat_front_addr":"172.21.15.53:6815/3878043918","state":["exists","up"]},{"osd":4,"uuid":"773e8080-1028-48fe-8f6e-ed6beca60211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2373040976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6801","nonce":2373040976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":2373040976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2373040976}]},"public_addr":"172.21.15.64:6800/2373040976","cluster_addr":"172.21.15.64:6801/2373040976","heartbeat_back_addr":"172.21.15.64:6803/2373040976","heartbeat_front_addr":"172.21.15.64:6802/2373040976","state":["exists","up"]},{"osd":5,"uuid":"d79dbfb8-8f92-4d65-92ab-4b00aaccdb07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2413553331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6805","nonce":2413553331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":2413553331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2413553331}]},"public_addr":"172.21.15.64:6804/2413553331","cluster_addr":"172.21.15.64:6805/2413553331","heartbeat_back_addr":"172.21.15.64:6807/2413553331","heartbeat_front_addr":"172.21.15.64:6806/2413553331","state":["exists","up"]},{"osd":6,"uuid":"ebbb9499-df1e-42b7-89bb-df6c5935f39e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3043274732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6809","nonce":3043274732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":3043274732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3043274732}]},"public_addr":"172.21.15.64:6808/3043274732","cluster_addr":"172.21.15.64:6809/3043274732","heartbeat_back_addr":"172.21.15.64:6811/3043274732","heartbeat_front_addr":"172.21.15.64:6810/3043274732","state":["exists","up"]},{"osd":7,"uuid":"cde36201-cb77-4967-9402-6fdbe633b40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2605082260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6813","nonce":2605082260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6815","nonce":2605082260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2605082260}]},"public_addr":"172.21.15.64:6812/2605082260","cluster_addr":"172.21.15.64:6813/2605082260","heartbeat_back_addr":"172.21.15.64:6815/2605082260","heartbeat_front_addr":"172.21.15.64:6814/2605082260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:46.029940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:57.555399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:25.314573+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:45.593652+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:51.402086+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:29.213021+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:39.375643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:55.585653+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/4161439976":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/2578736337":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/762516580":"2024-02-23T19:23:54.856898+0000","172.21.15.53:6800/3676085495":"2024-02-23T19:29:23.167786+0000","172.21.15.53:6800/3516945073":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/828421682":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/4127851809":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/1906929179":"2024-02-23T19:23:32.485117+0000","172.21.15.53:0/2640420245":"2024-02-23T19:23:32.485117+0000","172.21.15.53:6800/2854906057":"2024-02-23T19:23:32.485117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T19:30:00.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:29:59.188172+0000 mgr.y (mgr.14535) 35 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:30:00.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:00.000129+0000 mon.a (mon.0) 685 : cluster [INF] overall HEALTH_OK 2024-02-22T19:30:00.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:00.300255+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.53:0/3601866245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:30:00.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:29:59.188172+0000 mgr.y (mgr.14535) 35 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:30:00.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:00.000129+0000 mon.a (mon.0) 685 : cluster [INF] overall HEALTH_OK 2024-02-22T19:30:00.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:00.300255+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.53:0/3601866245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:30:00.990 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:00 smithi064 systemd[1]: Starting Ceph prometheus.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:30:00.990 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:29:59.188172+0000 mgr.y (mgr.14535) 35 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:30:00.990 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:00.000129+0000 mon.a (mon.0) 685 : cluster [INF] overall HEALTH_OK 2024-02-22T19:30:00.991 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:00.300255+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.53:0/3601866245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:30:01.003 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-22T19:30:01.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T19:30:01.289 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:01.602 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.174Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.174Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.174Z caller=main.go:518 level=info host_details="(Linux 4.18.0-540.el8.x86_64 #1 SMP Fri Feb 9 14:58:00 UTC 2024 x86_64 smithi064 (none))" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.174Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.174Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.176Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.176Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.177Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.178Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.178Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.716µs 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.178Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.178Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-22T19:30:01.603 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.178Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.81µs wal_replay_duration=175.926µs total_replay_duration=211.41µs 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.179Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.179Z caller=main.go:947 level=info msg="TSDB started" 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.179Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.197Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.788357ms db_storage=699ns remote_storage=953ns web_handler=292ns query_engine=443ns scrape=488.413µs scrape_sd=20.233µs notify=17.476µs notify_sd=5.455µs rules=17.012043ms 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:30:01.197Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 bash[63505]: 02669d4cb2d72b5af141c592dc0b88ea514ecc7469ba439a1e4ea71e6762b57e 2024-02-22T19:30:01.604 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:30:01 smithi064 systemd[1]: Started Ceph prometheus.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:30:01.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:01] ENGINE Bus STOPPING 2024-02-22T19:30:01.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:01] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:30:01.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:01] ENGINE Bus STOPPED 2024-02-22T19:30:01.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:01] ENGINE Bus STARTING 2024-02-22T19:30:01.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:01] ENGINE Serving on http://:::9283 2024-02-22T19:30:01.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:01] ENGINE Bus STARTED 2024-02-22T19:30:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:00.633194+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1b deep-scrub starts 2024-02-22T19:30:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:00.634046+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1b deep-scrub ok 2024-02-22T19:30:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:01.188524+0000 mgr.y (mgr.14535) 36 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:01.277236+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:01.391233+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:01.393876+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:01.396200+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:01.396561+0000 mgr.y (mgr.14535) 37 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:01.400024+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:30:01.410187+0000 mgr.y (mgr.14535) 38 : cephadm [INF] Deploying daemon grafana.a on smithi064 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:01.596053+0000 mon.a (mon.0) 691 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:00.633194+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1b deep-scrub starts 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:00.634046+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1b deep-scrub ok 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:01.188524+0000 mgr.y (mgr.14535) 36 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:01.277236+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:01.391233+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:01.393876+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:01.396200+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:01.396561+0000 mgr.y (mgr.14535) 37 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:01.400024+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:30:01.410187+0000 mgr.y (mgr.14535) 38 : cephadm [INF] Deploying daemon grafana.a on smithi064 2024-02-22T19:30:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:01.596053+0000 mon.a (mon.0) 691 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-22T19:30:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:00.633194+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1b deep-scrub starts 2024-02-22T19:30:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:00.634046+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1b deep-scrub ok 2024-02-22T19:30:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:01.188524+0000 mgr.y (mgr.14535) 36 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:01.277236+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:01.391233+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:01.393876+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:01.396200+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:01.396561+0000 mgr.y (mgr.14535) 37 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:01.400024+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:30:01.410187+0000 mgr.y (mgr.14535) 38 : cephadm [INF] Deploying daemon grafana.a on smithi064 2024-02-22T19:30:02.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:01.596053+0000 mon.a (mon.0) 691 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-22T19:30:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:30:03.252 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","created":"2024-02-22T19:22:59.746063+0000","modified":"2024-02-22T19:30:01.592035+0000","last_up_change":"2024-02-22T19:28:53.274045+0000","last_in_change":"2024-02-22T19:28:36.235896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-22T19:26:54.342689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-22T19:28:59.555543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-22T19:28:59.986190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-22T19:29:01.494051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-22T19:29:03.518927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-22T19:29:05.596341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"4326a76a-6144-4859-9b40-5ff69edbbcab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6801","nonce":162271081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":162271081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":162271081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":162271081}]},"public_addr":"172.21.15.53:6801/162271081","cluster_addr":"172.21.15.53:6802/162271081","heartbeat_back_addr":"172.21.15.53:6804/162271081","heartbeat_front_addr":"172.21.15.53:6803/162271081","state":["exists","up"]},{"osd":1,"uuid":"fc056bcd-1a3b-408c-b9d1-4992091df6c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6805","nonce":2973403044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2973403044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2973403044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2973403044}]},"public_addr":"172.21.15.53:6805/2973403044","cluster_addr":"172.21.15.53:6806/2973403044","heartbeat_back_addr":"172.21.15.53:6808/2973403044","heartbeat_front_addr":"172.21.15.53:6807/2973403044","state":["exists","up"]},{"osd":2,"uuid":"61f3f980-82cd-4e8e-9a3f-6d3f4559cc83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6809","nonce":936980935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":936980935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":936980935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":936980935}]},"public_addr":"172.21.15.53:6809/936980935","cluster_addr":"172.21.15.53:6810/936980935","heartbeat_back_addr":"172.21.15.53:6812/936980935","heartbeat_front_addr":"172.21.15.53:6811/936980935","state":["exists","up"]},{"osd":3,"uuid":"3f285320-aa28-4fc1-b1c7-456c9aecd993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":3878043918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3878043918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3878043918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6815","nonce":3878043918}]},"public_addr":"172.21.15.53:6813/3878043918","cluster_addr":"172.21.15.53:6814/3878043918","heartbeat_back_addr":"172.21.15.53:6816/3878043918","heartbeat_front_addr":"172.21.15.53:6815/3878043918","state":["exists","up"]},{"osd":4,"uuid":"773e8080-1028-48fe-8f6e-ed6beca60211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2373040976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6801","nonce":2373040976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":2373040976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2373040976}]},"public_addr":"172.21.15.64:6800/2373040976","cluster_addr":"172.21.15.64:6801/2373040976","heartbeat_back_addr":"172.21.15.64:6803/2373040976","heartbeat_front_addr":"172.21.15.64:6802/2373040976","state":["exists","up"]},{"osd":5,"uuid":"d79dbfb8-8f92-4d65-92ab-4b00aaccdb07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2413553331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6805","nonce":2413553331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":2413553331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2413553331}]},"public_addr":"172.21.15.64:6804/2413553331","cluster_addr":"172.21.15.64:6805/2413553331","heartbeat_back_addr":"172.21.15.64:6807/2413553331","heartbeat_front_addr":"172.21.15.64:6806/2413553331","state":["exists","up"]},{"osd":6,"uuid":"ebbb9499-df1e-42b7-89bb-df6c5935f39e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3043274732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6809","nonce":3043274732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":3043274732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3043274732}]},"public_addr":"172.21.15.64:6808/3043274732","cluster_addr":"172.21.15.64:6809/3043274732","heartbeat_back_addr":"172.21.15.64:6811/3043274732","heartbeat_front_addr":"172.21.15.64:6810/3043274732","state":["exists","up"]},{"osd":7,"uuid":"cde36201-cb77-4967-9402-6fdbe633b40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2605082260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6813","nonce":2605082260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6815","nonce":2605082260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2605082260}]},"public_addr":"172.21.15.64:6812/2605082260","cluster_addr":"172.21.15.64:6813/2605082260","heartbeat_back_addr":"172.21.15.64:6815/2605082260","heartbeat_front_addr":"172.21.15.64:6814/2605082260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:46.029940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:57.555399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:25.314573+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:45.593652+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:28:51.402086+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:29.213021+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:39.375643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:29:55.585653+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/4161439976":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/2578736337":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/762516580":"2024-02-23T19:23:54.856898+0000","172.21.15.53:6800/3676085495":"2024-02-23T19:29:23.167786+0000","172.21.15.53:6800/3516945073":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/828421682":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/4127851809":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/1906929179":"2024-02-23T19:23:32.485117+0000","172.21.15.53:0/2640420245":"2024-02-23T19:23:32.485117+0000","172.21.15.53:6800/2854906057":"2024-02-23T19:23:32.485117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T19:30:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:03.250702+0000 mon.a (mon.0) 692 : audit [DBG] from='client.? 172.21.15.53:0/531751900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:30:03.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:03.250702+0000 mon.a (mon.0) 692 : audit [DBG] from='client.? 172.21.15.53:0/531751900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:30:03.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:03.250702+0000 mon.a (mon.0) 692 : audit [DBG] from='client.? 172.21.15.53:0/531751900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:30:03.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-22T19:30:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:04.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:30:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:30:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.188885+0000 mgr.y (mgr.14535) 39 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:30:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:03.332772+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.439342+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.439416+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:04.665 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.188885+0000 mgr.y (mgr.14535) 39 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:03.332772+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.439342+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.439416+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.188885+0000 mgr.y (mgr.14535) 39 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 776 B/s rd, 0 op/s 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:03.332772+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.439342+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:04.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.439416+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.650452+0000 osd.0 (osd.0) 11 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:30:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.653345+0000 osd.0 (osd.0) 12 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:30:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.685135+0000 osd.7 (osd.7) 7 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:30:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:03.688020+0000 osd.7 (osd.7) 8 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:30:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:04.223416+0000 mgr.y (mgr.14535) 40 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:04.338506+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-22T19:30:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:04.604921+0000 osd.6 (osd.6) 5 : cluster [DBG] 3.a scrub starts 2024-02-22T19:30:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:04.608329+0000 osd.6 (osd.6) 6 : cluster [DBG] 3.a scrub ok 2024-02-22T19:30:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.650452+0000 osd.0 (osd.0) 11 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:30:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.653345+0000 osd.0 (osd.0) 12 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:30:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.685135+0000 osd.7 (osd.7) 7 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:30:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:03.688020+0000 osd.7 (osd.7) 8 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:30:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:04.223416+0000 mgr.y (mgr.14535) 40 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:04.338506+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:04.604921+0000 osd.6 (osd.6) 5 : cluster [DBG] 3.a scrub starts 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:04.608329+0000 osd.6 (osd.6) 6 : cluster [DBG] 3.a scrub ok 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.650452+0000 osd.0 (osd.0) 11 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.653345+0000 osd.0 (osd.0) 12 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.685135+0000 osd.7 (osd.7) 7 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:03.688020+0000 osd.7 (osd.7) 8 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:04.223416+0000 mgr.y (mgr.14535) 40 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:04.338506+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:04.604921+0000 osd.6 (osd.6) 5 : cluster [DBG] 3.a scrub starts 2024-02-22T19:30:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:04.608329+0000 osd.6 (osd.6) 6 : cluster [DBG] 3.a scrub ok 2024-02-22T19:30:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:05.189241+0000 mgr.y (mgr.14535) 41 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:30:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:05.471414+0000 osd.5 (osd.5) 5 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:30:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:05.473335+0000 osd.5 (osd.5) 6 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:30:06.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:05.189241+0000 mgr.y (mgr.14535) 41 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:30:06.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:05.471414+0000 osd.5 (osd.5) 5 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:30:06.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:05.473335+0000 osd.5 (osd.5) 6 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:30:06.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:05.189241+0000 mgr.y (mgr.14535) 41 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:30:06.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:05.471414+0000 osd.5 (osd.5) 5 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:30:06.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:05.473335+0000 osd.5 (osd.5) 6 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:30:06.922 INFO:teuthology.orchestra.run.smithi053.stdout:34359738419 2024-02-22T19:30:06.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-22T19:30:07.206 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:05.719593+0000 osd.3 (osd.3) 7 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:30:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:05.723053+0000 osd.3 (osd.3) 8 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:30:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.597581+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-22T19:30:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.605691+0000 osd.6 (osd.6) 7 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:30:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.607005+0000 osd.6 (osd.6) 8 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:30:07.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:05.719593+0000 osd.3 (osd.3) 7 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:30:07.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:05.723053+0000 osd.3 (osd.3) 8 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:30:07.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.597581+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.605691+0000 osd.6 (osd.6) 7 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.607005+0000 osd.6 (osd.6) 8 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:05.719593+0000 osd.3 (osd.3) 7 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:05.723053+0000 osd.3 (osd.3) 8 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.597581+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.605691+0000 osd.6 (osd.6) 7 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:30:07.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.607005+0000 osd.6 (osd.6) 8 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:30:08.392 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.717510+0000 osd.3 (osd.3) 9 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:30:08.392 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.720448+0000 osd.3 (osd.3) 10 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:30:08.392 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.955364+0000 osd.4 (osd.4) 5 : cluster [DBG] 4.e scrub starts 2024-02-22T19:30:08.392 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.957228+0000 osd.4 (osd.4) 6 : cluster [DBG] 4.e scrub ok 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.975196+0000 osd.1 (osd.1) 11 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:06.976103+0000 osd.1 (osd.1) 12 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.189596+0000 mgr.y (mgr.14535) 42 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.462308+0000 osd.5 (osd.5) 7 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.463119+0000 osd.5 (osd.5) 8 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.717510+0000 osd.3 (osd.3) 9 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.720448+0000 osd.3 (osd.3) 10 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.955364+0000 osd.4 (osd.4) 5 : cluster [DBG] 4.e scrub starts 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.957228+0000 osd.4 (osd.4) 6 : cluster [DBG] 4.e scrub ok 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.975196+0000 osd.1 (osd.1) 11 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:06.976103+0000 osd.1 (osd.1) 12 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.189596+0000 mgr.y (mgr.14535) 42 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:08.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.462308+0000 osd.5 (osd.5) 7 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:30:08.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.463119+0000 osd.5 (osd.5) 8 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.717510+0000 osd.3 (osd.3) 9 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.720448+0000 osd.3 (osd.3) 10 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.955364+0000 osd.4 (osd.4) 5 : cluster [DBG] 4.e scrub starts 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.957228+0000 osd.4 (osd.4) 6 : cluster [DBG] 4.e scrub ok 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.975196+0000 osd.1 (osd.1) 11 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:06.976103+0000 osd.1 (osd.1) 12 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.189596+0000 mgr.y (mgr.14535) 42 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.462308+0000 osd.5 (osd.5) 7 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:30:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.463119+0000 osd.5 (osd.5) 8 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:30:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.663668+0000 osd.0 (osd.0) 13 : cluster [DBG] 3.1f deep-scrub starts 2024-02-22T19:30:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.664418+0000 osd.0 (osd.0) 14 : cluster [DBG] 3.1f deep-scrub ok 2024-02-22T19:30:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.990608+0000 osd.1 (osd.1) 13 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:30:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:07.992667+0000 osd.1 (osd.1) 14 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:30:09.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.663668+0000 osd.0 (osd.0) 13 : cluster [DBG] 3.1f deep-scrub starts 2024-02-22T19:30:09.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.664418+0000 osd.0 (osd.0) 14 : cluster [DBG] 3.1f deep-scrub ok 2024-02-22T19:30:09.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.990608+0000 osd.1 (osd.1) 13 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:30:09.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:07.992667+0000 osd.1 (osd.1) 14 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:30:09.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.663668+0000 osd.0 (osd.0) 13 : cluster [DBG] 3.1f deep-scrub starts 2024-02-22T19:30:09.609 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.664418+0000 osd.0 (osd.0) 14 : cluster [DBG] 3.1f deep-scrub ok 2024-02-22T19:30:09.609 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.990608+0000 osd.1 (osd.1) 13 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:30:09.609 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:07.992667+0000 osd.1 (osd.1) 14 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:30:09.609 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:09] "GET /metrics HTTP/1.1" 200 172070 "" "Prometheus/2.33.4" 2024-02-22T19:30:09.628 INFO:teuthology.orchestra.run.smithi053.stdout:55834574894 2024-02-22T19:30:09.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-22T19:30:09.905 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:08.670357+0000 osd.0 (osd.0) 15 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:08.671232+0000 osd.0 (osd.0) 16 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:09.190446+0000 mgr.y (mgr.14535) 43 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:09.474613+0000 osd.5 (osd.5) 9 : cluster [DBG] 2.2 deep-scrub starts 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:09.477483+0000 osd.5 (osd.5) 10 : cluster [DBG] 2.2 deep-scrub ok 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:08.670357+0000 osd.0 (osd.0) 15 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:08.671232+0000 osd.0 (osd.0) 16 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:30:10.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:09.190446+0000 mgr.y (mgr.14535) 43 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:30:10.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:09.474613+0000 osd.5 (osd.5) 9 : cluster [DBG] 2.2 deep-scrub starts 2024-02-22T19:30:10.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:09.477483+0000 osd.5 (osd.5) 10 : cluster [DBG] 2.2 deep-scrub ok 2024-02-22T19:30:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:08.670357+0000 osd.0 (osd.0) 15 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:30:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:08.671232+0000 osd.0 (osd.0) 16 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:30:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:09.190446+0000 mgr.y (mgr.14535) 43 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:30:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:09.474613+0000 osd.5 (osd.5) 9 : cluster [DBG] 2.2 deep-scrub starts 2024-02-22T19:30:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:09.477483+0000 osd.5 (osd.5) 10 : cluster [DBG] 2.2 deep-scrub ok 2024-02-22T19:30:11.715 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:10.439219+0000 osd.5 (osd.5) 11 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:30:11.715 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:10.440055+0000 osd.5 (osd.5) 12 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:30:11.716 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:10.439219+0000 osd.5 (osd.5) 11 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:30:11.716 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:10.440055+0000 osd.5 (osd.5) 12 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:30:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:10.439219+0000 osd.5 (osd.5) 11 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:30:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:10.440055+0000 osd.5 (osd.5) 12 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:30:12.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:10.616794+0000 osd.7 (osd.7) 9 : cluster [DBG] 3.d scrub starts 2024-02-22T19:30:12.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:10.619680+0000 osd.7 (osd.7) 10 : cluster [DBG] 3.d scrub ok 2024-02-22T19:30:12.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:11.190989+0000 mgr.y (mgr.14535) 44 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:12.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:10.616794+0000 osd.7 (osd.7) 9 : cluster [DBG] 3.d scrub starts 2024-02-22T19:30:12.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:10.619680+0000 osd.7 (osd.7) 10 : cluster [DBG] 3.d scrub ok 2024-02-22T19:30:12.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:11.190989+0000 mgr.y (mgr.14535) 44 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:12.601 INFO:teuthology.orchestra.run.smithi053.stdout:77309411369 2024-02-22T19:30:12.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-22T19:30:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:10.616794+0000 osd.7 (osd.7) 9 : cluster [DBG] 3.d scrub starts 2024-02-22T19:30:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:10.619680+0000 osd.7 (osd.7) 10 : cluster [DBG] 3.d scrub ok 2024-02-22T19:30:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:11.190989+0000 mgr.y (mgr.14535) 44 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:13.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:11.659176+0000 osd.3 (osd.3) 11 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:30:13.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:11.662735+0000 osd.3 (osd.3) 12 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:30:13.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:12.057924+0000 osd.1 (osd.1) 15 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:30:13.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:12.059792+0000 osd.1 (osd.1) 16 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:30:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:11.659176+0000 osd.3 (osd.3) 11 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:30:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:11.662735+0000 osd.3 (osd.3) 12 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:30:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:12.057924+0000 osd.1 (osd.1) 15 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:30:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:12.059792+0000 osd.1 (osd.1) 16 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:30:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:11.659176+0000 osd.3 (osd.3) 11 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:30:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:11.662735+0000 osd.3 (osd.3) 12 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:30:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:12.057924+0000 osd.1 (osd.1) 15 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:30:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:12.059792+0000 osd.1 (osd.1) 16 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:30:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.175045+0000 osd.2 (osd.2) 7 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:30:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.177938+0000 osd.2 (osd.2) 8 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:30:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.191353+0000 mgr.y (mgr.14535) 45 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2024-02-22T19:30:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.175045+0000 osd.2 (osd.2) 7 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:30:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.177938+0000 osd.2 (osd.2) 8 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:30:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.191353+0000 mgr.y (mgr.14535) 45 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2024-02-22T19:30:14.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:30:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:30:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.175045+0000 osd.2 (osd.2) 7 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:30:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.177938+0000 osd.2 (osd.2) 8 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:30:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.191353+0000 mgr.y (mgr.14535) 45 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2024-02-22T19:30:15.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:30:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:30:15.166 INFO:teuthology.orchestra.run.smithi053.stdout:111669149732 2024-02-22T19:30:15.166 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-22T19:30:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:15.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.622221+0000 osd.7 (osd.7) 11 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.625115+0000 osd.7 (osd.7) 12 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.706811+0000 osd.3 (osd.3) 13 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.709687+0000 osd.3 (osd.3) 14 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.764544+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:13.766542+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.040248+0000 osd.1 (osd.1) 17 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.042054+0000 osd.1 (osd.1) 18 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.066010+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.069526+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:14.233409+0000 mgr.y (mgr.14535) 46 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.458968+0000 osd.5 (osd.5) 13 : cluster [DBG] 6.a scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.461853+0000 osd.5 (osd.5) 14 : cluster [DBG] 6.a scrub ok 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.655919+0000 osd.6 (osd.6) 9 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:30:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:14.658893+0000 osd.6 (osd.6) 10 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.622221+0000 osd.7 (osd.7) 11 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.625115+0000 osd.7 (osd.7) 12 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.706811+0000 osd.3 (osd.3) 13 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.709687+0000 osd.3 (osd.3) 14 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.764544+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:13.766542+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.040248+0000 osd.1 (osd.1) 17 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.042054+0000 osd.1 (osd.1) 18 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.066010+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.069526+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:14.233409+0000 mgr.y (mgr.14535) 46 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.458968+0000 osd.5 (osd.5) 13 : cluster [DBG] 6.a scrub starts 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.461853+0000 osd.5 (osd.5) 14 : cluster [DBG] 6.a scrub ok 2024-02-22T19:30:15.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.655919+0000 osd.6 (osd.6) 9 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:30:15.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:14.658893+0000 osd.6 (osd.6) 10 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:30:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.622221+0000 osd.7 (osd.7) 11 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:30:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.625115+0000 osd.7 (osd.7) 12 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.706811+0000 osd.3 (osd.3) 13 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.709687+0000 osd.3 (osd.3) 14 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.764544+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:13.766542+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.040248+0000 osd.1 (osd.1) 17 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.042054+0000 osd.1 (osd.1) 18 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.066010+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.069526+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:14.233409+0000 mgr.y (mgr.14535) 46 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.458968+0000 osd.5 (osd.5) 13 : cluster [DBG] 6.a scrub starts 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.461853+0000 osd.5 (osd.5) 14 : cluster [DBG] 6.a scrub ok 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.655919+0000 osd.6 (osd.6) 9 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:30:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:14.658893+0000 osd.6 (osd.6) 10 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:30:16.627 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:15.191958+0000 mgr.y (mgr.14535) 47 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:30:16.627 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:15.191958+0000 mgr.y (mgr.14535) 47 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:30:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:15.191958+0000 mgr.y (mgr.14535) 47 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:15.599171+0000 osd.7 (osd.7) 13 : cluster [DBG] 5.10 deep-scrub starts 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:15.602094+0000 osd.7 (osd.7) 14 : cluster [DBG] 5.10 deep-scrub ok 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.056498+0000 osd.4 (osd.4) 9 : cluster [DBG] 6.17 deep-scrub starts 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.057626+0000 osd.4 (osd.4) 10 : cluster [DBG] 6.17 deep-scrub ok 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.208303+0000 osd.4 (osd.4) 11 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.208364+0000 osd.4 (osd.4) 12 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.369399+0000 mon.a (mon.0) 696 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.599898+0000 mon.a (mon.0) 697 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.684948+0000 osd.6 (osd.6) 11 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:30:17.678 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.686042+0000 osd.6 (osd.6) 12 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:15.599171+0000 osd.7 (osd.7) 13 : cluster [DBG] 5.10 deep-scrub starts 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:15.602094+0000 osd.7 (osd.7) 14 : cluster [DBG] 5.10 deep-scrub ok 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.056498+0000 osd.4 (osd.4) 9 : cluster [DBG] 6.17 deep-scrub starts 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.057626+0000 osd.4 (osd.4) 10 : cluster [DBG] 6.17 deep-scrub ok 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.208303+0000 osd.4 (osd.4) 11 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.208364+0000 osd.4 (osd.4) 12 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.369399+0000 mon.a (mon.0) 696 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.599898+0000 mon.a (mon.0) 697 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.684948+0000 osd.6 (osd.6) 11 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:30:17.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.686042+0000 osd.6 (osd.6) 12 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:30:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:15.599171+0000 osd.7 (osd.7) 13 : cluster [DBG] 5.10 deep-scrub starts 2024-02-22T19:30:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:15.602094+0000 osd.7 (osd.7) 14 : cluster [DBG] 5.10 deep-scrub ok 2024-02-22T19:30:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.056498+0000 osd.4 (osd.4) 9 : cluster [DBG] 6.17 deep-scrub starts 2024-02-22T19:30:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.057626+0000 osd.4 (osd.4) 10 : cluster [DBG] 6.17 deep-scrub ok 2024-02-22T19:30:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.208303+0000 osd.4 (osd.4) 11 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.208364+0000 osd.4 (osd.4) 12 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.369399+0000 mon.a (mon.0) 696 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-22T19:30:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.599898+0000 mon.a (mon.0) 697 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-22T19:30:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.684948+0000 osd.6 (osd.6) 11 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:30:17.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.686042+0000 osd.6 (osd.6) 12 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:30:17.991 INFO:teuthology.orchestra.run.smithi053.stdout:137438953506 2024-02-22T19:30:17.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-22T19:30:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.729608+0000 osd.0 (osd.0) 19 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:30:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:16.731448+0000 osd.0 (osd.0) 20 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:30:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:17.192412+0000 mgr.y (mgr.14535) 48 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:30:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:17.465420+0000 osd.5 (osd.5) 15 : cluster [DBG] 3.c scrub starts 2024-02-22T19:30:18.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:17.466660+0000 osd.5 (osd.5) 16 : cluster [DBG] 3.c scrub ok 2024-02-22T19:30:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.729608+0000 osd.0 (osd.0) 19 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:30:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:16.731448+0000 osd.0 (osd.0) 20 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:30:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:17.192412+0000 mgr.y (mgr.14535) 48 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:30:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:17.465420+0000 osd.5 (osd.5) 15 : cluster [DBG] 3.c scrub starts 2024-02-22T19:30:18.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:17.466660+0000 osd.5 (osd.5) 16 : cluster [DBG] 3.c scrub ok 2024-02-22T19:30:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.729608+0000 osd.0 (osd.0) 19 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:30:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:16.731448+0000 osd.0 (osd.0) 20 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:30:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:17.192412+0000 mgr.y (mgr.14535) 48 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:30:18.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:17.465420+0000 osd.5 (osd.5) 15 : cluster [DBG] 3.c scrub starts 2024-02-22T19:30:18.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:17.466660+0000 osd.5 (osd.5) 16 : cluster [DBG] 3.c scrub ok 2024-02-22T19:30:19.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:19] "GET /metrics HTTP/1.1" 200 172070 "" "Prometheus/2.33.4" 2024-02-22T19:30:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:17.545567+0000 osd.7 (osd.7) 15 : cluster [DBG] 3.f deep-scrub starts 2024-02-22T19:30:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:17.547562+0000 osd.7 (osd.7) 16 : cluster [DBG] 3.f deep-scrub ok 2024-02-22T19:30:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.083470+0000 osd.4 (osd.4) 13 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:30:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.086935+0000 osd.4 (osd.4) 14 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:30:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.114187+0000 osd.2 (osd.2) 9 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.117093+0000 osd.2 (osd.2) 10 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.493844+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.496735+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:17.545567+0000 osd.7 (osd.7) 15 : cluster [DBG] 3.f deep-scrub starts 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:17.547562+0000 osd.7 (osd.7) 16 : cluster [DBG] 3.f deep-scrub ok 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.083470+0000 osd.4 (osd.4) 13 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.086935+0000 osd.4 (osd.4) 14 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.114187+0000 osd.2 (osd.2) 9 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.117093+0000 osd.2 (osd.2) 10 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.493844+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:30:19.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.496735+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:17.545567+0000 osd.7 (osd.7) 15 : cluster [DBG] 3.f deep-scrub starts 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:17.547562+0000 osd.7 (osd.7) 16 : cluster [DBG] 3.f deep-scrub ok 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.083470+0000 osd.4 (osd.4) 13 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.086935+0000 osd.4 (osd.4) 14 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.114187+0000 osd.2 (osd.2) 9 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.117093+0000 osd.2 (osd.2) 10 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.493844+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:30:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.496735+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:30:20.430 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.665165+0000 osd.3 (osd.3) 15 : cluster [DBG] 3.b scrub starts 2024-02-22T19:30:20.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.668708+0000 osd.3 (osd.3) 16 : cluster [DBG] 3.b scrub ok 2024-02-22T19:30:20.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.716418+0000 osd.0 (osd.0) 21 : cluster [DBG] 6.0 deep-scrub starts 2024-02-22T19:30:20.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:18.719306+0000 osd.0 (osd.0) 22 : cluster [DBG] 6.0 deep-scrub ok 2024-02-22T19:30:20.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:19.099840+0000 osd.4 (osd.4) 15 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:30:20.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:19.102756+0000 osd.4 (osd.4) 16 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:30:20.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:19.193119+0000 mgr.y (mgr.14535) 49 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:20.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.665165+0000 osd.3 (osd.3) 15 : cluster [DBG] 3.b scrub starts 2024-02-22T19:30:20.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.668708+0000 osd.3 (osd.3) 16 : cluster [DBG] 3.b scrub ok 2024-02-22T19:30:20.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.716418+0000 osd.0 (osd.0) 21 : cluster [DBG] 6.0 deep-scrub starts 2024-02-22T19:30:20.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:18.719306+0000 osd.0 (osd.0) 22 : cluster [DBG] 6.0 deep-scrub ok 2024-02-22T19:30:20.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:19.099840+0000 osd.4 (osd.4) 15 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:30:20.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:19.102756+0000 osd.4 (osd.4) 16 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:19.193119+0000 mgr.y (mgr.14535) 49 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.665165+0000 osd.3 (osd.3) 15 : cluster [DBG] 3.b scrub starts 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.668708+0000 osd.3 (osd.3) 16 : cluster [DBG] 3.b scrub ok 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.716418+0000 osd.0 (osd.0) 21 : cluster [DBG] 6.0 deep-scrub starts 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:18.719306+0000 osd.0 (osd.0) 22 : cluster [DBG] 6.0 deep-scrub ok 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:19.099840+0000 osd.4 (osd.4) 15 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:19.102756+0000 osd.4 (osd.4) 16 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:30:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:19.193119+0000 mgr.y (mgr.14535) 49 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:20.741 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:20 smithi064 systemd[1]: Starting Ceph grafana.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:30:20.843 INFO:teuthology.orchestra.run.smithi053.stdout:167503724573 2024-02-22T19:30:20.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-22T19:30:21.123 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:21.569 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:20.542488+0000 osd.5 (osd.5) 19 : cluster [DBG] 6.d scrub starts 2024-02-22T19:30:21.570 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:20.543395+0000 osd.5 (osd.5) 20 : cluster [DBG] 6.d scrub ok 2024-02-22T19:30:21.570 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:20.709854+0000 osd.6 (osd.6) 13 : cluster [DBG] 4.a scrub starts 2024-02-22T19:30:21.570 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:20.713326+0000 osd.6 (osd.6) 14 : cluster [DBG] 4.a scrub ok 2024-02-22T19:30:21.571 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-02-22T19:30:21.572 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="App mode production" logger=settings 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-02-22T19:30:21.573 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-02-22T19:30:21.574 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-02-22T19:30:21.575 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-02-22T19:30:21.576 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 bash[63869]: a4f400c041b95de0f0fdd529570362c94501fe84a26c7d745e6011676c68ce5f 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 systemd[1]: Started Ceph grafana.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-02-22T19:30:21.577 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-02-22T19:30:21.578 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-02-22T19:30:21.579 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-02-22T19:30:21.580 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-02-22T19:30:21.581 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-02-22T19:30:21.582 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-02-22T19:30:21.583 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-02-22T19:30:21.584 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-02-22T19:30:21.585 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-02-22T19:30:21.586 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:20.542488+0000 osd.5 (osd.5) 19 : cluster [DBG] 6.d scrub starts 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:20.543395+0000 osd.5 (osd.5) 20 : cluster [DBG] 6.d scrub ok 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:20.709854+0000 osd.6 (osd.6) 13 : cluster [DBG] 4.a scrub starts 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:20.713326+0000 osd.6 (osd.6) 14 : cluster [DBG] 4.a scrub ok 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:20.542488+0000 osd.5 (osd.5) 19 : cluster [DBG] 6.d scrub starts 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:20.543395+0000 osd.5 (osd.5) 20 : cluster [DBG] 6.d scrub ok 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:20.709854+0000 osd.6 (osd.6) 13 : cluster [DBG] 4.a scrub starts 2024-02-22T19:30:21.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:20.713326+0000 osd.6 (osd.6) 14 : cluster [DBG] 4.a scrub ok 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-02-22T19:30:21.823 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-02-22T19:30:21.824 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-22T19:30:21.825 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-02-22T19:30:21.826 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-02-22T19:30:21.827 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-02-22T19:30:21.828 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-02-22T19:30:21.829 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-02-22T19:30:21.830 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-02-22T19:30:21.831 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-02-22T19:30:21.832 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-02-22T19:30:21.833 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-02-22T19:30:21.834 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-02-22T19:30:21.835 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=500.642629ms 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-02-22T19:30:21.836 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-02-22T19:30:22.102 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-02-22T19:30:22.103 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:30:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:30:21+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-02-22T19:30:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:20.547364+0000 osd.7 (osd.7) 17 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:30:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:20.550832+0000 osd.7 (osd.7) 18 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:30:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.093519+0000 osd.1 (osd.1) 19 : cluster [DBG] 5.c scrub starts 2024-02-22T19:30:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.094362+0000 osd.1 (osd.1) 20 : cluster [DBG] 5.c scrub ok 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.113462+0000 osd.4 (osd.4) 17 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.116353+0000 osd.4 (osd.4) 18 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.193631+0000 mgr.y (mgr.14535) 50 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:21.437355+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:21.438422+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.532315+0000 osd.5 (osd.5) 21 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.533442+0000 osd.5 (osd.5) 22 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:20.547364+0000 osd.7 (osd.7) 17 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:20.550832+0000 osd.7 (osd.7) 18 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.093519+0000 osd.1 (osd.1) 19 : cluster [DBG] 5.c scrub starts 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.094362+0000 osd.1 (osd.1) 20 : cluster [DBG] 5.c scrub ok 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.113462+0000 osd.4 (osd.4) 17 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:30:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.116353+0000 osd.4 (osd.4) 18 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:30:22.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.193631+0000 mgr.y (mgr.14535) 50 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:22.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:21.437355+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:22.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:21.438422+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:22.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.532315+0000 osd.5 (osd.5) 21 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:30:22.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.533442+0000 osd.5 (osd.5) 22 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:30:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:20.547364+0000 osd.7 (osd.7) 17 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:20.550832+0000 osd.7 (osd.7) 18 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.093519+0000 osd.1 (osd.1) 19 : cluster [DBG] 5.c scrub starts 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.094362+0000 osd.1 (osd.1) 20 : cluster [DBG] 5.c scrub ok 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.113462+0000 osd.4 (osd.4) 17 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.116353+0000 osd.4 (osd.4) 18 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.193631+0000 mgr.y (mgr.14535) 50 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:21.437355+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:21.438422+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.532315+0000 osd.5 (osd.5) 21 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:30:22.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.533442+0000 osd.5 (osd.5) 22 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:30:23.620 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.679922+0000 osd.0 (osd.0) 23 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:30:23.620 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:21.682828+0000 osd.0 (osd.0) 24 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:30:23.620 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.121072+0000 osd.4 (osd.4) 19 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:30:23.620 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.123980+0000 osd.4 (osd.4) 20 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:30:23.620 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.573266+0000 osd.5 (osd.5) 23 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:30:23.620 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.574746+0000 osd.5 (osd.5) 24 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.188524+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.188719+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.188877+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.345428+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.679922+0000 osd.0 (osd.0) 23 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:21.682828+0000 osd.0 (osd.0) 24 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.121072+0000 osd.4 (osd.4) 19 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.123980+0000 osd.4 (osd.4) 20 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.573266+0000 osd.5 (osd.5) 23 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.574746+0000 osd.5 (osd.5) 24 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.188524+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.188719+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.622 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.188877+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.622 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.345428+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:23.640 INFO:teuthology.orchestra.run.smithi053.stdout:193273528344 2024-02-22T19:30:23.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-22T19:30:23.730 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.679922+0000 osd.0 (osd.0) 23 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:30:23.730 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:21.682828+0000 osd.0 (osd.0) 24 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.121072+0000 osd.4 (osd.4) 19 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.123980+0000 osd.4 (osd.4) 20 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.573266+0000 osd.5 (osd.5) 23 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.574746+0000 osd.5 (osd.5) 24 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.188524+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.188719+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.188877+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-22T19:30:23.731 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.345428+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:24.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:30:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.589833+0000 osd.7 (osd.7) 19 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:30:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.592765+0000 osd.7 (osd.7) 20 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.681450+0000 osd.0 (osd.0) 25 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.683506+0000 osd.0 (osd.0) 26 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.749473+0000 osd.3 (osd.3) 17 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:22.752361+0000 osd.3 (osd.3) 18 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.096704+0000 osd.2 (osd.2) 11 : cluster [DBG] 5.d scrub starts 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.100172+0000 osd.2 (osd.2) 12 : cluster [DBG] 5.d scrub ok 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.193995+0000 mgr.y (mgr.14535) 51 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.405506+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.405587+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:23.405669+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.405729+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:24.055167+0000 osd.1 (osd.1) 21 : cluster [DBG] 5.19 scrub starts 2024-02-22T19:30:24.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:24.056063+0000 osd.1 (osd.1) 22 : cluster [DBG] 5.19 scrub ok 2024-02-22T19:30:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.589833+0000 osd.7 (osd.7) 19 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:30:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.592765+0000 osd.7 (osd.7) 20 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:30:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.681450+0000 osd.0 (osd.0) 25 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:30:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.683506+0000 osd.0 (osd.0) 26 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.749473+0000 osd.3 (osd.3) 17 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:22.752361+0000 osd.3 (osd.3) 18 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.096704+0000 osd.2 (osd.2) 11 : cluster [DBG] 5.d scrub starts 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.100172+0000 osd.2 (osd.2) 12 : cluster [DBG] 5.d scrub ok 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.193995+0000 mgr.y (mgr.14535) 51 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.405506+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.405587+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:23.405669+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.405729+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:24.055167+0000 osd.1 (osd.1) 21 : cluster [DBG] 5.19 scrub starts 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:24.056063+0000 osd.1 (osd.1) 22 : cluster [DBG] 5.19 scrub ok 2024-02-22T19:30:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.589833+0000 osd.7 (osd.7) 19 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.592765+0000 osd.7 (osd.7) 20 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.681450+0000 osd.0 (osd.0) 25 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.683506+0000 osd.0 (osd.0) 26 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.749473+0000 osd.3 (osd.3) 17 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:22.752361+0000 osd.3 (osd.3) 18 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.096704+0000 osd.2 (osd.2) 11 : cluster [DBG] 5.d scrub starts 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.100172+0000 osd.2 (osd.2) 12 : cluster [DBG] 5.d scrub ok 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.193995+0000 mgr.y (mgr.14535) 51 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.405506+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.405587+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:23.405669+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.405729+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-22T19:30:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:24.055167+0000 osd.1 (osd.1) 21 : cluster [DBG] 5.19 scrub starts 2024-02-22T19:30:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:24.056063+0000 osd.1 (osd.1) 22 : cluster [DBG] 5.19 scrub ok 2024-02-22T19:30:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:30:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:30:25.446 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.635742+0000 osd.0 (osd.0) 27 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:30:25.446 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.636543+0000 osd.0 (osd.0) 28 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:30:25.446 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.785412+0000 osd.3 (osd.3) 19 : cluster [DBG] 6.1b deep-scrub starts 2024-02-22T19:30:25.446 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:23.788363+0000 osd.3 (osd.3) 20 : cluster [DBG] 6.1b deep-scrub ok 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:24.242214+0000 mgr.y (mgr.14535) 52 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:24.405990+0000 mon.a (mon.0) 708 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.635742+0000 osd.0 (osd.0) 27 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.636543+0000 osd.0 (osd.0) 28 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.785412+0000 osd.3 (osd.3) 19 : cluster [DBG] 6.1b deep-scrub starts 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:23.788363+0000 osd.3 (osd.3) 20 : cluster [DBG] 6.1b deep-scrub ok 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:24.242214+0000 mgr.y (mgr.14535) 52 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:25.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:24.405990+0000 mon.a (mon.0) 708 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-22T19:30:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.635742+0000 osd.0 (osd.0) 27 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:30:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.636543+0000 osd.0 (osd.0) 28 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:30:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.785412+0000 osd.3 (osd.3) 19 : cluster [DBG] 6.1b deep-scrub starts 2024-02-22T19:30:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:23.788363+0000 osd.3 (osd.3) 20 : cluster [DBG] 6.1b deep-scrub ok 2024-02-22T19:30:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:24.242214+0000 mgr.y (mgr.14535) 52 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:24.405990+0000 mon.a (mon.0) 708 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-22T19:30:26.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:25.194337+0000 mgr.y (mgr.14535) 53 : cluster [DBG] pgmap v53: 132 pgs: 3 peering, 129 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:30:26.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:25.194337+0000 mgr.y (mgr.14535) 53 : cluster [DBG] pgmap v53: 132 pgs: 3 peering, 129 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:30:26.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:25.194337+0000 mgr.y (mgr.14535) 53 : cluster [DBG] pgmap v53: 132 pgs: 3 peering, 129 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:30:26.977 INFO:teuthology.orchestra.run.smithi053.stdout:227633266708 2024-02-22T19:30:26.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-22T19:30:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:26.525064+0000 osd.5 (osd.5) 25 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:30:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:26.527960+0000 osd.5 (osd.5) 26 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:26.820164+0000 osd.3 (osd.3) 21 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:26.823359+0000 osd.3 (osd.3) 22 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:27.070073+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.171264+0000 osd.2 (osd.2) 13 : cluster [DBG] 4.f scrub starts 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.174775+0000 osd.2 (osd.2) 14 : cluster [DBG] 4.f scrub ok 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.194788+0000 mgr.y (mgr.14535) 54 : cluster [DBG] pgmap v54: 132 pgs: 3 peering, 129 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.239583+0000 osd.2 (osd.2) 15 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:28.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.239629+0000 osd.2 (osd.2) 16 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:28.446 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:26.525064+0000 osd.5 (osd.5) 25 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:26.527960+0000 osd.5 (osd.5) 26 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:26.820164+0000 osd.3 (osd.3) 21 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:26.823359+0000 osd.3 (osd.3) 22 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:27.070073+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.171264+0000 osd.2 (osd.2) 13 : cluster [DBG] 4.f scrub starts 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.174775+0000 osd.2 (osd.2) 14 : cluster [DBG] 4.f scrub ok 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.194788+0000 mgr.y (mgr.14535) 54 : cluster [DBG] pgmap v54: 132 pgs: 3 peering, 129 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.239583+0000 osd.2 (osd.2) 15 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:28.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.239629+0000 osd.2 (osd.2) 16 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:26.525064+0000 osd.5 (osd.5) 25 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:26.527960+0000 osd.5 (osd.5) 26 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:26.820164+0000 osd.3 (osd.3) 21 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:26.823359+0000 osd.3 (osd.3) 22 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:27.070073+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.171264+0000 osd.2 (osd.2) 13 : cluster [DBG] 4.f scrub starts 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.174775+0000 osd.2 (osd.2) 14 : cluster [DBG] 4.f scrub ok 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.194788+0000 mgr.y (mgr.14535) 54 : cluster [DBG] pgmap v54: 132 pgs: 3 peering, 129 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.239583+0000 osd.2 (osd.2) 15 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:28.448 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.239629+0000 osd.2 (osd.2) 16 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.573999+0000 osd.5 (osd.5) 27 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:27.575246+0000 osd.5 (osd.5) 28 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.076774+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.092757+0000 osd.4 (osd.4) 21 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.096662+0000 osd.4 (osd.4) 22 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.125280+0000 osd.2 (osd.2) 17 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.128748+0000 osd.2 (osd.2) 18 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:28.345435+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:28.813687+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:28.814315+0000 mon.a (mon.0) 713 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:29.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:28.818481+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.573999+0000 osd.5 (osd.5) 27 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:27.575246+0000 osd.5 (osd.5) 28 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.076774+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.092757+0000 osd.4 (osd.4) 21 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.096662+0000 osd.4 (osd.4) 22 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.125280+0000 osd.2 (osd.2) 17 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.128748+0000 osd.2 (osd.2) 18 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:28.345435+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:28.813687+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:28.814315+0000 mon.a (mon.0) 713 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:29.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:28.818481+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.573999+0000 osd.5 (osd.5) 27 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:30:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:27.575246+0000 osd.5 (osd.5) 28 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:30:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.076774+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-22T19:30:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.092757+0000 osd.4 (osd.4) 21 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:30:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.096662+0000 osd.4 (osd.4) 22 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:30:29.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.125280+0000 osd.2 (osd.2) 17 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:30:29.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.128748+0000 osd.2 (osd.2) 18 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:30:29.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:28.345435+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:29.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:28.813687+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:29.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:28.814315+0000 mon.a (mon.0) 713 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:29.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:28.818481+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:29.434 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:29] "GET /metrics HTTP/1.1" 200 172087 "" "Prometheus/2.33.4" 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.598048+0000 osd.5 (osd.5) 29 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:28.601560+0000 osd.5 (osd.5) 30 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:30:28.821740+0000 mgr.y (mgr.14535) 55 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:30:28.824472+0000 mgr.y (mgr.14535) 56 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi053 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.096535+0000 osd.4 (osd.4) 23 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.099678+0000 osd.4 (osd.4) 24 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.107466+0000 osd.2 (osd.2) 19 : cluster [DBG] 5.b scrub starts 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.110509+0000 osd.2 (osd.2) 20 : cluster [DBG] 5.b scrub ok 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.195438+0000 mgr.y (mgr.14535) 57 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 91 B/s, 2 objects/s recovering 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.655317+0000 osd.6 (osd.6) 15 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:30:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.656239+0000 osd.6 (osd.6) 16 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.598048+0000 osd.5 (osd.5) 29 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:28.601560+0000 osd.5 (osd.5) 30 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:30:28.821740+0000 mgr.y (mgr.14535) 55 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:30:28.824472+0000 mgr.y (mgr.14535) 56 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi053 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.096535+0000 osd.4 (osd.4) 23 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.099678+0000 osd.4 (osd.4) 24 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.107466+0000 osd.2 (osd.2) 19 : cluster [DBG] 5.b scrub starts 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.110509+0000 osd.2 (osd.2) 20 : cluster [DBG] 5.b scrub ok 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.195438+0000 mgr.y (mgr.14535) 57 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 91 B/s, 2 objects/s recovering 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.655317+0000 osd.6 (osd.6) 15 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:30:30.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.656239+0000 osd.6 (osd.6) 16 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:30:30.261 INFO:teuthology.orchestra.run.smithi053.stdout:34359738424 2024-02-22T19:30:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.598048+0000 osd.5 (osd.5) 29 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:30:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:28.601560+0000 osd.5 (osd.5) 30 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:30:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:30:28.821740+0000 mgr.y (mgr.14535) 55 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-22T19:30:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:30:28.824472+0000 mgr.y (mgr.14535) 56 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi053 2024-02-22T19:30:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.096535+0000 osd.4 (osd.4) 23 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:30:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.099678+0000 osd.4 (osd.4) 24 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:30:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.107466+0000 osd.2 (osd.2) 19 : cluster [DBG] 5.b scrub starts 2024-02-22T19:30:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.110509+0000 osd.2 (osd.2) 20 : cluster [DBG] 5.b scrub ok 2024-02-22T19:30:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.195438+0000 mgr.y (mgr.14535) 57 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 91 B/s, 2 objects/s recovering 2024-02-22T19:30:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.655317+0000 osd.6 (osd.6) 15 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:30:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.656239+0000 osd.6 (osd.6) 16 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:30:31.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.615650+0000 osd.5 (osd.5) 31 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:30:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:29.616632+0000 osd.5 (osd.5) 32 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:30:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:30.262012+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.53:0/41987615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T19:30:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.615650+0000 osd.5 (osd.5) 31 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:30:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:29.616632+0000 osd.5 (osd.5) 32 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:30:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:30.262012+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.53:0/41987615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T19:30:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.615650+0000 osd.5 (osd.5) 31 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:30:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:29.616632+0000 osd.5 (osd.5) 32 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:30:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:30.262012+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.53:0/41987615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T19:30:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:30.873700+0000 osd.3 (osd.3) 23 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:30.877406+0000 osd.3 (osd.3) 24 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.069067+0000 osd.1 (osd.1) 23 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.070012+0000 osd.1 (osd.1) 24 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.098988+0000 osd.2 (osd.2) 21 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.102057+0000 osd.2 (osd.2) 22 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.133991+0000 osd.4 (osd.4) 25 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.136955+0000 osd.4 (osd.4) 26 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:30:32.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.195882+0000 mgr.y (mgr.14535) 58 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 70 B/s, 2 objects/s recovering 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:30.873700+0000 osd.3 (osd.3) 23 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:30.877406+0000 osd.3 (osd.3) 24 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.069067+0000 osd.1 (osd.1) 23 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.070012+0000 osd.1 (osd.1) 24 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.098988+0000 osd.2 (osd.2) 21 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.102057+0000 osd.2 (osd.2) 22 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:30:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.133991+0000 osd.4 (osd.4) 25 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.136955+0000 osd.4 (osd.4) 26 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.195882+0000 mgr.y (mgr.14535) 58 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 70 B/s, 2 objects/s recovering 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:30.873700+0000 osd.3 (osd.3) 23 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:30.877406+0000 osd.3 (osd.3) 24 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.069067+0000 osd.1 (osd.1) 23 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.070012+0000 osd.1 (osd.1) 24 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.098988+0000 osd.2 (osd.2) 21 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.102057+0000 osd.2 (osd.2) 22 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.133991+0000 osd.4 (osd.4) 25 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.136955+0000 osd.4 (osd.4) 26 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:30:32.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.195882+0000 mgr.y (mgr.14535) 58 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 70 B/s, 2 objects/s recovering 2024-02-22T19:30:32.823 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:32 smithi053 systemd[1]: Stopping Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:30:32.855 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738424 for osd.0 2024-02-22T19:30:32.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-22T19:30:33.075 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:32 smithi053 bash[100851]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager.a" found: no such container 2024-02-22T19:30:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.554393+0000 osd.7 (osd.7) 21 : cluster [DBG] 3.e scrub starts 2024-02-22T19:30:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.557413+0000 osd.7 (osd.7) 22 : cluster [DBG] 3.e scrub ok 2024-02-22T19:30:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.645629+0000 osd.5 (osd.5) 33 : cluster [DBG] 5.f scrub starts 2024-02-22T19:30:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:31.646591+0000 osd.5 (osd.5) 34 : cluster [DBG] 5.f scrub ok 2024-02-22T19:30:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:32.141184+0000 osd.2 (osd.2) 23 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:30:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:32.144773+0000 osd.2 (osd.2) 24 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:30:33.406 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[96125]: level=info ts=2024-02-22T19:30:33.074Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-22T19:30:33.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.554393+0000 osd.7 (osd.7) 21 : cluster [DBG] 3.e scrub starts 2024-02-22T19:30:33.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.557413+0000 osd.7 (osd.7) 22 : cluster [DBG] 3.e scrub ok 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.645629+0000 osd.5 (osd.5) 33 : cluster [DBG] 5.f scrub starts 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:31.646591+0000 osd.5 (osd.5) 34 : cluster [DBG] 5.f scrub ok 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:32.141184+0000 osd.2 (osd.2) 23 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:32.144773+0000 osd.2 (osd.2) 24 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.554393+0000 osd.7 (osd.7) 21 : cluster [DBG] 3.e scrub starts 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.557413+0000 osd.7 (osd.7) 22 : cluster [DBG] 3.e scrub ok 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.645629+0000 osd.5 (osd.5) 33 : cluster [DBG] 5.f scrub starts 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:31.646591+0000 osd.5 (osd.5) 34 : cluster [DBG] 5.f scrub ok 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:32.141184+0000 osd.2 (osd.2) 23 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:30:33.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:32.144773+0000 osd.2 (osd.2) 24 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:30:33.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:33 smithi053 bash[100919]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a 2024-02-22T19:30:34.000 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:34.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:32.840447+0000 osd.3 (osd.3) 25 : cluster [DBG] 4.14 deep-scrub starts 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:32.846330+0000 osd.3 (osd.3) 26 : cluster [DBG] 4.14 deep-scrub ok 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:33.112564+0000 osd.1 (osd.1) 25 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:33.113516+0000 osd.1 (osd.1) 26 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:33.196407+0000 mgr.y (mgr.14535) 59 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 465 B/s wr, 10 op/s; 62 B/s, 1 objects/s recovering 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:32.840447+0000 osd.3 (osd.3) 25 : cluster [DBG] 4.14 deep-scrub starts 2024-02-22T19:30:34.353 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:32.846330+0000 osd.3 (osd.3) 26 : cluster [DBG] 4.14 deep-scrub ok 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:33.112564+0000 osd.1 (osd.1) 25 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:33.113516+0000 osd.1 (osd.1) 26 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:33.196407+0000 mgr.y (mgr.14535) 59 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 465 B/s wr, 10 op/s; 62 B/s, 1 objects/s recovering 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:32.840447+0000 osd.3 (osd.3) 25 : cluster [DBG] 4.14 deep-scrub starts 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:32.846330+0000 osd.3 (osd.3) 26 : cluster [DBG] 4.14 deep-scrub ok 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:33.112564+0000 osd.1 (osd.1) 25 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:33.113516+0000 osd.1 (osd.1) 26 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:30:34.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:33.196407+0000 mgr.y (mgr.14535) 59 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 465 B/s wr, 10 op/s; 62 B/s, 1 objects/s recovering 2024-02-22T19:30:34.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:34 smithi053 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-22T19:30:34.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:34 smithi053 systemd[1]: Stopped Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:30:34.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:34 smithi053 systemd[1]: Starting Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:30:35.090 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:30:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:30:35.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:33.583844+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:33.584817+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:34.058604+0000 osd.4 (osd.4) 27 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:34.059551+0000 osd.4 (osd.4) 28 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:34.148847+0000 osd.1 (osd.1) 27 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:34.149664+0000 osd.1 (osd.1) 28 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:34.253288+0000 mgr.y (mgr.14535) 60 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:34.560437+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.5 deep-scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:34.566435+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.5 deep-scrub ok 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:33.583844+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:33.584817+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:34.058604+0000 osd.4 (osd.4) 27 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:34.059551+0000 osd.4 (osd.4) 28 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:34.148847+0000 osd.1 (osd.1) 27 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:30:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:34.149664+0000 osd.1 (osd.1) 28 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:30:35.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:34.253288+0000 mgr.y (mgr.14535) 60 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:35.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:34.560437+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.5 deep-scrub starts 2024-02-22T19:30:35.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:34.566435+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.5 deep-scrub ok 2024-02-22T19:30:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:33.583844+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:30:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:33.584817+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:30:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:34.058604+0000 osd.4 (osd.4) 27 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:30:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:34.059551+0000 osd.4 (osd.4) 28 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:30:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:34.148847+0000 osd.1 (osd.1) 27 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:30:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:34.149664+0000 osd.1 (osd.1) 28 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:30:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:34.253288+0000 mgr.y (mgr.14535) 60 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:34.560437+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.5 deep-scrub starts 2024-02-22T19:30:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:34.566435+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.5 deep-scrub ok 2024-02-22T19:30:35.950 INFO:teuthology.orchestra.run.smithi053.stdout:55834574899 2024-02-22T19:30:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.028144+0000 osd.4 (osd.4) 29 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:30:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.031810+0000 osd.4 (osd.4) 30 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.178702+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.f scrub starts 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.181812+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.f scrub ok 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.197250+0000 mgr.y (mgr.14535) 61 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 409 B/s wr, 9 op/s; 54 B/s, 1 objects/s recovering 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.536813+0000 osd.6 (osd.6) 19 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.539765+0000 osd.6 (osd.6) 20 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:35.951205+0000 mon.a (mon.0) 715 : audit [DBG] from='client.? 172.21.15.53:0/150848654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.028144+0000 osd.4 (osd.4) 29 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.031810+0000 osd.4 (osd.4) 30 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.178702+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.f scrub starts 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.181812+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.f scrub ok 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.197250+0000 mgr.y (mgr.14535) 61 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 409 B/s wr, 9 op/s; 54 B/s, 1 objects/s recovering 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.536813+0000 osd.6 (osd.6) 19 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:30:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.539765+0000 osd.6 (osd.6) 20 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:30:36.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:35.951205+0000 mon.a (mon.0) 715 : audit [DBG] from='client.? 172.21.15.53:0/150848654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T19:30:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.028144+0000 osd.4 (osd.4) 29 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:30:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.031810+0000 osd.4 (osd.4) 30 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:30:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.178702+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.f scrub starts 2024-02-22T19:30:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.181812+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.f scrub ok 2024-02-22T19:30:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.197250+0000 mgr.y (mgr.14535) 61 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 409 B/s wr, 9 op/s; 54 B/s, 1 objects/s recovering 2024-02-22T19:30:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.536813+0000 osd.6 (osd.6) 19 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:30:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.539765+0000 osd.6 (osd.6) 20 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:30:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:35.951205+0000 mon.a (mon.0) 715 : audit [DBG] from='client.? 172.21.15.53:0/150848654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T19:30:36.807 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STOPPING 2024-02-22T19:30:36.807 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.665Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.665Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.666Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.53 port=9094 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.667Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.692Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.692Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.695Z caller=main.go:518 msg=Listening address=:9093 2024-02-22T19:30:36.807 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:36.695Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-22T19:30:36.808 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 bash[101204]: c06b015105d3f35ff4e6af6ce2ec605d5013b7415d8f36af4da5a0fe850743f4 2024-02-22T19:30:36.808 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:36 smithi053 systemd[1]: Started Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STOPPED 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STARTING 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Serving on http://:::9283 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STARTED 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STOPPING 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STOPPED 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:36] ENGINE Bus STARTING 2024-02-22T19:30:37.081 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Serving on http://:::9283 2024-02-22T19:30:37.082 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Bus STARTED 2024-02-22T19:30:37.082 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Bus STOPPING 2024-02-22T19:30:37.082 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:30:37.082 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Bus STOPPED 2024-02-22T19:30:37.082 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Bus STARTING 2024-02-22T19:30:37.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574894 got 55834574899 for osd.1 2024-02-22T19:30:37.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-22T19:30:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.555016+0000 osd.5 (osd.5) 37 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:30:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.556015+0000 osd.5 (osd.5) 38 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:30:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.997617+0000 osd.4 (osd.4) 31 : cluster [DBG] 5.e scrub starts 2024-02-22T19:30:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:35.998946+0000 osd.4 (osd.4) 32 : cluster [DBG] 5.e scrub ok 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.106046+0000 osd.1 (osd.1) 29 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.106916+0000 osd.1 (osd.1) 30 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.173593+0000 osd.2 (osd.2) 27 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.176485+0000 osd.2 (osd.2) 28 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.775903+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.779126+0000 mon.a (mon.0) 717 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.780334+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.784356+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.790047+0000 mon.a (mon.0) 720 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.793641+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.797907+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.802672+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.805275+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T19:30:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.810571+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.813851+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:37.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Serving on http://:::9283 2024-02-22T19:30:37.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:30:37] ENGINE Bus STARTED 2024-02-22T19:30:37.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.555016+0000 osd.5 (osd.5) 37 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:30:37.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.556015+0000 osd.5 (osd.5) 38 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:30:37.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.997617+0000 osd.4 (osd.4) 31 : cluster [DBG] 5.e scrub starts 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:35.998946+0000 osd.4 (osd.4) 32 : cluster [DBG] 5.e scrub ok 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.106046+0000 osd.1 (osd.1) 29 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.106916+0000 osd.1 (osd.1) 30 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.173593+0000 osd.2 (osd.2) 27 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.176485+0000 osd.2 (osd.2) 28 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.775903+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.779126+0000 mon.a (mon.0) 717 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.780334+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.784356+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.790047+0000 mon.a (mon.0) 720 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.793641+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.797907+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.802672+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.805275+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.810571+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.813851+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.555016+0000 osd.5 (osd.5) 37 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.556015+0000 osd.5 (osd.5) 38 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.997617+0000 osd.4 (osd.4) 31 : cluster [DBG] 5.e scrub starts 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:35.998946+0000 osd.4 (osd.4) 32 : cluster [DBG] 5.e scrub ok 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.106046+0000 osd.1 (osd.1) 29 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.106916+0000 osd.1 (osd.1) 30 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.173593+0000 osd.2 (osd.2) 27 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.176485+0000 osd.2 (osd.2) 28 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:30:37.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.775903+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.779126+0000 mon.a (mon.0) 717 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.780334+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.784356+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.790047+0000 mon.a (mon.0) 720 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.793641+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.797907+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.802672+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.805275+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.810571+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:37.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.813851+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:37.855 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:38.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.617138+0000 osd.7 (osd.7) 23 : cluster [DBG] 6.5 deep-scrub starts 2024-02-22T19:30:38.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.620210+0000 osd.7 (osd.7) 24 : cluster [DBG] 6.5 deep-scrub ok 2024-02-22T19:30:38.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.779555+0000 mgr.y (mgr.14535) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T19:30:38.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.780581+0000 mgr.y (mgr.14535) 63 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T19:30:38.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.790395+0000 mgr.y (mgr.14535) 64 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T19:30:38.441 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.793999+0000 mgr.y (mgr.14535) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T19:30:38.442 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.803008+0000 mgr.y (mgr.14535) 66 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T19:30:38.442 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:36.805565+0000 mgr.y (mgr.14535) 67 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T19:30:38.442 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.901841+0000 osd.3 (osd.3) 27 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:30:38.442 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:36.904712+0000 osd.3 (osd.3) 28 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:30:38.442 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:37.197575+0000 mgr.y (mgr.14535) 68 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.3 KiB/s rd, 409 B/s wr, 9 op/s; 54 B/s, 1 objects/s recovering 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.617138+0000 osd.7 (osd.7) 23 : cluster [DBG] 6.5 deep-scrub starts 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.620210+0000 osd.7 (osd.7) 24 : cluster [DBG] 6.5 deep-scrub ok 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.779555+0000 mgr.y (mgr.14535) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.780581+0000 mgr.y (mgr.14535) 63 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.790395+0000 mgr.y (mgr.14535) 64 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.793999+0000 mgr.y (mgr.14535) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.803008+0000 mgr.y (mgr.14535) 66 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:36.805565+0000 mgr.y (mgr.14535) 67 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.901841+0000 osd.3 (osd.3) 27 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:36.904712+0000 osd.3 (osd.3) 28 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:30:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:37.197575+0000 mgr.y (mgr.14535) 68 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.3 KiB/s rd, 409 B/s wr, 9 op/s; 54 B/s, 1 objects/s recovering 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.617138+0000 osd.7 (osd.7) 23 : cluster [DBG] 6.5 deep-scrub starts 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.620210+0000 osd.7 (osd.7) 24 : cluster [DBG] 6.5 deep-scrub ok 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.779555+0000 mgr.y (mgr.14535) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.780581+0000 mgr.y (mgr.14535) 63 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi053.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.790395+0000 mgr.y (mgr.14535) 64 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.793999+0000 mgr.y (mgr.14535) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.803008+0000 mgr.y (mgr.14535) 66 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:36.805565+0000 mgr.y (mgr.14535) 67 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.901841+0000 osd.3 (osd.3) 27 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:36.904712+0000 osd.3 (osd.3) 28 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:30:38.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:37.197575+0000 mgr.y (mgr.14535) 68 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.3 KiB/s rd, 409 B/s wr, 9 op/s; 54 B/s, 1 objects/s recovering 2024-02-22T19:30:38.991 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:38.668Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000378891s 2024-02-22T19:30:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:37.614246+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:30:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:37.617749+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:30:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:37.614246+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:30:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:37.617749+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:30:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:37.614246+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:30:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:37.617749+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:30:39.712 INFO:teuthology.orchestra.run.smithi053.stdout:77309411374 2024-02-22T19:30:39.742 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:39] "GET /metrics HTTP/1.1" 200 172096 "" "Prometheus/2.33.4" 2024-02-22T19:30:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:38.628809+0000 osd.7 (osd.7) 27 : cluster [DBG] 6.3 deep-scrub starts 2024-02-22T19:30:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:38.631693+0000 osd.7 (osd.7) 28 : cluster [DBG] 6.3 deep-scrub ok 2024-02-22T19:30:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.198223+0000 mgr.y (mgr.14535) 69 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.3 KiB/s rd, 368 B/s wr, 8 op/s; 49 B/s, 1 objects/s recovering 2024-02-22T19:30:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.207046+0000 osd.2 (osd.2) 29 : cluster [DBG] 5.a scrub starts 2024-02-22T19:30:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.210158+0000 osd.2 (osd.2) 30 : cluster [DBG] 5.a scrub ok 2024-02-22T19:30:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:39.712706+0000 mon.b (mon.1) 29 : audit [DBG] from='client.? 172.21.15.53:0/4035805447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T19:30:40.438 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:38.628809+0000 osd.7 (osd.7) 27 : cluster [DBG] 6.3 deep-scrub starts 2024-02-22T19:30:40.438 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:38.631693+0000 osd.7 (osd.7) 28 : cluster [DBG] 6.3 deep-scrub ok 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.198223+0000 mgr.y (mgr.14535) 69 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.3 KiB/s rd, 368 B/s wr, 8 op/s; 49 B/s, 1 objects/s recovering 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.207046+0000 osd.2 (osd.2) 29 : cluster [DBG] 5.a scrub starts 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.210158+0000 osd.2 (osd.2) 30 : cluster [DBG] 5.a scrub ok 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:39.712706+0000 mon.b (mon.1) 29 : audit [DBG] from='client.? 172.21.15.53:0/4035805447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:38.628809+0000 osd.7 (osd.7) 27 : cluster [DBG] 6.3 deep-scrub starts 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:38.631693+0000 osd.7 (osd.7) 28 : cluster [DBG] 6.3 deep-scrub ok 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.198223+0000 mgr.y (mgr.14535) 69 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.3 KiB/s rd, 368 B/s wr, 8 op/s; 49 B/s, 1 objects/s recovering 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.207046+0000 osd.2 (osd.2) 29 : cluster [DBG] 5.a scrub starts 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.210158+0000 osd.2 (osd.2) 30 : cluster [DBG] 5.a scrub ok 2024-02-22T19:30:40.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:39.712706+0000 mon.b (mon.1) 29 : audit [DBG] from='client.? 172.21.15.53:0/4035805447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T19:30:40.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411374 for osd.2 2024-02-22T19:30:40.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-22T19:30:41.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.593268+0000 osd.7 (osd.7) 29 : cluster [DBG] 3.13 deep-scrub starts 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.596161+0000 osd.7 (osd.7) 30 : cluster [DBG] 3.13 deep-scrub ok 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.659553+0000 osd.0 (osd.0) 29 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:39.660546+0000 osd.0 (osd.0) 30 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:40.013022+0000 osd.4 (osd.4) 33 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:40.016716+0000 osd.4 (osd.4) 34 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:41.022908+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.593268+0000 osd.7 (osd.7) 29 : cluster [DBG] 3.13 deep-scrub starts 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.596161+0000 osd.7 (osd.7) 30 : cluster [DBG] 3.13 deep-scrub ok 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.659553+0000 osd.0 (osd.0) 29 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:39.660546+0000 osd.0 (osd.0) 30 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:30:41.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:40.013022+0000 osd.4 (osd.4) 33 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:30:41.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:40.016716+0000 osd.4 (osd.4) 34 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:30:41.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:41.022908+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.593268+0000 osd.7 (osd.7) 29 : cluster [DBG] 3.13 deep-scrub starts 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.596161+0000 osd.7 (osd.7) 30 : cluster [DBG] 3.13 deep-scrub ok 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.659553+0000 osd.0 (osd.0) 29 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:39.660546+0000 osd.0 (osd.0) 30 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:40.013022+0000 osd.4 (osd.4) 33 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:40.016716+0000 osd.4 (osd.4) 34 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:30:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:41.022908+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:41.626 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:42.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:40.621557+0000 osd.7 (osd.7) 31 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:40.624538+0000 osd.7 (osd.7) 32 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.005084+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.008625+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.198527+0000 mgr.y (mgr.14535) 70 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:41.672575+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:41.673391+0000 mon.a (mon.0) 729 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:41.674053+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:41.677816+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:41.681895+0000 mon.a (mon.0) 732 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:40.621557+0000 osd.7 (osd.7) 31 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:40.624538+0000 osd.7 (osd.7) 32 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:30:42.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.005084+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.008625+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.198527+0000 mgr.y (mgr.14535) 70 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:41.672575+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:41.673391+0000 mon.a (mon.0) 729 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:41.674053+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:41.677816+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:41.681895+0000 mon.a (mon.0) 732 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:40.621557+0000 osd.7 (osd.7) 31 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:30:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:40.624538+0000 osd.7 (osd.7) 32 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:30:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.005084+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:30:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.008625+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:30:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.198527+0000 mgr.y (mgr.14535) 70 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:41.672575+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:41.673391+0000 mon.a (mon.0) 729 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:41.674053+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:41.677816+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:41.681895+0000 mon.a (mon.0) 732 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:30:43.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.616265+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.1a deep-scrub starts 2024-02-22T19:30:43.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.619437+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.1a deep-scrub ok 2024-02-22T19:30:43.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.966707+0000 osd.4 (osd.4) 37 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:30:43.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.969600+0000 osd.4 (osd.4) 38 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:30:43.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.980599+0000 osd.1 (osd.1) 31 : cluster [DBG] 5.1d deep-scrub starts 2024-02-22T19:30:43.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:41.981344+0000 osd.1 (osd.1) 32 : cluster [DBG] 5.1d deep-scrub ok 2024-02-22T19:30:43.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.616265+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.1a deep-scrub starts 2024-02-22T19:30:43.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.619437+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.1a deep-scrub ok 2024-02-22T19:30:43.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.966707+0000 osd.4 (osd.4) 37 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:30:43.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.969600+0000 osd.4 (osd.4) 38 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:30:43.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.980599+0000 osd.1 (osd.1) 31 : cluster [DBG] 5.1d deep-scrub starts 2024-02-22T19:30:43.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:41.981344+0000 osd.1 (osd.1) 32 : cluster [DBG] 5.1d deep-scrub ok 2024-02-22T19:30:43.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.616265+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.1a deep-scrub starts 2024-02-22T19:30:43.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.619437+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.1a deep-scrub ok 2024-02-22T19:30:43.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.966707+0000 osd.4 (osd.4) 37 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:30:43.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.969600+0000 osd.4 (osd.4) 38 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:30:43.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.980599+0000 osd.1 (osd.1) 31 : cluster [DBG] 5.1d deep-scrub starts 2024-02-22T19:30:43.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:41.981344+0000 osd.1 (osd.1) 32 : cluster [DBG] 5.1d deep-scrub ok 2024-02-22T19:30:43.761 INFO:teuthology.orchestra.run.smithi053.stdout:111669149737 2024-02-22T19:30:44.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:30:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:42.900250+0000 osd.3 (osd.3) 29 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:30:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:42.903789+0000 osd.3 (osd.3) 30 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:30:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:42.994012+0000 osd.1 (osd.1) 33 : cluster [DBG] 6.1 deep-scrub starts 2024-02-22T19:30:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:42.994915+0000 osd.1 (osd.1) 34 : cluster [DBG] 6.1 deep-scrub ok 2024-02-22T19:30:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:43.198860+0000 mgr.y (mgr.14535) 71 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:43.761811+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.53:0/389612051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T19:30:44.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:42.900250+0000 osd.3 (osd.3) 29 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:30:44.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:42.903789+0000 osd.3 (osd.3) 30 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:30:44.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:42.994012+0000 osd.1 (osd.1) 33 : cluster [DBG] 6.1 deep-scrub starts 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:42.994915+0000 osd.1 (osd.1) 34 : cluster [DBG] 6.1 deep-scrub ok 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:43.198860+0000 mgr.y (mgr.14535) 71 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:43.761811+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.53:0/389612051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:42.900250+0000 osd.3 (osd.3) 29 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:42.903789+0000 osd.3 (osd.3) 30 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:42.994012+0000 osd.1 (osd.1) 33 : cluster [DBG] 6.1 deep-scrub starts 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:42.994915+0000 osd.1 (osd.1) 34 : cluster [DBG] 6.1 deep-scrub ok 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:43.198860+0000 mgr.y (mgr.14535) 71 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:44.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:43.761811+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.53:0/389612051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T19:30:44.916 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149732 got 111669149737 for osd.3 2024-02-22T19:30:44.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-22T19:30:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:30:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:30:45.241 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:45.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:44.264320+0000 mgr.y (mgr.14535) 72 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:44.264320+0000 mgr.y (mgr.14535) 72 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:45.439 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:44.264320+0000 mgr.y (mgr.14535) 72 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:44.572586+0000 osd.0 (osd.0) 31 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:44.574350+0000 osd.0 (osd.0) 32 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:44.973751+0000 osd.4 (osd.4) 39 : cluster [DBG] 5.1c scrub starts 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:44.976647+0000 osd.4 (osd.4) 40 : cluster [DBG] 5.1c scrub ok 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.020580+0000 osd.1 (osd.1) 35 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.021515+0000 osd.1 (osd.1) 36 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.199451+0000 mgr.y (mgr.14535) 73 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:46.394 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:45.890252+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:44.572586+0000 osd.0 (osd.0) 31 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:44.574350+0000 osd.0 (osd.0) 32 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:44.973751+0000 osd.4 (osd.4) 39 : cluster [DBG] 5.1c scrub starts 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:44.976647+0000 osd.4 (osd.4) 40 : cluster [DBG] 5.1c scrub ok 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.020580+0000 osd.1 (osd.1) 35 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.021515+0000 osd.1 (osd.1) 36 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.199451+0000 mgr.y (mgr.14535) 73 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:46.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:45.890252+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:44.572586+0000 osd.0 (osd.0) 31 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:44.574350+0000 osd.0 (osd.0) 32 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:44.973751+0000 osd.4 (osd.4) 39 : cluster [DBG] 5.1c scrub starts 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:44.976647+0000 osd.4 (osd.4) 40 : cluster [DBG] 5.1c scrub ok 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.020580+0000 osd.1 (osd.1) 35 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.021515+0000 osd.1 (osd.1) 36 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.199451+0000 mgr.y (mgr.14535) 73 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:45.890252+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:47.004 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:30:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:30:46.669Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.001884463s 2024-02-22T19:30:47.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.530529+0000 osd.0 (osd.0) 33 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.533620+0000 osd.0 (osd.0) 34 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.656416+0000 osd.7 (osd.7) 35 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.659889+0000 osd.7 (osd.7) 36 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.906242+0000 osd.3 (osd.3) 31 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:45.909285+0000 osd.3 (osd.3) 32 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.530529+0000 osd.0 (osd.0) 33 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.533620+0000 osd.0 (osd.0) 34 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.656416+0000 osd.7 (osd.7) 35 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.659889+0000 osd.7 (osd.7) 36 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.906242+0000 osd.3 (osd.3) 31 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:30:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:45.909285+0000 osd.3 (osd.3) 32 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:30:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.530529+0000 osd.0 (osd.0) 33 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:30:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.533620+0000 osd.0 (osd.0) 34 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:30:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.656416+0000 osd.7 (osd.7) 35 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:30:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.659889+0000 osd.7 (osd.7) 36 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:30:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.906242+0000 osd.3 (osd.3) 31 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:30:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:45.909285+0000 osd.3 (osd.3) 32 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:30:47.739 INFO:teuthology.orchestra.run.smithi053.stdout:137438953512 2024-02-22T19:30:48.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:46.482134+0000 osd.0 (osd.0) 35 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:30:48.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:46.483252+0000 osd.0 (osd.0) 36 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:30:48.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:46.949416+0000 osd.4 (osd.4) 41 : cluster [DBG] 4.d scrub starts 2024-02-22T19:30:48.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:46.953056+0000 osd.4 (osd.4) 42 : cluster [DBG] 4.d scrub ok 2024-02-22T19:30:48.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:47.199967+0000 mgr.y (mgr.14535) 74 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:48.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:47.739836+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.53:0/3304370601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T19:30:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:46.482134+0000 osd.0 (osd.0) 35 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:30:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:46.483252+0000 osd.0 (osd.0) 36 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:30:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:46.949416+0000 osd.4 (osd.4) 41 : cluster [DBG] 4.d scrub starts 2024-02-22T19:30:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:46.953056+0000 osd.4 (osd.4) 42 : cluster [DBG] 4.d scrub ok 2024-02-22T19:30:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:47.199967+0000 mgr.y (mgr.14535) 74 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:47.739836+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.53:0/3304370601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T19:30:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:46.482134+0000 osd.0 (osd.0) 35 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:30:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:46.483252+0000 osd.0 (osd.0) 36 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:30:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:46.949416+0000 osd.4 (osd.4) 41 : cluster [DBG] 4.d scrub starts 2024-02-22T19:30:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:46.953056+0000 osd.4 (osd.4) 42 : cluster [DBG] 4.d scrub ok 2024-02-22T19:30:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:47.199967+0000 mgr.y (mgr.14535) 74 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:47.739836+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.53:0/3304370601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T19:30:48.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953506 got 137438953512 for osd.4 2024-02-22T19:30:48.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-22T19:30:49.146 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:49.661 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:49] "GET /metrics HTTP/1.1" 200 172096 "" "Prometheus/2.33.4" 2024-02-22T19:30:49.661 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:47.926735+0000 osd.3 (osd.3) 33 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:30:49.661 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:47.930236+0000 osd.3 (osd.3) 34 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:30:49.661 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:48.437234+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:49.661 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:48.438098+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:48.439093+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:48.442440+0000 osd.0 (osd.0) 37 : cluster [DBG] 4.b scrub starts 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:48.443711+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:48.444347+0000 osd.0 (osd.0) 38 : cluster [DBG] 4.b scrub ok 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:47.926735+0000 osd.3 (osd.3) 33 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:47.930236+0000 osd.3 (osd.3) 34 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:48.437234+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:48.438098+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:48.439093+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:48.442440+0000 osd.0 (osd.0) 37 : cluster [DBG] 4.b scrub starts 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:48.443711+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:49.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:48.444347+0000 osd.0 (osd.0) 38 : cluster [DBG] 4.b scrub ok 2024-02-22T19:30:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:47.926735+0000 osd.3 (osd.3) 33 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:30:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:47.930236+0000 osd.3 (osd.3) 34 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:30:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:48.437234+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:48.438098+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:30:49.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:48.439093+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:30:49.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:48.442440+0000 osd.0 (osd.0) 37 : cluster [DBG] 4.b scrub starts 2024-02-22T19:30:49.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:48.443711+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:30:49.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:48.444347+0000 osd.0 (osd.0) 38 : cluster [DBG] 4.b scrub ok 2024-02-22T19:30:50.693 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:48.877343+0000 osd.3 (osd.3) 35 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:30:50.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:48.880272+0000 osd.3 (osd.3) 36 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:30:50.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:49.200563+0000 mgr.y (mgr.14535) 75 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:50.694 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:48.877343+0000 osd.3 (osd.3) 35 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:30:50.694 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:48.880272+0000 osd.3 (osd.3) 36 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:30:50.694 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:49.200563+0000 mgr.y (mgr.14535) 75 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:48.877343+0000 osd.3 (osd.3) 35 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:30:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:48.880272+0000 osd.3 (osd.3) 36 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:30:50.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:49.200563+0000 mgr.y (mgr.14535) 75 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:51.134 INFO:teuthology.orchestra.run.smithi053.stdout:167503724579 2024-02-22T19:30:51.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:49.655308+0000 osd.7 (osd.7) 37 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:30:51.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:49.658221+0000 osd.7 (osd.7) 38 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:30:51.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:51.134568+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? 172.21.15.53:0/665947698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T19:30:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:49.655308+0000 osd.7 (osd.7) 37 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:30:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:49.658221+0000 osd.7 (osd.7) 38 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:30:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:51.134568+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? 172.21.15.53:0/665947698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T19:30:51.808 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724573 got 167503724579 for osd.5 2024-02-22T19:30:51.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-22T19:30:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:49.655308+0000 osd.7 (osd.7) 37 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:30:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:49.658221+0000 osd.7 (osd.7) 38 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:30:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:51.134568+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? 172.21.15.53:0/665947698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T19:30:52.092 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:52.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:50.476663+0000 osd.0 (osd.0) 39 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:30:52.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:50.477732+0000 osd.0 (osd.0) 40 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:30:52.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:50.933093+0000 osd.4 (osd.4) 43 : cluster [DBG] 4.c scrub starts 2024-02-22T19:30:52.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:50.936704+0000 osd.4 (osd.4) 44 : cluster [DBG] 4.c scrub ok 2024-02-22T19:30:52.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:51.200967+0000 mgr.y (mgr.14535) 76 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:52.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:50.476663+0000 osd.0 (osd.0) 39 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:30:52.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:50.477732+0000 osd.0 (osd.0) 40 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:30:52.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:50.933093+0000 osd.4 (osd.4) 43 : cluster [DBG] 4.c scrub starts 2024-02-22T19:30:52.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:50.936704+0000 osd.4 (osd.4) 44 : cluster [DBG] 4.c scrub ok 2024-02-22T19:30:52.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:51.200967+0000 mgr.y (mgr.14535) 76 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:50.476663+0000 osd.0 (osd.0) 39 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:30:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:50.477732+0000 osd.0 (osd.0) 40 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:30:52.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:50.933093+0000 osd.4 (osd.4) 43 : cluster [DBG] 4.c scrub starts 2024-02-22T19:30:52.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:50.936704+0000 osd.4 (osd.4) 44 : cluster [DBG] 4.c scrub ok 2024-02-22T19:30:52.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:51.200967+0000 mgr.y (mgr.14535) 76 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:53.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:51.887630+0000 osd.3 (osd.3) 37 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:30:53.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:51.890608+0000 osd.3 (osd.3) 38 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:30:53.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:51.887630+0000 osd.3 (osd.3) 37 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:30:53.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:51.890608+0000 osd.3 (osd.3) 38 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:30:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:51.887630+0000 osd.3 (osd.3) 37 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:30:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:51.890608+0000 osd.3 (osd.3) 38 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:30:54.206 INFO:teuthology.orchestra.run.smithi053.stdout:193273528350 2024-02-22T19:30:54.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:52.982502+0000 osd.4 (osd.4) 45 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:30:54.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:52.985406+0000 osd.4 (osd.4) 46 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:30:54.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:53.201347+0000 mgr.y (mgr.14535) 77 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:54.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:54.206577+0000 mon.a (mon.0) 738 : audit [DBG] from='client.? 172.21.15.53:0/2205005064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-22T19:30:54.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:52.982502+0000 osd.4 (osd.4) 45 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:30:54.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:52.985406+0000 osd.4 (osd.4) 46 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:30:54.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:53.201347+0000 mgr.y (mgr.14535) 77 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:54.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:54.206577+0000 mon.a (mon.0) 738 : audit [DBG] from='client.? 172.21.15.53:0/2205005064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-22T19:30:54.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:30:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:30:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:52.982502+0000 osd.4 (osd.4) 45 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:30:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:52.985406+0000 osd.4 (osd.4) 46 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:30:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:53.201347+0000 mgr.y (mgr.14535) 77 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:54.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:54.206577+0000 mon.a (mon.0) 738 : audit [DBG] from='client.? 172.21.15.53:0/2205005064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-22T19:30:54.826 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528344 got 193273528350 for osd.6 2024-02-22T19:30:54.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-22T19:30:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:30:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:30:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:55.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:53.874540+0000 osd.3 (osd.3) 39 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:30:55.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:53.877996+0000 osd.3 (osd.3) 40 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:30:55.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:54.015227+0000 osd.4 (osd.4) 47 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:30:55.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:54.022447+0000 osd.4 (osd.4) 48 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:30:55.586 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:54.271387+0000 mgr.y (mgr.14535) 78 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:53.874540+0000 osd.3 (osd.3) 39 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:30:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:53.877996+0000 osd.3 (osd.3) 40 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:30:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:54.015227+0000 osd.4 (osd.4) 47 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:30:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:54.022447+0000 osd.4 (osd.4) 48 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:30:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:54.271387+0000 mgr.y (mgr.14535) 78 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:53.874540+0000 osd.3 (osd.3) 39 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:30:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:53.877996+0000 osd.3 (osd.3) 40 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:30:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:54.015227+0000 osd.4 (osd.4) 47 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:30:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:54.022447+0000 osd.4 (osd.4) 48 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:30:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:54.271387+0000 mgr.y (mgr.14535) 78 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:30:56.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:55.017577+0000 osd.4 (osd.4) 49 : cluster [DBG] 6.e deep-scrub starts 2024-02-22T19:30:56.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:55.019983+0000 osd.4 (osd.4) 50 : cluster [DBG] 6.e deep-scrub ok 2024-02-22T19:30:56.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:55.202090+0000 mgr.y (mgr.14535) 79 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:56.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:55.534335+0000 osd.0 (osd.0) 41 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:30:56.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:55.536035+0000 osd.0 (osd.0) 42 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:30:56.721 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:55.017577+0000 osd.4 (osd.4) 49 : cluster [DBG] 6.e deep-scrub starts 2024-02-22T19:30:56.721 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:55.019983+0000 osd.4 (osd.4) 50 : cluster [DBG] 6.e deep-scrub ok 2024-02-22T19:30:56.721 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:55.202090+0000 mgr.y (mgr.14535) 79 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:56.721 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:55.534335+0000 osd.0 (osd.0) 41 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:30:56.721 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:55.536035+0000 osd.0 (osd.0) 42 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:30:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:55.017577+0000 osd.4 (osd.4) 49 : cluster [DBG] 6.e deep-scrub starts 2024-02-22T19:30:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:55.019983+0000 osd.4 (osd.4) 50 : cluster [DBG] 6.e deep-scrub ok 2024-02-22T19:30:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:55.202090+0000 mgr.y (mgr.14535) 79 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:30:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:55.534335+0000 osd.0 (osd.0) 41 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:30:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:55.536035+0000 osd.0 (osd.0) 42 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:30:57.148 INFO:teuthology.orchestra.run.smithi053.stdout:227633266714 2024-02-22T19:30:57.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:57.148616+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? 172.21.15.53:0/1157188233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-22T19:30:57.753 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:57.148616+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? 172.21.15.53:0/1157188233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-22T19:30:57.773 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266708 got 227633266714 for osd.7 2024-02-22T19:30:57.774 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-22T19:30:57.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:30:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:57.148616+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? 172.21.15.53:0/1157188233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-22T19:30:58.052 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:30:58.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:56.949931+0000 osd.3 (osd.3) 41 : cluster [DBG] 3.1e deep-scrub starts 2024-02-22T19:30:58.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:56.953180+0000 osd.3 (osd.3) 42 : cluster [DBG] 3.1e deep-scrub ok 2024-02-22T19:30:58.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:57.202526+0000 mgr.y (mgr.14535) 80 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:58.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:57.858389+0000 osd.5 (osd.5) 39 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:58.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:57.858444+0000 osd.5 (osd.5) 40 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:56.949931+0000 osd.3 (osd.3) 41 : cluster [DBG] 3.1e deep-scrub starts 2024-02-22T19:30:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:56.953180+0000 osd.3 (osd.3) 42 : cluster [DBG] 3.1e deep-scrub ok 2024-02-22T19:30:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:57.202526+0000 mgr.y (mgr.14535) 80 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:57.858389+0000 osd.5 (osd.5) 39 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:57.858444+0000 osd.5 (osd.5) 40 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:56.949931+0000 osd.3 (osd.3) 41 : cluster [DBG] 3.1e deep-scrub starts 2024-02-22T19:30:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:56.953180+0000 osd.3 (osd.3) 42 : cluster [DBG] 3.1e deep-scrub ok 2024-02-22T19:30:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:57.202526+0000 mgr.y (mgr.14535) 80 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:30:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:57.858389+0000 osd.5 (osd.5) 39 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:30:58.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:57.858444+0000 osd.5 (osd.5) 40 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:30:59.526 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:57.985045+0000 osd.3 (osd.3) 43 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:30:59.526 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:57.987948+0000 osd.3 (osd.3) 44 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:30:59.526 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:58.473953+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-22T19:30:59.526 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:30:59] "GET /metrics HTTP/1.1" 200 172103 "" "Prometheus/2.33.4" 2024-02-22T19:30:59.526 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:57.985045+0000 osd.3 (osd.3) 43 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:30:59.527 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:57.987948+0000 osd.3 (osd.3) 44 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:30:59.527 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:30:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:58.473953+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-22T19:30:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:57.985045+0000 osd.3 (osd.3) 43 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:30:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:57.987948+0000 osd.3 (osd.3) 44 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:30:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:30:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:58.473953+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-22T19:30:59.972 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:30:59.974 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:31:00.685 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:30:59.203024+0000 mgr.y (mgr.14535) 81 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:00.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:30:59.203024+0000 mgr.y (mgr.14535) 81 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:00.710 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-02-22T19:30:59.202670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":831,"num_read_kb":583,"num_write":428,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":488,"ondisk_log_size":488,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381736,"kb_used_data":4968,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579480,"statfs":{"total":767960285184,"available":765521387520,"internally_reserved":0,"allocated":5087232,"data_stored":2334313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002473"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096862+0000","last_change":"2024-02-22T19:30:28.096813+0000","last_active":"2024-02-22T19:30:28.096862+0000","last_peered":"2024-02-22T19:30:28.096862+0000","last_clean":"2024-02-22T19:30:28.096862+0000","last_became_active":"2024-02-22T19:29:07.099281+0000","last_became_peered":"2024-02-22T19:29:07.099281+0000","last_unstale":"2024-02-22T19:30:28.096862+0000","last_undegraded":"2024-02-22T19:30:28.096862+0000","last_fullsized":"2024-02-22T19:30:28.096862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.074360+0000","last_change":"2024-02-22T19:30:24.056145+0000","last_active":"2024-02-22T19:30:28.074360+0000","last_peered":"2024-02-22T19:30:28.074360+0000","last_clean":"2024-02-22T19:30:28.074360+0000","last_became_active":"2024-02-22T19:29:05.091467+0000","last_became_peered":"2024-02-22T19:29:05.091467+0000","last_unstale":"2024-02-22T19:30:28.074360+0000","last_undegraded":"2024-02-22T19:30:28.074360+0000","last_fullsized":"2024-02-22T19:30:28.074360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128363+0000","last_change":"2024-02-22T19:30:07.664488+0000","last_active":"2024-02-22T19:30:28.128363+0000","last_peered":"2024-02-22T19:30:28.128363+0000","last_clean":"2024-02-22T19:30:28.128363+0000","last_became_active":"2024-02-22T19:29:01.002834+0000","last_became_peered":"2024-02-22T19:29:01.002834+0000","last_unstale":"2024-02-22T19:30:28.128363+0000","last_undegraded":"2024-02-22T19:30:28.128363+0000","last_fullsized":"2024-02-22T19:30:28.128363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.664424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093600+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:30:28.093600+0000","last_peered":"2024-02-22T19:30:28.093600+0000","last_clean":"2024-02-22T19:30:28.093600+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:30:28.093600+0000","last_undegraded":"2024-02-22T19:30:28.093600+0000","last_fullsized":"2024-02-22T19:30:28.093600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077101+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:30:28.077101+0000","last_peered":"2024-02-22T19:30:28.077101+0000","last_clean":"2024-02-22T19:30:28.077101+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:30:28.077101+0000","last_undegraded":"2024-02-22T19:30:28.077101+0000","last_fullsized":"2024-02-22T19:30:28.077101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:52.985523+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:30:52.985523+0000","last_peered":"2024-02-22T19:30:52.985523+0000","last_clean":"2024-02-22T19:30:52.985523+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:30:52.985523+0000","last_undegraded":"2024-02-22T19:30:52.985523+0000","last_fullsized":"2024-02-22T19:30:52.985523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:56.953343+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:30:56.953343+0000","last_peered":"2024-02-22T19:30:56.953343+0000","last_clean":"2024-02-22T19:30:56.953343+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:30:56.953343+0000","last_undegraded":"2024-02-22T19:30:56.953343+0000","last_fullsized":"2024-02-22T19:30:56.953343+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:30.877529+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:30:30.877529+0000","last_peered":"2024-02-22T19:30:30.877529+0000","last_clean":"2024-02-22T19:30:30.877529+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:30:30.877529+0000","last_undegraded":"2024-02-22T19:30:30.877529+0000","last_fullsized":"2024-02-22T19:30:30.877529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127622+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:30:28.127622+0000","last_peered":"2024-02-22T19:30:28.127622+0000","last_clean":"2024-02-22T19:30:28.127622+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:30:28.127622+0000","last_undegraded":"2024-02-22T19:30:28.127622+0000","last_fullsized":"2024-02-22T19:30:28.127622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096341+0000","last_change":"2024-02-22T19:30:26.528062+0000","last_active":"2024-02-22T19:30:28.096341+0000","last_peered":"2024-02-22T19:30:28.096341+0000","last_clean":"2024-02-22T19:30:28.096341+0000","last_became_active":"2024-02-22T19:29:05.091567+0000","last_became_peered":"2024-02-22T19:29:05.091567+0000","last_unstale":"2024-02-22T19:30:28.096341+0000","last_undegraded":"2024-02-22T19:30:28.096341+0000","last_fullsized":"2024-02-22T19:30:28.096341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.527965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.527965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096622+0000","last_change":"2024-02-22T19:30:10.440133+0000","last_active":"2024-02-22T19:30:28.096622+0000","last_peered":"2024-02-22T19:30:28.096622+0000","last_clean":"2024-02-22T19:30:28.096622+0000","last_became_active":"2024-02-22T19:29:01.005184+0000","last_became_peered":"2024-02-22T19:29:01.005184+0000","last_unstale":"2024-02-22T19:30:28.096622+0000","last_undegraded":"2024-02-22T19:30:28.096622+0000","last_fullsized":"2024-02-22T19:30:28.096622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.440058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.440058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093874+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:30:28.093874+0000","last_peered":"2024-02-22T19:30:28.093874+0000","last_clean":"2024-02-22T19:30:28.093874+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:30:28.093874+0000","last_undegraded":"2024-02-22T19:30:28.093874+0000","last_fullsized":"2024-02-22T19:30:28.093874+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096064+0000","last_change":"2024-02-22T19:30:18.496805+0000","last_active":"2024-02-22T19:30:28.096064+0000","last_peered":"2024-02-22T19:30:28.096064+0000","last_clean":"2024-02-22T19:30:28.096064+0000","last_became_active":"2024-02-22T19:29:07.100382+0000","last_became_peered":"2024-02-22T19:29:07.100382+0000","last_unstale":"2024-02-22T19:30:28.096064+0000","last_undegraded":"2024-02-22T19:30:28.096064+0000","last_fullsized":"2024-02-22T19:30:28.096064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.619660+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:30:41.619660+0000","last_peered":"2024-02-22T19:30:41.619660+0000","last_clean":"2024-02-22T19:30:41.619660+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:30:41.619660+0000","last_undegraded":"2024-02-22T19:30:41.619660+0000","last_fullsized":"2024-02-22T19:30:41.619660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096551+0000","last_change":"2024-02-22T19:30:22.574826+0000","last_active":"2024-02-22T19:30:28.096551+0000","last_peered":"2024-02-22T19:30:28.096551+0000","last_clean":"2024-02-22T19:30:28.096551+0000","last_became_active":"2024-02-22T19:29:01.007689+0000","last_became_peered":"2024-02-22T19:29:01.007689+0000","last_unstale":"2024-02-22T19:30:28.096551+0000","last_undegraded":"2024-02-22T19:30:28.096551+0000","last_fullsized":"2024-02-22T19:30:28.096551+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:22.574750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.574750+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:40.016875+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:30:40.016875+0000","last_peered":"2024-02-22T19:30:40.016875+0000","last_clean":"2024-02-22T19:30:40.016875+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:30:40.016875+0000","last_undegraded":"2024-02-22T19:30:40.016875+0000","last_fullsized":"2024-02-22T19:30:40.016875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:34.059668+0000","last_change":"2024-02-22T19:30:34.059638+0000","last_active":"2024-02-22T19:30:34.059668+0000","last_peered":"2024-02-22T19:30:34.059668+0000","last_clean":"2024-02-22T19:30:34.059668+0000","last_became_active":"2024-02-22T19:29:07.197580+0000","last_became_peered":"2024-02-22T19:29:07.197580+0000","last_unstale":"2024-02-22T19:30:34.059668+0000","last_undegraded":"2024-02-22T19:30:34.059668+0000","last_fullsized":"2024-02-22T19:30:34.059668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.981453+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:30:41.981453+0000","last_peered":"2024-02-22T19:30:41.981453+0000","last_clean":"2024-02-22T19:30:41.981453+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:30:41.981453+0000","last_undegraded":"2024-02-22T19:30:41.981453+0000","last_fullsized":"2024-02-22T19:30:41.981453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127884+0000","last_change":"2024-02-22T19:30:00.634139+0000","last_active":"2024-02-22T19:30:28.127884+0000","last_peered":"2024-02-22T19:30:28.127884+0000","last_clean":"2024-02-22T19:30:28.127884+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:30:28.127884+0000","last_undegraded":"2024-02-22T19:30:28.127884+0000","last_fullsized":"2024-02-22T19:30:28.127884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:30:00.634060+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:32.144975+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:30:32.144975+0000","last_peered":"2024-02-22T19:30:32.144975+0000","last_clean":"2024-02-22T19:30:32.144975+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:30:32.144975+0000","last_undegraded":"2024-02-22T19:30:32.144975+0000","last_fullsized":"2024-02-22T19:30:32.144975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077996+0000","last_change":"2024-02-22T19:30:26.823434+0000","last_active":"2024-02-22T19:30:28.077996+0000","last_peered":"2024-02-22T19:30:28.077996+0000","last_clean":"2024-02-22T19:30:28.077996+0000","last_became_active":"2024-02-22T19:29:07.197840+0000","last_became_peered":"2024-02-22T19:29:07.197840+0000","last_unstale":"2024-02-22T19:30:28.077996+0000","last_undegraded":"2024-02-22T19:30:28.077996+0000","last_fullsized":"2024-02-22T19:30:28.077996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.823363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.823363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:44.976744+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:30:44.976744+0000","last_peered":"2024-02-22T19:30:44.976744+0000","last_clean":"2024-02-22T19:30:44.976744+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:30:44.976744+0000","last_undegraded":"2024-02-22T19:30:44.976744+0000","last_fullsized":"2024-02-22T19:30:44.976744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.137069+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:30:31.137069+0000","last_peered":"2024-02-22T19:30:31.137069+0000","last_clean":"2024-02-22T19:30:31.137069+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:30:31.137069+0000","last_undegraded":"2024-02-22T19:30:31.137069+0000","last_fullsized":"2024-02-22T19:30:31.137069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077293+0000","last_change":"2024-02-22T19:29:03.083045+0000","last_active":"2024-02-22T19:30:28.077293+0000","last_peered":"2024-02-22T19:30:28.077293+0000","last_clean":"2024-02-22T19:30:28.077293+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:30:28.077293+0000","last_undegraded":"2024-02-22T19:30:28.077293+0000","last_fullsized":"2024-02-22T19:30:28.077293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:29:02.056447+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:37.617849+0000","last_change":"2024-02-22T19:30:37.617824+0000","last_active":"2024-02-22T19:30:37.617849+0000","last_peered":"2024-02-22T19:30:37.617849+0000","last_clean":"2024-02-22T19:30:37.617849+0000","last_became_active":"2024-02-22T19:29:07.104332+0000","last_became_peered":"2024-02-22T19:29:07.104332+0000","last_unstale":"2024-02-22T19:30:37.617849+0000","last_undegraded":"2024-02-22T19:30:37.617849+0000","last_fullsized":"2024-02-22T19:30:37.617849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.147697+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:30:59.147697+0000","last_peered":"2024-02-22T19:30:59.147697+0000","last_clean":"2024-02-22T19:30:59.147697+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:30:59.147697+0000","last_undegraded":"2024-02-22T19:30:59.147697+0000","last_fullsized":"2024-02-22T19:30:59.147697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.074437+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:30:28.074437+0000","last_peered":"2024-02-22T19:30:28.074437+0000","last_clean":"2024-02-22T19:30:28.074437+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:30:28.074437+0000","last_undegraded":"2024-02-22T19:30:28.074437+0000","last_fullsized":"2024-02-22T19:30:28.074437+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128138+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:30:28.128138+0000","last_peered":"2024-02-22T19:30:28.128138+0000","last_clean":"2024-02-22T19:30:28.128138+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:30:28.128138+0000","last_undegraded":"2024-02-22T19:30:28.128138+0000","last_fullsized":"2024-02-22T19:30:28.128138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.070129+0000","last_change":"2024-02-22T19:30:31.070100+0000","last_active":"2024-02-22T19:30:31.070129+0000","last_peered":"2024-02-22T19:30:31.070129+0000","last_clean":"2024-02-22T19:30:31.070129+0000","last_became_active":"2024-02-22T19:29:07.089971+0000","last_became_peered":"2024-02-22T19:29:07.089971+0000","last_unstale":"2024-02-22T19:30:31.070129+0000","last_undegraded":"2024-02-22T19:30:31.070129+0000","last_fullsized":"2024-02-22T19:30:31.070129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:39.660635+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:30:39.660635+0000","last_peered":"2024-02-22T19:30:39.660635+0000","last_clean":"2024-02-22T19:30:39.660635+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:30:39.660635+0000","last_undegraded":"2024-02-22T19:30:39.660635+0000","last_fullsized":"2024-02-22T19:30:39.660635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:57.988120+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:30:57.988120+0000","last_peered":"2024-02-22T19:30:57.988120+0000","last_clean":"2024-02-22T19:30:57.988120+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:30:57.988120+0000","last_undegraded":"2024-02-22T19:30:57.988120+0000","last_fullsized":"2024-02-22T19:30:57.988120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077812+0000","last_change":"2024-02-22T19:30:06.607076+0000","last_active":"2024-02-22T19:30:28.077812+0000","last_peered":"2024-02-22T19:30:28.077812+0000","last_clean":"2024-02-22T19:30:28.077812+0000","last_became_active":"2024-02-22T19:29:03.087740+0000","last_became_peered":"2024-02-22T19:29:03.087740+0000","last_unstale":"2024-02-22T19:30:28.077812+0000","last_undegraded":"2024-02-22T19:30:28.077812+0000","last_fullsized":"2024-02-22T19:30:28.077812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.607011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.607011+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:40.624644+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:30:40.624644+0000","last_peered":"2024-02-22T19:30:40.624644+0000","last_clean":"2024-02-22T19:30:40.624644+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:30:40.624644+0000","last_undegraded":"2024-02-22T19:30:40.624644+0000","last_fullsized":"2024-02-22T19:30:40.624644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.539925+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:30:35.539925+0000","last_peered":"2024-02-22T19:30:35.539925+0000","last_clean":"2024-02-22T19:30:35.539925+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:30:35.539925+0000","last_undegraded":"2024-02-22T19:30:35.539925+0000","last_fullsized":"2024-02-22T19:30:35.539925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128291+0000","last_change":"2024-02-22T19:30:08.671313+0000","last_active":"2024-02-22T19:30:28.128291+0000","last_peered":"2024-02-22T19:30:28.128291+0000","last_clean":"2024-02-22T19:30:28.128291+0000","last_became_active":"2024-02-22T19:29:01.009788+0000","last_became_peered":"2024-02-22T19:29:01.009788+0000","last_unstale":"2024-02-22T19:30:28.128291+0000","last_undegraded":"2024-02-22T19:30:28.128291+0000","last_fullsized":"2024-02-22T19:30:28.128291+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:08.671238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:08.671238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:47.930394+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:30:47.930394+0000","last_peered":"2024-02-22T19:30:47.930394+0000","last_clean":"2024-02-22T19:30:47.930394+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:30:47.930394+0000","last_undegraded":"2024-02-22T19:30:47.930394+0000","last_fullsized":"2024-02-22T19:30:47.930394+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076912+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:30:28.076912+0000","last_peered":"2024-02-22T19:30:28.076912+0000","last_clean":"2024-02-22T19:30:28.076912+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:30:28.076912+0000","last_undegraded":"2024-02-22T19:30:28.076912+0000","last_fullsized":"2024-02-22T19:30:28.076912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597591+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:30:28.597591+0000","last_peered":"2024-02-22T19:30:28.597591+0000","last_clean":"2024-02-22T19:30:28.597591+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:30:28.597591+0000","last_undegraded":"2024-02-22T19:30:28.597591+0000","last_fullsized":"2024-02-22T19:30:28.597591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096826+0000","last_change":"2024-02-22T19:30:07.463186+0000","last_active":"2024-02-22T19:30:28.096826+0000","last_peered":"2024-02-22T19:30:28.096826+0000","last_clean":"2024-02-22T19:30:28.096826+0000","last_became_active":"2024-02-22T19:29:01.006896+0000","last_became_peered":"2024-02-22T19:29:01.006896+0000","last_unstale":"2024-02-22T19:30:28.096826+0000","last_undegraded":"2024-02-22T19:30:28.096826+0000","last_fullsized":"2024-02-22T19:30:28.096826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.463122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.463122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:42.903912+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:30:42.903912+0000","last_peered":"2024-02-22T19:30:42.903912+0000","last_clean":"2024-02-22T19:30:42.903912+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:30:42.903912+0000","last_undegraded":"2024-02-22T19:30:42.903912+0000","last_fullsized":"2024-02-22T19:30:42.903912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127355+0000","last_change":"2024-02-22T19:29:07.102062+0000","last_active":"2024-02-22T19:30:28.127355+0000","last_peered":"2024-02-22T19:30:28.127355+0000","last_clean":"2024-02-22T19:30:28.127355+0000","last_became_active":"2024-02-22T19:29:07.101369+0000","last_became_peered":"2024-02-22T19:29:07.101369+0000","last_unstale":"2024-02-22T19:30:28.127355+0000","last_undegraded":"2024-02-22T19:30:28.127355+0000","last_fullsized":"2024-02-22T19:30:28.127355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:48.880406+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:30:48.880406+0000","last_peered":"2024-02-22T19:30:48.880406+0000","last_clean":"2024-02-22T19:30:48.880406+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:30:48.880406+0000","last_undegraded":"2024-02-22T19:30:48.880406+0000","last_fullsized":"2024-02-22T19:30:48.880406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597569+0000","last_change":"2024-02-22T19:30:03.688086+0000","last_active":"2024-02-22T19:30:28.597569+0000","last_peered":"2024-02-22T19:30:28.597569+0000","last_clean":"2024-02-22T19:30:28.597569+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:30:28.597569+0000","last_undegraded":"2024-02-22T19:30:28.597569+0000","last_fullsized":"2024-02-22T19:30:28.597569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.688024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.688024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.073965+0000","last_change":"2024-02-22T19:30:14.042137+0000","last_active":"2024-02-22T19:30:28.073965+0000","last_peered":"2024-02-22T19:30:28.073965+0000","last_clean":"2024-02-22T19:30:28.073965+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:30:28.073965+0000","last_undegraded":"2024-02-22T19:30:28.073965+0000","last_fullsized":"2024-02-22T19:30:28.073965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.042059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.042059+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076592+0000","last_change":"2024-02-22T19:30:13.709779+0000","last_active":"2024-02-22T19:30:28.076592+0000","last_peered":"2024-02-22T19:30:28.076592+0000","last_clean":"2024-02-22T19:30:28.076592+0000","last_became_active":"2024-02-22T19:29:07.102604+0000","last_became_peered":"2024-02-22T19:29:07.102604+0000","last_unstale":"2024-02-22T19:30:28.076592+0000","last_undegraded":"2024-02-22T19:30:28.076592+0000","last_fullsized":"2024-02-22T19:30:28.076592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.709691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.709691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:45.021630+0000","last_change":"2024-02-22T19:30:45.021593+0000","last_active":"2024-02-22T19:30:45.021630+0000","last_peered":"2024-02-22T19:30:45.021630+0000","last_clean":"2024-02-22T19:30:45.021630+0000","last_became_active":"2024-02-22T19:29:05.091550+0000","last_became_peered":"2024-02-22T19:29:05.091550+0000","last_unstale":"2024-02-22T19:30:45.021630+0000","last_undegraded":"2024-02-22T19:30:45.021630+0000","last_fullsized":"2024-02-22T19:30:45.021630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.969712+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:30:41.969712+0000","last_peered":"2024-02-22T19:30:41.969712+0000","last_clean":"2024-02-22T19:30:41.969712+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:30:41.969712+0000","last_undegraded":"2024-02-22T19:30:41.969712+0000","last_fullsized":"2024-02-22T19:30:41.969712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.008739+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:30:41.008739+0000","last_peered":"2024-02-22T19:30:41.008739+0000","last_clean":"2024-02-22T19:30:41.008739+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:30:41.008739+0000","last_undegraded":"2024-02-22T19:30:41.008739+0000","last_fullsized":"2024-02-22T19:30:41.008739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:46.483352+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:30:46.483352+0000","last_peered":"2024-02-22T19:30:46.483352+0000","last_clean":"2024-02-22T19:30:46.483352+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:30:46.483352+0000","last_undegraded":"2024-02-22T19:30:46.483352+0000","last_fullsized":"2024-02-22T19:30:46.483352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.150020+0000","last_change":"2024-02-22T19:30:21.533563+0000","last_active":"2024-02-22T19:30:59.150020+0000","last_peered":"2024-02-22T19:30:59.150020+0000","last_clean":"2024-02-22T19:30:59.150020+0000","last_became_active":"2024-02-22T19:29:05.092444+0000","last_became_peered":"2024-02-22T19:29:05.092444+0000","last_unstale":"2024-02-22T19:30:59.150020+0000","last_undegraded":"2024-02-22T19:30:59.150020+0000","last_fullsized":"2024-02-22T19:30:59.150020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:21.533452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.533452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:39.596263+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:30:39.596263+0000","last_peered":"2024-02-22T19:30:39.596263+0000","last_clean":"2024-02-22T19:30:39.596263+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:30:39.596263+0000","last_undegraded":"2024-02-22T19:30:39.596263+0000","last_fullsized":"2024-02-22T19:30:39.596263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:32.846447+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:30:32.846447+0000","last_peered":"2024-02-22T19:30:32.846447+0000","last_clean":"2024-02-22T19:30:32.846447+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:30:32.846447+0000","last_undegraded":"2024-02-22T19:30:32.846447+0000","last_fullsized":"2024-02-22T19:30:32.846447+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.094734+0000","last_change":"2024-02-22T19:30:16.057704+0000","last_active":"2024-02-22T19:30:28.094734+0000","last_peered":"2024-02-22T19:30:28.094734+0000","last_clean":"2024-02-22T19:30:28.094734+0000","last_became_active":"2024-02-22T19:29:07.097503+0000","last_became_peered":"2024-02-22T19:29:07.097503+0000","last_unstale":"2024-02-22T19:30:28.094734+0000","last_undegraded":"2024-02-22T19:30:28.094734+0000","last_fullsized":"2024-02-22T19:30:28.094734+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.628292+0000","last_change":"2024-02-22T19:29:05.095926+0000","last_active":"2024-02-22T19:30:58.628292+0000","last_peered":"2024-02-22T19:30:58.628292+0000","last_clean":"2024-02-22T19:30:58.628292+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:30:58.628292+0000","last_undegraded":"2024-02-22T19:30:58.628292+0000","last_fullsized":"2024-02-22T19:30:58.628292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:29:04.066424+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127607+0000","last_change":"2024-02-22T19:30:03.653428+0000","last_active":"2024-02-22T19:30:28.127607+0000","last_peered":"2024-02-22T19:30:28.127607+0000","last_clean":"2024-02-22T19:30:28.127607+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:30:28.127607+0000","last_undegraded":"2024-02-22T19:30:28.127607+0000","last_fullsized":"2024-02-22T19:30:28.127607+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.653348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.653348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'9","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096872+0000","last_change":"2024-02-22T19:30:05.473412+0000","last_active":"2024-02-22T19:30:28.096872+0000","last_peered":"2024-02-22T19:30:28.096872+0000","last_clean":"2024-02-22T19:30:28.096872+0000","last_became_active":"2024-02-22T19:29:03.448670+0000","last_became_peered":"2024-02-22T19:29:03.448670+0000","last_unstale":"2024-02-22T19:30:28.096872+0000","last_undegraded":"2024-02-22T19:30:28.096872+0000","last_fullsized":"2024-02-22T19:30:28.096872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:05.473341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.473341+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.126398+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:30:28.126398+0000","last_peered":"2024-02-22T19:30:28.126398+0000","last_clean":"2024-02-22T19:30:28.126398+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:30:28.126398+0000","last_undegraded":"2024-02-22T19:30:28.126398+0000","last_fullsized":"2024-02-22T19:30:28.126398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077921+0000","last_change":"2024-02-22T19:29:05.096571+0000","last_active":"2024-02-22T19:30:28.077921+0000","last_peered":"2024-02-22T19:30:28.077921+0000","last_clean":"2024-02-22T19:30:28.077921+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:30:28.077921+0000","last_undegraded":"2024-02-22T19:30:28.077921+0000","last_fullsized":"2024-02-22T19:30:28.077921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:29:04.066424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597146+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:30:28.597146+0000","last_peered":"2024-02-22T19:30:28.597146+0000","last_clean":"2024-02-22T19:30:28.597146+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:30:28.597146+0000","last_undegraded":"2024-02-22T19:30:28.597146+0000","last_fullsized":"2024-02-22T19:30:28.597146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128181+0000","last_change":"2024-02-22T19:30:13.766624+0000","last_active":"2024-02-22T19:30:28.128181+0000","last_peered":"2024-02-22T19:30:28.128181+0000","last_clean":"2024-02-22T19:30:28.128181+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:30:28.128181+0000","last_undegraded":"2024-02-22T19:30:28.128181+0000","last_fullsized":"2024-02-22T19:30:28.128181+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:13.766546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.766546+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597324+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:30:28.597324+0000","last_peered":"2024-02-22T19:30:28.597324+0000","last_clean":"2024-02-22T19:30:28.597324+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:30:28.597324+0000","last_undegraded":"2024-02-22T19:30:28.597324+0000","last_fullsized":"2024-02-22T19:30:28.597324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096869+0000","last_change":"2024-02-22T19:30:27.575360+0000","last_active":"2024-02-22T19:30:28.096869+0000","last_peered":"2024-02-22T19:30:28.096869+0000","last_clean":"2024-02-22T19:30:28.096869+0000","last_became_active":"2024-02-22T19:29:05.092190+0000","last_became_peered":"2024-02-22T19:29:05.092190+0000","last_unstale":"2024-02-22T19:30:28.096869+0000","last_undegraded":"2024-02-22T19:30:28.096869+0000","last_fullsized":"2024-02-22T19:30:28.096869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.656363+0000","last_change":"2024-02-22T19:30:29.656325+0000","last_active":"2024-02-22T19:30:29.656363+0000","last_peered":"2024-02-22T19:30:29.656363+0000","last_clean":"2024-02-22T19:30:29.656363+0000","last_became_active":"2024-02-22T19:29:01.007053+0000","last_became_peered":"2024-02-22T19:29:01.007053+0000","last_unstale":"2024-02-22T19:30:29.656363+0000","last_undegraded":"2024-02-22T19:30:29.656363+0000","last_fullsized":"2024-02-22T19:30:29.656363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.078077+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:30:28.078077+0000","last_peered":"2024-02-22T19:30:28.078077+0000","last_clean":"2024-02-22T19:30:28.078077+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:30:28.078077+0000","last_undegraded":"2024-02-22T19:30:28.078077+0000","last_fullsized":"2024-02-22T19:30:28.078077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096374+0000","last_change":"2024-02-22T19:30:14.461934+0000","last_active":"2024-02-22T19:30:28.096374+0000","last_peered":"2024-02-22T19:30:28.096374+0000","last_clean":"2024-02-22T19:30:28.096374+0000","last_became_active":"2024-02-22T19:29:07.197976+0000","last_became_peered":"2024-02-22T19:29:07.197976+0000","last_unstale":"2024-02-22T19:30:28.096374+0000","last_undegraded":"2024-02-22T19:30:28.096374+0000","last_fullsized":"2024-02-22T19:30:28.096374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597893+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:30:28.597893+0000","last_peered":"2024-02-22T19:30:28.597893+0000","last_clean":"2024-02-22T19:30:28.597893+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:30:28.597893+0000","last_undegraded":"2024-02-22T19:30:28.597893+0000","last_fullsized":"2024-02-22T19:30:28.597893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.601696+0000","last_change":"2024-02-22T19:30:28.601662+0000","last_active":"2024-02-22T19:30:28.601696+0000","last_peered":"2024-02-22T19:30:28.601696+0000","last_clean":"2024-02-22T19:30:28.601696+0000","last_became_active":"2024-02-22T19:29:03.447032+0000","last_became_peered":"2024-02-22T19:29:03.447032+0000","last_unstale":"2024-02-22T19:30:28.601696+0000","last_undegraded":"2024-02-22T19:30:28.601696+0000","last_fullsized":"2024-02-22T19:30:28.601696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.147208+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:30:59.147208+0000","last_peered":"2024-02-22T19:30:59.147208+0000","last_clean":"2024-02-22T19:30:59.147208+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:30:59.147208+0000","last_undegraded":"2024-02-22T19:30:59.147208+0000","last_fullsized":"2024-02-22T19:30:59.147208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077004+0000","last_change":"2024-02-22T19:29:07.101872+0000","last_active":"2024-02-22T19:30:28.077004+0000","last_peered":"2024-02-22T19:30:28.077004+0000","last_clean":"2024-02-22T19:30:28.077004+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:30:28.077004+0000","last_undegraded":"2024-02-22T19:30:28.077004+0000","last_fullsized":"2024-02-22T19:30:28.077004+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.557564+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:30:31.557564+0000","last_peered":"2024-02-22T19:30:31.557564+0000","last_clean":"2024-02-22T19:30:31.557564+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:30:31.557564+0000","last_undegraded":"2024-02-22T19:30:31.557564+0000","last_fullsized":"2024-02-22T19:30:31.557564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.031939+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:30:35.031939+0000","last_peered":"2024-02-22T19:30:35.031939+0000","last_clean":"2024-02-22T19:30:35.031939+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:30:35.031939+0000","last_undegraded":"2024-02-22T19:30:35.031939+0000","last_fullsized":"2024-02-22T19:30:35.031939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.176592+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:30:36.176592+0000","last_peered":"2024-02-22T19:30:36.176592+0000","last_clean":"2024-02-22T19:30:36.176592+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:30:36.176592+0000","last_undegraded":"2024-02-22T19:30:36.176592+0000","last_fullsized":"2024-02-22T19:30:36.176592+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.596913+0000","last_change":"2024-02-22T19:29:07.104657+0000","last_active":"2024-02-22T19:30:28.596913+0000","last_peered":"2024-02-22T19:30:28.596913+0000","last_clean":"2024-02-22T19:30:28.596913+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:30:28.596913+0000","last_undegraded":"2024-02-22T19:30:28.596913+0000","last_fullsized":"2024-02-22T19:30:28.596913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.110598+0000","last_change":"2024-02-22T19:30:29.110575+0000","last_active":"2024-02-22T19:30:29.110598+0000","last_peered":"2024-02-22T19:30:29.110598+0000","last_clean":"2024-02-22T19:30:29.110598+0000","last_became_active":"2024-02-22T19:29:05.091952+0000","last_became_peered":"2024-02-22T19:29:05.091952+0000","last_unstale":"2024-02-22T19:30:29.110598+0000","last_undegraded":"2024-02-22T19:30:29.110598+0000","last_fullsized":"2024-02-22T19:30:29.110598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.110513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.110513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597675+0000","last_change":"2024-02-22T19:30:10.619750+0000","last_active":"2024-02-22T19:30:28.597675+0000","last_peered":"2024-02-22T19:30:28.597675+0000","last_clean":"2024-02-22T19:30:28.597675+0000","last_became_active":"2024-02-22T19:29:01.004850+0000","last_became_peered":"2024-02-22T19:29:01.004850+0000","last_unstale":"2024-02-22T19:30:28.597675+0000","last_undegraded":"2024-02-22T19:30:28.597675+0000","last_fullsized":"2024-02-22T19:30:28.597675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.619685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.619685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077301+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:30:28.077301+0000","last_peered":"2024-02-22T19:30:28.077301+0000","last_clean":"2024-02-22T19:30:28.077301+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:30:28.077301+0000","last_undegraded":"2024-02-22T19:30:28.077301+0000","last_fullsized":"2024-02-22T19:30:28.077301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127426+0000","last_change":"2024-02-22T19:29:07.101872+0000","last_active":"2024-02-22T19:30:28.127426+0000","last_peered":"2024-02-22T19:30:28.127426+0000","last_clean":"2024-02-22T19:30:28.127426+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:30:28.127426+0000","last_undegraded":"2024-02-22T19:30:28.127426+0000","last_fullsized":"2024-02-22T19:30:28.127426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:39.210253+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:30:39.210253+0000","last_peered":"2024-02-22T19:30:39.210253+0000","last_clean":"2024-02-22T19:30:39.210253+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:30:39.210253+0000","last_undegraded":"2024-02-22T19:30:39.210253+0000","last_fullsized":"2024-02-22T19:30:39.210253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096625+0000","last_change":"2024-02-22T19:30:17.466789+0000","last_active":"2024-02-22T19:30:28.096625+0000","last_peered":"2024-02-22T19:30:28.096625+0000","last_clean":"2024-02-22T19:30:28.096625+0000","last_became_active":"2024-02-22T19:29:01.010917+0000","last_became_peered":"2024-02-22T19:29:01.010917+0000","last_unstale":"2024-02-22T19:30:28.096625+0000","last_undegraded":"2024-02-22T19:30:28.096625+0000","last_fullsized":"2024-02-22T19:30:28.096625+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:48.444523+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:30:48.444523+0000","last_peered":"2024-02-22T19:30:48.444523+0000","last_clean":"2024-02-22T19:30:48.444523+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:30:48.444523+0000","last_undegraded":"2024-02-22T19:30:48.444523+0000","last_fullsized":"2024-02-22T19:30:48.444523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:51.890712+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:30:51.890712+0000","last_peered":"2024-02-22T19:30:51.890712+0000","last_clean":"2024-02-22T19:30:51.890712+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:30:51.890712+0000","last_undegraded":"2024-02-22T19:30:51.890712+0000","last_fullsized":"2024-02-22T19:30:51.890712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096206+0000","last_change":"2024-02-22T19:30:09.477558+0000","last_active":"2024-02-22T19:30:28.096206+0000","last_peered":"2024-02-22T19:30:28.096206+0000","last_clean":"2024-02-22T19:30:28.096206+0000","last_became_active":"2024-02-22T19:29:01.006282+0000","last_became_peered":"2024-02-22T19:29:01.006282+0000","last_unstale":"2024-02-22T19:30:28.096206+0000","last_undegraded":"2024-02-22T19:30:28.096206+0000","last_fullsized":"2024-02-22T19:30:28.096206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_clean_scrub_stamp":"2024-02-22T19:30:09.477488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093820+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:30:28.093820+0000","last_peered":"2024-02-22T19:30:28.093820+0000","last_clean":"2024-02-22T19:30:28.093820+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:30:28.093820+0000","last_undegraded":"2024-02-22T19:30:28.093820+0000","last_fullsized":"2024-02-22T19:30:28.093820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":113,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.474336+0000","last_change":"2024-02-22T19:30:07.992740+0000","last_active":"2024-02-22T19:30:58.474336+0000","last_peered":"2024-02-22T19:30:58.474336+0000","last_clean":"2024-02-22T19:30:58.474336+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:30:58.474336+0000","last_undegraded":"2024-02-22T19:30:58.474336+0000","last_fullsized":"2024-02-22T19:30:58.474336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:30:07.992671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.992671+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127924+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:30:28.127924+0000","last_peered":"2024-02-22T19:30:28.127924+0000","last_clean":"2024-02-22T19:30:28.127924+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:30:28.127924+0000","last_undegraded":"2024-02-22T19:30:28.127924+0000","last_fullsized":"2024-02-22T19:30:28.127924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.620324+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:30:36.620324+0000","last_peered":"2024-02-22T19:30:36.620324+0000","last_clean":"2024-02-22T19:30:36.620324+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:30:36.620324+0000","last_undegraded":"2024-02-22T19:30:36.620324+0000","last_fullsized":"2024-02-22T19:30:36.620324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.107010+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:30:36.107010+0000","last_peered":"2024-02-22T19:30:36.107010+0000","last_clean":"2024-02-22T19:30:36.107010+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:30:36.107010+0000","last_undegraded":"2024-02-22T19:30:36.107010+0000","last_fullsized":"2024-02-22T19:30:36.107010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.125988+0000","last_change":"2024-02-22T19:30:18.117213+0000","last_active":"2024-02-22T19:30:28.125988+0000","last_peered":"2024-02-22T19:30:28.125988+0000","last_clean":"2024-02-22T19:30:28.125988+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:30:28.125988+0000","last_undegraded":"2024-02-22T19:30:28.125988+0000","last_fullsized":"2024-02-22T19:30:28.125988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.117111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128836+0000","last_change":"2024-02-22T19:30:28.128814+0000","last_active":"2024-02-22T19:30:28.128836+0000","last_peered":"2024-02-22T19:30:28.128836+0000","last_clean":"2024-02-22T19:30:28.128836+0000","last_became_active":"2024-02-22T19:30:24.410718+0000","last_became_peered":"2024-02-22T19:30:24.410718+0000","last_unstale":"2024-02-22T19:30:28.128836+0000","last_undegraded":"2024-02-22T19:30:28.128836+0000","last_fullsized":"2024-02-22T19:30:28.128836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.102231+0000","last_change":"2024-02-22T19:30:31.102193+0000","last_active":"2024-02-22T19:30:31.102231+0000","last_peered":"2024-02-22T19:30:31.102231+0000","last_clean":"2024-02-22T19:30:31.102231+0000","last_became_active":"2024-02-22T19:29:05.098656+0000","last_became_peered":"2024-02-22T19:29:05.098656+0000","last_unstale":"2024-02-22T19:30:31.102231+0000","last_undegraded":"2024-02-22T19:30:31.102231+0000","last_fullsized":"2024-02-22T19:30:31.102231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.102065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.102065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.556128+0000","last_change":"2024-02-22T19:30:35.556098+0000","last_active":"2024-02-22T19:30:35.556128+0000","last_peered":"2024-02-22T19:30:35.556128+0000","last_clean":"2024-02-22T19:30:35.556128+0000","last_became_active":"2024-02-22T19:29:07.098131+0000","last_became_peered":"2024-02-22T19:29:07.098131+0000","last_unstale":"2024-02-22T19:30:35.556128+0000","last_undegraded":"2024-02-22T19:30:35.556128+0000","last_fullsized":"2024-02-22T19:30:35.556128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597272+0000","last_change":"2024-02-22T19:29:15.492725+0000","last_active":"2024-02-22T19:30:28.597272+0000","last_peered":"2024-02-22T19:30:28.597272+0000","last_clean":"2024-02-22T19:30:28.597272+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:30:28.597272+0000","last_undegraded":"2024-02-22T19:30:28.597272+0000","last_fullsized":"2024-02-22T19:30:28.597272+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:15.492609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:29:15.492609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:45.909387+0000","last_change":"2024-02-22T19:30:45.909363+0000","last_active":"2024-02-22T19:30:45.909387+0000","last_peered":"2024-02-22T19:30:45.909387+0000","last_clean":"2024-02-22T19:30:45.909387+0000","last_became_active":"2024-02-22T19:29:01.010904+0000","last_became_peered":"2024-02-22T19:29:01.010904+0000","last_unstale":"2024-02-22T19:30:45.909387+0000","last_undegraded":"2024-02-22T19:30:45.909387+0000","last_fullsized":"2024-02-22T19:30:45.909387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:34.566573+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:30:34.566573+0000","last_peered":"2024-02-22T19:30:34.566573+0000","last_clean":"2024-02-22T19:30:34.566573+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:30:34.566573+0000","last_undegraded":"2024-02-22T19:30:34.566573+0000","last_fullsized":"2024-02-22T19:30:34.566573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:49.658335+0000","last_change":"2024-02-22T19:30:49.658305+0000","last_active":"2024-02-22T19:30:49.658335+0000","last_peered":"2024-02-22T19:30:49.658335+0000","last_clean":"2024-02-22T19:30:49.658335+0000","last_became_active":"2024-02-22T19:29:05.092427+0000","last_became_peered":"2024-02-22T19:29:05.092427+0000","last_unstale":"2024-02-22T19:30:49.658335+0000","last_undegraded":"2024-02-22T19:30:49.658335+0000","last_fullsized":"2024-02-22T19:30:49.658335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:33.113637+0000","last_change":"2024-02-22T19:30:33.113603+0000","last_active":"2024-02-22T19:30:33.113637+0000","last_peered":"2024-02-22T19:30:33.113637+0000","last_clean":"2024-02-22T19:30:33.113637+0000","last_became_active":"2024-02-22T19:29:07.095700+0000","last_became_peered":"2024-02-22T19:29:07.095700+0000","last_unstale":"2024-02-22T19:30:33.113637+0000","last_undegraded":"2024-02-22T19:30:33.113637+0000","last_fullsized":"2024-02-22T19:30:33.113637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:45.533761+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:30:45.533761+0000","last_peered":"2024-02-22T19:30:45.533761+0000","last_clean":"2024-02-22T19:30:45.533761+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:30:45.533761+0000","last_undegraded":"2024-02-22T19:30:45.533761+0000","last_fullsized":"2024-02-22T19:30:45.533761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":156,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.371259+0000","last_change":"2024-02-22T19:30:20.550910+0000","last_active":"2024-02-22T19:30:58.371259+0000","last_peered":"2024-02-22T19:30:58.371259+0000","last_clean":"2024-02-22T19:30:58.371259+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:30:58.371259+0000","last_undegraded":"2024-02-22T19:30:58.371259+0000","last_fullsized":"2024-02-22T19:30:58.371259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:20.550837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.550837+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":112,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:55.536143+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:30:55.536143+0000","last_peered":"2024-02-22T19:30:55.536143+0000","last_clean":"2024-02-22T19:30:55.536143+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:30:55.536143+0000","last_undegraded":"2024-02-22T19:30:55.536143+0000","last_fullsized":"2024-02-22T19:30:55.536143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:33.584973+0000","last_change":"2024-02-22T19:30:33.584923+0000","last_active":"2024-02-22T19:30:33.584973+0000","last_peered":"2024-02-22T19:30:33.584973+0000","last_clean":"2024-02-22T19:30:33.584973+0000","last_became_active":"2024-02-22T19:29:05.091758+0000","last_became_peered":"2024-02-22T19:29:05.091758+0000","last_unstale":"2024-02-22T19:30:33.584973+0000","last_undegraded":"2024-02-22T19:30:33.584973+0000","last_fullsized":"2024-02-22T19:30:33.584973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:42.995014+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:30:42.995014+0000","last_peered":"2024-02-22T19:30:42.995014+0000","last_clean":"2024-02-22T19:30:42.995014+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:30:42.995014+0000","last_undegraded":"2024-02-22T19:30:42.995014+0000","last_fullsized":"2024-02-22T19:30:42.995014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:34.149787+0000","last_change":"2024-02-22T19:30:34.149749+0000","last_active":"2024-02-22T19:30:34.149787+0000","last_peered":"2024-02-22T19:30:34.149787+0000","last_clean":"2024-02-22T19:30:34.149787+0000","last_became_active":"2024-02-22T19:29:01.004572+0000","last_became_peered":"2024-02-22T19:29:01.004572+0000","last_unstale":"2024-02-22T19:30:34.149787+0000","last_undegraded":"2024-02-22T19:30:34.149787+0000","last_fullsized":"2024-02-22T19:30:34.149787+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":75,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127812+0000","last_change":"2024-02-22T19:30:22.683587+0000","last_active":"2024-02-22T19:30:28.127812+0000","last_peered":"2024-02-22T19:30:28.127812+0000","last_clean":"2024-02-22T19:30:28.127812+0000","last_became_active":"2024-02-22T19:29:03.446818+0000","last_became_peered":"2024-02-22T19:29:03.446818+0000","last_unstale":"2024-02-22T19:30:28.127812+0000","last_undegraded":"2024-02-22T19:30:28.127812+0000","last_fullsized":"2024-02-22T19:30:28.127812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077411+0000","last_change":"2024-02-22T19:30:14.659000+0000","last_active":"2024-02-22T19:30:28.077411+0000","last_peered":"2024-02-22T19:30:28.077411+0000","last_clean":"2024-02-22T19:30:28.077411+0000","last_became_active":"2024-02-22T19:29:05.091867+0000","last_became_peered":"2024-02-22T19:29:05.091867+0000","last_unstale":"2024-02-22T19:30:28.077411+0000","last_undegraded":"2024-02-22T19:30:28.077411+0000","last_fullsized":"2024-02-22T19:30:28.077411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127371+0000","last_change":"2024-02-22T19:30:18.719372+0000","last_active":"2024-02-22T19:30:28.127371+0000","last_peered":"2024-02-22T19:30:28.127371+0000","last_clean":"2024-02-22T19:30:28.127371+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:30:28.127371+0000","last_undegraded":"2024-02-22T19:30:28.127371+0000","last_fullsized":"2024-02-22T19:30:28.127371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.719310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.616750+0000","last_change":"2024-02-22T19:30:29.616721+0000","last_active":"2024-02-22T19:30:29.616750+0000","last_peered":"2024-02-22T19:30:29.616750+0000","last_clean":"2024-02-22T19:30:29.616750+0000","last_became_active":"2024-02-22T19:29:01.012034+0000","last_became_peered":"2024-02-22T19:29:01.012034+0000","last_unstale":"2024-02-22T19:30:29.616750+0000","last_undegraded":"2024-02-22T19:30:29.616750+0000","last_fullsized":"2024-02-22T19:30:29.616750+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.073911+0000","last_change":"2024-02-22T19:30:12.059865+0000","last_active":"2024-02-22T19:30:28.073911+0000","last_peered":"2024-02-22T19:30:28.073911+0000","last_clean":"2024-02-22T19:30:28.073911+0000","last_became_active":"2024-02-22T19:29:03.086071+0000","last_became_peered":"2024-02-22T19:29:03.086071+0000","last_unstale":"2024-02-22T19:30:28.073911+0000","last_undegraded":"2024-02-22T19:30:28.073911+0000","last_fullsized":"2024-02-22T19:30:28.073911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:12.059797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:12.059797+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.147458+0000","last_change":"2024-02-22T19:30:44.574471+0000","last_active":"2024-02-22T19:30:59.147458+0000","last_peered":"2024-02-22T19:30:59.147458+0000","last_clean":"2024-02-22T19:30:59.147458+0000","last_became_active":"2024-02-22T19:29:05.094532+0000","last_became_peered":"2024-02-22T19:29:05.094532+0000","last_unstale":"2024-02-22T19:30:59.147458+0000","last_undegraded":"2024-02-22T19:30:59.147458+0000","last_fullsized":"2024-02-22T19:30:59.147458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:38.631799+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:30:38.631799+0000","last_peered":"2024-02-22T19:30:38.631799+0000","last_clean":"2024-02-22T19:30:38.631799+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:30:38.631799+0000","last_undegraded":"2024-02-22T19:30:38.631799+0000","last_fullsized":"2024-02-22T19:30:38.631799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:50.477842+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:30:50.477842+0000","last_peered":"2024-02-22T19:30:50.477842+0000","last_clean":"2024-02-22T19:30:50.477842+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:30:50.477842+0000","last_undegraded":"2024-02-22T19:30:50.477842+0000","last_fullsized":"2024-02-22T19:30:50.477842+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:54.022567+0000","last_change":"2024-02-22T19:30:54.022533+0000","last_active":"2024-02-22T19:30:54.022567+0000","last_peered":"2024-02-22T19:30:54.022567+0000","last_clean":"2024-02-22T19:30:54.022567+0000","last_became_active":"2024-02-22T19:29:03.084036+0000","last_became_peered":"2024-02-22T19:29:03.084036+0000","last_unstale":"2024-02-22T19:30:54.022567+0000","last_undegraded":"2024-02-22T19:30:54.022567+0000","last_fullsized":"2024-02-22T19:30:54.022567+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.628022+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:30:58.628022+0000","last_peered":"2024-02-22T19:30:58.628022+0000","last_clean":"2024-02-22T19:30:58.628022+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:30:58.628022+0000","last_undegraded":"2024-02-22T19:30:58.628022+0000","last_fullsized":"2024-02-22T19:30:58.628022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.099811+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:30:29.099811+0000","last_peered":"2024-02-22T19:30:29.099811+0000","last_clean":"2024-02-22T19:30:29.099811+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:30:29.099811+0000","last_undegraded":"2024-02-22T19:30:29.099811+0000","last_fullsized":"2024-02-22T19:30:29.099811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.904799+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:30:36.904799+0000","last_peered":"2024-02-22T19:30:36.904799+0000","last_clean":"2024-02-22T19:30:36.904799+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:30:36.904799+0000","last_undegraded":"2024-02-22T19:30:36.904799+0000","last_fullsized":"2024-02-22T19:30:36.904799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076692+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:30:28.076692+0000","last_peered":"2024-02-22T19:30:28.076692+0000","last_clean":"2024-02-22T19:30:28.076692+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:30:28.076692+0000","last_undegraded":"2024-02-22T19:30:28.076692+0000","last_fullsized":"2024-02-22T19:30:28.076692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093639+0000","last_change":"2024-02-22T19:30:21.116431+0000","last_active":"2024-02-22T19:30:28.093639+0000","last_peered":"2024-02-22T19:30:28.093639+0000","last_clean":"2024-02-22T19:30:28.093639+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:30:28.093639+0000","last_undegraded":"2024-02-22T19:30:28.093639+0000","last_fullsized":"2024-02-22T19:30:28.093639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.116358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096477+0000","last_change":"2024-02-22T19:30:20.543466+0000","last_active":"2024-02-22T19:30:28.096477+0000","last_peered":"2024-02-22T19:30:28.096477+0000","last_clean":"2024-02-22T19:30:28.096477+0000","last_became_active":"2024-02-22T19:29:07.095452+0000","last_became_peered":"2024-02-22T19:29:07.095452+0000","last_unstale":"2024-02-22T19:30:28.096477+0000","last_undegraded":"2024-02-22T19:30:28.096477+0000","last_fullsized":"2024-02-22T19:30:28.096477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:20.543398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.543398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.479363+0000","last_change":"2024-02-22T19:30:35.999034+0000","last_active":"2024-02-22T19:30:58.479363+0000","last_peered":"2024-02-22T19:30:58.479363+0000","last_clean":"2024-02-22T19:30:58.479363+0000","last_became_active":"2024-02-22T19:29:05.095818+0000","last_became_peered":"2024-02-22T19:29:05.095818+0000","last_unstale":"2024-02-22T19:30:58.479363+0000","last_undegraded":"2024-02-22T19:30:58.479363+0000","last_fullsized":"2024-02-22T19:30:58.479363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076638+0000","last_change":"2024-02-22T19:30:06.720523+0000","last_active":"2024-02-22T19:30:28.076638+0000","last_peered":"2024-02-22T19:30:28.076638+0000","last_clean":"2024-02-22T19:30:28.076638+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:30:28.076638+0000","last_undegraded":"2024-02-22T19:30:28.076638+0000","last_fullsized":"2024-02-22T19:30:28.076638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.720452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.720452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.126355+0000","last_change":"2024-02-22T19:30:27.174847+0000","last_active":"2024-02-22T19:30:28.126355+0000","last_peered":"2024-02-22T19:30:28.126355+0000","last_clean":"2024-02-22T19:30:28.126355+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:30:28.126355+0000","last_undegraded":"2024-02-22T19:30:28.126355+0000","last_fullsized":"2024-02-22T19:30:28.126355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:27.174779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.174779+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077193+0000","last_change":"2024-02-22T19:29:07.198513+0000","last_active":"2024-02-22T19:30:28.077193+0000","last_peered":"2024-02-22T19:30:28.077193+0000","last_clean":"2024-02-22T19:30:28.077193+0000","last_became_active":"2024-02-22T19:29:07.198153+0000","last_became_peered":"2024-02-22T19:29:07.198153+0000","last_unstale":"2024-02-22T19:30:28.077193+0000","last_undegraded":"2024-02-22T19:30:28.077193+0000","last_fullsized":"2024-02-22T19:30:28.077193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.646705+0000","last_change":"2024-02-22T19:30:31.646674+0000","last_active":"2024-02-22T19:30:31.646705+0000","last_peered":"2024-02-22T19:30:31.646705+0000","last_clean":"2024-02-22T19:30:31.646705+0000","last_became_active":"2024-02-22T19:29:05.096026+0000","last_became_peered":"2024-02-22T19:29:05.096026+0000","last_unstale":"2024-02-22T19:30:31.646705+0000","last_undegraded":"2024-02-22T19:30:31.646705+0000","last_fullsized":"2024-02-22T19:30:31.646705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093673+0000","last_change":"2024-02-22T19:30:19.102902+0000","last_active":"2024-02-22T19:30:28.093673+0000","last_peered":"2024-02-22T19:30:28.093673+0000","last_clean":"2024-02-22T19:30:28.093673+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:30:28.093673+0000","last_undegraded":"2024-02-22T19:30:28.093673+0000","last_fullsized":"2024-02-22T19:30:28.093673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:19.102765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:19.102765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.094387+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:30:28.094387+0000","last_peered":"2024-02-22T19:30:28.094387+0000","last_clean":"2024-02-22T19:30:28.094387+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:30:28.094387+0000","last_undegraded":"2024-02-22T19:30:28.094387+0000","last_fullsized":"2024-02-22T19:30:28.094387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.181920+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:30:35.181920+0000","last_peered":"2024-02-22T19:30:35.181920+0000","last_clean":"2024-02-22T19:30:35.181920+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:30:35.181920+0000","last_undegraded":"2024-02-22T19:30:35.181920+0000","last_fullsized":"2024-02-22T19:30:35.181920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.075252+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:30:28.075252+0000","last_peered":"2024-02-22T19:30:28.075252+0000","last_clean":"2024-02-22T19:30:28.075252+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:30:28.075252+0000","last_undegraded":"2024-02-22T19:30:28.075252+0000","last_fullsized":"2024-02-22T19:30:28.075252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077068+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:30:28.077068+0000","last_peered":"2024-02-22T19:30:28.077068+0000","last_clean":"2024-02-22T19:30:28.077068+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:30:28.077068+0000","last_undegraded":"2024-02-22T19:30:28.077068+0000","last_fullsized":"2024-02-22T19:30:28.077068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:46.953185+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:30:46.953185+0000","last_peered":"2024-02-22T19:30:46.953185+0000","last_clean":"2024-02-22T19:30:46.953185+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:30:46.953185+0000","last_undegraded":"2024-02-22T19:30:46.953185+0000","last_fullsized":"2024-02-22T19:30:46.953185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:55.020116+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:30:55.020116+0000","last_peered":"2024-02-22T19:30:55.020116+0000","last_clean":"2024-02-22T19:30:55.020116+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:30:55.020116+0000","last_undegraded":"2024-02-22T19:30:55.020116+0000","last_fullsized":"2024-02-22T19:30:55.020116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.473166+0000","last_change":"2024-02-22T19:30:23.100251+0000","last_active":"2024-02-22T19:30:58.473166+0000","last_peered":"2024-02-22T19:30:58.473166+0000","last_clean":"2024-02-22T19:30:58.473166+0000","last_became_active":"2024-02-22T19:29:05.092069+0000","last_became_peered":"2024-02-22T19:29:05.092069+0000","last_unstale":"2024-02-22T19:30:58.473166+0000","last_undegraded":"2024-02-22T19:30:58.473166+0000","last_fullsized":"2024-02-22T19:30:58.473166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:23.100177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.100177+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077282+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:30:28.077282+0000","last_peered":"2024-02-22T19:30:28.077282+0000","last_clean":"2024-02-22T19:30:28.077282+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:30:28.077282+0000","last_undegraded":"2024-02-22T19:30:28.077282+0000","last_fullsized":"2024-02-22T19:30:28.077282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:50.936816+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:30:50.936816+0000","last_peered":"2024-02-22T19:30:50.936816+0000","last_clean":"2024-02-22T19:30:50.936816+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:30:50.936816+0000","last_undegraded":"2024-02-22T19:30:50.936816+0000","last_fullsized":"2024-02-22T19:30:50.936816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":410,"ondisk_log_size":410,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":112,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266715,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297712,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":630784,"data_stored":289404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65100000000000002,"5min":0.871,"15min":0.871},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63100000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68700000000000006,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.55200000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58199999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68500000000000005,"5min":1.98,"15min":1.98},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64300000000000002,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71099999999999997,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.753,"15min":0.753},"last":0.749},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72899999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.72899999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297684,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447468,"statfs":{"total":95995035648,"available":95690207232,"internally_reserved":0,"allocated":602112,"data_stored":288019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":1.024,"15min":1.024},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73599999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.309,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.36799999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":7,"last update":"Thu Feb 22 19:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":296548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:40 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36399999999999999,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52900000000000003,"5min":0.62,"15min":0.62},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.378,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.41199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.502}]},{"osd":2,"last update":"Thu Feb 22 19:30:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56200000000000006,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.40100000000000002,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.42699999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Feb 22 19:30:27 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.43099999999999999,"15min":0.45500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.441,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.48999999999999999,"15min":0.50900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54300000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.623}]},{"osd":4,"last update":"Thu Feb 22 19:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54300000000000004,"15min":0.56599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.58799999999999997,"15min":0.60099999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.754}]},{"osd":7,"last update":"Thu Feb 22 19:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.76600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297716,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":634880,"data_stored":289974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.52400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.376}]},{"osd":2,"last update":"Thu Feb 22 19:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.23300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53900000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.311},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.377,"15min":0.374},"min":{"1min":0.252,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53200000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.40799999999999997,"15min":0.42599999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.443,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.42699999999999999,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46899999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70899999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.995,"15min":0.995},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Feb 22 19:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67600000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.624}]},{"osd":6,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.745}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297712,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":630784,"data_stored":289030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.48399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.376,"15min":0.36599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:30:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48999999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.29299999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.31},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.442},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.50800000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58899999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297740,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447412,"statfs":{"total":95995035648,"available":95690149888,"internally_reserved":0,"allocated":659456,"data_stored":296456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41599999999999998,"15min":0.432},"min":{"1min":0.20799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.45000000000000001,"15min":0.46400000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.47599999999999998,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.46999999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46500000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.626}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78300000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.82899999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953515,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297736,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447416,"statfs":{"total":95995035648,"available":95690153984,"internally_reserved":0,"allocated":655360,"data_stored":296495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":1,"15min":1},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.34200000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.40999999999999998},"min":{"1min":0.36499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Feb 22 19:29:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.626}]}]},{"osd":5,"up_from":39,"seq":167503724581,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297688,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447464,"statfs":{"total":95995035648,"available":95690203136,"internally_reserved":0,"allocated":606208,"data_stored":288387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.5229999999999997,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.35599999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78000000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.59499999999999997,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.5529999999999999,"5min":5.5529999999999999,"15min":5.5529999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.219,"15min":0.219},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.26600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.42499999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.46500000000000002},"min":{"1min":0.35299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":1.071,"15min":1.071},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:31:00.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:31:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:30:59.203024+0000 mgr.y (mgr.14535) 81 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:01.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:30:59.956412+0000 mgr.y (mgr.14535) 82 : audit [DBG] from='client.24641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:01.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:30:59.956412+0000 mgr.y (mgr.14535) 82 : audit [DBG] from='client.24641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:30:59.956412+0000 mgr.y (mgr.14535) 82 : audit [DBG] from='client.24641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:02.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:01.009569+0000 osd.3 (osd.3) 45 : cluster [DBG] 5.17 scrub starts 2024-02-22T19:31:02.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:01.012485+0000 osd.3 (osd.3) 46 : cluster [DBG] 5.17 scrub ok 2024-02-22T19:31:02.626 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:01.203434+0000 mgr.y (mgr.14535) 83 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:02.626 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:01.009569+0000 osd.3 (osd.3) 45 : cluster [DBG] 5.17 scrub starts 2024-02-22T19:31:02.626 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:01.012485+0000 osd.3 (osd.3) 46 : cluster [DBG] 5.17 scrub ok 2024-02-22T19:31:02.626 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:01.203434+0000 mgr.y (mgr.14535) 83 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:01.009569+0000 osd.3 (osd.3) 45 : cluster [DBG] 5.17 scrub starts 2024-02-22T19:31:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:01.012485+0000 osd.3 (osd.3) 46 : cluster [DBG] 5.17 scrub ok 2024-02-22T19:31:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:01.203434+0000 mgr.y (mgr.14535) 83 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:03.072 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:31:03.074 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:31:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:02.924326+0000 osd.0 (osd.0) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:02.924380+0000 osd.0 (osd.0) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:03.716 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:02.924326+0000 osd.0 (osd.0) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:03.716 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:02.924380+0000 osd.0 (osd.0) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:03.740 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-02-22T19:31:01.203235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":831,"num_read_kb":583,"num_write":428,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":488,"ondisk_log_size":488,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381736,"kb_used_data":4968,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579480,"statfs":{"total":767960285184,"available":765521387520,"internally_reserved":0,"allocated":5087232,"data_stored":2334313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002499"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096862+0000","last_change":"2024-02-22T19:30:28.096813+0000","last_active":"2024-02-22T19:30:28.096862+0000","last_peered":"2024-02-22T19:30:28.096862+0000","last_clean":"2024-02-22T19:30:28.096862+0000","last_became_active":"2024-02-22T19:29:07.099281+0000","last_became_peered":"2024-02-22T19:29:07.099281+0000","last_unstale":"2024-02-22T19:30:28.096862+0000","last_undegraded":"2024-02-22T19:30:28.096862+0000","last_fullsized":"2024-02-22T19:30:28.096862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.074360+0000","last_change":"2024-02-22T19:30:24.056145+0000","last_active":"2024-02-22T19:30:28.074360+0000","last_peered":"2024-02-22T19:30:28.074360+0000","last_clean":"2024-02-22T19:30:28.074360+0000","last_became_active":"2024-02-22T19:29:05.091467+0000","last_became_peered":"2024-02-22T19:29:05.091467+0000","last_unstale":"2024-02-22T19:30:28.074360+0000","last_undegraded":"2024-02-22T19:30:28.074360+0000","last_fullsized":"2024-02-22T19:30:28.074360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128363+0000","last_change":"2024-02-22T19:30:07.664488+0000","last_active":"2024-02-22T19:30:28.128363+0000","last_peered":"2024-02-22T19:30:28.128363+0000","last_clean":"2024-02-22T19:30:28.128363+0000","last_became_active":"2024-02-22T19:29:01.002834+0000","last_became_peered":"2024-02-22T19:29:01.002834+0000","last_unstale":"2024-02-22T19:30:28.128363+0000","last_undegraded":"2024-02-22T19:30:28.128363+0000","last_fullsized":"2024-02-22T19:30:28.128363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.664424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093600+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:30:28.093600+0000","last_peered":"2024-02-22T19:30:28.093600+0000","last_clean":"2024-02-22T19:30:28.093600+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:30:28.093600+0000","last_undegraded":"2024-02-22T19:30:28.093600+0000","last_fullsized":"2024-02-22T19:30:28.093600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077101+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:30:28.077101+0000","last_peered":"2024-02-22T19:30:28.077101+0000","last_clean":"2024-02-22T19:30:28.077101+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:30:28.077101+0000","last_undegraded":"2024-02-22T19:30:28.077101+0000","last_fullsized":"2024-02-22T19:30:28.077101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:52.985523+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:30:52.985523+0000","last_peered":"2024-02-22T19:30:52.985523+0000","last_clean":"2024-02-22T19:30:52.985523+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:30:52.985523+0000","last_undegraded":"2024-02-22T19:30:52.985523+0000","last_fullsized":"2024-02-22T19:30:52.985523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:56.953343+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:30:56.953343+0000","last_peered":"2024-02-22T19:30:56.953343+0000","last_clean":"2024-02-22T19:30:56.953343+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:30:56.953343+0000","last_undegraded":"2024-02-22T19:30:56.953343+0000","last_fullsized":"2024-02-22T19:30:56.953343+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:30.877529+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:30:30.877529+0000","last_peered":"2024-02-22T19:30:30.877529+0000","last_clean":"2024-02-22T19:30:30.877529+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:30:30.877529+0000","last_undegraded":"2024-02-22T19:30:30.877529+0000","last_fullsized":"2024-02-22T19:30:30.877529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127622+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:30:28.127622+0000","last_peered":"2024-02-22T19:30:28.127622+0000","last_clean":"2024-02-22T19:30:28.127622+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:30:28.127622+0000","last_undegraded":"2024-02-22T19:30:28.127622+0000","last_fullsized":"2024-02-22T19:30:28.127622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096341+0000","last_change":"2024-02-22T19:30:26.528062+0000","last_active":"2024-02-22T19:30:28.096341+0000","last_peered":"2024-02-22T19:30:28.096341+0000","last_clean":"2024-02-22T19:30:28.096341+0000","last_became_active":"2024-02-22T19:29:05.091567+0000","last_became_peered":"2024-02-22T19:29:05.091567+0000","last_unstale":"2024-02-22T19:30:28.096341+0000","last_undegraded":"2024-02-22T19:30:28.096341+0000","last_fullsized":"2024-02-22T19:30:28.096341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.527965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.527965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096622+0000","last_change":"2024-02-22T19:30:10.440133+0000","last_active":"2024-02-22T19:30:28.096622+0000","last_peered":"2024-02-22T19:30:28.096622+0000","last_clean":"2024-02-22T19:30:28.096622+0000","last_became_active":"2024-02-22T19:29:01.005184+0000","last_became_peered":"2024-02-22T19:29:01.005184+0000","last_unstale":"2024-02-22T19:30:28.096622+0000","last_undegraded":"2024-02-22T19:30:28.096622+0000","last_fullsized":"2024-02-22T19:30:28.096622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.440058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.440058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093874+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:30:28.093874+0000","last_peered":"2024-02-22T19:30:28.093874+0000","last_clean":"2024-02-22T19:30:28.093874+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:30:28.093874+0000","last_undegraded":"2024-02-22T19:30:28.093874+0000","last_fullsized":"2024-02-22T19:30:28.093874+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096064+0000","last_change":"2024-02-22T19:30:18.496805+0000","last_active":"2024-02-22T19:30:28.096064+0000","last_peered":"2024-02-22T19:30:28.096064+0000","last_clean":"2024-02-22T19:30:28.096064+0000","last_became_active":"2024-02-22T19:29:07.100382+0000","last_became_peered":"2024-02-22T19:29:07.100382+0000","last_unstale":"2024-02-22T19:30:28.096064+0000","last_undegraded":"2024-02-22T19:30:28.096064+0000","last_fullsized":"2024-02-22T19:30:28.096064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.619660+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:30:41.619660+0000","last_peered":"2024-02-22T19:30:41.619660+0000","last_clean":"2024-02-22T19:30:41.619660+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:30:41.619660+0000","last_undegraded":"2024-02-22T19:30:41.619660+0000","last_fullsized":"2024-02-22T19:30:41.619660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096551+0000","last_change":"2024-02-22T19:30:22.574826+0000","last_active":"2024-02-22T19:30:28.096551+0000","last_peered":"2024-02-22T19:30:28.096551+0000","last_clean":"2024-02-22T19:30:28.096551+0000","last_became_active":"2024-02-22T19:29:01.007689+0000","last_became_peered":"2024-02-22T19:29:01.007689+0000","last_unstale":"2024-02-22T19:30:28.096551+0000","last_undegraded":"2024-02-22T19:30:28.096551+0000","last_fullsized":"2024-02-22T19:30:28.096551+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:22.574750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.574750+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:40.016875+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:30:40.016875+0000","last_peered":"2024-02-22T19:30:40.016875+0000","last_clean":"2024-02-22T19:30:40.016875+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:30:40.016875+0000","last_undegraded":"2024-02-22T19:30:40.016875+0000","last_fullsized":"2024-02-22T19:30:40.016875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:34.059668+0000","last_change":"2024-02-22T19:30:34.059638+0000","last_active":"2024-02-22T19:30:34.059668+0000","last_peered":"2024-02-22T19:30:34.059668+0000","last_clean":"2024-02-22T19:30:34.059668+0000","last_became_active":"2024-02-22T19:29:07.197580+0000","last_became_peered":"2024-02-22T19:29:07.197580+0000","last_unstale":"2024-02-22T19:30:34.059668+0000","last_undegraded":"2024-02-22T19:30:34.059668+0000","last_fullsized":"2024-02-22T19:30:34.059668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.981453+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:30:41.981453+0000","last_peered":"2024-02-22T19:30:41.981453+0000","last_clean":"2024-02-22T19:30:41.981453+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:30:41.981453+0000","last_undegraded":"2024-02-22T19:30:41.981453+0000","last_fullsized":"2024-02-22T19:30:41.981453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127884+0000","last_change":"2024-02-22T19:30:00.634139+0000","last_active":"2024-02-22T19:30:28.127884+0000","last_peered":"2024-02-22T19:30:28.127884+0000","last_clean":"2024-02-22T19:30:28.127884+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:30:28.127884+0000","last_undegraded":"2024-02-22T19:30:28.127884+0000","last_fullsized":"2024-02-22T19:30:28.127884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:30:00.634060+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:32.144975+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:30:32.144975+0000","last_peered":"2024-02-22T19:30:32.144975+0000","last_clean":"2024-02-22T19:30:32.144975+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:30:32.144975+0000","last_undegraded":"2024-02-22T19:30:32.144975+0000","last_fullsized":"2024-02-22T19:30:32.144975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077996+0000","last_change":"2024-02-22T19:30:26.823434+0000","last_active":"2024-02-22T19:30:28.077996+0000","last_peered":"2024-02-22T19:30:28.077996+0000","last_clean":"2024-02-22T19:30:28.077996+0000","last_became_active":"2024-02-22T19:29:07.197840+0000","last_became_peered":"2024-02-22T19:29:07.197840+0000","last_unstale":"2024-02-22T19:30:28.077996+0000","last_undegraded":"2024-02-22T19:30:28.077996+0000","last_fullsized":"2024-02-22T19:30:28.077996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.823363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.823363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:44.976744+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:30:44.976744+0000","last_peered":"2024-02-22T19:30:44.976744+0000","last_clean":"2024-02-22T19:30:44.976744+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:30:44.976744+0000","last_undegraded":"2024-02-22T19:30:44.976744+0000","last_fullsized":"2024-02-22T19:30:44.976744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.137069+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:30:31.137069+0000","last_peered":"2024-02-22T19:30:31.137069+0000","last_clean":"2024-02-22T19:30:31.137069+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:30:31.137069+0000","last_undegraded":"2024-02-22T19:30:31.137069+0000","last_fullsized":"2024-02-22T19:30:31.137069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077293+0000","last_change":"2024-02-22T19:29:03.083045+0000","last_active":"2024-02-22T19:30:28.077293+0000","last_peered":"2024-02-22T19:30:28.077293+0000","last_clean":"2024-02-22T19:30:28.077293+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:30:28.077293+0000","last_undegraded":"2024-02-22T19:30:28.077293+0000","last_fullsized":"2024-02-22T19:30:28.077293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:29:02.056447+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:37.617849+0000","last_change":"2024-02-22T19:30:37.617824+0000","last_active":"2024-02-22T19:30:37.617849+0000","last_peered":"2024-02-22T19:30:37.617849+0000","last_clean":"2024-02-22T19:30:37.617849+0000","last_became_active":"2024-02-22T19:29:07.104332+0000","last_became_peered":"2024-02-22T19:29:07.104332+0000","last_unstale":"2024-02-22T19:30:37.617849+0000","last_undegraded":"2024-02-22T19:30:37.617849+0000","last_fullsized":"2024-02-22T19:30:37.617849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.147697+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:30:59.147697+0000","last_peered":"2024-02-22T19:30:59.147697+0000","last_clean":"2024-02-22T19:30:59.147697+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:30:59.147697+0000","last_undegraded":"2024-02-22T19:30:59.147697+0000","last_fullsized":"2024-02-22T19:30:59.147697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.074437+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:30:28.074437+0000","last_peered":"2024-02-22T19:30:28.074437+0000","last_clean":"2024-02-22T19:30:28.074437+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:30:28.074437+0000","last_undegraded":"2024-02-22T19:30:28.074437+0000","last_fullsized":"2024-02-22T19:30:28.074437+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128138+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:30:28.128138+0000","last_peered":"2024-02-22T19:30:28.128138+0000","last_clean":"2024-02-22T19:30:28.128138+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:30:28.128138+0000","last_undegraded":"2024-02-22T19:30:28.128138+0000","last_fullsized":"2024-02-22T19:30:28.128138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.070129+0000","last_change":"2024-02-22T19:30:31.070100+0000","last_active":"2024-02-22T19:30:31.070129+0000","last_peered":"2024-02-22T19:30:31.070129+0000","last_clean":"2024-02-22T19:30:31.070129+0000","last_became_active":"2024-02-22T19:29:07.089971+0000","last_became_peered":"2024-02-22T19:29:07.089971+0000","last_unstale":"2024-02-22T19:30:31.070129+0000","last_undegraded":"2024-02-22T19:30:31.070129+0000","last_fullsized":"2024-02-22T19:30:31.070129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:39.660635+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:30:39.660635+0000","last_peered":"2024-02-22T19:30:39.660635+0000","last_clean":"2024-02-22T19:30:39.660635+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:30:39.660635+0000","last_undegraded":"2024-02-22T19:30:39.660635+0000","last_fullsized":"2024-02-22T19:30:39.660635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:57.988120+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:30:57.988120+0000","last_peered":"2024-02-22T19:30:57.988120+0000","last_clean":"2024-02-22T19:30:57.988120+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:30:57.988120+0000","last_undegraded":"2024-02-22T19:30:57.988120+0000","last_fullsized":"2024-02-22T19:30:57.988120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077812+0000","last_change":"2024-02-22T19:30:06.607076+0000","last_active":"2024-02-22T19:30:28.077812+0000","last_peered":"2024-02-22T19:30:28.077812+0000","last_clean":"2024-02-22T19:30:28.077812+0000","last_became_active":"2024-02-22T19:29:03.087740+0000","last_became_peered":"2024-02-22T19:29:03.087740+0000","last_unstale":"2024-02-22T19:30:28.077812+0000","last_undegraded":"2024-02-22T19:30:28.077812+0000","last_fullsized":"2024-02-22T19:30:28.077812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.607011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.607011+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:40.624644+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:30:40.624644+0000","last_peered":"2024-02-22T19:30:40.624644+0000","last_clean":"2024-02-22T19:30:40.624644+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:30:40.624644+0000","last_undegraded":"2024-02-22T19:30:40.624644+0000","last_fullsized":"2024-02-22T19:30:40.624644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.539925+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:30:35.539925+0000","last_peered":"2024-02-22T19:30:35.539925+0000","last_clean":"2024-02-22T19:30:35.539925+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:30:35.539925+0000","last_undegraded":"2024-02-22T19:30:35.539925+0000","last_fullsized":"2024-02-22T19:30:35.539925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128291+0000","last_change":"2024-02-22T19:30:08.671313+0000","last_active":"2024-02-22T19:30:28.128291+0000","last_peered":"2024-02-22T19:30:28.128291+0000","last_clean":"2024-02-22T19:30:28.128291+0000","last_became_active":"2024-02-22T19:29:01.009788+0000","last_became_peered":"2024-02-22T19:29:01.009788+0000","last_unstale":"2024-02-22T19:30:28.128291+0000","last_undegraded":"2024-02-22T19:30:28.128291+0000","last_fullsized":"2024-02-22T19:30:28.128291+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:08.671238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:08.671238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:47.930394+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:30:47.930394+0000","last_peered":"2024-02-22T19:30:47.930394+0000","last_clean":"2024-02-22T19:30:47.930394+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:30:47.930394+0000","last_undegraded":"2024-02-22T19:30:47.930394+0000","last_fullsized":"2024-02-22T19:30:47.930394+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076912+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:30:28.076912+0000","last_peered":"2024-02-22T19:30:28.076912+0000","last_clean":"2024-02-22T19:30:28.076912+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:30:28.076912+0000","last_undegraded":"2024-02-22T19:30:28.076912+0000","last_fullsized":"2024-02-22T19:30:28.076912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597591+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:30:28.597591+0000","last_peered":"2024-02-22T19:30:28.597591+0000","last_clean":"2024-02-22T19:30:28.597591+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:30:28.597591+0000","last_undegraded":"2024-02-22T19:30:28.597591+0000","last_fullsized":"2024-02-22T19:30:28.597591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096826+0000","last_change":"2024-02-22T19:30:07.463186+0000","last_active":"2024-02-22T19:30:28.096826+0000","last_peered":"2024-02-22T19:30:28.096826+0000","last_clean":"2024-02-22T19:30:28.096826+0000","last_became_active":"2024-02-22T19:29:01.006896+0000","last_became_peered":"2024-02-22T19:29:01.006896+0000","last_unstale":"2024-02-22T19:30:28.096826+0000","last_undegraded":"2024-02-22T19:30:28.096826+0000","last_fullsized":"2024-02-22T19:30:28.096826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.463122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.463122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:42.903912+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:30:42.903912+0000","last_peered":"2024-02-22T19:30:42.903912+0000","last_clean":"2024-02-22T19:30:42.903912+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:30:42.903912+0000","last_undegraded":"2024-02-22T19:30:42.903912+0000","last_fullsized":"2024-02-22T19:30:42.903912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127355+0000","last_change":"2024-02-22T19:29:07.102062+0000","last_active":"2024-02-22T19:30:28.127355+0000","last_peered":"2024-02-22T19:30:28.127355+0000","last_clean":"2024-02-22T19:30:28.127355+0000","last_became_active":"2024-02-22T19:29:07.101369+0000","last_became_peered":"2024-02-22T19:29:07.101369+0000","last_unstale":"2024-02-22T19:30:28.127355+0000","last_undegraded":"2024-02-22T19:30:28.127355+0000","last_fullsized":"2024-02-22T19:30:28.127355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:48.880406+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:30:48.880406+0000","last_peered":"2024-02-22T19:30:48.880406+0000","last_clean":"2024-02-22T19:30:48.880406+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:30:48.880406+0000","last_undegraded":"2024-02-22T19:30:48.880406+0000","last_fullsized":"2024-02-22T19:30:48.880406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597569+0000","last_change":"2024-02-22T19:30:03.688086+0000","last_active":"2024-02-22T19:30:28.597569+0000","last_peered":"2024-02-22T19:30:28.597569+0000","last_clean":"2024-02-22T19:30:28.597569+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:30:28.597569+0000","last_undegraded":"2024-02-22T19:30:28.597569+0000","last_fullsized":"2024-02-22T19:30:28.597569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.688024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.688024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.073965+0000","last_change":"2024-02-22T19:30:14.042137+0000","last_active":"2024-02-22T19:30:28.073965+0000","last_peered":"2024-02-22T19:30:28.073965+0000","last_clean":"2024-02-22T19:30:28.073965+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:30:28.073965+0000","last_undegraded":"2024-02-22T19:30:28.073965+0000","last_fullsized":"2024-02-22T19:30:28.073965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.042059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.042059+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076592+0000","last_change":"2024-02-22T19:30:13.709779+0000","last_active":"2024-02-22T19:30:28.076592+0000","last_peered":"2024-02-22T19:30:28.076592+0000","last_clean":"2024-02-22T19:30:28.076592+0000","last_became_active":"2024-02-22T19:29:07.102604+0000","last_became_peered":"2024-02-22T19:29:07.102604+0000","last_unstale":"2024-02-22T19:30:28.076592+0000","last_undegraded":"2024-02-22T19:30:28.076592+0000","last_fullsized":"2024-02-22T19:30:28.076592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.709691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.709691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:45.021630+0000","last_change":"2024-02-22T19:30:45.021593+0000","last_active":"2024-02-22T19:30:45.021630+0000","last_peered":"2024-02-22T19:30:45.021630+0000","last_clean":"2024-02-22T19:30:45.021630+0000","last_became_active":"2024-02-22T19:29:05.091550+0000","last_became_peered":"2024-02-22T19:29:05.091550+0000","last_unstale":"2024-02-22T19:30:45.021630+0000","last_undegraded":"2024-02-22T19:30:45.021630+0000","last_fullsized":"2024-02-22T19:30:45.021630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.969712+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:30:41.969712+0000","last_peered":"2024-02-22T19:30:41.969712+0000","last_clean":"2024-02-22T19:30:41.969712+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:30:41.969712+0000","last_undegraded":"2024-02-22T19:30:41.969712+0000","last_fullsized":"2024-02-22T19:30:41.969712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:41.008739+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:30:41.008739+0000","last_peered":"2024-02-22T19:30:41.008739+0000","last_clean":"2024-02-22T19:30:41.008739+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:30:41.008739+0000","last_undegraded":"2024-02-22T19:30:41.008739+0000","last_fullsized":"2024-02-22T19:30:41.008739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:46.483352+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:30:46.483352+0000","last_peered":"2024-02-22T19:30:46.483352+0000","last_clean":"2024-02-22T19:30:46.483352+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:30:46.483352+0000","last_undegraded":"2024-02-22T19:30:46.483352+0000","last_fullsized":"2024-02-22T19:30:46.483352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.150020+0000","last_change":"2024-02-22T19:30:21.533563+0000","last_active":"2024-02-22T19:30:59.150020+0000","last_peered":"2024-02-22T19:30:59.150020+0000","last_clean":"2024-02-22T19:30:59.150020+0000","last_became_active":"2024-02-22T19:29:05.092444+0000","last_became_peered":"2024-02-22T19:29:05.092444+0000","last_unstale":"2024-02-22T19:30:59.150020+0000","last_undegraded":"2024-02-22T19:30:59.150020+0000","last_fullsized":"2024-02-22T19:30:59.150020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:21.533452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.533452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:39.596263+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:30:39.596263+0000","last_peered":"2024-02-22T19:30:39.596263+0000","last_clean":"2024-02-22T19:30:39.596263+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:30:39.596263+0000","last_undegraded":"2024-02-22T19:30:39.596263+0000","last_fullsized":"2024-02-22T19:30:39.596263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:32.846447+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:30:32.846447+0000","last_peered":"2024-02-22T19:30:32.846447+0000","last_clean":"2024-02-22T19:30:32.846447+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:30:32.846447+0000","last_undegraded":"2024-02-22T19:30:32.846447+0000","last_fullsized":"2024-02-22T19:30:32.846447+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.094734+0000","last_change":"2024-02-22T19:30:16.057704+0000","last_active":"2024-02-22T19:30:28.094734+0000","last_peered":"2024-02-22T19:30:28.094734+0000","last_clean":"2024-02-22T19:30:28.094734+0000","last_became_active":"2024-02-22T19:29:07.097503+0000","last_became_peered":"2024-02-22T19:29:07.097503+0000","last_unstale":"2024-02-22T19:30:28.094734+0000","last_undegraded":"2024-02-22T19:30:28.094734+0000","last_fullsized":"2024-02-22T19:30:28.094734+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.628292+0000","last_change":"2024-02-22T19:29:05.095926+0000","last_active":"2024-02-22T19:30:58.628292+0000","last_peered":"2024-02-22T19:30:58.628292+0000","last_clean":"2024-02-22T19:30:58.628292+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:30:58.628292+0000","last_undegraded":"2024-02-22T19:30:58.628292+0000","last_fullsized":"2024-02-22T19:30:58.628292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:29:04.066424+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127607+0000","last_change":"2024-02-22T19:30:03.653428+0000","last_active":"2024-02-22T19:30:28.127607+0000","last_peered":"2024-02-22T19:30:28.127607+0000","last_clean":"2024-02-22T19:30:28.127607+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:30:28.127607+0000","last_undegraded":"2024-02-22T19:30:28.127607+0000","last_fullsized":"2024-02-22T19:30:28.127607+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.653348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.653348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'9","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096872+0000","last_change":"2024-02-22T19:30:05.473412+0000","last_active":"2024-02-22T19:30:28.096872+0000","last_peered":"2024-02-22T19:30:28.096872+0000","last_clean":"2024-02-22T19:30:28.096872+0000","last_became_active":"2024-02-22T19:29:03.448670+0000","last_became_peered":"2024-02-22T19:29:03.448670+0000","last_unstale":"2024-02-22T19:30:28.096872+0000","last_undegraded":"2024-02-22T19:30:28.096872+0000","last_fullsized":"2024-02-22T19:30:28.096872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:05.473341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.473341+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.126398+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:30:28.126398+0000","last_peered":"2024-02-22T19:30:28.126398+0000","last_clean":"2024-02-22T19:30:28.126398+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:30:28.126398+0000","last_undegraded":"2024-02-22T19:30:28.126398+0000","last_fullsized":"2024-02-22T19:30:28.126398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077921+0000","last_change":"2024-02-22T19:29:05.096571+0000","last_active":"2024-02-22T19:30:28.077921+0000","last_peered":"2024-02-22T19:30:28.077921+0000","last_clean":"2024-02-22T19:30:28.077921+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:30:28.077921+0000","last_undegraded":"2024-02-22T19:30:28.077921+0000","last_fullsized":"2024-02-22T19:30:28.077921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:29:04.066424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597146+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:30:28.597146+0000","last_peered":"2024-02-22T19:30:28.597146+0000","last_clean":"2024-02-22T19:30:28.597146+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:30:28.597146+0000","last_undegraded":"2024-02-22T19:30:28.597146+0000","last_fullsized":"2024-02-22T19:30:28.597146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128181+0000","last_change":"2024-02-22T19:30:13.766624+0000","last_active":"2024-02-22T19:30:28.128181+0000","last_peered":"2024-02-22T19:30:28.128181+0000","last_clean":"2024-02-22T19:30:28.128181+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:30:28.128181+0000","last_undegraded":"2024-02-22T19:30:28.128181+0000","last_fullsized":"2024-02-22T19:30:28.128181+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:13.766546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.766546+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597324+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:30:28.597324+0000","last_peered":"2024-02-22T19:30:28.597324+0000","last_clean":"2024-02-22T19:30:28.597324+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:30:28.597324+0000","last_undegraded":"2024-02-22T19:30:28.597324+0000","last_fullsized":"2024-02-22T19:30:28.597324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096869+0000","last_change":"2024-02-22T19:30:27.575360+0000","last_active":"2024-02-22T19:30:28.096869+0000","last_peered":"2024-02-22T19:30:28.096869+0000","last_clean":"2024-02-22T19:30:28.096869+0000","last_became_active":"2024-02-22T19:29:05.092190+0000","last_became_peered":"2024-02-22T19:29:05.092190+0000","last_unstale":"2024-02-22T19:30:28.096869+0000","last_undegraded":"2024-02-22T19:30:28.096869+0000","last_fullsized":"2024-02-22T19:30:28.096869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.656363+0000","last_change":"2024-02-22T19:30:29.656325+0000","last_active":"2024-02-22T19:30:29.656363+0000","last_peered":"2024-02-22T19:30:29.656363+0000","last_clean":"2024-02-22T19:30:29.656363+0000","last_became_active":"2024-02-22T19:29:01.007053+0000","last_became_peered":"2024-02-22T19:29:01.007053+0000","last_unstale":"2024-02-22T19:30:29.656363+0000","last_undegraded":"2024-02-22T19:30:29.656363+0000","last_fullsized":"2024-02-22T19:30:29.656363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.078077+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:30:28.078077+0000","last_peered":"2024-02-22T19:30:28.078077+0000","last_clean":"2024-02-22T19:30:28.078077+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:30:28.078077+0000","last_undegraded":"2024-02-22T19:30:28.078077+0000","last_fullsized":"2024-02-22T19:30:28.078077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096374+0000","last_change":"2024-02-22T19:30:14.461934+0000","last_active":"2024-02-22T19:30:28.096374+0000","last_peered":"2024-02-22T19:30:28.096374+0000","last_clean":"2024-02-22T19:30:28.096374+0000","last_became_active":"2024-02-22T19:29:07.197976+0000","last_became_peered":"2024-02-22T19:29:07.197976+0000","last_unstale":"2024-02-22T19:30:28.096374+0000","last_undegraded":"2024-02-22T19:30:28.096374+0000","last_fullsized":"2024-02-22T19:30:28.096374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597893+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:30:28.597893+0000","last_peered":"2024-02-22T19:30:28.597893+0000","last_clean":"2024-02-22T19:30:28.597893+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:30:28.597893+0000","last_undegraded":"2024-02-22T19:30:28.597893+0000","last_fullsized":"2024-02-22T19:30:28.597893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.601696+0000","last_change":"2024-02-22T19:30:28.601662+0000","last_active":"2024-02-22T19:30:28.601696+0000","last_peered":"2024-02-22T19:30:28.601696+0000","last_clean":"2024-02-22T19:30:28.601696+0000","last_became_active":"2024-02-22T19:29:03.447032+0000","last_became_peered":"2024-02-22T19:29:03.447032+0000","last_unstale":"2024-02-22T19:30:28.601696+0000","last_undegraded":"2024-02-22T19:30:28.601696+0000","last_fullsized":"2024-02-22T19:30:28.601696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.147208+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:30:59.147208+0000","last_peered":"2024-02-22T19:30:59.147208+0000","last_clean":"2024-02-22T19:30:59.147208+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:30:59.147208+0000","last_undegraded":"2024-02-22T19:30:59.147208+0000","last_fullsized":"2024-02-22T19:30:59.147208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077004+0000","last_change":"2024-02-22T19:29:07.101872+0000","last_active":"2024-02-22T19:30:28.077004+0000","last_peered":"2024-02-22T19:30:28.077004+0000","last_clean":"2024-02-22T19:30:28.077004+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:30:28.077004+0000","last_undegraded":"2024-02-22T19:30:28.077004+0000","last_fullsized":"2024-02-22T19:30:28.077004+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.557564+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:30:31.557564+0000","last_peered":"2024-02-22T19:30:31.557564+0000","last_clean":"2024-02-22T19:30:31.557564+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:30:31.557564+0000","last_undegraded":"2024-02-22T19:30:31.557564+0000","last_fullsized":"2024-02-22T19:30:31.557564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.031939+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:30:35.031939+0000","last_peered":"2024-02-22T19:30:35.031939+0000","last_clean":"2024-02-22T19:30:35.031939+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:30:35.031939+0000","last_undegraded":"2024-02-22T19:30:35.031939+0000","last_fullsized":"2024-02-22T19:30:35.031939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.176592+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:30:36.176592+0000","last_peered":"2024-02-22T19:30:36.176592+0000","last_clean":"2024-02-22T19:30:36.176592+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:30:36.176592+0000","last_undegraded":"2024-02-22T19:30:36.176592+0000","last_fullsized":"2024-02-22T19:30:36.176592+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.596913+0000","last_change":"2024-02-22T19:29:07.104657+0000","last_active":"2024-02-22T19:30:28.596913+0000","last_peered":"2024-02-22T19:30:28.596913+0000","last_clean":"2024-02-22T19:30:28.596913+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:30:28.596913+0000","last_undegraded":"2024-02-22T19:30:28.596913+0000","last_fullsized":"2024-02-22T19:30:28.596913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.110598+0000","last_change":"2024-02-22T19:30:29.110575+0000","last_active":"2024-02-22T19:30:29.110598+0000","last_peered":"2024-02-22T19:30:29.110598+0000","last_clean":"2024-02-22T19:30:29.110598+0000","last_became_active":"2024-02-22T19:29:05.091952+0000","last_became_peered":"2024-02-22T19:29:05.091952+0000","last_unstale":"2024-02-22T19:30:29.110598+0000","last_undegraded":"2024-02-22T19:30:29.110598+0000","last_fullsized":"2024-02-22T19:30:29.110598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.110513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.110513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597675+0000","last_change":"2024-02-22T19:30:10.619750+0000","last_active":"2024-02-22T19:30:28.597675+0000","last_peered":"2024-02-22T19:30:28.597675+0000","last_clean":"2024-02-22T19:30:28.597675+0000","last_became_active":"2024-02-22T19:29:01.004850+0000","last_became_peered":"2024-02-22T19:29:01.004850+0000","last_unstale":"2024-02-22T19:30:28.597675+0000","last_undegraded":"2024-02-22T19:30:28.597675+0000","last_fullsized":"2024-02-22T19:30:28.597675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.619685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.619685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077301+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:30:28.077301+0000","last_peered":"2024-02-22T19:30:28.077301+0000","last_clean":"2024-02-22T19:30:28.077301+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:30:28.077301+0000","last_undegraded":"2024-02-22T19:30:28.077301+0000","last_fullsized":"2024-02-22T19:30:28.077301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127426+0000","last_change":"2024-02-22T19:29:07.101872+0000","last_active":"2024-02-22T19:30:28.127426+0000","last_peered":"2024-02-22T19:30:28.127426+0000","last_clean":"2024-02-22T19:30:28.127426+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:30:28.127426+0000","last_undegraded":"2024-02-22T19:30:28.127426+0000","last_fullsized":"2024-02-22T19:30:28.127426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:39.210253+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:30:39.210253+0000","last_peered":"2024-02-22T19:30:39.210253+0000","last_clean":"2024-02-22T19:30:39.210253+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:30:39.210253+0000","last_undegraded":"2024-02-22T19:30:39.210253+0000","last_fullsized":"2024-02-22T19:30:39.210253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096625+0000","last_change":"2024-02-22T19:30:17.466789+0000","last_active":"2024-02-22T19:30:28.096625+0000","last_peered":"2024-02-22T19:30:28.096625+0000","last_clean":"2024-02-22T19:30:28.096625+0000","last_became_active":"2024-02-22T19:29:01.010917+0000","last_became_peered":"2024-02-22T19:29:01.010917+0000","last_unstale":"2024-02-22T19:30:28.096625+0000","last_undegraded":"2024-02-22T19:30:28.096625+0000","last_fullsized":"2024-02-22T19:30:28.096625+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:48.444523+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:30:48.444523+0000","last_peered":"2024-02-22T19:30:48.444523+0000","last_clean":"2024-02-22T19:30:48.444523+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:30:48.444523+0000","last_undegraded":"2024-02-22T19:30:48.444523+0000","last_fullsized":"2024-02-22T19:30:48.444523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:51.890712+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:30:51.890712+0000","last_peered":"2024-02-22T19:30:51.890712+0000","last_clean":"2024-02-22T19:30:51.890712+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:30:51.890712+0000","last_undegraded":"2024-02-22T19:30:51.890712+0000","last_fullsized":"2024-02-22T19:30:51.890712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096206+0000","last_change":"2024-02-22T19:30:09.477558+0000","last_active":"2024-02-22T19:30:28.096206+0000","last_peered":"2024-02-22T19:30:28.096206+0000","last_clean":"2024-02-22T19:30:28.096206+0000","last_became_active":"2024-02-22T19:29:01.006282+0000","last_became_peered":"2024-02-22T19:29:01.006282+0000","last_unstale":"2024-02-22T19:30:28.096206+0000","last_undegraded":"2024-02-22T19:30:28.096206+0000","last_fullsized":"2024-02-22T19:30:28.096206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_clean_scrub_stamp":"2024-02-22T19:30:09.477488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093820+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:30:28.093820+0000","last_peered":"2024-02-22T19:30:28.093820+0000","last_clean":"2024-02-22T19:30:28.093820+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:30:28.093820+0000","last_undegraded":"2024-02-22T19:30:28.093820+0000","last_fullsized":"2024-02-22T19:30:28.093820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":113,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.474336+0000","last_change":"2024-02-22T19:30:07.992740+0000","last_active":"2024-02-22T19:30:58.474336+0000","last_peered":"2024-02-22T19:30:58.474336+0000","last_clean":"2024-02-22T19:30:58.474336+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:30:58.474336+0000","last_undegraded":"2024-02-22T19:30:58.474336+0000","last_fullsized":"2024-02-22T19:30:58.474336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:30:07.992671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.992671+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127924+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:30:28.127924+0000","last_peered":"2024-02-22T19:30:28.127924+0000","last_clean":"2024-02-22T19:30:28.127924+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:30:28.127924+0000","last_undegraded":"2024-02-22T19:30:28.127924+0000","last_fullsized":"2024-02-22T19:30:28.127924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.620324+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:30:36.620324+0000","last_peered":"2024-02-22T19:30:36.620324+0000","last_clean":"2024-02-22T19:30:36.620324+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:30:36.620324+0000","last_undegraded":"2024-02-22T19:30:36.620324+0000","last_fullsized":"2024-02-22T19:30:36.620324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.107010+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:30:36.107010+0000","last_peered":"2024-02-22T19:30:36.107010+0000","last_clean":"2024-02-22T19:30:36.107010+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:30:36.107010+0000","last_undegraded":"2024-02-22T19:30:36.107010+0000","last_fullsized":"2024-02-22T19:30:36.107010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.125988+0000","last_change":"2024-02-22T19:30:18.117213+0000","last_active":"2024-02-22T19:30:28.125988+0000","last_peered":"2024-02-22T19:30:28.125988+0000","last_clean":"2024-02-22T19:30:28.125988+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:30:28.125988+0000","last_undegraded":"2024-02-22T19:30:28.125988+0000","last_fullsized":"2024-02-22T19:30:28.125988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.117111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.128836+0000","last_change":"2024-02-22T19:30:28.128814+0000","last_active":"2024-02-22T19:30:28.128836+0000","last_peered":"2024-02-22T19:30:28.128836+0000","last_clean":"2024-02-22T19:30:28.128836+0000","last_became_active":"2024-02-22T19:30:24.410718+0000","last_became_peered":"2024-02-22T19:30:24.410718+0000","last_unstale":"2024-02-22T19:30:28.128836+0000","last_undegraded":"2024-02-22T19:30:28.128836+0000","last_fullsized":"2024-02-22T19:30:28.128836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.102231+0000","last_change":"2024-02-22T19:30:31.102193+0000","last_active":"2024-02-22T19:30:31.102231+0000","last_peered":"2024-02-22T19:30:31.102231+0000","last_clean":"2024-02-22T19:30:31.102231+0000","last_became_active":"2024-02-22T19:29:05.098656+0000","last_became_peered":"2024-02-22T19:29:05.098656+0000","last_unstale":"2024-02-22T19:30:31.102231+0000","last_undegraded":"2024-02-22T19:30:31.102231+0000","last_fullsized":"2024-02-22T19:30:31.102231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.102065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.102065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.556128+0000","last_change":"2024-02-22T19:30:35.556098+0000","last_active":"2024-02-22T19:30:35.556128+0000","last_peered":"2024-02-22T19:30:35.556128+0000","last_clean":"2024-02-22T19:30:35.556128+0000","last_became_active":"2024-02-22T19:29:07.098131+0000","last_became_peered":"2024-02-22T19:29:07.098131+0000","last_unstale":"2024-02-22T19:30:35.556128+0000","last_undegraded":"2024-02-22T19:30:35.556128+0000","last_fullsized":"2024-02-22T19:30:35.556128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.597272+0000","last_change":"2024-02-22T19:29:15.492725+0000","last_active":"2024-02-22T19:30:28.597272+0000","last_peered":"2024-02-22T19:30:28.597272+0000","last_clean":"2024-02-22T19:30:28.597272+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:30:28.597272+0000","last_undegraded":"2024-02-22T19:30:28.597272+0000","last_fullsized":"2024-02-22T19:30:28.597272+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:15.492609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:29:15.492609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:45.909387+0000","last_change":"2024-02-22T19:30:45.909363+0000","last_active":"2024-02-22T19:30:45.909387+0000","last_peered":"2024-02-22T19:30:45.909387+0000","last_clean":"2024-02-22T19:30:45.909387+0000","last_became_active":"2024-02-22T19:29:01.010904+0000","last_became_peered":"2024-02-22T19:29:01.010904+0000","last_unstale":"2024-02-22T19:30:45.909387+0000","last_undegraded":"2024-02-22T19:30:45.909387+0000","last_fullsized":"2024-02-22T19:30:45.909387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:34.566573+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:30:34.566573+0000","last_peered":"2024-02-22T19:30:34.566573+0000","last_clean":"2024-02-22T19:30:34.566573+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:30:34.566573+0000","last_undegraded":"2024-02-22T19:30:34.566573+0000","last_fullsized":"2024-02-22T19:30:34.566573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:49.658335+0000","last_change":"2024-02-22T19:30:49.658305+0000","last_active":"2024-02-22T19:30:49.658335+0000","last_peered":"2024-02-22T19:30:49.658335+0000","last_clean":"2024-02-22T19:30:49.658335+0000","last_became_active":"2024-02-22T19:29:05.092427+0000","last_became_peered":"2024-02-22T19:29:05.092427+0000","last_unstale":"2024-02-22T19:30:49.658335+0000","last_undegraded":"2024-02-22T19:30:49.658335+0000","last_fullsized":"2024-02-22T19:30:49.658335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:33.113637+0000","last_change":"2024-02-22T19:30:33.113603+0000","last_active":"2024-02-22T19:30:33.113637+0000","last_peered":"2024-02-22T19:30:33.113637+0000","last_clean":"2024-02-22T19:30:33.113637+0000","last_became_active":"2024-02-22T19:29:07.095700+0000","last_became_peered":"2024-02-22T19:29:07.095700+0000","last_unstale":"2024-02-22T19:30:33.113637+0000","last_undegraded":"2024-02-22T19:30:33.113637+0000","last_fullsized":"2024-02-22T19:30:33.113637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:45.533761+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:30:45.533761+0000","last_peered":"2024-02-22T19:30:45.533761+0000","last_clean":"2024-02-22T19:30:45.533761+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:30:45.533761+0000","last_undegraded":"2024-02-22T19:30:45.533761+0000","last_fullsized":"2024-02-22T19:30:45.533761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":156,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.371259+0000","last_change":"2024-02-22T19:30:20.550910+0000","last_active":"2024-02-22T19:30:58.371259+0000","last_peered":"2024-02-22T19:30:58.371259+0000","last_clean":"2024-02-22T19:30:58.371259+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:30:58.371259+0000","last_undegraded":"2024-02-22T19:30:58.371259+0000","last_fullsized":"2024-02-22T19:30:58.371259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:20.550837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.550837+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":112,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:55.536143+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:30:55.536143+0000","last_peered":"2024-02-22T19:30:55.536143+0000","last_clean":"2024-02-22T19:30:55.536143+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:30:55.536143+0000","last_undegraded":"2024-02-22T19:30:55.536143+0000","last_fullsized":"2024-02-22T19:30:55.536143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:33.584973+0000","last_change":"2024-02-22T19:30:33.584923+0000","last_active":"2024-02-22T19:30:33.584973+0000","last_peered":"2024-02-22T19:30:33.584973+0000","last_clean":"2024-02-22T19:30:33.584973+0000","last_became_active":"2024-02-22T19:29:05.091758+0000","last_became_peered":"2024-02-22T19:29:05.091758+0000","last_unstale":"2024-02-22T19:30:33.584973+0000","last_undegraded":"2024-02-22T19:30:33.584973+0000","last_fullsized":"2024-02-22T19:30:33.584973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:42.995014+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:30:42.995014+0000","last_peered":"2024-02-22T19:30:42.995014+0000","last_clean":"2024-02-22T19:30:42.995014+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:30:42.995014+0000","last_undegraded":"2024-02-22T19:30:42.995014+0000","last_fullsized":"2024-02-22T19:30:42.995014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:34.149787+0000","last_change":"2024-02-22T19:30:34.149749+0000","last_active":"2024-02-22T19:30:34.149787+0000","last_peered":"2024-02-22T19:30:34.149787+0000","last_clean":"2024-02-22T19:30:34.149787+0000","last_became_active":"2024-02-22T19:29:01.004572+0000","last_became_peered":"2024-02-22T19:29:01.004572+0000","last_unstale":"2024-02-22T19:30:34.149787+0000","last_undegraded":"2024-02-22T19:30:34.149787+0000","last_fullsized":"2024-02-22T19:30:34.149787+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":75,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127812+0000","last_change":"2024-02-22T19:30:22.683587+0000","last_active":"2024-02-22T19:30:28.127812+0000","last_peered":"2024-02-22T19:30:28.127812+0000","last_clean":"2024-02-22T19:30:28.127812+0000","last_became_active":"2024-02-22T19:29:03.446818+0000","last_became_peered":"2024-02-22T19:29:03.446818+0000","last_unstale":"2024-02-22T19:30:28.127812+0000","last_undegraded":"2024-02-22T19:30:28.127812+0000","last_fullsized":"2024-02-22T19:30:28.127812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077411+0000","last_change":"2024-02-22T19:30:14.659000+0000","last_active":"2024-02-22T19:30:28.077411+0000","last_peered":"2024-02-22T19:30:28.077411+0000","last_clean":"2024-02-22T19:30:28.077411+0000","last_became_active":"2024-02-22T19:29:05.091867+0000","last_became_peered":"2024-02-22T19:29:05.091867+0000","last_unstale":"2024-02-22T19:30:28.077411+0000","last_undegraded":"2024-02-22T19:30:28.077411+0000","last_fullsized":"2024-02-22T19:30:28.077411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.127371+0000","last_change":"2024-02-22T19:30:18.719372+0000","last_active":"2024-02-22T19:30:28.127371+0000","last_peered":"2024-02-22T19:30:28.127371+0000","last_clean":"2024-02-22T19:30:28.127371+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:30:28.127371+0000","last_undegraded":"2024-02-22T19:30:28.127371+0000","last_fullsized":"2024-02-22T19:30:28.127371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.719310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.616750+0000","last_change":"2024-02-22T19:30:29.616721+0000","last_active":"2024-02-22T19:30:29.616750+0000","last_peered":"2024-02-22T19:30:29.616750+0000","last_clean":"2024-02-22T19:30:29.616750+0000","last_became_active":"2024-02-22T19:29:01.012034+0000","last_became_peered":"2024-02-22T19:29:01.012034+0000","last_unstale":"2024-02-22T19:30:29.616750+0000","last_undegraded":"2024-02-22T19:30:29.616750+0000","last_fullsized":"2024-02-22T19:30:29.616750+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.073911+0000","last_change":"2024-02-22T19:30:12.059865+0000","last_active":"2024-02-22T19:30:28.073911+0000","last_peered":"2024-02-22T19:30:28.073911+0000","last_clean":"2024-02-22T19:30:28.073911+0000","last_became_active":"2024-02-22T19:29:03.086071+0000","last_became_peered":"2024-02-22T19:29:03.086071+0000","last_unstale":"2024-02-22T19:30:28.073911+0000","last_undegraded":"2024-02-22T19:30:28.073911+0000","last_fullsized":"2024-02-22T19:30:28.073911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:12.059797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:12.059797+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:59.147458+0000","last_change":"2024-02-22T19:30:44.574471+0000","last_active":"2024-02-22T19:30:59.147458+0000","last_peered":"2024-02-22T19:30:59.147458+0000","last_clean":"2024-02-22T19:30:59.147458+0000","last_became_active":"2024-02-22T19:29:05.094532+0000","last_became_peered":"2024-02-22T19:29:05.094532+0000","last_unstale":"2024-02-22T19:30:59.147458+0000","last_undegraded":"2024-02-22T19:30:59.147458+0000","last_fullsized":"2024-02-22T19:30:59.147458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:38.631799+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:30:38.631799+0000","last_peered":"2024-02-22T19:30:38.631799+0000","last_clean":"2024-02-22T19:30:38.631799+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:30:38.631799+0000","last_undegraded":"2024-02-22T19:30:38.631799+0000","last_fullsized":"2024-02-22T19:30:38.631799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:50.477842+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:30:50.477842+0000","last_peered":"2024-02-22T19:30:50.477842+0000","last_clean":"2024-02-22T19:30:50.477842+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:30:50.477842+0000","last_undegraded":"2024-02-22T19:30:50.477842+0000","last_fullsized":"2024-02-22T19:30:50.477842+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:54.022567+0000","last_change":"2024-02-22T19:30:54.022533+0000","last_active":"2024-02-22T19:30:54.022567+0000","last_peered":"2024-02-22T19:30:54.022567+0000","last_clean":"2024-02-22T19:30:54.022567+0000","last_became_active":"2024-02-22T19:29:03.084036+0000","last_became_peered":"2024-02-22T19:29:03.084036+0000","last_unstale":"2024-02-22T19:30:54.022567+0000","last_undegraded":"2024-02-22T19:30:54.022567+0000","last_fullsized":"2024-02-22T19:30:54.022567+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.628022+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:30:58.628022+0000","last_peered":"2024-02-22T19:30:58.628022+0000","last_clean":"2024-02-22T19:30:58.628022+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:30:58.628022+0000","last_undegraded":"2024-02-22T19:30:58.628022+0000","last_fullsized":"2024-02-22T19:30:58.628022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:29.099811+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:30:29.099811+0000","last_peered":"2024-02-22T19:30:29.099811+0000","last_clean":"2024-02-22T19:30:29.099811+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:30:29.099811+0000","last_undegraded":"2024-02-22T19:30:29.099811+0000","last_fullsized":"2024-02-22T19:30:29.099811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:36.904799+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:30:36.904799+0000","last_peered":"2024-02-22T19:30:36.904799+0000","last_clean":"2024-02-22T19:30:36.904799+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:30:36.904799+0000","last_undegraded":"2024-02-22T19:30:36.904799+0000","last_fullsized":"2024-02-22T19:30:36.904799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076692+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:30:28.076692+0000","last_peered":"2024-02-22T19:30:28.076692+0000","last_clean":"2024-02-22T19:30:28.076692+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:30:28.076692+0000","last_undegraded":"2024-02-22T19:30:28.076692+0000","last_fullsized":"2024-02-22T19:30:28.076692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093639+0000","last_change":"2024-02-22T19:30:21.116431+0000","last_active":"2024-02-22T19:30:28.093639+0000","last_peered":"2024-02-22T19:30:28.093639+0000","last_clean":"2024-02-22T19:30:28.093639+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:30:28.093639+0000","last_undegraded":"2024-02-22T19:30:28.093639+0000","last_fullsized":"2024-02-22T19:30:28.093639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.116358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.096477+0000","last_change":"2024-02-22T19:30:20.543466+0000","last_active":"2024-02-22T19:30:28.096477+0000","last_peered":"2024-02-22T19:30:28.096477+0000","last_clean":"2024-02-22T19:30:28.096477+0000","last_became_active":"2024-02-22T19:29:07.095452+0000","last_became_peered":"2024-02-22T19:29:07.095452+0000","last_unstale":"2024-02-22T19:30:28.096477+0000","last_undegraded":"2024-02-22T19:30:28.096477+0000","last_fullsized":"2024-02-22T19:30:28.096477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:20.543398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.543398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.479363+0000","last_change":"2024-02-22T19:30:35.999034+0000","last_active":"2024-02-22T19:30:58.479363+0000","last_peered":"2024-02-22T19:30:58.479363+0000","last_clean":"2024-02-22T19:30:58.479363+0000","last_became_active":"2024-02-22T19:29:05.095818+0000","last_became_peered":"2024-02-22T19:29:05.095818+0000","last_unstale":"2024-02-22T19:30:58.479363+0000","last_undegraded":"2024-02-22T19:30:58.479363+0000","last_fullsized":"2024-02-22T19:30:58.479363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.076638+0000","last_change":"2024-02-22T19:30:06.720523+0000","last_active":"2024-02-22T19:30:28.076638+0000","last_peered":"2024-02-22T19:30:28.076638+0000","last_clean":"2024-02-22T19:30:28.076638+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:30:28.076638+0000","last_undegraded":"2024-02-22T19:30:28.076638+0000","last_fullsized":"2024-02-22T19:30:28.076638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.720452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.720452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.126355+0000","last_change":"2024-02-22T19:30:27.174847+0000","last_active":"2024-02-22T19:30:28.126355+0000","last_peered":"2024-02-22T19:30:28.126355+0000","last_clean":"2024-02-22T19:30:28.126355+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:30:28.126355+0000","last_undegraded":"2024-02-22T19:30:28.126355+0000","last_fullsized":"2024-02-22T19:30:28.126355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:27.174779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.174779+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077193+0000","last_change":"2024-02-22T19:29:07.198513+0000","last_active":"2024-02-22T19:30:28.077193+0000","last_peered":"2024-02-22T19:30:28.077193+0000","last_clean":"2024-02-22T19:30:28.077193+0000","last_became_active":"2024-02-22T19:29:07.198153+0000","last_became_peered":"2024-02-22T19:29:07.198153+0000","last_unstale":"2024-02-22T19:30:28.077193+0000","last_undegraded":"2024-02-22T19:30:28.077193+0000","last_fullsized":"2024-02-22T19:30:28.077193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:29:06.076443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:31.646705+0000","last_change":"2024-02-22T19:30:31.646674+0000","last_active":"2024-02-22T19:30:31.646705+0000","last_peered":"2024-02-22T19:30:31.646705+0000","last_clean":"2024-02-22T19:30:31.646705+0000","last_became_active":"2024-02-22T19:29:05.096026+0000","last_became_peered":"2024-02-22T19:29:05.096026+0000","last_unstale":"2024-02-22T19:30:31.646705+0000","last_undegraded":"2024-02-22T19:30:31.646705+0000","last_fullsized":"2024-02-22T19:30:31.646705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.093673+0000","last_change":"2024-02-22T19:30:19.102902+0000","last_active":"2024-02-22T19:30:28.093673+0000","last_peered":"2024-02-22T19:30:28.093673+0000","last_clean":"2024-02-22T19:30:28.093673+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:30:28.093673+0000","last_undegraded":"2024-02-22T19:30:28.093673+0000","last_fullsized":"2024-02-22T19:30:28.093673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:19.102765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:19.102765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.094387+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:30:28.094387+0000","last_peered":"2024-02-22T19:30:28.094387+0000","last_clean":"2024-02-22T19:30:28.094387+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:30:28.094387+0000","last_undegraded":"2024-02-22T19:30:28.094387+0000","last_fullsized":"2024-02-22T19:30:28.094387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:35.181920+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:30:35.181920+0000","last_peered":"2024-02-22T19:30:35.181920+0000","last_clean":"2024-02-22T19:30:35.181920+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:30:35.181920+0000","last_undegraded":"2024-02-22T19:30:35.181920+0000","last_fullsized":"2024-02-22T19:30:35.181920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.075252+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:30:28.075252+0000","last_peered":"2024-02-22T19:30:28.075252+0000","last_clean":"2024-02-22T19:30:28.075252+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:30:28.075252+0000","last_undegraded":"2024-02-22T19:30:28.075252+0000","last_fullsized":"2024-02-22T19:30:28.075252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077068+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:30:28.077068+0000","last_peered":"2024-02-22T19:30:28.077068+0000","last_clean":"2024-02-22T19:30:28.077068+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:30:28.077068+0000","last_undegraded":"2024-02-22T19:30:28.077068+0000","last_fullsized":"2024-02-22T19:30:28.077068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:46.953185+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:30:46.953185+0000","last_peered":"2024-02-22T19:30:46.953185+0000","last_clean":"2024-02-22T19:30:46.953185+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:30:46.953185+0000","last_undegraded":"2024-02-22T19:30:46.953185+0000","last_fullsized":"2024-02-22T19:30:46.953185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:55.020116+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:30:55.020116+0000","last_peered":"2024-02-22T19:30:55.020116+0000","last_clean":"2024-02-22T19:30:55.020116+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:30:55.020116+0000","last_undegraded":"2024-02-22T19:30:55.020116+0000","last_fullsized":"2024-02-22T19:30:55.020116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-22T19:30:58.473166+0000","last_change":"2024-02-22T19:30:23.100251+0000","last_active":"2024-02-22T19:30:58.473166+0000","last_peered":"2024-02-22T19:30:58.473166+0000","last_clean":"2024-02-22T19:30:58.473166+0000","last_became_active":"2024-02-22T19:29:05.092069+0000","last_became_peered":"2024-02-22T19:29:05.092069+0000","last_unstale":"2024-02-22T19:30:58.473166+0000","last_undegraded":"2024-02-22T19:30:58.473166+0000","last_fullsized":"2024-02-22T19:30:58.473166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:23.100177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.100177+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:28.077282+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:30:28.077282+0000","last_peered":"2024-02-22T19:30:28.077282+0000","last_clean":"2024-02-22T19:30:28.077282+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:30:28.077282+0000","last_undegraded":"2024-02-22T19:30:28.077282+0000","last_fullsized":"2024-02-22T19:30:28.077282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-22T19:30:50.936816+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:30:50.936816+0000","last_peered":"2024-02-22T19:30:50.936816+0000","last_clean":"2024-02-22T19:30:50.936816+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:30:50.936816+0000","last_undegraded":"2024-02-22T19:30:50.936816+0000","last_fullsized":"2024-02-22T19:30:50.936816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":410,"ondisk_log_size":410,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":112,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266715,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297712,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":630784,"data_stored":289404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65100000000000002,"5min":0.871,"15min":0.871},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63100000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68700000000000006,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.55200000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58199999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68500000000000005,"5min":1.98,"15min":1.98},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64300000000000002,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71099999999999997,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.753,"15min":0.753},"last":0.749},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72899999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.72899999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297684,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447468,"statfs":{"total":95995035648,"available":95690207232,"internally_reserved":0,"allocated":602112,"data_stored":288019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":1.024,"15min":1.024},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73599999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.309,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.36799999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":7,"last update":"Thu Feb 22 19:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":296548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:40 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36399999999999999,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52900000000000003,"5min":0.62,"15min":0.62},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.378,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.41199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.502}]},{"osd":2,"last update":"Thu Feb 22 19:30:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56200000000000006,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.40100000000000002,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.42699999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Feb 22 19:30:27 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.43099999999999999,"15min":0.45500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.441,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.48999999999999999,"15min":0.50900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54300000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.623}]},{"osd":4,"last update":"Thu Feb 22 19:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54300000000000004,"15min":0.56599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.58799999999999997,"15min":0.60099999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.754}]},{"osd":7,"last update":"Thu Feb 22 19:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.76600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297716,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":634880,"data_stored":289974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.52400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.376}]},{"osd":2,"last update":"Thu Feb 22 19:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.23300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53900000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.311},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.377,"15min":0.374},"min":{"1min":0.252,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53200000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.40799999999999997,"15min":0.42599999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.443,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.42699999999999999,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46899999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70899999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.995,"15min":0.995},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Feb 22 19:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67600000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.624}]},{"osd":6,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.745}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297712,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":630784,"data_stored":289030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.48399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.376,"15min":0.36599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:30:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48999999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.29299999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.31},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.442},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.50800000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58899999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297740,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447412,"statfs":{"total":95995035648,"available":95690149888,"internally_reserved":0,"allocated":659456,"data_stored":296456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41599999999999998,"15min":0.432},"min":{"1min":0.20799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.45000000000000001,"15min":0.46400000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.47599999999999998,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.46999999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46500000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.626}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78300000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.82899999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953515,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297736,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447416,"statfs":{"total":95995035648,"available":95690153984,"internally_reserved":0,"allocated":655360,"data_stored":296495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":1,"15min":1},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.34200000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.40999999999999998},"min":{"1min":0.36499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Feb 22 19:29:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.626}]}]},{"osd":5,"up_from":39,"seq":167503724581,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297688,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447464,"statfs":{"total":95995035648,"available":95690203136,"internally_reserved":0,"allocated":606208,"data_stored":288387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.5229999999999997,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.35599999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78000000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.59499999999999997,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.5529999999999999,"5min":5.5529999999999999,"15min":5.5529999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.219,"15min":0.219},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.26600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.42499999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.46500000000000002},"min":{"1min":0.35299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":1.071,"15min":1.071},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:31:03.746 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-22T19:31:03.746 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-22T19:31:03.746 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-22T19:31:03.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph health --format=json 2024-02-22T19:31:03.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:02.924326+0000 osd.0 (osd.0) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:03.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:02.924380+0000 osd.0 (osd.0) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:04.034 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:02.748785+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:02.751776+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:03.056901+0000 mgr.y (mgr.14535) 84 : audit [DBG] from='client.24647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:03.203945+0000 mgr.y (mgr.14535) 85 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:03.491827+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:03.769392+0000 osd.6 (osd.6) 21 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:03.769450+0000 osd.6 (osd.6) 22 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:02.748785+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:02.751776+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:03.056901+0000 mgr.y (mgr.14535) 84 : audit [DBG] from='client.24647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:03.203945+0000 mgr.y (mgr.14535) 85 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:03.491827+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:03.769392+0000 osd.6 (osd.6) 21 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:03.769450+0000 osd.6 (osd.6) 22 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:04.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:31:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:02.748785+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:31:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:02.751776+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:31:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:03.056901+0000 mgr.y (mgr.14535) 84 : audit [DBG] from='client.24647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:03.203945+0000 mgr.y (mgr.14535) 85 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:03.491827+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-22T19:31:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:03.769392+0000 osd.6 (osd.6) 21 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:03.769450+0000 osd.6 (osd.6) 22 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:31:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:31:05.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:04.007011+0000 osd.3 (osd.3) 47 : cluster [DBG] 4.1d deep-scrub starts 2024-02-22T19:31:05.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:04.012654+0000 osd.3 (osd.3) 48 : cluster [DBG] 4.1d deep-scrub ok 2024-02-22T19:31:05.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:04.282294+0000 mgr.y (mgr.14535) 86 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:05.608 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:04.496741+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-22T19:31:05.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:04.007011+0000 osd.3 (osd.3) 47 : cluster [DBG] 4.1d deep-scrub starts 2024-02-22T19:31:05.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:04.012654+0000 osd.3 (osd.3) 48 : cluster [DBG] 4.1d deep-scrub ok 2024-02-22T19:31:05.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:04.282294+0000 mgr.y (mgr.14535) 86 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:05.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:04.496741+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-22T19:31:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:04.007011+0000 osd.3 (osd.3) 47 : cluster [DBG] 4.1d deep-scrub starts 2024-02-22T19:31:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:04.012654+0000 osd.3 (osd.3) 48 : cluster [DBG] 4.1d deep-scrub ok 2024-02-22T19:31:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:04.282294+0000 mgr.y (mgr.14535) 86 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:04.496741+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-22T19:31:06.053 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:31:06.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:05.204317+0000 mgr.y (mgr.14535) 87 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-22T19:31:06.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:06.053159+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.53:0/4083539255' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-22T19:31:06.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:05.204317+0000 mgr.y (mgr.14535) 87 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-22T19:31:06.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:06.053159+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.53:0/4083539255' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-22T19:31:06.816 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-22T19:31:06.816 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-22T19:31:06.817 INFO:tasks.cephadm:Setup complete, yielding 2024-02-22T19:31:06.817 INFO:teuthology.run_tasks:Running task thrashosds... 2024-02-22T19:31:06.828 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-02-22T19:31:06.828 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'pacific-release'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', 'flag\\(s\\) set', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(PG_', '\\(OSD_', 'mons down'], 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-ksirivad-pacific-release-whitelist', 'sha1': '391ea41edcaa85aa674d6e816fb2469974fc7d8e'}} 2024-02-22T19:31:06.828 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-02-22T19:31:06.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-02-22T19:31:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:05.204317+0000 mgr.y (mgr.14535) 87 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-22T19:31:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:06.053159+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.53:0/4083539255' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-22T19:31:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:07.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:06.062952+0000 osd.3 (osd.3) 49 : cluster [DBG] 6.b scrub starts 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:06.066243+0000 osd.3 (osd.3) 50 : cluster [DBG] 6.b scrub ok 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:06.493026+0000 osd.0 (osd.0) 45 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:06.495872+0000 osd.0 (osd.0) 46 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:06.062952+0000 osd.3 (osd.3) 49 : cluster [DBG] 6.b scrub starts 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:06.066243+0000 osd.3 (osd.3) 50 : cluster [DBG] 6.b scrub ok 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:06.493026+0000 osd.0 (osd.0) 45 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:31:07.605 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:06.495872+0000 osd.0 (osd.0) 46 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:31:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:06.062952+0000 osd.3 (osd.3) 49 : cluster [DBG] 6.b scrub starts 2024-02-22T19:31:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:06.066243+0000 osd.3 (osd.3) 50 : cluster [DBG] 6.b scrub ok 2024-02-22T19:31:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:06.493026+0000 osd.0 (osd.0) 45 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:31:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:06.495872+0000 osd.0 (osd.0) 46 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:31:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:07.204636+0000 mgr.y (mgr.14535) 88 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:07.484876+0000 osd.0 (osd.0) 47 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:31:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:07.487776+0000 osd.0 (osd.0) 48 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:31:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:08.042094+0000 osd.3 (osd.3) 51 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:31:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:08.045614+0000 osd.3 (osd.3) 52 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:31:09.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:07.204636+0000 mgr.y (mgr.14535) 88 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:07.484876+0000 osd.0 (osd.0) 47 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:31:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:07.487776+0000 osd.0 (osd.0) 48 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:31:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:08.042094+0000 osd.3 (osd.3) 51 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:31:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:08.045614+0000 osd.3 (osd.3) 52 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:31:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:07.204636+0000 mgr.y (mgr.14535) 88 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:07.484876+0000 osd.0 (osd.0) 47 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:31:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:07.487776+0000 osd.0 (osd.0) 48 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:31:09.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:08.042094+0000 osd.3 (osd.3) 51 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:31:09.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:08.045614+0000 osd.3 (osd.3) 52 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:31:09.395 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:31:09] ENGINE Bus STOPPING 2024-02-22T19:31:09.395 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:31:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:31:09.395 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:31:09] ENGINE Bus STOPPED 2024-02-22T19:31:09.396 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:31:09] ENGINE Bus STARTING 2024-02-22T19:31:09.396 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:31:09] ENGINE Serving on http://:::9283 2024-02-22T19:31:09.396 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:31:09] ENGINE Bus STARTED 2024-02-22T19:31:09.396 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:09] "GET /metrics HTTP/1.1" 200 172102 "" "Prometheus/2.33.4" 2024-02-22T19:31:09.734 INFO:tasks.thrashosds:Beginning thrashosds... 2024-02-22T19:31:09.734 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-22T19:31:09.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:31:10.017 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:09.061641+0000 osd.3 (osd.3) 53 : cluster [DBG] 6.c scrub starts 2024-02-22T19:31:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:09.064533+0000 osd.3 (osd.3) 54 : cluster [DBG] 6.c scrub ok 2024-02-22T19:31:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:09.082546+0000 mon.a (mon.0) 743 : audit [INF] from='client.? 172.21.15.53:0/2144433615' entity='client.admin' 2024-02-22T19:31:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:09.109716+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:31:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:09.111039+0000 mon.a (mon.0) 745 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:31:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:09.111740+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:31:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:09.116809+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:09.205391+0000 mgr.y (mgr.14535) 89 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:10.502 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:09.061641+0000 osd.3 (osd.3) 53 : cluster [DBG] 6.c scrub starts 2024-02-22T19:31:10.502 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:09.064533+0000 osd.3 (osd.3) 54 : cluster [DBG] 6.c scrub ok 2024-02-22T19:31:10.502 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:09.082546+0000 mon.a (mon.0) 743 : audit [INF] from='client.? 172.21.15.53:0/2144433615' entity='client.admin' 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:09.109716+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:09.111039+0000 mon.a (mon.0) 745 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:09.111740+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:09.116809+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:09.205391+0000 mgr.y (mgr.14535) 89 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:09.061641+0000 osd.3 (osd.3) 53 : cluster [DBG] 6.c scrub starts 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:09.064533+0000 osd.3 (osd.3) 54 : cluster [DBG] 6.c scrub ok 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:09.082546+0000 mon.a (mon.0) 743 : audit [INF] from='client.? 172.21.15.53:0/2144433615' entity='client.admin' 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:09.109716+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:09.111039+0000 mon.a (mon.0) 745 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:09.111740+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:31:10.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:09.116809+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:10.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:09.205391+0000 mgr.y (mgr.14535) 89 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:11.928 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:31:11.930 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:31:12.573 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:11.205927+0000 mgr.y (mgr.14535) 90 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:12.573 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:11.412727+0000 osd.0 (osd.0) 49 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:31:12.573 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:11.415713+0000 osd.0 (osd.0) 50 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:31:12.574 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:11.205927+0000 mgr.y (mgr.14535) 90 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:12.574 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:11.412727+0000 osd.0 (osd.0) 49 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:31:12.574 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:11.415713+0000 osd.0 (osd.0) 50 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:31:12.594 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-02-22T19:31:11.205599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":841,"num_read_kb":593,"num_write":428,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":493,"ondisk_log_size":493,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381988,"kb_used_data":5220,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579228,"statfs":{"total":767960285184,"available":765521129472,"internally_reserved":0,"allocated":5345280,"data_stored":2468398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001965"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490994+0000","last_change":"2024-02-22T19:30:28.096813+0000","last_active":"2024-02-22T19:31:03.490994+0000","last_peered":"2024-02-22T19:31:03.490994+0000","last_clean":"2024-02-22T19:31:03.490994+0000","last_became_active":"2024-02-22T19:29:07.099281+0000","last_became_peered":"2024-02-22T19:29:07.099281+0000","last_unstale":"2024-02-22T19:31:03.490994+0000","last_undegraded":"2024-02-22T19:31:03.490994+0000","last_fullsized":"2024-02-22T19:31:03.490994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490939+0000","last_change":"2024-02-22T19:30:24.056145+0000","last_active":"2024-02-22T19:31:03.490939+0000","last_peered":"2024-02-22T19:31:03.490939+0000","last_clean":"2024-02-22T19:31:03.490939+0000","last_became_active":"2024-02-22T19:29:05.091467+0000","last_became_peered":"2024-02-22T19:29:05.091467+0000","last_unstale":"2024-02-22T19:31:03.490939+0000","last_undegraded":"2024-02-22T19:31:03.490939+0000","last_fullsized":"2024-02-22T19:31:03.490939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148520+0000","last_change":"2024-02-22T19:30:07.664488+0000","last_active":"2024-02-22T19:31:04.148520+0000","last_peered":"2024-02-22T19:31:04.148520+0000","last_clean":"2024-02-22T19:31:04.148520+0000","last_became_active":"2024-02-22T19:29:01.002834+0000","last_became_peered":"2024-02-22T19:29:01.002834+0000","last_unstale":"2024-02-22T19:31:04.148520+0000","last_undegraded":"2024-02-22T19:31:04.148520+0000","last_fullsized":"2024-02-22T19:31:04.148520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.664424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490859+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:31:03.490859+0000","last_peered":"2024-02-22T19:31:03.490859+0000","last_clean":"2024-02-22T19:31:03.490859+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:31:03.490859+0000","last_undegraded":"2024-02-22T19:31:03.490859+0000","last_fullsized":"2024-02-22T19:31:03.490859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493005+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:31:03.493005+0000","last_peered":"2024-02-22T19:31:03.493005+0000","last_clean":"2024-02-22T19:31:03.493005+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:31:03.493005+0000","last_undegraded":"2024-02-22T19:31:03.493005+0000","last_fullsized":"2024-02-22T19:31:03.493005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490843+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:31:03.490843+0000","last_peered":"2024-02-22T19:31:03.490843+0000","last_clean":"2024-02-22T19:31:03.490843+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:31:03.490843+0000","last_undegraded":"2024-02-22T19:31:03.490843+0000","last_fullsized":"2024-02-22T19:31:03.490843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493460+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:31:03.493460+0000","last_peered":"2024-02-22T19:31:03.493460+0000","last_clean":"2024-02-22T19:31:03.493460+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:31:03.493460+0000","last_undegraded":"2024-02-22T19:31:03.493460+0000","last_fullsized":"2024-02-22T19:31:03.493460+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492869+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:31:03.492869+0000","last_peered":"2024-02-22T19:31:03.492869+0000","last_clean":"2024-02-22T19:31:03.492869+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:31:03.492869+0000","last_undegraded":"2024-02-22T19:31:03.492869+0000","last_fullsized":"2024-02-22T19:31:03.492869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148018+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:31:04.148018+0000","last_peered":"2024-02-22T19:31:04.148018+0000","last_clean":"2024-02-22T19:31:04.148018+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:31:04.148018+0000","last_undegraded":"2024-02-22T19:31:04.148018+0000","last_fullsized":"2024-02-22T19:31:04.148018+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149656+0000","last_change":"2024-02-22T19:30:26.528062+0000","last_active":"2024-02-22T19:31:04.149656+0000","last_peered":"2024-02-22T19:31:04.149656+0000","last_clean":"2024-02-22T19:31:04.149656+0000","last_became_active":"2024-02-22T19:29:05.091567+0000","last_became_peered":"2024-02-22T19:29:05.091567+0000","last_unstale":"2024-02-22T19:31:04.149656+0000","last_undegraded":"2024-02-22T19:31:04.149656+0000","last_fullsized":"2024-02-22T19:31:04.149656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.527965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.527965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149550+0000","last_change":"2024-02-22T19:30:10.440133+0000","last_active":"2024-02-22T19:31:04.149550+0000","last_peered":"2024-02-22T19:31:04.149550+0000","last_clean":"2024-02-22T19:31:04.149550+0000","last_became_active":"2024-02-22T19:29:01.005184+0000","last_became_peered":"2024-02-22T19:29:01.005184+0000","last_unstale":"2024-02-22T19:31:04.149550+0000","last_undegraded":"2024-02-22T19:31:04.149550+0000","last_fullsized":"2024-02-22T19:31:04.149550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.440058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.440058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491173+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:31:03.491173+0000","last_peered":"2024-02-22T19:31:03.491173+0000","last_clean":"2024-02-22T19:31:03.491173+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:31:03.491173+0000","last_undegraded":"2024-02-22T19:31:03.491173+0000","last_fullsized":"2024-02-22T19:31:03.491173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148637+0000","last_change":"2024-02-22T19:30:18.496805+0000","last_active":"2024-02-22T19:31:04.148637+0000","last_peered":"2024-02-22T19:31:04.148637+0000","last_clean":"2024-02-22T19:31:04.148637+0000","last_became_active":"2024-02-22T19:29:07.100382+0000","last_became_peered":"2024-02-22T19:29:07.100382+0000","last_unstale":"2024-02-22T19:31:04.148637+0000","last_undegraded":"2024-02-22T19:31:04.148637+0000","last_fullsized":"2024-02-22T19:31:04.148637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149144+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:31:04.149144+0000","last_peered":"2024-02-22T19:31:04.149144+0000","last_clean":"2024-02-22T19:31:04.149144+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:31:04.149144+0000","last_undegraded":"2024-02-22T19:31:04.149144+0000","last_fullsized":"2024-02-22T19:31:04.149144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149644+0000","last_change":"2024-02-22T19:30:22.574826+0000","last_active":"2024-02-22T19:31:04.149644+0000","last_peered":"2024-02-22T19:31:04.149644+0000","last_clean":"2024-02-22T19:31:04.149644+0000","last_became_active":"2024-02-22T19:29:01.007689+0000","last_became_peered":"2024-02-22T19:29:01.007689+0000","last_unstale":"2024-02-22T19:31:04.149644+0000","last_undegraded":"2024-02-22T19:31:04.149644+0000","last_fullsized":"2024-02-22T19:31:04.149644+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:22.574750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.574750+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491451+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:31:03.491451+0000","last_peered":"2024-02-22T19:31:03.491451+0000","last_clean":"2024-02-22T19:31:03.491451+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:31:03.491451+0000","last_undegraded":"2024-02-22T19:31:03.491451+0000","last_fullsized":"2024-02-22T19:31:03.491451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492016+0000","last_change":"2024-02-22T19:30:34.059638+0000","last_active":"2024-02-22T19:31:03.492016+0000","last_peered":"2024-02-22T19:31:03.492016+0000","last_clean":"2024-02-22T19:31:03.492016+0000","last_became_active":"2024-02-22T19:29:07.197580+0000","last_became_peered":"2024-02-22T19:29:07.197580+0000","last_unstale":"2024-02-22T19:31:03.492016+0000","last_undegraded":"2024-02-22T19:31:03.492016+0000","last_fullsized":"2024-02-22T19:31:03.492016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491449+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:31:03.491449+0000","last_peered":"2024-02-22T19:31:03.491449+0000","last_clean":"2024-02-22T19:31:03.491449+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:31:03.491449+0000","last_undegraded":"2024-02-22T19:31:03.491449+0000","last_fullsized":"2024-02-22T19:31:03.491449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148433+0000","last_change":"2024-02-22T19:30:00.634139+0000","last_active":"2024-02-22T19:31:04.148433+0000","last_peered":"2024-02-22T19:31:04.148433+0000","last_clean":"2024-02-22T19:31:04.148433+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:31:04.148433+0000","last_undegraded":"2024-02-22T19:31:04.148433+0000","last_fullsized":"2024-02-22T19:31:04.148433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:30:00.634060+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147946+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:31:04.147946+0000","last_peered":"2024-02-22T19:31:04.147946+0000","last_clean":"2024-02-22T19:31:04.147946+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:31:04.147946+0000","last_undegraded":"2024-02-22T19:31:04.147946+0000","last_fullsized":"2024-02-22T19:31:04.147946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493627+0000","last_change":"2024-02-22T19:30:26.823434+0000","last_active":"2024-02-22T19:31:03.493627+0000","last_peered":"2024-02-22T19:31:03.493627+0000","last_clean":"2024-02-22T19:31:03.493627+0000","last_became_active":"2024-02-22T19:29:07.197840+0000","last_became_peered":"2024-02-22T19:29:07.197840+0000","last_unstale":"2024-02-22T19:31:03.493627+0000","last_undegraded":"2024-02-22T19:31:03.493627+0000","last_fullsized":"2024-02-22T19:31:03.493627+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.823363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.823363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491395+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:31:03.491395+0000","last_peered":"2024-02-22T19:31:03.491395+0000","last_clean":"2024-02-22T19:31:03.491395+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:31:03.491395+0000","last_undegraded":"2024-02-22T19:31:03.491395+0000","last_fullsized":"2024-02-22T19:31:03.491395+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491246+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:31:03.491246+0000","last_peered":"2024-02-22T19:31:03.491246+0000","last_clean":"2024-02-22T19:31:03.491246+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:31:03.491246+0000","last_undegraded":"2024-02-22T19:31:03.491246+0000","last_fullsized":"2024-02-22T19:31:03.491246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.012802+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:31:04.012802+0000","last_peered":"2024-02-22T19:31:04.012802+0000","last_clean":"2024-02-22T19:31:04.012802+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:31:04.012802+0000","last_undegraded":"2024-02-22T19:31:04.012802+0000","last_fullsized":"2024-02-22T19:31:04.012802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148685+0000","last_change":"2024-02-22T19:30:37.617824+0000","last_active":"2024-02-22T19:31:04.148685+0000","last_peered":"2024-02-22T19:31:04.148685+0000","last_clean":"2024-02-22T19:31:04.148685+0000","last_became_active":"2024-02-22T19:29:07.104332+0000","last_became_peered":"2024-02-22T19:29:07.104332+0000","last_unstale":"2024-02-22T19:31:04.148685+0000","last_undegraded":"2024-02-22T19:31:04.148685+0000","last_fullsized":"2024-02-22T19:31:04.148685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.695544+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:31:04.695544+0000","last_peered":"2024-02-22T19:31:04.695544+0000","last_clean":"2024-02-22T19:31:04.695544+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:31:04.695544+0000","last_undegraded":"2024-02-22T19:31:04.695544+0000","last_fullsized":"2024-02-22T19:31:04.695544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491021+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:31:03.491021+0000","last_peered":"2024-02-22T19:31:03.491021+0000","last_clean":"2024-02-22T19:31:03.491021+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:31:03.491021+0000","last_undegraded":"2024-02-22T19:31:03.491021+0000","last_fullsized":"2024-02-22T19:31:03.491021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148853+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:31:04.148853+0000","last_peered":"2024-02-22T19:31:04.148853+0000","last_clean":"2024-02-22T19:31:04.148853+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:31:04.148853+0000","last_undegraded":"2024-02-22T19:31:04.148853+0000","last_fullsized":"2024-02-22T19:31:04.148853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491531+0000","last_change":"2024-02-22T19:30:31.070100+0000","last_active":"2024-02-22T19:31:03.491531+0000","last_peered":"2024-02-22T19:31:03.491531+0000","last_clean":"2024-02-22T19:31:03.491531+0000","last_became_active":"2024-02-22T19:29:07.089971+0000","last_became_peered":"2024-02-22T19:29:07.089971+0000","last_unstale":"2024-02-22T19:31:03.491531+0000","last_undegraded":"2024-02-22T19:31:03.491531+0000","last_fullsized":"2024-02-22T19:31:03.491531+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149089+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:31:04.149089+0000","last_peered":"2024-02-22T19:31:04.149089+0000","last_clean":"2024-02-22T19:31:04.149089+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:31:04.149089+0000","last_undegraded":"2024-02-22T19:31:04.149089+0000","last_fullsized":"2024-02-22T19:31:04.149089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492689+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:31:03.492689+0000","last_peered":"2024-02-22T19:31:03.492689+0000","last_clean":"2024-02-22T19:31:03.492689+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:31:03.492689+0000","last_undegraded":"2024-02-22T19:31:03.492689+0000","last_fullsized":"2024-02-22T19:31:03.492689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150273+0000","last_change":"2024-02-22T19:30:06.607076+0000","last_active":"2024-02-22T19:31:04.150273+0000","last_peered":"2024-02-22T19:31:04.150273+0000","last_clean":"2024-02-22T19:31:04.150273+0000","last_became_active":"2024-02-22T19:29:03.087740+0000","last_became_peered":"2024-02-22T19:29:03.087740+0000","last_unstale":"2024-02-22T19:31:04.150273+0000","last_undegraded":"2024-02-22T19:31:04.150273+0000","last_fullsized":"2024-02-22T19:31:04.150273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.607011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.607011+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149046+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:31:04.149046+0000","last_peered":"2024-02-22T19:31:04.149046+0000","last_clean":"2024-02-22T19:31:04.149046+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:31:04.149046+0000","last_undegraded":"2024-02-22T19:31:04.149046+0000","last_fullsized":"2024-02-22T19:31:04.149046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147968+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:31:04.147968+0000","last_peered":"2024-02-22T19:31:04.147968+0000","last_clean":"2024-02-22T19:31:04.147968+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:31:04.147968+0000","last_undegraded":"2024-02-22T19:31:04.147968+0000","last_fullsized":"2024-02-22T19:31:04.147968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148519+0000","last_change":"2024-02-22T19:30:08.671313+0000","last_active":"2024-02-22T19:31:04.148519+0000","last_peered":"2024-02-22T19:31:04.148519+0000","last_clean":"2024-02-22T19:31:04.148519+0000","last_became_active":"2024-02-22T19:29:01.009788+0000","last_became_peered":"2024-02-22T19:29:01.009788+0000","last_unstale":"2024-02-22T19:31:04.148519+0000","last_undegraded":"2024-02-22T19:31:04.148519+0000","last_fullsized":"2024-02-22T19:31:04.148519+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:08.671238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:08.671238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492696+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:31:03.492696+0000","last_peered":"2024-02-22T19:31:03.492696+0000","last_clean":"2024-02-22T19:31:03.492696+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:31:03.492696+0000","last_undegraded":"2024-02-22T19:31:03.492696+0000","last_fullsized":"2024-02-22T19:31:03.492696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492917+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:31:03.492917+0000","last_peered":"2024-02-22T19:31:03.492917+0000","last_clean":"2024-02-22T19:31:03.492917+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:31:03.492917+0000","last_undegraded":"2024-02-22T19:31:03.492917+0000","last_fullsized":"2024-02-22T19:31:03.492917+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150440+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:31:04.150440+0000","last_peered":"2024-02-22T19:31:04.150440+0000","last_clean":"2024-02-22T19:31:04.150440+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:31:04.150440+0000","last_undegraded":"2024-02-22T19:31:04.150440+0000","last_fullsized":"2024-02-22T19:31:04.150440+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148716+0000","last_change":"2024-02-22T19:30:07.463186+0000","last_active":"2024-02-22T19:31:04.148716+0000","last_peered":"2024-02-22T19:31:04.148716+0000","last_clean":"2024-02-22T19:31:04.148716+0000","last_became_active":"2024-02-22T19:29:01.006896+0000","last_became_peered":"2024-02-22T19:29:01.006896+0000","last_unstale":"2024-02-22T19:31:04.148716+0000","last_undegraded":"2024-02-22T19:31:04.148716+0000","last_fullsized":"2024-02-22T19:31:04.148716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.463122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.463122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492820+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:31:03.492820+0000","last_peered":"2024-02-22T19:31:03.492820+0000","last_clean":"2024-02-22T19:31:03.492820+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:31:03.492820+0000","last_undegraded":"2024-02-22T19:31:03.492820+0000","last_fullsized":"2024-02-22T19:31:03.492820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:07.487841+0000","last_change":"2024-02-22T19:31:07.487841+0000","last_active":"2024-02-22T19:31:07.487841+0000","last_peered":"2024-02-22T19:31:07.487841+0000","last_clean":"2024-02-22T19:31:07.487841+0000","last_became_active":"2024-02-22T19:29:07.101369+0000","last_became_peered":"2024-02-22T19:29:07.101369+0000","last_unstale":"2024-02-22T19:31:07.487841+0000","last_undegraded":"2024-02-22T19:31:07.487841+0000","last_fullsized":"2024-02-22T19:31:07.487841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493046+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:31:03.493046+0000","last_peered":"2024-02-22T19:31:03.493046+0000","last_clean":"2024-02-22T19:31:03.493046+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:31:03.493046+0000","last_undegraded":"2024-02-22T19:31:03.493046+0000","last_fullsized":"2024-02-22T19:31:03.493046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149806+0000","last_change":"2024-02-22T19:30:03.688086+0000","last_active":"2024-02-22T19:31:04.149806+0000","last_peered":"2024-02-22T19:31:04.149806+0000","last_clean":"2024-02-22T19:31:04.149806+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:31:04.149806+0000","last_undegraded":"2024-02-22T19:31:04.149806+0000","last_fullsized":"2024-02-22T19:31:04.149806+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.688024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.688024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491215+0000","last_change":"2024-02-22T19:30:14.042137+0000","last_active":"2024-02-22T19:31:03.491215+0000","last_peered":"2024-02-22T19:31:03.491215+0000","last_clean":"2024-02-22T19:31:03.491215+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:31:03.491215+0000","last_undegraded":"2024-02-22T19:31:03.491215+0000","last_fullsized":"2024-02-22T19:31:03.491215+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.042059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.042059+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492607+0000","last_change":"2024-02-22T19:30:13.709779+0000","last_active":"2024-02-22T19:31:03.492607+0000","last_peered":"2024-02-22T19:31:03.492607+0000","last_clean":"2024-02-22T19:31:03.492607+0000","last_became_active":"2024-02-22T19:29:07.102604+0000","last_became_peered":"2024-02-22T19:29:07.102604+0000","last_unstale":"2024-02-22T19:31:03.492607+0000","last_undegraded":"2024-02-22T19:31:03.492607+0000","last_fullsized":"2024-02-22T19:31:03.492607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.709691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.709691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491046+0000","last_change":"2024-02-22T19:30:45.021593+0000","last_active":"2024-02-22T19:31:03.491046+0000","last_peered":"2024-02-22T19:31:03.491046+0000","last_clean":"2024-02-22T19:31:03.491046+0000","last_became_active":"2024-02-22T19:29:05.091550+0000","last_became_peered":"2024-02-22T19:29:05.091550+0000","last_unstale":"2024-02-22T19:31:03.491046+0000","last_undegraded":"2024-02-22T19:31:03.491046+0000","last_fullsized":"2024-02-22T19:31:03.491046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491722+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:31:03.491722+0000","last_peered":"2024-02-22T19:31:03.491722+0000","last_clean":"2024-02-22T19:31:03.491722+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:31:03.491722+0000","last_undegraded":"2024-02-22T19:31:03.491722+0000","last_fullsized":"2024-02-22T19:31:03.491722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491427+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:31:03.491427+0000","last_peered":"2024-02-22T19:31:03.491427+0000","last_clean":"2024-02-22T19:31:03.491427+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:31:03.491427+0000","last_undegraded":"2024-02-22T19:31:03.491427+0000","last_fullsized":"2024-02-22T19:31:03.491427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148616+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:31:04.148616+0000","last_peered":"2024-02-22T19:31:04.148616+0000","last_clean":"2024-02-22T19:31:04.148616+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:31:04.148616+0000","last_undegraded":"2024-02-22T19:31:04.148616+0000","last_fullsized":"2024-02-22T19:31:04.148616+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.694721+0000","last_change":"2024-02-22T19:30:21.533563+0000","last_active":"2024-02-22T19:31:04.694721+0000","last_peered":"2024-02-22T19:31:04.694721+0000","last_clean":"2024-02-22T19:31:04.694721+0000","last_became_active":"2024-02-22T19:29:05.092444+0000","last_became_peered":"2024-02-22T19:29:05.092444+0000","last_unstale":"2024-02-22T19:31:04.694721+0000","last_undegraded":"2024-02-22T19:31:04.694721+0000","last_fullsized":"2024-02-22T19:31:04.694721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:21.533452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.533452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148953+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:31:04.148953+0000","last_peered":"2024-02-22T19:31:04.148953+0000","last_clean":"2024-02-22T19:31:04.148953+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:31:04.148953+0000","last_undegraded":"2024-02-22T19:31:04.148953+0000","last_fullsized":"2024-02-22T19:31:04.148953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493557+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:31:03.493557+0000","last_peered":"2024-02-22T19:31:03.493557+0000","last_clean":"2024-02-22T19:31:03.493557+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:31:03.493557+0000","last_undegraded":"2024-02-22T19:31:03.493557+0000","last_fullsized":"2024-02-22T19:31:03.493557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492256+0000","last_change":"2024-02-22T19:30:16.057704+0000","last_active":"2024-02-22T19:31:03.492256+0000","last_peered":"2024-02-22T19:31:03.492256+0000","last_clean":"2024-02-22T19:31:03.492256+0000","last_became_active":"2024-02-22T19:29:07.097503+0000","last_became_peered":"2024-02-22T19:29:07.097503+0000","last_unstale":"2024-02-22T19:31:03.492256+0000","last_undegraded":"2024-02-22T19:31:03.492256+0000","last_fullsized":"2024-02-22T19:31:03.492256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:08.045723+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:31:08.045723+0000","last_peered":"2024-02-22T19:31:08.045723+0000","last_clean":"2024-02-22T19:31:08.045723+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:31:08.045723+0000","last_undegraded":"2024-02-22T19:31:08.045723+0000","last_fullsized":"2024-02-22T19:31:08.045723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148135+0000","last_change":"2024-02-22T19:30:03.653428+0000","last_active":"2024-02-22T19:31:04.148135+0000","last_peered":"2024-02-22T19:31:04.148135+0000","last_clean":"2024-02-22T19:31:04.148135+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:31:04.148135+0000","last_undegraded":"2024-02-22T19:31:04.148135+0000","last_fullsized":"2024-02-22T19:31:04.148135+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.653348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.653348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'9","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149987+0000","last_change":"2024-02-22T19:30:05.473412+0000","last_active":"2024-02-22T19:31:04.149987+0000","last_peered":"2024-02-22T19:31:04.149987+0000","last_clean":"2024-02-22T19:31:04.149987+0000","last_became_active":"2024-02-22T19:29:03.448670+0000","last_became_peered":"2024-02-22T19:29:03.448670+0000","last_unstale":"2024-02-22T19:31:04.149987+0000","last_undegraded":"2024-02-22T19:31:04.149987+0000","last_fullsized":"2024-02-22T19:31:04.149987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:05.473341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.473341+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148211+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:31:04.148211+0000","last_peered":"2024-02-22T19:31:04.148211+0000","last_clean":"2024-02-22T19:31:04.148211+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:31:04.148211+0000","last_undegraded":"2024-02-22T19:31:04.148211+0000","last_fullsized":"2024-02-22T19:31:04.148211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493566+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:31:03.493566+0000","last_peered":"2024-02-22T19:31:03.493566+0000","last_clean":"2024-02-22T19:31:03.493566+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:31:03.493566+0000","last_undegraded":"2024-02-22T19:31:03.493566+0000","last_fullsized":"2024-02-22T19:31:03.493566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149725+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:31:04.149725+0000","last_peered":"2024-02-22T19:31:04.149725+0000","last_clean":"2024-02-22T19:31:04.149725+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:31:04.149725+0000","last_undegraded":"2024-02-22T19:31:04.149725+0000","last_fullsized":"2024-02-22T19:31:04.149725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148938+0000","last_change":"2024-02-22T19:30:13.766624+0000","last_active":"2024-02-22T19:31:04.148938+0000","last_peered":"2024-02-22T19:31:04.148938+0000","last_clean":"2024-02-22T19:31:04.148938+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:31:04.148938+0000","last_undegraded":"2024-02-22T19:31:04.148938+0000","last_fullsized":"2024-02-22T19:31:04.148938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:13.766546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.766546+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148605+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:31:04.148605+0000","last_peered":"2024-02-22T19:31:04.148605+0000","last_clean":"2024-02-22T19:31:04.148605+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:31:04.148605+0000","last_undegraded":"2024-02-22T19:31:04.148605+0000","last_fullsized":"2024-02-22T19:31:04.148605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148809+0000","last_change":"2024-02-22T19:30:27.575360+0000","last_active":"2024-02-22T19:31:04.148809+0000","last_peered":"2024-02-22T19:31:04.148809+0000","last_clean":"2024-02-22T19:31:04.148809+0000","last_became_active":"2024-02-22T19:29:05.092190+0000","last_became_peered":"2024-02-22T19:29:05.092190+0000","last_unstale":"2024-02-22T19:31:04.148809+0000","last_undegraded":"2024-02-22T19:31:04.148809+0000","last_fullsized":"2024-02-22T19:31:04.148809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147912+0000","last_change":"2024-02-22T19:30:29.656325+0000","last_active":"2024-02-22T19:31:04.147912+0000","last_peered":"2024-02-22T19:31:04.147912+0000","last_clean":"2024-02-22T19:31:04.147912+0000","last_became_active":"2024-02-22T19:29:01.007053+0000","last_became_peered":"2024-02-22T19:29:01.007053+0000","last_unstale":"2024-02-22T19:31:04.147912+0000","last_undegraded":"2024-02-22T19:31:04.147912+0000","last_fullsized":"2024-02-22T19:31:04.147912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493655+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:31:03.493655+0000","last_peered":"2024-02-22T19:31:03.493655+0000","last_clean":"2024-02-22T19:31:03.493655+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:31:03.493655+0000","last_undegraded":"2024-02-22T19:31:03.493655+0000","last_fullsized":"2024-02-22T19:31:03.493655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149312+0000","last_change":"2024-02-22T19:30:14.461934+0000","last_active":"2024-02-22T19:31:04.149312+0000","last_peered":"2024-02-22T19:31:04.149312+0000","last_clean":"2024-02-22T19:31:04.149312+0000","last_became_active":"2024-02-22T19:29:07.197976+0000","last_became_peered":"2024-02-22T19:29:07.197976+0000","last_unstale":"2024-02-22T19:31:04.149312+0000","last_undegraded":"2024-02-22T19:31:04.149312+0000","last_fullsized":"2024-02-22T19:31:04.149312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150723+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:31:04.150723+0000","last_peered":"2024-02-22T19:31:04.150723+0000","last_clean":"2024-02-22T19:31:04.150723+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:31:04.150723+0000","last_undegraded":"2024-02-22T19:31:04.150723+0000","last_fullsized":"2024-02-22T19:31:04.150723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148276+0000","last_change":"2024-02-22T19:30:28.601662+0000","last_active":"2024-02-22T19:31:04.148276+0000","last_peered":"2024-02-22T19:31:04.148276+0000","last_clean":"2024-02-22T19:31:04.148276+0000","last_became_active":"2024-02-22T19:29:03.447032+0000","last_became_peered":"2024-02-22T19:29:03.447032+0000","last_unstale":"2024-02-22T19:31:04.148276+0000","last_undegraded":"2024-02-22T19:31:04.148276+0000","last_fullsized":"2024-02-22T19:31:04.148276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.695763+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:31:04.695763+0000","last_peered":"2024-02-22T19:31:04.695763+0000","last_clean":"2024-02-22T19:31:04.695763+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:31:04.695763+0000","last_undegraded":"2024-02-22T19:31:04.695763+0000","last_fullsized":"2024-02-22T19:31:04.695763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:06.066327+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:31:06.066327+0000","last_peered":"2024-02-22T19:31:06.066327+0000","last_clean":"2024-02-22T19:31:06.066327+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:31:06.066327+0000","last_undegraded":"2024-02-22T19:31:06.066327+0000","last_fullsized":"2024-02-22T19:31:06.066327+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150366+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:31:04.150366+0000","last_peered":"2024-02-22T19:31:04.150366+0000","last_clean":"2024-02-22T19:31:04.150366+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:31:04.150366+0000","last_undegraded":"2024-02-22T19:31:04.150366+0000","last_fullsized":"2024-02-22T19:31:04.150366+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491266+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:31:03.491266+0000","last_peered":"2024-02-22T19:31:03.491266+0000","last_clean":"2024-02-22T19:31:03.491266+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:31:03.491266+0000","last_undegraded":"2024-02-22T19:31:03.491266+0000","last_fullsized":"2024-02-22T19:31:03.491266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147777+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:31:04.147777+0000","last_peered":"2024-02-22T19:31:04.147777+0000","last_clean":"2024-02-22T19:31:04.147777+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:31:04.147777+0000","last_undegraded":"2024-02-22T19:31:04.147777+0000","last_fullsized":"2024-02-22T19:31:04.147777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148190+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:31:04.148190+0000","last_peered":"2024-02-22T19:31:04.148190+0000","last_clean":"2024-02-22T19:31:04.148190+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:31:04.148190+0000","last_undegraded":"2024-02-22T19:31:04.148190+0000","last_fullsized":"2024-02-22T19:31:04.148190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148379+0000","last_change":"2024-02-22T19:30:29.110575+0000","last_active":"2024-02-22T19:31:04.148379+0000","last_peered":"2024-02-22T19:31:04.148379+0000","last_clean":"2024-02-22T19:31:04.148379+0000","last_became_active":"2024-02-22T19:29:05.091952+0000","last_became_peered":"2024-02-22T19:29:05.091952+0000","last_unstale":"2024-02-22T19:31:04.148379+0000","last_undegraded":"2024-02-22T19:31:04.148379+0000","last_fullsized":"2024-02-22T19:31:04.148379+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.110513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.110513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150818+0000","last_change":"2024-02-22T19:30:10.619750+0000","last_active":"2024-02-22T19:31:04.150818+0000","last_peered":"2024-02-22T19:31:04.150818+0000","last_clean":"2024-02-22T19:31:04.150818+0000","last_became_active":"2024-02-22T19:29:01.004850+0000","last_became_peered":"2024-02-22T19:29:01.004850+0000","last_unstale":"2024-02-22T19:31:04.150818+0000","last_undegraded":"2024-02-22T19:31:04.150818+0000","last_fullsized":"2024-02-22T19:31:04.150818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.619685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.619685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149996+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:31:04.149996+0000","last_peered":"2024-02-22T19:31:04.149996+0000","last_clean":"2024-02-22T19:31:04.149996+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:31:04.149996+0000","last_undegraded":"2024-02-22T19:31:04.149996+0000","last_fullsized":"2024-02-22T19:31:04.149996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:06.495959+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:31:06.495959+0000","last_peered":"2024-02-22T19:31:06.495959+0000","last_clean":"2024-02-22T19:31:06.495959+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:31:06.495959+0000","last_undegraded":"2024-02-22T19:31:06.495959+0000","last_fullsized":"2024-02-22T19:31:06.495959+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149146+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:31:04.149146+0000","last_peered":"2024-02-22T19:31:04.149146+0000","last_clean":"2024-02-22T19:31:04.149146+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:31:04.149146+0000","last_undegraded":"2024-02-22T19:31:04.149146+0000","last_fullsized":"2024-02-22T19:31:04.149146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150057+0000","last_change":"2024-02-22T19:30:17.466789+0000","last_active":"2024-02-22T19:31:04.150057+0000","last_peered":"2024-02-22T19:31:04.150057+0000","last_clean":"2024-02-22T19:31:04.150057+0000","last_became_active":"2024-02-22T19:29:01.010917+0000","last_became_peered":"2024-02-22T19:29:01.010917+0000","last_unstale":"2024-02-22T19:31:04.150057+0000","last_undegraded":"2024-02-22T19:31:04.150057+0000","last_fullsized":"2024-02-22T19:31:04.150057+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148585+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:31:04.148585+0000","last_peered":"2024-02-22T19:31:04.148585+0000","last_clean":"2024-02-22T19:31:04.148585+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:31:04.148585+0000","last_undegraded":"2024-02-22T19:31:04.148585+0000","last_fullsized":"2024-02-22T19:31:04.148585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493259+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:31:03.493259+0000","last_peered":"2024-02-22T19:31:03.493259+0000","last_clean":"2024-02-22T19:31:03.493259+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:31:03.493259+0000","last_undegraded":"2024-02-22T19:31:03.493259+0000","last_fullsized":"2024-02-22T19:31:03.493259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149039+0000","last_change":"2024-02-22T19:30:09.477558+0000","last_active":"2024-02-22T19:31:04.149039+0000","last_peered":"2024-02-22T19:31:04.149039+0000","last_clean":"2024-02-22T19:31:04.149039+0000","last_became_active":"2024-02-22T19:29:01.006282+0000","last_became_peered":"2024-02-22T19:29:01.006282+0000","last_unstale":"2024-02-22T19:31:04.149039+0000","last_undegraded":"2024-02-22T19:31:04.149039+0000","last_fullsized":"2024-02-22T19:31:04.149039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_clean_scrub_stamp":"2024-02-22T19:30:09.477488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491226+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:31:03.491226+0000","last_peered":"2024-02-22T19:31:03.491226+0000","last_clean":"2024-02-22T19:31:03.491226+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:31:03.491226+0000","last_undegraded":"2024-02-22T19:31:03.491226+0000","last_fullsized":"2024-02-22T19:31:03.491226+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.495939+0000","last_change":"2024-02-22T19:30:07.992740+0000","last_active":"2024-02-22T19:31:04.495939+0000","last_peered":"2024-02-22T19:31:04.495939+0000","last_clean":"2024-02-22T19:31:04.495939+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:31:04.495939+0000","last_undegraded":"2024-02-22T19:31:04.495939+0000","last_fullsized":"2024-02-22T19:31:04.495939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:30:07.992671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.992671+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148142+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:31:04.148142+0000","last_peered":"2024-02-22T19:31:04.148142+0000","last_clean":"2024-02-22T19:31:04.148142+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:31:04.148142+0000","last_undegraded":"2024-02-22T19:31:04.148142+0000","last_fullsized":"2024-02-22T19:31:04.148142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150945+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:31:04.150945+0000","last_peered":"2024-02-22T19:31:04.150945+0000","last_clean":"2024-02-22T19:31:04.150945+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:31:04.150945+0000","last_undegraded":"2024-02-22T19:31:04.150945+0000","last_fullsized":"2024-02-22T19:31:04.150945+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490671+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:31:03.490671+0000","last_peered":"2024-02-22T19:31:03.490671+0000","last_clean":"2024-02-22T19:31:03.490671+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:31:03.490671+0000","last_undegraded":"2024-02-22T19:31:03.490671+0000","last_fullsized":"2024-02-22T19:31:03.490671+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148038+0000","last_change":"2024-02-22T19:30:18.117213+0000","last_active":"2024-02-22T19:31:04.148038+0000","last_peered":"2024-02-22T19:31:04.148038+0000","last_clean":"2024-02-22T19:31:04.148038+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:31:04.148038+0000","last_undegraded":"2024-02-22T19:31:04.148038+0000","last_fullsized":"2024-02-22T19:31:04.148038+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.117111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148728+0000","last_change":"2024-02-22T19:30:28.128814+0000","last_active":"2024-02-22T19:31:04.148728+0000","last_peered":"2024-02-22T19:31:04.148728+0000","last_clean":"2024-02-22T19:31:04.148728+0000","last_became_active":"2024-02-22T19:30:24.410718+0000","last_became_peered":"2024-02-22T19:30:24.410718+0000","last_unstale":"2024-02-22T19:31:04.148728+0000","last_undegraded":"2024-02-22T19:31:04.148728+0000","last_fullsized":"2024-02-22T19:31:04.148728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148628+0000","last_change":"2024-02-22T19:30:31.102193+0000","last_active":"2024-02-22T19:31:04.148628+0000","last_peered":"2024-02-22T19:31:04.148628+0000","last_clean":"2024-02-22T19:31:04.148628+0000","last_became_active":"2024-02-22T19:29:05.098656+0000","last_became_peered":"2024-02-22T19:29:05.098656+0000","last_unstale":"2024-02-22T19:31:04.148628+0000","last_undegraded":"2024-02-22T19:31:04.148628+0000","last_fullsized":"2024-02-22T19:31:04.148628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.102065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.102065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149698+0000","last_change":"2024-02-22T19:30:35.556098+0000","last_active":"2024-02-22T19:31:04.149698+0000","last_peered":"2024-02-22T19:31:04.149698+0000","last_clean":"2024-02-22T19:31:04.149698+0000","last_became_active":"2024-02-22T19:29:07.098131+0000","last_became_peered":"2024-02-22T19:29:07.098131+0000","last_unstale":"2024-02-22T19:31:04.149698+0000","last_undegraded":"2024-02-22T19:31:04.149698+0000","last_fullsized":"2024-02-22T19:31:04.149698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150275+0000","last_change":"2024-02-22T19:29:15.492725+0000","last_active":"2024-02-22T19:31:04.150275+0000","last_peered":"2024-02-22T19:31:04.150275+0000","last_clean":"2024-02-22T19:31:04.150275+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:31:04.150275+0000","last_undegraded":"2024-02-22T19:31:04.150275+0000","last_fullsized":"2024-02-22T19:31:04.150275+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:15.492609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:29:15.492609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492866+0000","last_change":"2024-02-22T19:30:45.909363+0000","last_active":"2024-02-22T19:31:03.492866+0000","last_peered":"2024-02-22T19:31:03.492866+0000","last_clean":"2024-02-22T19:31:03.492866+0000","last_became_active":"2024-02-22T19:29:01.010904+0000","last_became_peered":"2024-02-22T19:29:01.010904+0000","last_unstale":"2024-02-22T19:31:03.492866+0000","last_undegraded":"2024-02-22T19:31:03.492866+0000","last_fullsized":"2024-02-22T19:31:03.492866+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149281+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:31:04.149281+0000","last_peered":"2024-02-22T19:31:04.149281+0000","last_clean":"2024-02-22T19:31:04.149281+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:31:04.149281+0000","last_undegraded":"2024-02-22T19:31:04.149281+0000","last_fullsized":"2024-02-22T19:31:04.149281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148855+0000","last_change":"2024-02-22T19:30:49.658305+0000","last_active":"2024-02-22T19:31:04.148855+0000","last_peered":"2024-02-22T19:31:04.148855+0000","last_clean":"2024-02-22T19:31:04.148855+0000","last_became_active":"2024-02-22T19:29:05.092427+0000","last_became_peered":"2024-02-22T19:29:05.092427+0000","last_unstale":"2024-02-22T19:31:04.148855+0000","last_undegraded":"2024-02-22T19:31:04.148855+0000","last_fullsized":"2024-02-22T19:31:04.148855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490952+0000","last_change":"2024-02-22T19:30:33.113603+0000","last_active":"2024-02-22T19:31:03.490952+0000","last_peered":"2024-02-22T19:31:03.490952+0000","last_clean":"2024-02-22T19:31:03.490952+0000","last_became_active":"2024-02-22T19:29:07.095700+0000","last_became_peered":"2024-02-22T19:29:07.095700+0000","last_unstale":"2024-02-22T19:31:03.490952+0000","last_undegraded":"2024-02-22T19:31:03.490952+0000","last_fullsized":"2024-02-22T19:31:03.490952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147963+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:31:04.147963+0000","last_peered":"2024-02-22T19:31:04.147963+0000","last_clean":"2024-02-22T19:31:04.147963+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:31:04.147963+0000","last_undegraded":"2024-02-22T19:31:04.147963+0000","last_fullsized":"2024-02-22T19:31:04.147963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":168,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:07.385287+0000","last_change":"2024-02-22T19:30:20.550910+0000","last_active":"2024-02-22T19:31:07.385287+0000","last_peered":"2024-02-22T19:31:07.385287+0000","last_clean":"2024-02-22T19:31:07.385287+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:31:07.385287+0000","last_undegraded":"2024-02-22T19:31:07.385287+0000","last_fullsized":"2024-02-22T19:31:07.385287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:20.550837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.550837+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148696+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:31:04.148696+0000","last_peered":"2024-02-22T19:31:04.148696+0000","last_clean":"2024-02-22T19:31:04.148696+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:31:04.148696+0000","last_undegraded":"2024-02-22T19:31:04.148696+0000","last_fullsized":"2024-02-22T19:31:04.148696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150803+0000","last_change":"2024-02-22T19:30:33.584923+0000","last_active":"2024-02-22T19:31:04.150803+0000","last_peered":"2024-02-22T19:31:04.150803+0000","last_clean":"2024-02-22T19:31:04.150803+0000","last_became_active":"2024-02-22T19:29:05.091758+0000","last_became_peered":"2024-02-22T19:29:05.091758+0000","last_unstale":"2024-02-22T19:31:04.150803+0000","last_undegraded":"2024-02-22T19:31:04.150803+0000","last_fullsized":"2024-02-22T19:31:04.150803+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490848+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:31:03.490848+0000","last_peered":"2024-02-22T19:31:03.490848+0000","last_clean":"2024-02-22T19:31:03.490848+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:31:03.490848+0000","last_undegraded":"2024-02-22T19:31:03.490848+0000","last_fullsized":"2024-02-22T19:31:03.490848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491189+0000","last_change":"2024-02-22T19:30:34.149749+0000","last_active":"2024-02-22T19:31:03.491189+0000","last_peered":"2024-02-22T19:31:03.491189+0000","last_clean":"2024-02-22T19:31:03.491189+0000","last_became_active":"2024-02-22T19:29:01.004572+0000","last_became_peered":"2024-02-22T19:29:01.004572+0000","last_unstale":"2024-02-22T19:31:03.491189+0000","last_undegraded":"2024-02-22T19:31:03.491189+0000","last_fullsized":"2024-02-22T19:31:03.491189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148353+0000","last_change":"2024-02-22T19:30:22.683587+0000","last_active":"2024-02-22T19:31:04.148353+0000","last_peered":"2024-02-22T19:31:04.148353+0000","last_clean":"2024-02-22T19:31:04.148353+0000","last_became_active":"2024-02-22T19:29:03.446818+0000","last_became_peered":"2024-02-22T19:29:03.446818+0000","last_unstale":"2024-02-22T19:31:04.148353+0000","last_undegraded":"2024-02-22T19:31:04.148353+0000","last_fullsized":"2024-02-22T19:31:04.148353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150088+0000","last_change":"2024-02-22T19:30:14.659000+0000","last_active":"2024-02-22T19:31:04.150088+0000","last_peered":"2024-02-22T19:31:04.150088+0000","last_clean":"2024-02-22T19:31:04.150088+0000","last_became_active":"2024-02-22T19:29:05.091867+0000","last_became_peered":"2024-02-22T19:29:05.091867+0000","last_unstale":"2024-02-22T19:31:04.150088+0000","last_undegraded":"2024-02-22T19:31:04.150088+0000","last_fullsized":"2024-02-22T19:31:04.150088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147697+0000","last_change":"2024-02-22T19:30:18.719372+0000","last_active":"2024-02-22T19:31:04.147697+0000","last_peered":"2024-02-22T19:31:04.147697+0000","last_clean":"2024-02-22T19:31:04.147697+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:31:04.147697+0000","last_undegraded":"2024-02-22T19:31:04.147697+0000","last_fullsized":"2024-02-22T19:31:04.147697+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.719310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149794+0000","last_change":"2024-02-22T19:30:29.616721+0000","last_active":"2024-02-22T19:31:04.149794+0000","last_peered":"2024-02-22T19:31:04.149794+0000","last_clean":"2024-02-22T19:31:04.149794+0000","last_became_active":"2024-02-22T19:29:01.012034+0000","last_became_peered":"2024-02-22T19:29:01.012034+0000","last_unstale":"2024-02-22T19:31:04.149794+0000","last_undegraded":"2024-02-22T19:31:04.149794+0000","last_fullsized":"2024-02-22T19:31:04.149794+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491135+0000","last_change":"2024-02-22T19:30:12.059865+0000","last_active":"2024-02-22T19:31:03.491135+0000","last_peered":"2024-02-22T19:31:03.491135+0000","last_clean":"2024-02-22T19:31:03.491135+0000","last_became_active":"2024-02-22T19:29:03.086071+0000","last_became_peered":"2024-02-22T19:29:03.086071+0000","last_unstale":"2024-02-22T19:31:03.491135+0000","last_undegraded":"2024-02-22T19:31:03.491135+0000","last_fullsized":"2024-02-22T19:31:03.491135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:12.059797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:12.059797+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.497407+0000","last_change":"2024-02-22T19:30:44.574471+0000","last_active":"2024-02-22T19:31:04.497407+0000","last_peered":"2024-02-22T19:31:04.497407+0000","last_clean":"2024-02-22T19:31:04.497407+0000","last_became_active":"2024-02-22T19:29:05.094532+0000","last_became_peered":"2024-02-22T19:29:05.094532+0000","last_unstale":"2024-02-22T19:31:04.497407+0000","last_undegraded":"2024-02-22T19:31:04.497407+0000","last_fullsized":"2024-02-22T19:31:04.497407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149234+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:31:04.149234+0000","last_peered":"2024-02-22T19:31:04.149234+0000","last_clean":"2024-02-22T19:31:04.149234+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:31:04.149234+0000","last_undegraded":"2024-02-22T19:31:04.149234+0000","last_fullsized":"2024-02-22T19:31:04.149234+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148772+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:31:04.148772+0000","last_peered":"2024-02-22T19:31:04.148772+0000","last_clean":"2024-02-22T19:31:04.148772+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:31:04.148772+0000","last_undegraded":"2024-02-22T19:31:04.148772+0000","last_fullsized":"2024-02-22T19:31:04.148772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491108+0000","last_change":"2024-02-22T19:30:54.022533+0000","last_active":"2024-02-22T19:31:03.491108+0000","last_peered":"2024-02-22T19:31:03.491108+0000","last_clean":"2024-02-22T19:31:03.491108+0000","last_became_active":"2024-02-22T19:29:03.084036+0000","last_became_peered":"2024-02-22T19:29:03.084036+0000","last_unstale":"2024-02-22T19:31:03.491108+0000","last_undegraded":"2024-02-22T19:31:03.491108+0000","last_fullsized":"2024-02-22T19:31:03.491108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.693788+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:31:04.693788+0000","last_peered":"2024-02-22T19:31:04.693788+0000","last_clean":"2024-02-22T19:31:04.693788+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:31:04.693788+0000","last_undegraded":"2024-02-22T19:31:04.693788+0000","last_fullsized":"2024-02-22T19:31:04.693788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491112+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:31:03.491112+0000","last_peered":"2024-02-22T19:31:03.491112+0000","last_clean":"2024-02-22T19:31:03.491112+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:31:03.491112+0000","last_undegraded":"2024-02-22T19:31:03.491112+0000","last_fullsized":"2024-02-22T19:31:03.491112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493481+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:31:03.493481+0000","last_peered":"2024-02-22T19:31:03.493481+0000","last_clean":"2024-02-22T19:31:03.493481+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:31:03.493481+0000","last_undegraded":"2024-02-22T19:31:03.493481+0000","last_fullsized":"2024-02-22T19:31:03.493481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492821+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:31:03.492821+0000","last_peered":"2024-02-22T19:31:03.492821+0000","last_clean":"2024-02-22T19:31:03.492821+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:31:03.492821+0000","last_undegraded":"2024-02-22T19:31:03.492821+0000","last_fullsized":"2024-02-22T19:31:03.492821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490954+0000","last_change":"2024-02-22T19:30:21.116431+0000","last_active":"2024-02-22T19:31:03.490954+0000","last_peered":"2024-02-22T19:31:03.490954+0000","last_clean":"2024-02-22T19:31:03.490954+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:31:03.490954+0000","last_undegraded":"2024-02-22T19:31:03.490954+0000","last_fullsized":"2024-02-22T19:31:03.490954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.116358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149857+0000","last_change":"2024-02-22T19:30:20.543466+0000","last_active":"2024-02-22T19:31:04.149857+0000","last_peered":"2024-02-22T19:31:04.149857+0000","last_clean":"2024-02-22T19:31:04.149857+0000","last_became_active":"2024-02-22T19:29:07.095452+0000","last_became_peered":"2024-02-22T19:29:07.095452+0000","last_unstale":"2024-02-22T19:31:04.149857+0000","last_undegraded":"2024-02-22T19:31:04.149857+0000","last_fullsized":"2024-02-22T19:31:04.149857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:20.543398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.543398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.497628+0000","last_change":"2024-02-22T19:30:35.999034+0000","last_active":"2024-02-22T19:31:04.497628+0000","last_peered":"2024-02-22T19:31:04.497628+0000","last_clean":"2024-02-22T19:31:04.497628+0000","last_became_active":"2024-02-22T19:29:05.095818+0000","last_became_peered":"2024-02-22T19:29:05.095818+0000","last_unstale":"2024-02-22T19:31:04.497628+0000","last_undegraded":"2024-02-22T19:31:04.497628+0000","last_fullsized":"2024-02-22T19:31:04.497628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492824+0000","last_change":"2024-02-22T19:30:06.720523+0000","last_active":"2024-02-22T19:31:03.492824+0000","last_peered":"2024-02-22T19:31:03.492824+0000","last_clean":"2024-02-22T19:31:03.492824+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:31:03.492824+0000","last_undegraded":"2024-02-22T19:31:03.492824+0000","last_fullsized":"2024-02-22T19:31:03.492824+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.720452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.720452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148703+0000","last_change":"2024-02-22T19:30:27.174847+0000","last_active":"2024-02-22T19:31:04.148703+0000","last_peered":"2024-02-22T19:31:04.148703+0000","last_clean":"2024-02-22T19:31:04.148703+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:31:04.148703+0000","last_undegraded":"2024-02-22T19:31:04.148703+0000","last_fullsized":"2024-02-22T19:31:04.148703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:27.174779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.174779+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:09.064602+0000","last_change":"2024-02-22T19:31:09.064602+0000","last_active":"2024-02-22T19:31:09.064602+0000","last_peered":"2024-02-22T19:31:09.064602+0000","last_clean":"2024-02-22T19:31:09.064602+0000","last_became_active":"2024-02-22T19:29:07.198153+0000","last_became_peered":"2024-02-22T19:29:07.198153+0000","last_unstale":"2024-02-22T19:31:09.064602+0000","last_undegraded":"2024-02-22T19:31:09.064602+0000","last_fullsized":"2024-02-22T19:31:09.064602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150707+0000","last_change":"2024-02-22T19:30:31.646674+0000","last_active":"2024-02-22T19:31:04.150707+0000","last_peered":"2024-02-22T19:31:04.150707+0000","last_clean":"2024-02-22T19:31:04.150707+0000","last_became_active":"2024-02-22T19:29:05.096026+0000","last_became_peered":"2024-02-22T19:29:05.096026+0000","last_unstale":"2024-02-22T19:31:04.150707+0000","last_undegraded":"2024-02-22T19:31:04.150707+0000","last_fullsized":"2024-02-22T19:31:04.150707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490967+0000","last_change":"2024-02-22T19:30:19.102902+0000","last_active":"2024-02-22T19:31:03.490967+0000","last_peered":"2024-02-22T19:31:03.490967+0000","last_clean":"2024-02-22T19:31:03.490967+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:31:03.490967+0000","last_undegraded":"2024-02-22T19:31:03.490967+0000","last_fullsized":"2024-02-22T19:31:03.490967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:19.102765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:19.102765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491954+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:31:03.491954+0000","last_peered":"2024-02-22T19:31:03.491954+0000","last_clean":"2024-02-22T19:31:03.491954+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:31:03.491954+0000","last_undegraded":"2024-02-22T19:31:03.491954+0000","last_fullsized":"2024-02-22T19:31:03.491954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148791+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:31:04.148791+0000","last_peered":"2024-02-22T19:31:04.148791+0000","last_clean":"2024-02-22T19:31:04.148791+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:31:04.148791+0000","last_undegraded":"2024-02-22T19:31:04.148791+0000","last_fullsized":"2024-02-22T19:31:04.148791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491416+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:31:03.491416+0000","last_peered":"2024-02-22T19:31:03.491416+0000","last_clean":"2024-02-22T19:31:03.491416+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:31:03.491416+0000","last_undegraded":"2024-02-22T19:31:03.491416+0000","last_fullsized":"2024-02-22T19:31:03.491416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149831+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:31:04.149831+0000","last_peered":"2024-02-22T19:31:04.149831+0000","last_clean":"2024-02-22T19:31:04.149831+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:31:04.149831+0000","last_undegraded":"2024-02-22T19:31:04.149831+0000","last_fullsized":"2024-02-22T19:31:04.149831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491950+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:31:03.491950+0000","last_peered":"2024-02-22T19:31:03.491950+0000","last_clean":"2024-02-22T19:31:03.491950+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:31:03.491950+0000","last_undegraded":"2024-02-22T19:31:03.491950+0000","last_fullsized":"2024-02-22T19:31:03.491950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492090+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:31:03.492090+0000","last_peered":"2024-02-22T19:31:03.492090+0000","last_clean":"2024-02-22T19:31:03.492090+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:31:03.492090+0000","last_undegraded":"2024-02-22T19:31:03.492090+0000","last_fullsized":"2024-02-22T19:31:03.492090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.694174+0000","last_change":"2024-02-22T19:30:23.100251+0000","last_active":"2024-02-22T19:31:04.694174+0000","last_peered":"2024-02-22T19:31:04.694174+0000","last_clean":"2024-02-22T19:31:04.694174+0000","last_became_active":"2024-02-22T19:29:05.092069+0000","last_became_peered":"2024-02-22T19:29:05.092069+0000","last_unstale":"2024-02-22T19:31:04.694174+0000","last_undegraded":"2024-02-22T19:31:04.694174+0000","last_fullsized":"2024-02-22T19:31:04.694174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:23.100177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.100177+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493114+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:31:03.493114+0000","last_peered":"2024-02-22T19:31:03.493114+0000","last_clean":"2024-02-22T19:31:03.493114+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:31:03.493114+0000","last_undegraded":"2024-02-22T19:31:03.493114+0000","last_fullsized":"2024-02-22T19:31:03.493114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491607+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:31:03.491607+0000","last_peered":"2024-02-22T19:31:03.491607+0000","last_clean":"2024-02-22T19:31:03.491607+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:31:03.491607+0000","last_undegraded":"2024-02-22T19:31:03.491607+0000","last_fullsized":"2024-02-22T19:31:03.491607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":415,"ondisk_log_size":415,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266717,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":308559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65100000000000002,"5min":0.871,"15min":0.871},"last":0.439}]},{"osd":1,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63100000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68700000000000006,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.55200000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58199999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68500000000000005,"5min":1.98,"15min":1.98},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64300000000000002,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71099999999999997,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.753,"15min":0.753},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.623},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72899999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.56000000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528354,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297720,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":638976,"data_stored":307174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":1.024,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73599999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.62},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.309,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.499},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.36799999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.55500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297772,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447380,"statfs":{"total":95995035648,"available":95690117120,"internally_reserved":0,"allocated":692224,"data_stored":309318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:40 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36399999999999999,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52900000000000003,"5min":0.62,"15min":0.62},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.378,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.41199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38}]},{"osd":2,"last update":"Thu Feb 22 19:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.254,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:27 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.43099999999999999,"15min":0.45500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.441,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.48999999999999999,"15min":0.50900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54300000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54300000000000004,"15min":0.56599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Feb 22 19:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78700000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.505},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297752,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447400,"statfs":{"total":95995035648,"available":95690137600,"internally_reserved":0,"allocated":671744,"data_stored":309129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.52400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.27600000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.46899999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.377,"15min":0.374},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.53900000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.40799999999999997,"15min":0.42599999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.443,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.307},{"interface":"front","average":{"1min":0.378,"5min":0.42699999999999999,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46899999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70899999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.995,"15min":0.995},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67600000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":308185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.373,"15min":0.37},"min":{"1min":0.23000000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.53000000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38,"15min":0.378},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55000000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.442},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.50800000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.5}]},{"osd":5,"last update":"Thu Feb 22 19:31:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.436,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.875,"5min":0.995,"15min":0.995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90200000000000002,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.624}]},{"osd":6,"last update":"Thu Feb 22 19:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.60999999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149743,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297764,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447388,"statfs":{"total":95995035648,"available":95690125312,"internally_reserved":0,"allocated":684032,"data_stored":309226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41599999999999998,"15min":0.432},"min":{"1min":0.20799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.193},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.45000000000000001,"15min":0.46400000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.317},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.47599999999999998,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.46999999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.31}]},{"osd":4,"last update":"Thu Feb 22 19:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78300000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41499999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953517,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447392,"statfs":{"total":95995035648,"available":95690129408,"internally_reserved":0,"allocated":679936,"data_stored":309265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.626}]},{"osd":1,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":1,"15min":1},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.34200000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.378},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.40999999999999998},"min":{"1min":0.36499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.5}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:31:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60899999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724583,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":307542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.5229999999999997,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.35599999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78000000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.59499999999999997,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.5529999999999999,"5min":5.5529999999999999,"15min":5.5529999999999999},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.219,"15min":0.219},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.26600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.42499999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.46500000000000002},"min":{"1min":0.35299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.878},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":1.071,"15min":1.071},"last":0.73899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:31:12.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:31:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:11.205927+0000 mgr.y (mgr.14535) 90 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:11.412727+0000 osd.0 (osd.0) 49 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:31:12.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:11.415713+0000 osd.0 (osd.0) 50 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:31:12.882 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:11.903641+0000 osd.7 (osd.7) 41 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:11.906557+0000 osd.7 (osd.7) 42 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:11.913912+0000 mgr.y (mgr.14535) 91 : audit [DBG] from='client.24665 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:12.444382+0000 osd.0 (osd.0) 51 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:12.445201+0000 osd.0 (osd.0) 52 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:11.903641+0000 osd.7 (osd.7) 41 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:11.906557+0000 osd.7 (osd.7) 42 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:11.913912+0000 mgr.y (mgr.14535) 91 : audit [DBG] from='client.24665 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:12.444382+0000 osd.0 (osd.0) 51 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:31:13.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:12.445201+0000 osd.0 (osd.0) 52 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:31:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:11.903641+0000 osd.7 (osd.7) 41 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:31:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:11.906557+0000 osd.7 (osd.7) 42 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:31:13.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:11.913912+0000 mgr.y (mgr.14535) 91 : audit [DBG] from='client.24665 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:13.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:12.444382+0000 osd.0 (osd.0) 51 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:31:13.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:12.445201+0000 osd.0 (osd.0) 52 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:31:14.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:13.206306+0000 mgr.y (mgr.14535) 92 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:31:14.356 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:13.206306+0000 mgr.y (mgr.14535) 92 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:31:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:13.206306+0000 mgr.y (mgr.14535) 92 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:31:14.603 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:31:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:31:14.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:31:14.802 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:31:15.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:31:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:31:15.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:14.070795+0000 osd.3 (osd.3) 55 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:31:15.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:14.073903+0000 osd.3 (osd.3) 56 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:31:15.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:14.292952+0000 mgr.y (mgr.14535) 93 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:15.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:14.070795+0000 osd.3 (osd.3) 55 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:31:15.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:14.073903+0000 osd.3 (osd.3) 56 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:31:15.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:14.292952+0000 mgr.y (mgr.14535) 93 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:15.528 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-02-22T19:31:13.206123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":841,"num_read_kb":593,"num_write":428,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":493,"ondisk_log_size":493,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381988,"kb_used_data":5220,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579228,"statfs":{"total":767960285184,"available":765521129472,"internally_reserved":0,"allocated":5345280,"data_stored":2468398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.716721"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490994+0000","last_change":"2024-02-22T19:30:28.096813+0000","last_active":"2024-02-22T19:31:03.490994+0000","last_peered":"2024-02-22T19:31:03.490994+0000","last_clean":"2024-02-22T19:31:03.490994+0000","last_became_active":"2024-02-22T19:29:07.099281+0000","last_became_peered":"2024-02-22T19:29:07.099281+0000","last_unstale":"2024-02-22T19:31:03.490994+0000","last_undegraded":"2024-02-22T19:31:03.490994+0000","last_fullsized":"2024-02-22T19:31:03.490994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490939+0000","last_change":"2024-02-22T19:30:24.056145+0000","last_active":"2024-02-22T19:31:03.490939+0000","last_peered":"2024-02-22T19:31:03.490939+0000","last_clean":"2024-02-22T19:31:03.490939+0000","last_became_active":"2024-02-22T19:29:05.091467+0000","last_became_peered":"2024-02-22T19:29:05.091467+0000","last_unstale":"2024-02-22T19:31:03.490939+0000","last_undegraded":"2024-02-22T19:31:03.490939+0000","last_fullsized":"2024-02-22T19:31:03.490939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148520+0000","last_change":"2024-02-22T19:30:07.664488+0000","last_active":"2024-02-22T19:31:04.148520+0000","last_peered":"2024-02-22T19:31:04.148520+0000","last_clean":"2024-02-22T19:31:04.148520+0000","last_became_active":"2024-02-22T19:29:01.002834+0000","last_became_peered":"2024-02-22T19:29:01.002834+0000","last_unstale":"2024-02-22T19:31:04.148520+0000","last_undegraded":"2024-02-22T19:31:04.148520+0000","last_fullsized":"2024-02-22T19:31:04.148520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.664424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490859+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:31:03.490859+0000","last_peered":"2024-02-22T19:31:03.490859+0000","last_clean":"2024-02-22T19:31:03.490859+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:31:03.490859+0000","last_undegraded":"2024-02-22T19:31:03.490859+0000","last_fullsized":"2024-02-22T19:31:03.490859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493005+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:31:03.493005+0000","last_peered":"2024-02-22T19:31:03.493005+0000","last_clean":"2024-02-22T19:31:03.493005+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:31:03.493005+0000","last_undegraded":"2024-02-22T19:31:03.493005+0000","last_fullsized":"2024-02-22T19:31:03.493005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490843+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:31:03.490843+0000","last_peered":"2024-02-22T19:31:03.490843+0000","last_clean":"2024-02-22T19:31:03.490843+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:31:03.490843+0000","last_undegraded":"2024-02-22T19:31:03.490843+0000","last_fullsized":"2024-02-22T19:31:03.490843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493460+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:31:03.493460+0000","last_peered":"2024-02-22T19:31:03.493460+0000","last_clean":"2024-02-22T19:31:03.493460+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:31:03.493460+0000","last_undegraded":"2024-02-22T19:31:03.493460+0000","last_fullsized":"2024-02-22T19:31:03.493460+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492869+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:31:03.492869+0000","last_peered":"2024-02-22T19:31:03.492869+0000","last_clean":"2024-02-22T19:31:03.492869+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:31:03.492869+0000","last_undegraded":"2024-02-22T19:31:03.492869+0000","last_fullsized":"2024-02-22T19:31:03.492869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148018+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:31:04.148018+0000","last_peered":"2024-02-22T19:31:04.148018+0000","last_clean":"2024-02-22T19:31:04.148018+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:31:04.148018+0000","last_undegraded":"2024-02-22T19:31:04.148018+0000","last_fullsized":"2024-02-22T19:31:04.148018+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149656+0000","last_change":"2024-02-22T19:30:26.528062+0000","last_active":"2024-02-22T19:31:04.149656+0000","last_peered":"2024-02-22T19:31:04.149656+0000","last_clean":"2024-02-22T19:31:04.149656+0000","last_became_active":"2024-02-22T19:29:05.091567+0000","last_became_peered":"2024-02-22T19:29:05.091567+0000","last_unstale":"2024-02-22T19:31:04.149656+0000","last_undegraded":"2024-02-22T19:31:04.149656+0000","last_fullsized":"2024-02-22T19:31:04.149656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.527965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.527965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149550+0000","last_change":"2024-02-22T19:30:10.440133+0000","last_active":"2024-02-22T19:31:04.149550+0000","last_peered":"2024-02-22T19:31:04.149550+0000","last_clean":"2024-02-22T19:31:04.149550+0000","last_became_active":"2024-02-22T19:29:01.005184+0000","last_became_peered":"2024-02-22T19:29:01.005184+0000","last_unstale":"2024-02-22T19:31:04.149550+0000","last_undegraded":"2024-02-22T19:31:04.149550+0000","last_fullsized":"2024-02-22T19:31:04.149550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.440058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.440058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491173+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:31:03.491173+0000","last_peered":"2024-02-22T19:31:03.491173+0000","last_clean":"2024-02-22T19:31:03.491173+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:31:03.491173+0000","last_undegraded":"2024-02-22T19:31:03.491173+0000","last_fullsized":"2024-02-22T19:31:03.491173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148637+0000","last_change":"2024-02-22T19:30:18.496805+0000","last_active":"2024-02-22T19:31:04.148637+0000","last_peered":"2024-02-22T19:31:04.148637+0000","last_clean":"2024-02-22T19:31:04.148637+0000","last_became_active":"2024-02-22T19:29:07.100382+0000","last_became_peered":"2024-02-22T19:29:07.100382+0000","last_unstale":"2024-02-22T19:31:04.148637+0000","last_undegraded":"2024-02-22T19:31:04.148637+0000","last_fullsized":"2024-02-22T19:31:04.148637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149144+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:31:04.149144+0000","last_peered":"2024-02-22T19:31:04.149144+0000","last_clean":"2024-02-22T19:31:04.149144+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:31:04.149144+0000","last_undegraded":"2024-02-22T19:31:04.149144+0000","last_fullsized":"2024-02-22T19:31:04.149144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149644+0000","last_change":"2024-02-22T19:30:22.574826+0000","last_active":"2024-02-22T19:31:04.149644+0000","last_peered":"2024-02-22T19:31:04.149644+0000","last_clean":"2024-02-22T19:31:04.149644+0000","last_became_active":"2024-02-22T19:29:01.007689+0000","last_became_peered":"2024-02-22T19:29:01.007689+0000","last_unstale":"2024-02-22T19:31:04.149644+0000","last_undegraded":"2024-02-22T19:31:04.149644+0000","last_fullsized":"2024-02-22T19:31:04.149644+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:22.574750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.574750+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491451+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:31:03.491451+0000","last_peered":"2024-02-22T19:31:03.491451+0000","last_clean":"2024-02-22T19:31:03.491451+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:31:03.491451+0000","last_undegraded":"2024-02-22T19:31:03.491451+0000","last_fullsized":"2024-02-22T19:31:03.491451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492016+0000","last_change":"2024-02-22T19:30:34.059638+0000","last_active":"2024-02-22T19:31:03.492016+0000","last_peered":"2024-02-22T19:31:03.492016+0000","last_clean":"2024-02-22T19:31:03.492016+0000","last_became_active":"2024-02-22T19:29:07.197580+0000","last_became_peered":"2024-02-22T19:29:07.197580+0000","last_unstale":"2024-02-22T19:31:03.492016+0000","last_undegraded":"2024-02-22T19:31:03.492016+0000","last_fullsized":"2024-02-22T19:31:03.492016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491449+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:31:03.491449+0000","last_peered":"2024-02-22T19:31:03.491449+0000","last_clean":"2024-02-22T19:31:03.491449+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:31:03.491449+0000","last_undegraded":"2024-02-22T19:31:03.491449+0000","last_fullsized":"2024-02-22T19:31:03.491449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148433+0000","last_change":"2024-02-22T19:30:00.634139+0000","last_active":"2024-02-22T19:31:04.148433+0000","last_peered":"2024-02-22T19:31:04.148433+0000","last_clean":"2024-02-22T19:31:04.148433+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:31:04.148433+0000","last_undegraded":"2024-02-22T19:31:04.148433+0000","last_fullsized":"2024-02-22T19:31:04.148433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:30:00.634060+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147946+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:31:04.147946+0000","last_peered":"2024-02-22T19:31:04.147946+0000","last_clean":"2024-02-22T19:31:04.147946+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:31:04.147946+0000","last_undegraded":"2024-02-22T19:31:04.147946+0000","last_fullsized":"2024-02-22T19:31:04.147946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493627+0000","last_change":"2024-02-22T19:30:26.823434+0000","last_active":"2024-02-22T19:31:03.493627+0000","last_peered":"2024-02-22T19:31:03.493627+0000","last_clean":"2024-02-22T19:31:03.493627+0000","last_became_active":"2024-02-22T19:29:07.197840+0000","last_became_peered":"2024-02-22T19:29:07.197840+0000","last_unstale":"2024-02-22T19:31:03.493627+0000","last_undegraded":"2024-02-22T19:31:03.493627+0000","last_fullsized":"2024-02-22T19:31:03.493627+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.823363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.823363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491395+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:31:03.491395+0000","last_peered":"2024-02-22T19:31:03.491395+0000","last_clean":"2024-02-22T19:31:03.491395+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:31:03.491395+0000","last_undegraded":"2024-02-22T19:31:03.491395+0000","last_fullsized":"2024-02-22T19:31:03.491395+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491246+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:31:03.491246+0000","last_peered":"2024-02-22T19:31:03.491246+0000","last_clean":"2024-02-22T19:31:03.491246+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:31:03.491246+0000","last_undegraded":"2024-02-22T19:31:03.491246+0000","last_fullsized":"2024-02-22T19:31:03.491246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.012802+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:31:04.012802+0000","last_peered":"2024-02-22T19:31:04.012802+0000","last_clean":"2024-02-22T19:31:04.012802+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:31:04.012802+0000","last_undegraded":"2024-02-22T19:31:04.012802+0000","last_fullsized":"2024-02-22T19:31:04.012802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148685+0000","last_change":"2024-02-22T19:30:37.617824+0000","last_active":"2024-02-22T19:31:04.148685+0000","last_peered":"2024-02-22T19:31:04.148685+0000","last_clean":"2024-02-22T19:31:04.148685+0000","last_became_active":"2024-02-22T19:29:07.104332+0000","last_became_peered":"2024-02-22T19:29:07.104332+0000","last_unstale":"2024-02-22T19:31:04.148685+0000","last_undegraded":"2024-02-22T19:31:04.148685+0000","last_fullsized":"2024-02-22T19:31:04.148685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.695544+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:31:04.695544+0000","last_peered":"2024-02-22T19:31:04.695544+0000","last_clean":"2024-02-22T19:31:04.695544+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:31:04.695544+0000","last_undegraded":"2024-02-22T19:31:04.695544+0000","last_fullsized":"2024-02-22T19:31:04.695544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491021+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:31:03.491021+0000","last_peered":"2024-02-22T19:31:03.491021+0000","last_clean":"2024-02-22T19:31:03.491021+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:31:03.491021+0000","last_undegraded":"2024-02-22T19:31:03.491021+0000","last_fullsized":"2024-02-22T19:31:03.491021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148853+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:31:04.148853+0000","last_peered":"2024-02-22T19:31:04.148853+0000","last_clean":"2024-02-22T19:31:04.148853+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:31:04.148853+0000","last_undegraded":"2024-02-22T19:31:04.148853+0000","last_fullsized":"2024-02-22T19:31:04.148853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491531+0000","last_change":"2024-02-22T19:30:31.070100+0000","last_active":"2024-02-22T19:31:03.491531+0000","last_peered":"2024-02-22T19:31:03.491531+0000","last_clean":"2024-02-22T19:31:03.491531+0000","last_became_active":"2024-02-22T19:29:07.089971+0000","last_became_peered":"2024-02-22T19:29:07.089971+0000","last_unstale":"2024-02-22T19:31:03.491531+0000","last_undegraded":"2024-02-22T19:31:03.491531+0000","last_fullsized":"2024-02-22T19:31:03.491531+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149089+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:31:04.149089+0000","last_peered":"2024-02-22T19:31:04.149089+0000","last_clean":"2024-02-22T19:31:04.149089+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:31:04.149089+0000","last_undegraded":"2024-02-22T19:31:04.149089+0000","last_fullsized":"2024-02-22T19:31:04.149089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492689+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:31:03.492689+0000","last_peered":"2024-02-22T19:31:03.492689+0000","last_clean":"2024-02-22T19:31:03.492689+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:31:03.492689+0000","last_undegraded":"2024-02-22T19:31:03.492689+0000","last_fullsized":"2024-02-22T19:31:03.492689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150273+0000","last_change":"2024-02-22T19:30:06.607076+0000","last_active":"2024-02-22T19:31:04.150273+0000","last_peered":"2024-02-22T19:31:04.150273+0000","last_clean":"2024-02-22T19:31:04.150273+0000","last_became_active":"2024-02-22T19:29:03.087740+0000","last_became_peered":"2024-02-22T19:29:03.087740+0000","last_unstale":"2024-02-22T19:31:04.150273+0000","last_undegraded":"2024-02-22T19:31:04.150273+0000","last_fullsized":"2024-02-22T19:31:04.150273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.607011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.607011+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149046+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:31:04.149046+0000","last_peered":"2024-02-22T19:31:04.149046+0000","last_clean":"2024-02-22T19:31:04.149046+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:31:04.149046+0000","last_undegraded":"2024-02-22T19:31:04.149046+0000","last_fullsized":"2024-02-22T19:31:04.149046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147968+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:31:04.147968+0000","last_peered":"2024-02-22T19:31:04.147968+0000","last_clean":"2024-02-22T19:31:04.147968+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:31:04.147968+0000","last_undegraded":"2024-02-22T19:31:04.147968+0000","last_fullsized":"2024-02-22T19:31:04.147968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148519+0000","last_change":"2024-02-22T19:30:08.671313+0000","last_active":"2024-02-22T19:31:04.148519+0000","last_peered":"2024-02-22T19:31:04.148519+0000","last_clean":"2024-02-22T19:31:04.148519+0000","last_became_active":"2024-02-22T19:29:01.009788+0000","last_became_peered":"2024-02-22T19:29:01.009788+0000","last_unstale":"2024-02-22T19:31:04.148519+0000","last_undegraded":"2024-02-22T19:31:04.148519+0000","last_fullsized":"2024-02-22T19:31:04.148519+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:08.671238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:08.671238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492696+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:31:03.492696+0000","last_peered":"2024-02-22T19:31:03.492696+0000","last_clean":"2024-02-22T19:31:03.492696+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:31:03.492696+0000","last_undegraded":"2024-02-22T19:31:03.492696+0000","last_fullsized":"2024-02-22T19:31:03.492696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492917+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:31:03.492917+0000","last_peered":"2024-02-22T19:31:03.492917+0000","last_clean":"2024-02-22T19:31:03.492917+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:31:03.492917+0000","last_undegraded":"2024-02-22T19:31:03.492917+0000","last_fullsized":"2024-02-22T19:31:03.492917+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150440+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:31:04.150440+0000","last_peered":"2024-02-22T19:31:04.150440+0000","last_clean":"2024-02-22T19:31:04.150440+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:31:04.150440+0000","last_undegraded":"2024-02-22T19:31:04.150440+0000","last_fullsized":"2024-02-22T19:31:04.150440+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148716+0000","last_change":"2024-02-22T19:30:07.463186+0000","last_active":"2024-02-22T19:31:04.148716+0000","last_peered":"2024-02-22T19:31:04.148716+0000","last_clean":"2024-02-22T19:31:04.148716+0000","last_became_active":"2024-02-22T19:29:01.006896+0000","last_became_peered":"2024-02-22T19:29:01.006896+0000","last_unstale":"2024-02-22T19:31:04.148716+0000","last_undegraded":"2024-02-22T19:31:04.148716+0000","last_fullsized":"2024-02-22T19:31:04.148716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.463122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.463122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492820+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:31:03.492820+0000","last_peered":"2024-02-22T19:31:03.492820+0000","last_clean":"2024-02-22T19:31:03.492820+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:31:03.492820+0000","last_undegraded":"2024-02-22T19:31:03.492820+0000","last_fullsized":"2024-02-22T19:31:03.492820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:07.487841+0000","last_change":"2024-02-22T19:31:07.487841+0000","last_active":"2024-02-22T19:31:07.487841+0000","last_peered":"2024-02-22T19:31:07.487841+0000","last_clean":"2024-02-22T19:31:07.487841+0000","last_became_active":"2024-02-22T19:29:07.101369+0000","last_became_peered":"2024-02-22T19:29:07.101369+0000","last_unstale":"2024-02-22T19:31:07.487841+0000","last_undegraded":"2024-02-22T19:31:07.487841+0000","last_fullsized":"2024-02-22T19:31:07.487841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493046+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:31:03.493046+0000","last_peered":"2024-02-22T19:31:03.493046+0000","last_clean":"2024-02-22T19:31:03.493046+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:31:03.493046+0000","last_undegraded":"2024-02-22T19:31:03.493046+0000","last_fullsized":"2024-02-22T19:31:03.493046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149806+0000","last_change":"2024-02-22T19:30:03.688086+0000","last_active":"2024-02-22T19:31:04.149806+0000","last_peered":"2024-02-22T19:31:04.149806+0000","last_clean":"2024-02-22T19:31:04.149806+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:31:04.149806+0000","last_undegraded":"2024-02-22T19:31:04.149806+0000","last_fullsized":"2024-02-22T19:31:04.149806+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.688024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.688024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491215+0000","last_change":"2024-02-22T19:30:14.042137+0000","last_active":"2024-02-22T19:31:03.491215+0000","last_peered":"2024-02-22T19:31:03.491215+0000","last_clean":"2024-02-22T19:31:03.491215+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:31:03.491215+0000","last_undegraded":"2024-02-22T19:31:03.491215+0000","last_fullsized":"2024-02-22T19:31:03.491215+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.042059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.042059+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492607+0000","last_change":"2024-02-22T19:30:13.709779+0000","last_active":"2024-02-22T19:31:03.492607+0000","last_peered":"2024-02-22T19:31:03.492607+0000","last_clean":"2024-02-22T19:31:03.492607+0000","last_became_active":"2024-02-22T19:29:07.102604+0000","last_became_peered":"2024-02-22T19:29:07.102604+0000","last_unstale":"2024-02-22T19:31:03.492607+0000","last_undegraded":"2024-02-22T19:31:03.492607+0000","last_fullsized":"2024-02-22T19:31:03.492607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.709691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.709691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491046+0000","last_change":"2024-02-22T19:30:45.021593+0000","last_active":"2024-02-22T19:31:03.491046+0000","last_peered":"2024-02-22T19:31:03.491046+0000","last_clean":"2024-02-22T19:31:03.491046+0000","last_became_active":"2024-02-22T19:29:05.091550+0000","last_became_peered":"2024-02-22T19:29:05.091550+0000","last_unstale":"2024-02-22T19:31:03.491046+0000","last_undegraded":"2024-02-22T19:31:03.491046+0000","last_fullsized":"2024-02-22T19:31:03.491046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491722+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:31:03.491722+0000","last_peered":"2024-02-22T19:31:03.491722+0000","last_clean":"2024-02-22T19:31:03.491722+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:31:03.491722+0000","last_undegraded":"2024-02-22T19:31:03.491722+0000","last_fullsized":"2024-02-22T19:31:03.491722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491427+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:31:03.491427+0000","last_peered":"2024-02-22T19:31:03.491427+0000","last_clean":"2024-02-22T19:31:03.491427+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:31:03.491427+0000","last_undegraded":"2024-02-22T19:31:03.491427+0000","last_fullsized":"2024-02-22T19:31:03.491427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148616+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:31:04.148616+0000","last_peered":"2024-02-22T19:31:04.148616+0000","last_clean":"2024-02-22T19:31:04.148616+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:31:04.148616+0000","last_undegraded":"2024-02-22T19:31:04.148616+0000","last_fullsized":"2024-02-22T19:31:04.148616+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.694721+0000","last_change":"2024-02-22T19:30:21.533563+0000","last_active":"2024-02-22T19:31:04.694721+0000","last_peered":"2024-02-22T19:31:04.694721+0000","last_clean":"2024-02-22T19:31:04.694721+0000","last_became_active":"2024-02-22T19:29:05.092444+0000","last_became_peered":"2024-02-22T19:29:05.092444+0000","last_unstale":"2024-02-22T19:31:04.694721+0000","last_undegraded":"2024-02-22T19:31:04.694721+0000","last_fullsized":"2024-02-22T19:31:04.694721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:21.533452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.533452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148953+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:31:04.148953+0000","last_peered":"2024-02-22T19:31:04.148953+0000","last_clean":"2024-02-22T19:31:04.148953+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:31:04.148953+0000","last_undegraded":"2024-02-22T19:31:04.148953+0000","last_fullsized":"2024-02-22T19:31:04.148953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493557+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:31:03.493557+0000","last_peered":"2024-02-22T19:31:03.493557+0000","last_clean":"2024-02-22T19:31:03.493557+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:31:03.493557+0000","last_undegraded":"2024-02-22T19:31:03.493557+0000","last_fullsized":"2024-02-22T19:31:03.493557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492256+0000","last_change":"2024-02-22T19:30:16.057704+0000","last_active":"2024-02-22T19:31:03.492256+0000","last_peered":"2024-02-22T19:31:03.492256+0000","last_clean":"2024-02-22T19:31:03.492256+0000","last_became_active":"2024-02-22T19:29:07.097503+0000","last_became_peered":"2024-02-22T19:29:07.097503+0000","last_unstale":"2024-02-22T19:31:03.492256+0000","last_undegraded":"2024-02-22T19:31:03.492256+0000","last_fullsized":"2024-02-22T19:31:03.492256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:08.045723+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:31:08.045723+0000","last_peered":"2024-02-22T19:31:08.045723+0000","last_clean":"2024-02-22T19:31:08.045723+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:31:08.045723+0000","last_undegraded":"2024-02-22T19:31:08.045723+0000","last_fullsized":"2024-02-22T19:31:08.045723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148135+0000","last_change":"2024-02-22T19:30:03.653428+0000","last_active":"2024-02-22T19:31:04.148135+0000","last_peered":"2024-02-22T19:31:04.148135+0000","last_clean":"2024-02-22T19:31:04.148135+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:31:04.148135+0000","last_undegraded":"2024-02-22T19:31:04.148135+0000","last_fullsized":"2024-02-22T19:31:04.148135+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:03.653348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:03.653348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'9","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149987+0000","last_change":"2024-02-22T19:30:05.473412+0000","last_active":"2024-02-22T19:31:04.149987+0000","last_peered":"2024-02-22T19:31:04.149987+0000","last_clean":"2024-02-22T19:31:04.149987+0000","last_became_active":"2024-02-22T19:29:03.448670+0000","last_became_peered":"2024-02-22T19:29:03.448670+0000","last_unstale":"2024-02-22T19:31:04.149987+0000","last_undegraded":"2024-02-22T19:31:04.149987+0000","last_fullsized":"2024-02-22T19:31:04.149987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:05.473341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.473341+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148211+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:31:04.148211+0000","last_peered":"2024-02-22T19:31:04.148211+0000","last_clean":"2024-02-22T19:31:04.148211+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:31:04.148211+0000","last_undegraded":"2024-02-22T19:31:04.148211+0000","last_fullsized":"2024-02-22T19:31:04.148211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493566+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:31:03.493566+0000","last_peered":"2024-02-22T19:31:03.493566+0000","last_clean":"2024-02-22T19:31:03.493566+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:31:03.493566+0000","last_undegraded":"2024-02-22T19:31:03.493566+0000","last_fullsized":"2024-02-22T19:31:03.493566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149725+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:31:04.149725+0000","last_peered":"2024-02-22T19:31:04.149725+0000","last_clean":"2024-02-22T19:31:04.149725+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:31:04.149725+0000","last_undegraded":"2024-02-22T19:31:04.149725+0000","last_fullsized":"2024-02-22T19:31:04.149725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148938+0000","last_change":"2024-02-22T19:30:13.766624+0000","last_active":"2024-02-22T19:31:04.148938+0000","last_peered":"2024-02-22T19:31:04.148938+0000","last_clean":"2024-02-22T19:31:04.148938+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:31:04.148938+0000","last_undegraded":"2024-02-22T19:31:04.148938+0000","last_fullsized":"2024-02-22T19:31:04.148938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:13.766546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.766546+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148605+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:31:04.148605+0000","last_peered":"2024-02-22T19:31:04.148605+0000","last_clean":"2024-02-22T19:31:04.148605+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:31:04.148605+0000","last_undegraded":"2024-02-22T19:31:04.148605+0000","last_fullsized":"2024-02-22T19:31:04.148605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148809+0000","last_change":"2024-02-22T19:30:27.575360+0000","last_active":"2024-02-22T19:31:04.148809+0000","last_peered":"2024-02-22T19:31:04.148809+0000","last_clean":"2024-02-22T19:31:04.148809+0000","last_became_active":"2024-02-22T19:29:05.092190+0000","last_became_peered":"2024-02-22T19:29:05.092190+0000","last_unstale":"2024-02-22T19:31:04.148809+0000","last_undegraded":"2024-02-22T19:31:04.148809+0000","last_fullsized":"2024-02-22T19:31:04.148809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147912+0000","last_change":"2024-02-22T19:30:29.656325+0000","last_active":"2024-02-22T19:31:04.147912+0000","last_peered":"2024-02-22T19:31:04.147912+0000","last_clean":"2024-02-22T19:31:04.147912+0000","last_became_active":"2024-02-22T19:29:01.007053+0000","last_became_peered":"2024-02-22T19:29:01.007053+0000","last_unstale":"2024-02-22T19:31:04.147912+0000","last_undegraded":"2024-02-22T19:31:04.147912+0000","last_fullsized":"2024-02-22T19:31:04.147912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493655+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:31:03.493655+0000","last_peered":"2024-02-22T19:31:03.493655+0000","last_clean":"2024-02-22T19:31:03.493655+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:31:03.493655+0000","last_undegraded":"2024-02-22T19:31:03.493655+0000","last_fullsized":"2024-02-22T19:31:03.493655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149312+0000","last_change":"2024-02-22T19:30:14.461934+0000","last_active":"2024-02-22T19:31:04.149312+0000","last_peered":"2024-02-22T19:31:04.149312+0000","last_clean":"2024-02-22T19:31:04.149312+0000","last_became_active":"2024-02-22T19:29:07.197976+0000","last_became_peered":"2024-02-22T19:29:07.197976+0000","last_unstale":"2024-02-22T19:31:04.149312+0000","last_undegraded":"2024-02-22T19:31:04.149312+0000","last_fullsized":"2024-02-22T19:31:04.149312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150723+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:31:04.150723+0000","last_peered":"2024-02-22T19:31:04.150723+0000","last_clean":"2024-02-22T19:31:04.150723+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:31:04.150723+0000","last_undegraded":"2024-02-22T19:31:04.150723+0000","last_fullsized":"2024-02-22T19:31:04.150723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148276+0000","last_change":"2024-02-22T19:30:28.601662+0000","last_active":"2024-02-22T19:31:04.148276+0000","last_peered":"2024-02-22T19:31:04.148276+0000","last_clean":"2024-02-22T19:31:04.148276+0000","last_became_active":"2024-02-22T19:29:03.447032+0000","last_became_peered":"2024-02-22T19:29:03.447032+0000","last_unstale":"2024-02-22T19:31:04.148276+0000","last_undegraded":"2024-02-22T19:31:04.148276+0000","last_fullsized":"2024-02-22T19:31:04.148276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.695763+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:31:04.695763+0000","last_peered":"2024-02-22T19:31:04.695763+0000","last_clean":"2024-02-22T19:31:04.695763+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:31:04.695763+0000","last_undegraded":"2024-02-22T19:31:04.695763+0000","last_fullsized":"2024-02-22T19:31:04.695763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:06.066327+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:31:06.066327+0000","last_peered":"2024-02-22T19:31:06.066327+0000","last_clean":"2024-02-22T19:31:06.066327+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:31:06.066327+0000","last_undegraded":"2024-02-22T19:31:06.066327+0000","last_fullsized":"2024-02-22T19:31:06.066327+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150366+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:31:04.150366+0000","last_peered":"2024-02-22T19:31:04.150366+0000","last_clean":"2024-02-22T19:31:04.150366+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:31:04.150366+0000","last_undegraded":"2024-02-22T19:31:04.150366+0000","last_fullsized":"2024-02-22T19:31:04.150366+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491266+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:31:03.491266+0000","last_peered":"2024-02-22T19:31:03.491266+0000","last_clean":"2024-02-22T19:31:03.491266+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:31:03.491266+0000","last_undegraded":"2024-02-22T19:31:03.491266+0000","last_fullsized":"2024-02-22T19:31:03.491266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147777+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:31:04.147777+0000","last_peered":"2024-02-22T19:31:04.147777+0000","last_clean":"2024-02-22T19:31:04.147777+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:31:04.147777+0000","last_undegraded":"2024-02-22T19:31:04.147777+0000","last_fullsized":"2024-02-22T19:31:04.147777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148190+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:31:04.148190+0000","last_peered":"2024-02-22T19:31:04.148190+0000","last_clean":"2024-02-22T19:31:04.148190+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:31:04.148190+0000","last_undegraded":"2024-02-22T19:31:04.148190+0000","last_fullsized":"2024-02-22T19:31:04.148190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148379+0000","last_change":"2024-02-22T19:30:29.110575+0000","last_active":"2024-02-22T19:31:04.148379+0000","last_peered":"2024-02-22T19:31:04.148379+0000","last_clean":"2024-02-22T19:31:04.148379+0000","last_became_active":"2024-02-22T19:29:05.091952+0000","last_became_peered":"2024-02-22T19:29:05.091952+0000","last_unstale":"2024-02-22T19:31:04.148379+0000","last_undegraded":"2024-02-22T19:31:04.148379+0000","last_fullsized":"2024-02-22T19:31:04.148379+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.110513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.110513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150818+0000","last_change":"2024-02-22T19:30:10.619750+0000","last_active":"2024-02-22T19:31:04.150818+0000","last_peered":"2024-02-22T19:31:04.150818+0000","last_clean":"2024-02-22T19:31:04.150818+0000","last_became_active":"2024-02-22T19:29:01.004850+0000","last_became_peered":"2024-02-22T19:29:01.004850+0000","last_unstale":"2024-02-22T19:31:04.150818+0000","last_undegraded":"2024-02-22T19:31:04.150818+0000","last_fullsized":"2024-02-22T19:31:04.150818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.619685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.619685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149996+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:31:04.149996+0000","last_peered":"2024-02-22T19:31:04.149996+0000","last_clean":"2024-02-22T19:31:04.149996+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:31:04.149996+0000","last_undegraded":"2024-02-22T19:31:04.149996+0000","last_fullsized":"2024-02-22T19:31:04.149996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:06.495959+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:31:06.495959+0000","last_peered":"2024-02-22T19:31:06.495959+0000","last_clean":"2024-02-22T19:31:06.495959+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:31:06.495959+0000","last_undegraded":"2024-02-22T19:31:06.495959+0000","last_fullsized":"2024-02-22T19:31:06.495959+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149146+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:31:04.149146+0000","last_peered":"2024-02-22T19:31:04.149146+0000","last_clean":"2024-02-22T19:31:04.149146+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:31:04.149146+0000","last_undegraded":"2024-02-22T19:31:04.149146+0000","last_fullsized":"2024-02-22T19:31:04.149146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150057+0000","last_change":"2024-02-22T19:30:17.466789+0000","last_active":"2024-02-22T19:31:04.150057+0000","last_peered":"2024-02-22T19:31:04.150057+0000","last_clean":"2024-02-22T19:31:04.150057+0000","last_became_active":"2024-02-22T19:29:01.010917+0000","last_became_peered":"2024-02-22T19:29:01.010917+0000","last_unstale":"2024-02-22T19:31:04.150057+0000","last_undegraded":"2024-02-22T19:31:04.150057+0000","last_fullsized":"2024-02-22T19:31:04.150057+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148585+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:31:04.148585+0000","last_peered":"2024-02-22T19:31:04.148585+0000","last_clean":"2024-02-22T19:31:04.148585+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:31:04.148585+0000","last_undegraded":"2024-02-22T19:31:04.148585+0000","last_fullsized":"2024-02-22T19:31:04.148585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493259+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:31:03.493259+0000","last_peered":"2024-02-22T19:31:03.493259+0000","last_clean":"2024-02-22T19:31:03.493259+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:31:03.493259+0000","last_undegraded":"2024-02-22T19:31:03.493259+0000","last_fullsized":"2024-02-22T19:31:03.493259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149039+0000","last_change":"2024-02-22T19:30:09.477558+0000","last_active":"2024-02-22T19:31:04.149039+0000","last_peered":"2024-02-22T19:31:04.149039+0000","last_clean":"2024-02-22T19:31:04.149039+0000","last_became_active":"2024-02-22T19:29:01.006282+0000","last_became_peered":"2024-02-22T19:29:01.006282+0000","last_unstale":"2024-02-22T19:31:04.149039+0000","last_undegraded":"2024-02-22T19:31:04.149039+0000","last_fullsized":"2024-02-22T19:31:04.149039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:09.477488+0000","last_clean_scrub_stamp":"2024-02-22T19:30:09.477488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491226+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:31:03.491226+0000","last_peered":"2024-02-22T19:31:03.491226+0000","last_clean":"2024-02-22T19:31:03.491226+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:31:03.491226+0000","last_undegraded":"2024-02-22T19:31:03.491226+0000","last_fullsized":"2024-02-22T19:31:03.491226+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.495939+0000","last_change":"2024-02-22T19:30:07.992740+0000","last_active":"2024-02-22T19:31:04.495939+0000","last_peered":"2024-02-22T19:31:04.495939+0000","last_clean":"2024-02-22T19:31:04.495939+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:31:04.495939+0000","last_undegraded":"2024-02-22T19:31:04.495939+0000","last_fullsized":"2024-02-22T19:31:04.495939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:30:07.992671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.992671+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148142+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:31:04.148142+0000","last_peered":"2024-02-22T19:31:04.148142+0000","last_clean":"2024-02-22T19:31:04.148142+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:31:04.148142+0000","last_undegraded":"2024-02-22T19:31:04.148142+0000","last_fullsized":"2024-02-22T19:31:04.148142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150945+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:31:04.150945+0000","last_peered":"2024-02-22T19:31:04.150945+0000","last_clean":"2024-02-22T19:31:04.150945+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:31:04.150945+0000","last_undegraded":"2024-02-22T19:31:04.150945+0000","last_fullsized":"2024-02-22T19:31:04.150945+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490671+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:31:03.490671+0000","last_peered":"2024-02-22T19:31:03.490671+0000","last_clean":"2024-02-22T19:31:03.490671+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:31:03.490671+0000","last_undegraded":"2024-02-22T19:31:03.490671+0000","last_fullsized":"2024-02-22T19:31:03.490671+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148038+0000","last_change":"2024-02-22T19:30:18.117213+0000","last_active":"2024-02-22T19:31:04.148038+0000","last_peered":"2024-02-22T19:31:04.148038+0000","last_clean":"2024-02-22T19:31:04.148038+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:31:04.148038+0000","last_undegraded":"2024-02-22T19:31:04.148038+0000","last_fullsized":"2024-02-22T19:31:04.148038+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.117111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148728+0000","last_change":"2024-02-22T19:30:28.128814+0000","last_active":"2024-02-22T19:31:04.148728+0000","last_peered":"2024-02-22T19:31:04.148728+0000","last_clean":"2024-02-22T19:31:04.148728+0000","last_became_active":"2024-02-22T19:30:24.410718+0000","last_became_peered":"2024-02-22T19:30:24.410718+0000","last_unstale":"2024-02-22T19:31:04.148728+0000","last_undegraded":"2024-02-22T19:31:04.148728+0000","last_fullsized":"2024-02-22T19:31:04.148728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148628+0000","last_change":"2024-02-22T19:30:31.102193+0000","last_active":"2024-02-22T19:31:04.148628+0000","last_peered":"2024-02-22T19:31:04.148628+0000","last_clean":"2024-02-22T19:31:04.148628+0000","last_became_active":"2024-02-22T19:29:05.098656+0000","last_became_peered":"2024-02-22T19:29:05.098656+0000","last_unstale":"2024-02-22T19:31:04.148628+0000","last_undegraded":"2024-02-22T19:31:04.148628+0000","last_fullsized":"2024-02-22T19:31:04.148628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.102065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.102065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149698+0000","last_change":"2024-02-22T19:30:35.556098+0000","last_active":"2024-02-22T19:31:04.149698+0000","last_peered":"2024-02-22T19:31:04.149698+0000","last_clean":"2024-02-22T19:31:04.149698+0000","last_became_active":"2024-02-22T19:29:07.098131+0000","last_became_peered":"2024-02-22T19:29:07.098131+0000","last_unstale":"2024-02-22T19:31:04.149698+0000","last_undegraded":"2024-02-22T19:31:04.149698+0000","last_fullsized":"2024-02-22T19:31:04.149698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150275+0000","last_change":"2024-02-22T19:29:15.492725+0000","last_active":"2024-02-22T19:31:04.150275+0000","last_peered":"2024-02-22T19:31:04.150275+0000","last_clean":"2024-02-22T19:31:04.150275+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:31:04.150275+0000","last_undegraded":"2024-02-22T19:31:04.150275+0000","last_fullsized":"2024-02-22T19:31:04.150275+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:29:15.492609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:29:15.492609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492866+0000","last_change":"2024-02-22T19:30:45.909363+0000","last_active":"2024-02-22T19:31:03.492866+0000","last_peered":"2024-02-22T19:31:03.492866+0000","last_clean":"2024-02-22T19:31:03.492866+0000","last_became_active":"2024-02-22T19:29:01.010904+0000","last_became_peered":"2024-02-22T19:29:01.010904+0000","last_unstale":"2024-02-22T19:31:03.492866+0000","last_undegraded":"2024-02-22T19:31:03.492866+0000","last_fullsized":"2024-02-22T19:31:03.492866+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149281+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:31:04.149281+0000","last_peered":"2024-02-22T19:31:04.149281+0000","last_clean":"2024-02-22T19:31:04.149281+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:31:04.149281+0000","last_undegraded":"2024-02-22T19:31:04.149281+0000","last_fullsized":"2024-02-22T19:31:04.149281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148855+0000","last_change":"2024-02-22T19:30:49.658305+0000","last_active":"2024-02-22T19:31:04.148855+0000","last_peered":"2024-02-22T19:31:04.148855+0000","last_clean":"2024-02-22T19:31:04.148855+0000","last_became_active":"2024-02-22T19:29:05.092427+0000","last_became_peered":"2024-02-22T19:29:05.092427+0000","last_unstale":"2024-02-22T19:31:04.148855+0000","last_undegraded":"2024-02-22T19:31:04.148855+0000","last_fullsized":"2024-02-22T19:31:04.148855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490952+0000","last_change":"2024-02-22T19:30:33.113603+0000","last_active":"2024-02-22T19:31:03.490952+0000","last_peered":"2024-02-22T19:31:03.490952+0000","last_clean":"2024-02-22T19:31:03.490952+0000","last_became_active":"2024-02-22T19:29:07.095700+0000","last_became_peered":"2024-02-22T19:29:07.095700+0000","last_unstale":"2024-02-22T19:31:03.490952+0000","last_undegraded":"2024-02-22T19:31:03.490952+0000","last_fullsized":"2024-02-22T19:31:03.490952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147963+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:31:04.147963+0000","last_peered":"2024-02-22T19:31:04.147963+0000","last_clean":"2024-02-22T19:31:04.147963+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:31:04.147963+0000","last_undegraded":"2024-02-22T19:31:04.147963+0000","last_fullsized":"2024-02-22T19:31:04.147963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":168,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:07.385287+0000","last_change":"2024-02-22T19:30:20.550910+0000","last_active":"2024-02-22T19:31:07.385287+0000","last_peered":"2024-02-22T19:31:07.385287+0000","last_clean":"2024-02-22T19:31:07.385287+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:31:07.385287+0000","last_undegraded":"2024-02-22T19:31:07.385287+0000","last_fullsized":"2024-02-22T19:31:07.385287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:20.550837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.550837+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148696+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:31:04.148696+0000","last_peered":"2024-02-22T19:31:04.148696+0000","last_clean":"2024-02-22T19:31:04.148696+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:31:04.148696+0000","last_undegraded":"2024-02-22T19:31:04.148696+0000","last_fullsized":"2024-02-22T19:31:04.148696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150803+0000","last_change":"2024-02-22T19:30:33.584923+0000","last_active":"2024-02-22T19:31:04.150803+0000","last_peered":"2024-02-22T19:31:04.150803+0000","last_clean":"2024-02-22T19:31:04.150803+0000","last_became_active":"2024-02-22T19:29:05.091758+0000","last_became_peered":"2024-02-22T19:29:05.091758+0000","last_unstale":"2024-02-22T19:31:04.150803+0000","last_undegraded":"2024-02-22T19:31:04.150803+0000","last_fullsized":"2024-02-22T19:31:04.150803+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490848+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:31:03.490848+0000","last_peered":"2024-02-22T19:31:03.490848+0000","last_clean":"2024-02-22T19:31:03.490848+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:31:03.490848+0000","last_undegraded":"2024-02-22T19:31:03.490848+0000","last_fullsized":"2024-02-22T19:31:03.490848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491189+0000","last_change":"2024-02-22T19:30:34.149749+0000","last_active":"2024-02-22T19:31:03.491189+0000","last_peered":"2024-02-22T19:31:03.491189+0000","last_clean":"2024-02-22T19:31:03.491189+0000","last_became_active":"2024-02-22T19:29:01.004572+0000","last_became_peered":"2024-02-22T19:29:01.004572+0000","last_unstale":"2024-02-22T19:31:03.491189+0000","last_undegraded":"2024-02-22T19:31:03.491189+0000","last_fullsized":"2024-02-22T19:31:03.491189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148353+0000","last_change":"2024-02-22T19:30:22.683587+0000","last_active":"2024-02-22T19:31:04.148353+0000","last_peered":"2024-02-22T19:31:04.148353+0000","last_clean":"2024-02-22T19:31:04.148353+0000","last_became_active":"2024-02-22T19:29:03.446818+0000","last_became_peered":"2024-02-22T19:29:03.446818+0000","last_unstale":"2024-02-22T19:31:04.148353+0000","last_undegraded":"2024-02-22T19:31:04.148353+0000","last_fullsized":"2024-02-22T19:31:04.148353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150088+0000","last_change":"2024-02-22T19:30:14.659000+0000","last_active":"2024-02-22T19:31:04.150088+0000","last_peered":"2024-02-22T19:31:04.150088+0000","last_clean":"2024-02-22T19:31:04.150088+0000","last_became_active":"2024-02-22T19:29:05.091867+0000","last_became_peered":"2024-02-22T19:29:05.091867+0000","last_unstale":"2024-02-22T19:31:04.150088+0000","last_undegraded":"2024-02-22T19:31:04.150088+0000","last_fullsized":"2024-02-22T19:31:04.150088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.147697+0000","last_change":"2024-02-22T19:30:18.719372+0000","last_active":"2024-02-22T19:31:04.147697+0000","last_peered":"2024-02-22T19:31:04.147697+0000","last_clean":"2024-02-22T19:31:04.147697+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:31:04.147697+0000","last_undegraded":"2024-02-22T19:31:04.147697+0000","last_fullsized":"2024-02-22T19:31:04.147697+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.719310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149794+0000","last_change":"2024-02-22T19:30:29.616721+0000","last_active":"2024-02-22T19:31:04.149794+0000","last_peered":"2024-02-22T19:31:04.149794+0000","last_clean":"2024-02-22T19:31:04.149794+0000","last_became_active":"2024-02-22T19:29:01.012034+0000","last_became_peered":"2024-02-22T19:29:01.012034+0000","last_unstale":"2024-02-22T19:31:04.149794+0000","last_undegraded":"2024-02-22T19:31:04.149794+0000","last_fullsized":"2024-02-22T19:31:04.149794+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491135+0000","last_change":"2024-02-22T19:30:12.059865+0000","last_active":"2024-02-22T19:31:03.491135+0000","last_peered":"2024-02-22T19:31:03.491135+0000","last_clean":"2024-02-22T19:31:03.491135+0000","last_became_active":"2024-02-22T19:29:03.086071+0000","last_became_peered":"2024-02-22T19:29:03.086071+0000","last_unstale":"2024-02-22T19:31:03.491135+0000","last_undegraded":"2024-02-22T19:31:03.491135+0000","last_fullsized":"2024-02-22T19:31:03.491135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:12.059797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:12.059797+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.497407+0000","last_change":"2024-02-22T19:30:44.574471+0000","last_active":"2024-02-22T19:31:04.497407+0000","last_peered":"2024-02-22T19:31:04.497407+0000","last_clean":"2024-02-22T19:31:04.497407+0000","last_became_active":"2024-02-22T19:29:05.094532+0000","last_became_peered":"2024-02-22T19:29:05.094532+0000","last_unstale":"2024-02-22T19:31:04.497407+0000","last_undegraded":"2024-02-22T19:31:04.497407+0000","last_fullsized":"2024-02-22T19:31:04.497407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149234+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:31:04.149234+0000","last_peered":"2024-02-22T19:31:04.149234+0000","last_clean":"2024-02-22T19:31:04.149234+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:31:04.149234+0000","last_undegraded":"2024-02-22T19:31:04.149234+0000","last_fullsized":"2024-02-22T19:31:04.149234+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148772+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:31:04.148772+0000","last_peered":"2024-02-22T19:31:04.148772+0000","last_clean":"2024-02-22T19:31:04.148772+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:31:04.148772+0000","last_undegraded":"2024-02-22T19:31:04.148772+0000","last_fullsized":"2024-02-22T19:31:04.148772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491108+0000","last_change":"2024-02-22T19:30:54.022533+0000","last_active":"2024-02-22T19:31:03.491108+0000","last_peered":"2024-02-22T19:31:03.491108+0000","last_clean":"2024-02-22T19:31:03.491108+0000","last_became_active":"2024-02-22T19:29:03.084036+0000","last_became_peered":"2024-02-22T19:29:03.084036+0000","last_unstale":"2024-02-22T19:31:03.491108+0000","last_undegraded":"2024-02-22T19:31:03.491108+0000","last_fullsized":"2024-02-22T19:31:03.491108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.693788+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:31:04.693788+0000","last_peered":"2024-02-22T19:31:04.693788+0000","last_clean":"2024-02-22T19:31:04.693788+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:31:04.693788+0000","last_undegraded":"2024-02-22T19:31:04.693788+0000","last_fullsized":"2024-02-22T19:31:04.693788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491112+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:31:03.491112+0000","last_peered":"2024-02-22T19:31:03.491112+0000","last_clean":"2024-02-22T19:31:03.491112+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:31:03.491112+0000","last_undegraded":"2024-02-22T19:31:03.491112+0000","last_fullsized":"2024-02-22T19:31:03.491112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493481+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:31:03.493481+0000","last_peered":"2024-02-22T19:31:03.493481+0000","last_clean":"2024-02-22T19:31:03.493481+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:31:03.493481+0000","last_undegraded":"2024-02-22T19:31:03.493481+0000","last_fullsized":"2024-02-22T19:31:03.493481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492821+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:31:03.492821+0000","last_peered":"2024-02-22T19:31:03.492821+0000","last_clean":"2024-02-22T19:31:03.492821+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:31:03.492821+0000","last_undegraded":"2024-02-22T19:31:03.492821+0000","last_fullsized":"2024-02-22T19:31:03.492821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490954+0000","last_change":"2024-02-22T19:30:21.116431+0000","last_active":"2024-02-22T19:31:03.490954+0000","last_peered":"2024-02-22T19:31:03.490954+0000","last_clean":"2024-02-22T19:31:03.490954+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:31:03.490954+0000","last_undegraded":"2024-02-22T19:31:03.490954+0000","last_fullsized":"2024-02-22T19:31:03.490954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.116358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149857+0000","last_change":"2024-02-22T19:30:20.543466+0000","last_active":"2024-02-22T19:31:04.149857+0000","last_peered":"2024-02-22T19:31:04.149857+0000","last_clean":"2024-02-22T19:31:04.149857+0000","last_became_active":"2024-02-22T19:29:07.095452+0000","last_became_peered":"2024-02-22T19:29:07.095452+0000","last_unstale":"2024-02-22T19:31:04.149857+0000","last_undegraded":"2024-02-22T19:31:04.149857+0000","last_fullsized":"2024-02-22T19:31:04.149857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:20.543398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.543398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.497628+0000","last_change":"2024-02-22T19:30:35.999034+0000","last_active":"2024-02-22T19:31:04.497628+0000","last_peered":"2024-02-22T19:31:04.497628+0000","last_clean":"2024-02-22T19:31:04.497628+0000","last_became_active":"2024-02-22T19:29:05.095818+0000","last_became_peered":"2024-02-22T19:29:05.095818+0000","last_unstale":"2024-02-22T19:31:04.497628+0000","last_undegraded":"2024-02-22T19:31:04.497628+0000","last_fullsized":"2024-02-22T19:31:04.497628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492824+0000","last_change":"2024-02-22T19:30:06.720523+0000","last_active":"2024-02-22T19:31:03.492824+0000","last_peered":"2024-02-22T19:31:03.492824+0000","last_clean":"2024-02-22T19:31:03.492824+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:31:03.492824+0000","last_undegraded":"2024-02-22T19:31:03.492824+0000","last_fullsized":"2024-02-22T19:31:03.492824+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.720452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.720452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148703+0000","last_change":"2024-02-22T19:30:27.174847+0000","last_active":"2024-02-22T19:31:04.148703+0000","last_peered":"2024-02-22T19:31:04.148703+0000","last_clean":"2024-02-22T19:31:04.148703+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:31:04.148703+0000","last_undegraded":"2024-02-22T19:31:04.148703+0000","last_fullsized":"2024-02-22T19:31:04.148703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:27.174779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.174779+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:09.064602+0000","last_change":"2024-02-22T19:31:09.064602+0000","last_active":"2024-02-22T19:31:09.064602+0000","last_peered":"2024-02-22T19:31:09.064602+0000","last_clean":"2024-02-22T19:31:09.064602+0000","last_became_active":"2024-02-22T19:29:07.198153+0000","last_became_peered":"2024-02-22T19:29:07.198153+0000","last_unstale":"2024-02-22T19:31:09.064602+0000","last_undegraded":"2024-02-22T19:31:09.064602+0000","last_fullsized":"2024-02-22T19:31:09.064602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.150707+0000","last_change":"2024-02-22T19:30:31.646674+0000","last_active":"2024-02-22T19:31:04.150707+0000","last_peered":"2024-02-22T19:31:04.150707+0000","last_clean":"2024-02-22T19:31:04.150707+0000","last_became_active":"2024-02-22T19:29:05.096026+0000","last_became_peered":"2024-02-22T19:29:05.096026+0000","last_unstale":"2024-02-22T19:31:04.150707+0000","last_undegraded":"2024-02-22T19:31:04.150707+0000","last_fullsized":"2024-02-22T19:31:04.150707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.490967+0000","last_change":"2024-02-22T19:30:19.102902+0000","last_active":"2024-02-22T19:31:03.490967+0000","last_peered":"2024-02-22T19:31:03.490967+0000","last_clean":"2024-02-22T19:31:03.490967+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:31:03.490967+0000","last_undegraded":"2024-02-22T19:31:03.490967+0000","last_fullsized":"2024-02-22T19:31:03.490967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:19.102765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:19.102765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491954+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:31:03.491954+0000","last_peered":"2024-02-22T19:31:03.491954+0000","last_clean":"2024-02-22T19:31:03.491954+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:31:03.491954+0000","last_undegraded":"2024-02-22T19:31:03.491954+0000","last_fullsized":"2024-02-22T19:31:03.491954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.148791+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:31:04.148791+0000","last_peered":"2024-02-22T19:31:04.148791+0000","last_clean":"2024-02-22T19:31:04.148791+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:31:04.148791+0000","last_undegraded":"2024-02-22T19:31:04.148791+0000","last_fullsized":"2024-02-22T19:31:04.148791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491416+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:31:03.491416+0000","last_peered":"2024-02-22T19:31:03.491416+0000","last_clean":"2024-02-22T19:31:03.491416+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:31:03.491416+0000","last_undegraded":"2024-02-22T19:31:03.491416+0000","last_fullsized":"2024-02-22T19:31:03.491416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.149831+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:31:04.149831+0000","last_peered":"2024-02-22T19:31:04.149831+0000","last_clean":"2024-02-22T19:31:04.149831+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:31:04.149831+0000","last_undegraded":"2024-02-22T19:31:04.149831+0000","last_fullsized":"2024-02-22T19:31:04.149831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491950+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:31:03.491950+0000","last_peered":"2024-02-22T19:31:03.491950+0000","last_clean":"2024-02-22T19:31:03.491950+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:31:03.491950+0000","last_undegraded":"2024-02-22T19:31:03.491950+0000","last_fullsized":"2024-02-22T19:31:03.491950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.492090+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:31:03.492090+0000","last_peered":"2024-02-22T19:31:03.492090+0000","last_clean":"2024-02-22T19:31:03.492090+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:31:03.492090+0000","last_undegraded":"2024-02-22T19:31:03.492090+0000","last_fullsized":"2024-02-22T19:31:03.492090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-22T19:31:04.694174+0000","last_change":"2024-02-22T19:30:23.100251+0000","last_active":"2024-02-22T19:31:04.694174+0000","last_peered":"2024-02-22T19:31:04.694174+0000","last_clean":"2024-02-22T19:31:04.694174+0000","last_became_active":"2024-02-22T19:29:05.092069+0000","last_became_peered":"2024-02-22T19:29:05.092069+0000","last_unstale":"2024-02-22T19:31:04.694174+0000","last_undegraded":"2024-02-22T19:31:04.694174+0000","last_fullsized":"2024-02-22T19:31:04.694174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:23.100177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.100177+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.493114+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:31:03.493114+0000","last_peered":"2024-02-22T19:31:03.493114+0000","last_clean":"2024-02-22T19:31:03.493114+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:31:03.493114+0000","last_undegraded":"2024-02-22T19:31:03.493114+0000","last_fullsized":"2024-02-22T19:31:03.493114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-22T19:31:03.491607+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:31:03.491607+0000","last_peered":"2024-02-22T19:31:03.491607+0000","last_clean":"2024-02-22T19:31:03.491607+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:31:03.491607+0000","last_undegraded":"2024-02-22T19:31:03.491607+0000","last_fullsized":"2024-02-22T19:31:03.491607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":415,"ondisk_log_size":415,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266717,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":308559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65100000000000002,"5min":0.871,"15min":0.871},"last":0.439}]},{"osd":1,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63100000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68700000000000006,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.55200000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58199999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68500000000000005,"5min":1.98,"15min":1.98},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64300000000000002,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71099999999999997,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.753,"15min":0.753},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.623},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72899999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.56000000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528354,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297720,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":638976,"data_stored":307174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":1.024,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73599999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.62},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.309,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.499},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.36799999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.32500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.55500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297772,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447380,"statfs":{"total":95995035648,"available":95690117120,"internally_reserved":0,"allocated":692224,"data_stored":309318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:40 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36399999999999999,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52900000000000003,"5min":0.62,"15min":0.62},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.378,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.41199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38}]},{"osd":2,"last update":"Thu Feb 22 19:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.254,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:27 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.43099999999999999,"15min":0.45500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.441,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.48999999999999999,"15min":0.50900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54300000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54300000000000004,"15min":0.56599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Feb 22 19:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78700000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.505},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297752,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447400,"statfs":{"total":95995035648,"available":95690137600,"internally_reserved":0,"allocated":671744,"data_stored":309129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":0.51100000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.52400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.27600000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.46899999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.377,"15min":0.374},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.53900000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.40799999999999997,"15min":0.42599999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.443,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.307},{"interface":"front","average":{"1min":0.378,"5min":0.42699999999999999,"15min":0.443},"min":{"1min":0.30399999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46899999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70899999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.995,"15min":0.995},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67600000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:30:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":308185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.373,"15min":0.37},"min":{"1min":0.23000000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.53000000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:31:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38,"15min":0.378},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55000000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.442},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.50800000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:30:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.5}]},{"osd":5,"last update":"Thu Feb 22 19:31:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.436,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.875,"5min":0.995,"15min":0.995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90200000000000002,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.624}]},{"osd":6,"last update":"Thu Feb 22 19:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.60999999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149743,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297764,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447388,"statfs":{"total":95995035648,"available":95690125312,"internally_reserved":0,"allocated":684032,"data_stored":309226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41599999999999998,"15min":0.432},"min":{"1min":0.20799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.193},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.45000000000000001,"15min":0.46400000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67600000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.317},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:30:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.47599999999999998,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.46999999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.31}]},{"osd":4,"last update":"Thu Feb 22 19:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78300000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41499999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953517,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447392,"statfs":{"total":95995035648,"available":95690129408,"internally_reserved":0,"allocated":679936,"data_stored":309265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.626}]},{"osd":1,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":1,"15min":1},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.65900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.34200000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.378},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.40999999999999998},"min":{"1min":0.36499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.5}]},{"osd":6,"last update":"Thu Feb 22 19:30:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:31:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60899999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724583,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":307542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.5229999999999997,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.35599999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78000000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.59499999999999997,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.5529999999999999,"5min":5.5529999999999999,"15min":5.5529999999999999},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.219,"15min":0.219},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.26600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.42499999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.46500000000000002},"min":{"1min":0.35299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.878},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":1.071,"15min":1.071},"last":0.73899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:31:15.533 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-22T19:31:15.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump 2024-02-22T19:31:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:14.070795+0000 osd.3 (osd.3) 55 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:31:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:14.073903+0000 osd.3 (osd.3) 56 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:31:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:14.292952+0000 mgr.y (mgr.14535) 93 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:16.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:14.689751+0000 osd.5 (osd.5) 41 : cluster [DBG] 2.2 deep-scrub starts 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:14.692689+0000 osd.5 (osd.5) 42 : cluster [DBG] 2.2 deep-scrub ok 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:14.700309+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:14.700352+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:14.786200+0000 mgr.y (mgr.14535) 94 : audit [DBG] from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:15.206883+0000 mgr.y (mgr.14535) 95 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 955 B/s rd, 0 op/s 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:15.273803+0000 mon.a (mon.0) 748 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:14.689751+0000 osd.5 (osd.5) 41 : cluster [DBG] 2.2 deep-scrub starts 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:14.692689+0000 osd.5 (osd.5) 42 : cluster [DBG] 2.2 deep-scrub ok 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:14.700309+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:14.700352+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:14.786200+0000 mgr.y (mgr.14535) 94 : audit [DBG] from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:16.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:15.206883+0000 mgr.y (mgr.14535) 95 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 955 B/s rd, 0 op/s 2024-02-22T19:31:16.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:15.273803+0000 mon.a (mon.0) 748 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:14.689751+0000 osd.5 (osd.5) 41 : cluster [DBG] 2.2 deep-scrub starts 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:14.692689+0000 osd.5 (osd.5) 42 : cluster [DBG] 2.2 deep-scrub ok 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:14.700309+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:14.700352+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:14.786200+0000 mgr.y (mgr.14535) 94 : audit [DBG] from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:15.206883+0000 mgr.y (mgr.14535) 95 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 955 B/s rd, 0 op/s 2024-02-22T19:31:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:15.273803+0000 mon.a (mon.0) 748 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-22T19:31:17.278 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:15.681185+0000 osd.5 (osd.5) 43 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:31:17.278 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:15.685290+0000 osd.5 (osd.5) 44 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:31:17.278 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:15.880081+0000 osd.7 (osd.7) 45 : cluster [DBG] 3.d scrub starts 2024-02-22T19:31:17.278 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:15.882988+0000 osd.7 (osd.7) 46 : cluster [DBG] 3.d scrub ok 2024-02-22T19:31:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:15.681185+0000 osd.5 (osd.5) 43 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:31:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:15.685290+0000 osd.5 (osd.5) 44 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:31:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:15.880081+0000 osd.7 (osd.7) 45 : cluster [DBG] 3.d scrub starts 2024-02-22T19:31:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:15.882988+0000 osd.7 (osd.7) 46 : cluster [DBG] 3.d scrub ok 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 89 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:fsid b6783888-d1b7-11ee-95bf-87774f69a715 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-02-22T19:22:59.746063+0000 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2024-02-22T19:31:15.265971+0000 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:crush_version 18 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:full_ratio 0.95 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:backfillfull_ratio 0.9 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:nearfull_ratio 0.85 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:require_min_compat_client luminous 2024-02-22T19:31:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:min_compat_client luminous 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:require_osd_release pacific 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:stretch_mode_enabled false 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:pool 1 'device_health_metrics' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 67 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr_devicehealth 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:pool 2 'iscsi' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 59 flags hashpspool stripe_width 0 application rbd 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:pool 3 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 58 flags hashpspool stripe_width 0 application rgw 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rgw 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 62 flags hashpspool stripe_width 0 application rgw 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 64 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:max_osd 8 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6801/162271081 v2:172.21.15.53:6802/162271081 exists,up 4326a76a-6144-4859-9b40-5ff69edbbcab 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 up in weight 1 up_from 13 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6805/2973403044 v2:172.21.15.53:6806/2973403044 exists,up fc056bcd-1a3b-408c-b9d1-4992091df6c2 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6809/936980935 v2:172.21.15.53:6810/936980935 exists,up 61f3f980-82cd-4e8e-9a3f-6d3f4559cc83 2024-02-22T19:31:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 up in weight 1 up_from 26 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6813/3878043918 v2:172.21.15.53:6814/3878043918 exists,up 3f285320-aa28-4fc1-b1c7-456c9aecd993 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 up in weight 1 up_from 32 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6800/2373040976 v2:172.21.15.64:6801/2373040976 exists,up 773e8080-1028-48fe-8f6e-ed6beca60211 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 up in weight 1 up_from 39 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6804/2413553331 v2:172.21.15.64:6805/2413553331 exists,up d79dbfb8-8f92-4d65-92ab-4b00aaccdb07 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 up in weight 1 up_from 45 up_thru 60 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6808/3043274732 v2:172.21.15.64:6809/3043274732 exists,up ebbb9499-df1e-42b7-89bb-df6c5935f39e 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 up in weight 1 up_from 53 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6812/2605082260 v2:172.21.15.64:6813/2605082260 exists,up cde36201-cb77-4967-9402-6fdbe633b40a 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 4.7 [1,2] 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 4.b [1,2] 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 4.f [1,2] 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/4161439976 expires 2024-02-23T19:29:23.167786+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/2578736337 expires 2024-02-23T19:29:23.167786+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/762516580 expires 2024-02-23T19:23:54.856898+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/3676085495 expires 2024-02-23T19:29:23.167786+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/3516945073 expires 2024-02-23T19:23:54.856898+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/828421682 expires 2024-02-23T19:23:54.856898+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/4127851809 expires 2024-02-23T19:29:23.167786+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/1906929179 expires 2024-02-23T19:23:32.485117+0000 2024-02-22T19:31:17.749 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/2640420245 expires 2024-02-23T19:23:32.485117+0000 2024-02-22T19:31:17.750 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/2854906057 expires 2024-02-23T19:23:32.485117+0000 2024-02-22T19:31:17.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:15.681185+0000 osd.5 (osd.5) 43 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:31:17.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:15.685290+0000 osd.5 (osd.5) 44 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:31:17.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:15.880081+0000 osd.7 (osd.7) 45 : cluster [DBG] 3.d scrub starts 2024-02-22T19:31:17.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:15.882988+0000 osd.7 (osd.7) 46 : cluster [DBG] 3.d scrub ok 2024-02-22T19:31:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:17.207403+0000 mgr.y (mgr.14535) 96 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:18.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:17.747137+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? 172.21.15.53:0/2006917346' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-22T19:31:18.457 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6801/162271081 v2:172.21.15.53:6802/162271081 exists,up 4326a76a-6144-4859-9b40-5ff69edbbcab', 'osd.1 up in weight 1 up_from 13 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6805/2973403044 v2:172.21.15.53:6806/2973403044 exists,up fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6809/936980935 v2:172.21.15.53:6810/936980935 exists,up 61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'osd.3 up in weight 1 up_from 26 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.53:6813/3878043918 v2:172.21.15.53:6814/3878043918 exists,up 3f285320-aa28-4fc1-b1c7-456c9aecd993', 'osd.4 up in weight 1 up_from 32 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6800/2373040976 v2:172.21.15.64:6801/2373040976 exists,up 773e8080-1028-48fe-8f6e-ed6beca60211', 'osd.5 up in weight 1 up_from 39 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6804/2413553331 v2:172.21.15.64:6805/2413553331 exists,up d79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'osd.6 up in weight 1 up_from 45 up_thru 60 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6808/3043274732 v2:172.21.15.64:6809/3043274732 exists,up ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'osd.7 up in weight 1 up_from 53 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.64:6812/2605082260 v2:172.21.15.64:6813/2605082260 exists,up cde36201-cb77-4967-9402-6fdbe633b40a'] 2024-02-22T19:31:18.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon mon.a config show 2024-02-22T19:31:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:17.207403+0000 mgr.y (mgr.14535) 96 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:17.747137+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? 172.21.15.53:0/2006917346' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-22T19:31:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:18.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:17.207403+0000 mgr.y (mgr.14535) 96 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:18.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:17.747137+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? 172.21.15.53:0/2006917346' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-22T19:31:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:17.902418+0000 osd.1 (osd.1) 37 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:17.902524+0000 osd.1 (osd.1) 38 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:18.281340+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-22T19:31:19.636 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:17.902418+0000 osd.1 (osd.1) 37 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:19.637 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:17.902524+0000 osd.1 (osd.1) 38 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:19.637 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:18.281340+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-22T19:31:19.637 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:17.902418+0000 osd.1 (osd.1) 37 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:19.637 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:17.902524+0000 osd.1 (osd.1) 38 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:19.637 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:18.281340+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-22T19:31:19.637 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:19] "GET /metrics HTTP/1.1" 200 172102 "" "Prometheus/2.33.4" 2024-02-22T19:31:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:19.207933+0000 mgr.y (mgr.14535) 97 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:19.207933+0000 mgr.y (mgr.14535) 97 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mon.a", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster": "ceph", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "admin_socket_mode": "", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_client_required": "cephx, none", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_cluster_required": "cephx", 2024-02-22T19:31:20.527 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_debug": "true", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_service_required": "cephx", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_service_ticket_ttl": "240.000000", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_supported": "", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio": "true", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_max_queue_depth": "1024", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_poll_ms": "250", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_reap_max": "16", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_async_discard": "false", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_block_size": "4096", 2024-02-22T19:31:20.528 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_inflight_ios": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_enable_discard": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_flock_retry": "3", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_flock_retry_interval": "0.100000", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_inject_crash": "0", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_inject_crash_flush_delay": "2", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring_hipri": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring_sqthread_poll": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_type": "", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_alloc_size": "1048576", 2024-02-22T19:31:20.529 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_allocator": "hybrid", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_buffered_io": "true", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_for_zeros": "false", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_volume_selector_often": "false", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_compact_log_sync": "false", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_compact_min_size": "16777216", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_replay_check_allocations": "true", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_max_log_runway": "4194304", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_max_prefetch": "1048576", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_min_flush_size": "524288", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_min_log_runway": "1048576", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_replay_recovery": "false", 2024-02-22T19:31:20.530 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_shared_alloc_size": "65536", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_sync_write": "false", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_allocator": "hybrid", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blobid_prealloc": "10240", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_create": "true", 2024-02-22T19:31:20.531 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_create": "false", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_path": "", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_size": "0", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_path": "", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_preallocate_file": "false", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_size": "107374182400", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_create": "false", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_path": "", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_size": "100663296", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs": "true", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_env_mirror": "false", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_autotune": "true", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-02-22T19:31:20.532 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size": "0", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_type": "2q", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_clone_cow": "true", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_algorithm": "snappy", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size": "0", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size": "0", 2024-02-22T19:31:20.533 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_mode": "none", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_csum_type": "crc32c", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_enforce_settings": "default", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_freelist": "false", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_fsck_abort": "false", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_bug21040": "false", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_read_err": "false", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_legacy_omap": "false", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-02-22T19:31:20.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_prefill": "0.000000", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_small_allocations": "0", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_default_buffered_read": "true", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_default_buffered_write": "false", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops": "0", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-02-22T19:31:20.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mkfs": "true", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mount": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_umount": "false", 2024-02-22T19:31:20.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_ignore_data_csum": "false", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_kvbackend": "rocksdb", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_op_age": "5.000000", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_alloc_size": "0", 2024-02-22T19:31:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size": "0", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_defer_interval": "3.000000", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_deferred_txc": "32", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size": "0", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_nid_prealloc": "1024", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size": "0", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_retry_disk_reads": "3", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_cf": "true", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-02-22T19:31:20.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_options_annex": "", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_coremask": "0x1", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_io_sleep": "5", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_max_io_completion": "0", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_mem": "512", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_sync_submit_transaction": "false", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_bytes": "67108864", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io": "0", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_tracing": "false", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_reserved": "0", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-02-22T19:31:20.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_action_update_interval": "2", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_mount_timeout": "10", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-02-22T19:31:20.540 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_cluster_require_signatures": "false", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_cluster_require_version": "2", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_require_signatures": "false", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_require_version": "2", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_service_require_signatures": "false", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_service_require_version": "2", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_sign_messages": "true", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "chdir": "", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_acl_type": "", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_asio_thread_count": "2", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_cache_mid": "0.750000", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_cache_size": "16384", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_caps_release_delay": "5", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_check_pool_perm": "true", 2024-02-22T19:31:20.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_collect_and_send_global_metrics": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_force_sync_read": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_getattr_caps": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_inject_tick_delay": "0", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_die_on_failed_remount": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_dirsize_rbytes": "true", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_force_lazyio": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_fs": "", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_inject_fixed_oldest_tid": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_inject_release_failure": "false", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_max_inline_size": "4096", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_max_retries_on_remount_failure": "5", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mds_namespace": "", 2024-02-22T19:31:20.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_metadata": "", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_gid": "-1", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_timeout": "300", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_uid": "-1", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mountpoint": "/", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_notify_timeout": "10", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc": "true", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_dirty": "104857600", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_dirty_age": "5.000000", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_objects": "1000", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_size": "209715200", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_target_dirty": "8388608", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_permissions": "true", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_quota": "true", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_quota_df": "true", 2024-02-22T19:31:20.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_max_bytes": "0", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_max_periods": "4", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_min": "131072", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_reconnect_stale": "false", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_shutdown_timeout": "30", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_snapdir": ".snap", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_tick_interval": "1", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_trace": "", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_try_dentry_invalidate": "false", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_use_faked_inos": "false", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_use_random_mds": "false", 2024-02-22T19:31:20.544 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog": "false", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog_port": "12201", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_monitors": "default=true", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog": "false", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog_level": "info", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "-", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_network": "", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_network_interface": "", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_isal": "false", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_level": "5", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_winsize": "-15", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zstd_level": "1", 2024-02-22T19:31:20.545 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_obc_lru_size": "10", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location": "", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location_hook": "", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location_hook_timeout": "10", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "daemonize": "false", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_allow_any_pool_priority": "false", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asok": "1/5", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asok_assert_abort": "false", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asserts_on_shutdown": "true", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_auth": "1/5", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bdev": "1/3", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bluefs": "1/5", 2024-02-22T19:31:20.546 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bluestore": "1/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_buffer": "0/1", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cephfs_mirror": "0/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cephsqlite": "0/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_civetweb": "1/10", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_client": "0/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_compressor": "1/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_context": "0/1", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crush": "1/1", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crypto": "1/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_deliberately_leak_memory": "false", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_disable_randomized_ping": "false", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_dpdk": "1/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_eventtrace": "1/5", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_filer": "0/1", 2024-02-22T19:31:20.547 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_filestore": "1/3", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_finisher": "1/1", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_fuse": "1/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_heartbeat_testing_span": "0", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_heartbeatmap": "1/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_immutable_obj_cache": "0/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_javaclient": "1/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_journal": "1/3", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_journaler": "0/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_kstore": "1/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_leveldb": "4/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_lockdep": "0/1", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds": "1/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_balancer": "1/5", 2024-02-22T19:31:20.548 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_locker": "1/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_log": "1/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_log_expire": "1/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_migrator": "1/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_memdb": "4/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mgr": "2/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mgrc": "1/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mon": "20/20", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_monc": "0/10", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_ms": "1/1", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_none": "0/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objclass": "0/5", 2024-02-22T19:31:20.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objectcacher": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objecter": "0/1", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_optracker": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_osd": "1/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_paxos": "20/20", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_perfcounter": "1/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_prioritycache": "1/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rados": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_mirror": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_pwl": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_replay": "0/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_refs": "0/0", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_reserver": "1/1", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw": "1/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_sync": "1/5", 2024-02-22T19:31:20.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rocksdb": "4/5", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_striper": "0/1", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_test": "0/5", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_throttle": "1/1", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_timer": "0/1", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_tp": "0/5", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "defer_client_eviction_on_laggy_osds": "true", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "device_failure_prediction_mode": "none", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_graylog": "false", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_stderr": "true", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_syslog": "false", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event_tracing": "false", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "fake_statfs_for_testing": "0", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "fatal_signal_handlers": "true", 2024-02-22T19:31:20.551 INFO:teuthology.orchestra.run.smithi053.stdout: "filer_max_purge_ops": "10", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filer_max_truncate_ops": "128", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_apply_finisher_threads": "1", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_blackhole": "false", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_btrfs_clone_range": "true", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_btrfs_snap": "true", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_caller_concurrency": "10", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_collect_device_partition_information": "true", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_commit_timeout": "600.000000", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_inject_read_err": "false", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_omap_check": "false", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_random_read_err": "0.000000", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_verify_split": "false", 2024-02-22T19:31:20.552 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_dump_file": "", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fadvise": "true", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fail_eio": "true", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fd_cache_shards": "16", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fd_cache_size": "128", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fiemap": "false", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fiemap_threshold": "4096", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_index_retry_probability": "0.000000", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_inject_stall": "0", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_parallel": "false", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_trailing": "false", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_writeahead": "false", 2024-02-22T19:31:20.553 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_kill_at": "0", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size": "0", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs": "0", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_other": "2", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_sync_interval": "5.000000", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size": "0", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-02-22T19:31:20.554 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_merge_threshold": "-10", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_min_sync_interval": "0.010000", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_odsync_write": "false", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_backend": "rocksdb", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_backend_path": "", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_header_cache_size": "1024", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_ondisk_finisher_threads": "1", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_thread_timeout": "60", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_threads": "2", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_punch_hole": "false", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-02-22T19:31:20.555 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_bytes": "104857600", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_ops": "50", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_seek_data_hole": "false", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_sloppy_crc": "false", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_splice": "false", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_split_multiple": "2", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_split_rand_factor": "20", 2024-02-22T19:31:20.556 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_update_to": "1000", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_enable": "true", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-02-22T19:31:20.557 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_xfs_extsize": "false", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_zfs_snap": "false", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fio_dir": "/tmp/fio", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_allow_other": "true", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_atomic_o_trunc": "true", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_big_writes": "true", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_debug": "false", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_default_permissions": "false", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_disable_pagecache": "false", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_max_write": "0", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_multithreaded": "true", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_require_active_mds": "true", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_set_user_groups": "true", 2024-02-22T19:31:20.558 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_move": "true", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_read": "true", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_write": "true", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_syncfs_on_mksnap": "true", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_use_invalidate_cb": "true", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "gss_target_name": "ceph", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_file": "", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_inject_failure": "0", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_interval": "5", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "host": "", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_max_size": "1073741824", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-02-22T19:31:20.559 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_watermark": "0.900000", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "inject_early_sigterm": "false", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_aio": "true", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_align_min_size": "65536", 2024-02-22T19:31:20.560 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_block_align": "true", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_block_size": "4096", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_dio": "true", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_discard": "false", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_force_aio": "false", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_ignore_corruption": "false", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_max_write_bytes": "10485760", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_max_write_entries": "100", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_replay_from": "0", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_high_multiple": "0.000000", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_max_multiple": "0.000000", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_write_header_frequency": "0", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_zero_on_create": "false", 2024-02-22T19:31:20.561 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_prefetch_periods": "10", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_prezero_periods": "5", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_write_head_interval": "15", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "keyfile": "", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_backend": "rocksdb", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_default_stripe_size": "65536", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_fsck_on_mount": "false", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_fsck_on_mount_deep": "true", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_max_bytes": "67108864", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_max_ops": "512", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_nid_prealloc": "1024", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_onode_map_size": "1024", 2024-02-22T19:31:20.562 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_sync_submit_transaction": "false", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_sync_transaction": "false", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_block_size": "65536", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_bloom_size": "0", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_cache_size": "536870912", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_compact_on_mount": "false", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_compression": "false", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_log": "", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_log_to_ceph_log": "true", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_max_open_files": "0", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_paranoid": "false", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_write_buffer_size": "33554432", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "librados_thread_count": "2", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "lockdep": "false", 2024-02-22T19:31:20.563 INFO:teuthology.orchestra.run.smithi053.stdout: "lockdep_force_backtrace": "false", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_coarse_timestamps": "true", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_flush_on_exit": "false", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_graylog_host": "127.0.0.1", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_graylog_port": "12201", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_max_new": "1000", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_max_recent": "10000", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_stderr_prefix": "debug ", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_stop_at_utilization": "0.970000", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": "true", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_graylog": "false", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_stderr": "false", 2024-02-22T19:31:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_syslog": "false", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "max_rotating_auth_attempts": "10", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_action_on_write_error": "1", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_alternate_name_max": "8192", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_asio_thread_count": "2", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_export_pin": "true", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_dirs": "true", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_interval": "5", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_size_max": "100000", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_idle_threshold": "0.000000", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_interval": "10", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_max": "-1", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_max_until": "-1", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_merge_size": "50", 2024-02-22T19:31:20.565 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_midchunk": "0.300000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_min_rebalance": "0.100000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_min_start": "0.200000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_minchunk": "0.001000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_mode": "0", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_need_max": "1.200000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_need_min": "0.800000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_sample_interval": "3.000000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_bits": "3", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_rd": "25000.000000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_size": "10000", 2024-02-22T19:31:20.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_wr": "10000.000000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_target_decay": "10.000000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_grace": "15.000000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_interval": "4.000000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_mon_down_grace": "60", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_memory_limit": "4294967296", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_mid": "0.700000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_release_free_interval": "10", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_reservation": "0.050000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_interval": "1", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_threshold": "262144", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-02-22T19:31:20.567 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_delegate_inos_pct": "50", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_prealloc_inos": "1000", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_connect_bootstrapping": "false", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_damage_table_max_entries": "10000", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_auth_pins": "false", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_frag": "false", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_scatterstat": "false", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_subtrees": "false", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_decay_halflife": "5.000000", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_default_dir_hash": "2", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_defer_session_stale": "true", 2024-02-22T19:31:20.568 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_deny_all_reconnect": "false", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_keys_per_op": "16384", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_max_commit_size": "10", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_max_entries": "0", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dirstat_min_interval": "1.000000", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_after_rejoin": "false", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_on_map": "false", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_threshold_file": "0", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_early_reply": "true", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_enable_op_tracker": "true", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_enforce_unique_name": "true", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_distributed": "true", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_random": "true", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_forward_all_requests_to_auth": "false", 2024-02-22T19:31:20.569 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_health_cache_threshold": "1.500000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_health_summarize_threshold": "10", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_heartbeat_grace": "15.000000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_heartbeat_reset_grace": "1000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_health_dummy": "false", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_migrator_session_race": "false", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_join_fs": "", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_journal_format": "1", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_create_at": "0", 2024-02-22T19:31:20.570 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_export_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_import_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_expire_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_replay_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_link_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_mdstable_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_openc_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_rename_at": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_events_per_segment": "1024", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_max_events": "-1", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_max_segments": "128", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_pause": "false", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_segment_size": "0", 2024-02-22T19:31:20.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_skip_corrupt_events": "false", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_warn_factor": "2.000000", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_caps_per_client": "1048576", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_completed_flushes": "100000", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_completed_requests": "100000", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_export_size": "20971520", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_file_recover": "32", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_files": "64", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_ops": "8192", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_snaps_per_dir": "100", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_xattr_pairs_size": "65536", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_metrics_update_interval": "2", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_min_caps_per_client": "100", 2024-02-22T19:31:20.572 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_min_caps_working_set": "10000", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_numa_node": "-1", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_complaint_time": "30.000000", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_duration": "600", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_size": "20", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_log_threshold": "5", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_ping_grace": "15", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_ping_interval": "5", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_caps": "30000", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-02-22T19:31:20.573 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_decay_threshold": "131072", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_warning_threshold": "262144", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_reconnect_timeout": "45.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_replay_interval": "1.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_root_ino_gid": "0", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_root_ino_uid": "0", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_blocklist_on_evict": "true", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_blocklist_on_timeout": "true", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-02-22T19:31:20.574 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_metadata_threshold": "16777216", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_shutdown_check": "0", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_skip_ino": "0", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sleep_rank_change": "0.000000", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_max_uid": "4294967294", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_min_uid": "0", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_rstat": "false", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_standby_replay_damaged": "false", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_task_status_update_interval": "2.000000", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_thrash_exports": "0", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_thrash_fragments": "0", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_tick_interval": "5.000000", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_valgrind_exit": "false", 2024-02-22T19:31:20.575 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_verify_backtrace": "1", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_verify_scatter": "false", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_wipe_ino_prealloc": "false", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_wipe_sessions": "false", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool_debug": "false", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_debug_omit_block_device_write": "false", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_device_bytes": "1073741824", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_page_set": "false", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_page_size": "65536", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_bytes": "134217728", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_messages": "512", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_connect_retry_interval": "1.000000", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-02-22T19:31:20.576 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_disabled_modules": "", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_max_pg_num_change": "128", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mds_bytes": "134217728", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mds_messages": "128", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mon_bytes": "134217728", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mon_messages": "128", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_osd_bytes": "536870912", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_osd_messages": "8192", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_service_beacon_grace": "60.000000", 2024-02-22T19:31:20.577 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_standby_modules": "true", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_stats_period": "5", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_stats_threshold": "5", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_tick_period": "2", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_accept_timeout_factor": "2.000000", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_allow_pool_delete": "true", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_allow_pool_size_one": "false", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_auth_validate_all_caps": "true", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_bytes": "104857600", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_directed_command_retry": "2", 2024-02-22T19:31:20.578 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval": "3.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_parallel": "3", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_log_interval": "1.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_ping_interval": "10.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_ping_timeout": "30.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clock_drift_allowed": "1.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_file_level": "debug", 2024-02-22T19:31:20.579 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_file": "true", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog": "false", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_stderr": "true", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_bootstrap": "false", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_start": "false", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_trim": "true", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_con_tracker_persist_interval": "10", 2024-02-22T19:31:20.580 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_con_tracker_score_halflife": "43200", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_config_key_max_entry_size": "65536", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cpu_threads": "4", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_crush_min_required_version": "hammer", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_daemon_bytes": "419430400", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_avail_crit": "5", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_avail_warn": "5", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_size_warn": "16106127360", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_block_osdmap_trim": "false", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_json": "false", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_transactions": "false", 2024-02-22T19:31:20.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_extra_checks": "false", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_octopus": "false", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_pacific": "false", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_delta_reset_interval": "10.000000", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_dns_srv_name": "ceph-mon", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_down_mkfs_grace": "60", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_election_default_strategy": "1", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_election_timeout": "5.000000", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ping_divisor": "2", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ping_timeout": "2.000000", 2024-02-22T19:31:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_enable_op_tracker": "true", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_fake_pool_delete": "false", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_force_quorum_join": "false", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_globalid_prealloc": "10000", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_detail_to_clog": "true", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_log_update_period": "5", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_max_detail": "50", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog": "true", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog_interval": "600", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_host": "[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_host_override": "", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_initial_members": "", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-02-22T19:31:20.583 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_keyvaluedb": "rocksdb", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease": "5.000000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_max_summary": "50", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_log_entries_per_event": "4096", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_log_epochs": "500", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_mdsmap_epochs": "500", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_mgrmap_epochs": "500", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_osd": "10000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_pg_per_osd": "10000", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_pool_pg_num": "65536", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-02-22T19:31:20.584 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_force_trim_to": "0", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_skip_sanity": "false", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_memory_autotune": "true", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_memory_target": "2147483648", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_beacon_grace": "30", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_digest_period": "5", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_inactive_grace": "60", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_mkfs_grace": "240", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_min_osdmap_epochs": "50", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_complaint_time": "30", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_duration": "600", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_size": "20", 2024-02-22T19:31:20.585 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_slow_op_size": "20", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_slow_op_threshold": "10", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_log_threshold": "5", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_in": "false", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_new_in": "true", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_cache_size": "500", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_cache_size_min": "134217728", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_crush_smoke_test": "true", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_destroyed_out_interval": "600", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_down_out_interval": "600", 2024-02-22T19:31:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_force_trim_to": "0", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_full_ratio": "0.950000", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_halflife": "3600", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_max_interval": "300", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_weight": "0.300000", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_max_creating_pgs": "1024", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_max_initial_pgs": "1024", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_down_reporters": "2", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-02-22T19:31:20.587 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp": "true", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_report_timeout": "900", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_warn_num_repaired": "10", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_warn_op_age": "32.000000", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_interval": "2", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_min": "15", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_stuck_threshold": "60", 2024-02-22T19:31:20.588 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_objects": "10000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_per_osd": "0", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pool_quota_crit_threshold": "0", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pool_quota_warn_threshold": "0", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_probe_timeout": "2.000000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_max_change": "0.050000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_max_osds": "4", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_interval": "86400", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_max_keys": "100", 2024-02-22T19:31:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_timeout": "300", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_session_timeout": "300", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_smart_report_timeout": "5", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stat_smooth_intervals": "6", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_pool_min_size": "2", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_pool_size": "4", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_subscribe_interval": "86400.000000", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_debug": "false", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_max_payload_keys": "2000", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_max_payload_size": "1048576", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_provider_kill_at": "0", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_requester_kill_at": "0", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_timeout": "60.000000", 2024-02-22T19:31:20.590 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_target_pg_per_osd": "100", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_tick_interval": "5", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_timecheck_interval": "300.000000", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_older_version_delay": "604800", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_misplaced": "false", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-02-22T19:31:20.591 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_older_version": "true", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_no_app": "true", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_too_few_osds": "false", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "monmap": "", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_op_threads": "3", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_buffer_size": "131072", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_cm": "false", 2024-02-22T19:31:20.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_device_name": "", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_dscp": "96", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_gid_idx": "0", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_local_gid": "", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_polling_us": "1000", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_port_num": "1", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_roce_ver": "1", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_send_buffers": "1024", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_sl": "3", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_support_srq": "true", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_type": "ib", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_reap_threshold": "5", 2024-02-22T19:31:20.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_before_connect": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_ipv4": "true", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_ipv6": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_msgr1": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_msgr2": "true", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_port_max": "7300", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_port_min": "6800", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_prefer_ipv4": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_retry_count": "3", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_retry_delay": "5", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_client": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mds": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mgr": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mon": "false", 2024-02-22T19:31:20.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_osd": "false", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_client_mode": "crc secure", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_cluster_mode": "crc secure", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_cluster_type": "", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_connection_idle_timeout": "900", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_connection_ready_timeout": "10", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_crc_data": "true", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_crc_header": "true", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_bad_msg": "false", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_bug": "true", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_old_message": "true", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_skipped_message": "false", 2024-02-22T19:31:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_unhandled_msg": "false", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_coremask": "0xF", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hugepages": "", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hw_flow_control": "true", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_lro": "true", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_memory_channel": "4", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-02-22T19:31:20.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_pmd": "", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_port_id": "0", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dump_corrupt_message_level": "1", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dump_on_send": "false", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_initial_backoff": "0.200000", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_max": "1.000000", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_probability": "0.000000", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_type": "", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_internal_delays": "0.000000", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_socket_failures": "0", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_learn_addr_from_peer": "true", 2024-02-22T19:31:20.597 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_max_accept_failures": "4", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_max_backoff": "15.000000", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_client_mode": "secure crc", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_cluster_mode": "secure crc", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_service_mode": "secure crc", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_pq_min_cost": "65536", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_public_type": "", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_service_mode": "crc secure", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_listen_backlog": "512", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_nodelay": "true", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_rcvbuf": "0", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_type": "async+posix", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "no_config_file": "false", 2024-02-22T19:31:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_completion_locks_per_session": "32", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_debug_inject_relock_delay": "false", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inflight_op_bytes": "104857600", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inflight_ops": "1024", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inject_no_watch_ping": "false", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_tick_interval": "5.000000", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_timeout": "10.000000", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "objectstore_blackhole": "false", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "openssl_engine_opts": "", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_delay_time": "5.000000", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_hist_halflife": "1000", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_max_low_ops": "2", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_max_ops": "4", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-02-22T19:31:20.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_quantize_effort": "0.100000", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_slop": "0.020000", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_allow_recovery_below_min_size": "true", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_asio_thread_count": "2", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_async_recovery_min_cost": "100", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_auto_mark_unfound_lost": "false", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_retry_interval": "30.000000", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_scan_max": "512", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_scan_min": "64", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_degraded": "false", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_peering": "false", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_unfound": "true", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_beacon_report_interval": "300", 2024-02-22T19:31:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_duration": "30", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_max_block_size": "67108864", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_small_size_max_iops": "100", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_blkin_trace_all": "false", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_check_for_log_corruption": "false", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_update_on_start": "true", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_message_cap": "256", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_message_size_cap": "524288000", 2024-02-22T19:31:20.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_op_priority": "63", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_watch_timeout": "30", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_command_max_records": "256", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_command_thread_suicide_timeout": "900", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_command_thread_timeout": "600", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_compact_on_start": "false", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_chooseleaf_type": "1", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_initial_weight": "-1.000000", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_update_on_start": "true", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_update_weight_set": "true", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-02-22T19:31:20.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_drop_ping_duration": "0", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_feed_pullee": "-1", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_misdirected_ops": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_no_acting_change": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_no_purge_strays": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_op_order": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_pg_log_writeout": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_pretend_recovery_active": "false", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-02-22T19:31:20.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_shutdown": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_cached_snaps": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_missing_on_start": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_snaps": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_keys": "1024", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-02-22T19:31:20.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_stride": "524288", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_default_data_pool_replay_window": "45", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_default_notify_timeout": "30", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep": "0.000000", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_discard_disconnected_ops": "true", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_enable_op_tracker": "true", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_info": "true", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown": "true", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-02-22T19:31:20.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_function_tracing": "false", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_grace": "20", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_interval": "6", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_peers": "10", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_size": "2000", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_stale": "600", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_max_size": "100000", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_min_size": "1000", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-02-22T19:31:20.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal_flush_on_shutdown": "true", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal_size": "5120", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_kill_backfill_at": "0", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_loop_before_reset_tphandle": "64", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_cache_size": "50", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_dedup": "true", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_message_max": "40", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_message_max_bytes": "10485760", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_share_max_epochs": "40", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_attr_name_len": "100", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_attr_size": "0", 2024-02-22T19:31:20.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_backfills": "1", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_markdown_count": "5", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_markdown_period": "600", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_name_len": "2048", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_namespace_len": "256", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_size": "134217728", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_omap_entries_per_request": "1024", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_blocked_by": "16", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_log_entries": "10000", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pgls": "1024", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_push_cost": "8388608", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_push_objects": "10", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_scrubs": "1", 2024-02-22T19:31:20.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_trimming_pgs": "2", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_write_op_reply_len": "64", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_write_size": "90", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_cost_per_byte_usec": "0.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_cost_per_byte_usec_hdd": "5.200000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_cost_per_byte_usec_ssd": "0.011000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_cost_per_io_usec": "0.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_cost_per_io_usec_hdd": "25000.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_cost_per_io_usec_ssd": "50.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_profile": "high_client_ops", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-02-22T19:31:20.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_lim": "999999", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_res": "1", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_skip_benchmark": "false", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_base": "805306368", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_cache_min": "134217728", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target": "4294967296", 2024-02-22T19:31:20.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target_autotune": "false", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_min_pg_log_entries": "250", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_min_recovery_priority": "0", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_heartbeat_interval": "30", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_report_interval": "5", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_report_max_in_flight": "2", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_num_cache_shards": "32", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_num_op_tracker_shard": "32", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_auto_affinity": "true", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_node": "-1", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_prefer_iface": "true", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objecter_finishers": "1", 2024-02-22T19:31:20.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore": "bluestore", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore_fuse": "false", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore_tracing": "false", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_complaint_time": "30.000000", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_duration": "600", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_size": "20", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_slow_op_size": "20", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_log_threshold": "5", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards": "0", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards_hdd": "5", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards_ssd": "8", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard": "0", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-02-22T19:31:20.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_pq_min_cost": "65536", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_queue": "wpq", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_queue_cut_off": "high", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_thread_suicide_timeout": "150", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_thread_timeout": "15", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_open_classes_on_start": "true", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_os_flags": "0", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_peering_op_priority": "255", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_delete_cost": "1048576", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_delete_priority": "5", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_dups_tracked": "3000", 2024-02-22T19:31:20.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_trim_max": "10000", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_trim_min": "100", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_object_context_cache_count": "64", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_stat_report_interval_max": "500", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_crush_rule": "-1", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_ec_fast_read": "false", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_bulk": "false", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-02-22T19:31:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nodelete": "false", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flags": "0", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_min_size": "0", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pg_num": "32", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pgp_num": "0", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_size": "3", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_type": "replicated", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-02-22T19:31:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_use_gmt_hitset": "true", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_push_per_object_cost": "1000", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_read_ec_check_for_errors": "false", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recover_clone_overlap": "true", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recover_clone_overlap_limit": "10", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_cost": "20971520", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_delay_start": "0.000000", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active": "0", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active_hdd": "3", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active_ssd": "10", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_chunk": "8388608", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_single_start": "1", 2024-02-22T19:31:20.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_op_priority": "3", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_op_warn_multiple": "16", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_priority": "5", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_retry_interval": "30.000000", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep": "0.000000", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_repair_during_recovery": "false", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_requested_scrub_priority": "120", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_rollback_to_cluster_snap": "", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_auto_repair": "false", 2024-02-22T19:31:20.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_begin_hour": "0", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_begin_week_day": "0", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_chunk_max": "25", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_chunk_min": "5", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_cost": "52428800", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_during_recovery": "false", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_end_hour": "0", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_end_week_day": "0", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_invalid_stats": "true", 2024-02-22T19:31:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_load_threshold": "0.500000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_max_interval": "604800.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_max_preemptions": "5", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_min_interval": "86400.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_priority": "5", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_sleep": "0.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shutdown_pgref_assert": "false", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_skip_data_digest": "false", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_smart_report_timeout": "5", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_cost": "1048576", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_priority": "5", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep": "0.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-02-22T19:31:20.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_target_transaction_size": "30", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_mode": "writeback", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tracing": "false", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_use_stale_snap": "false", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osdc_blkin_trace_all": "false", 2024-02-22T19:31:20.620 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_kill_at": "0", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_max_join_drift": "10", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_min": "500", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_min_wait": "0.050000", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_propose_interval": "1.000000", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_max": "500", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_max_multiplier": "20", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_min": "10", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_stash_full_interval": "25", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_trim_max": "500", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_trim_min": "250", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "perf": "true", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "pid_file": "", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "-", 2024-02-22T19:31:20.621 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrv": "", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "public_bind_addr": "-", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "public_network": "", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "public_network_interface": "", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "qat_compressor_enabled": "false", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_mon_op_timeout": "0", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_osd_op_timeout": "0", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_tracing": "false", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_atime_update_interval": "60", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_balance_parent_reads": "false", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_balance_snap_reads": "false", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blkin_trace_all": "false", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blocklist_expire_seconds": "0", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blocklist_on_break_lock": "true", 2024-02-22T19:31:20.622 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache": "true", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_block_writes_upfront": "false", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty": "25165824", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty_object": "0", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_policy": "writearound", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_size": "33554432", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_target_dirty": "16777216", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_clone_copy_on_read": "false", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_compression_hint": "none", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_concurrent_management_ops": "10", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_clone_format": "auto", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_data_pool": "", 2024-02-22T19:31:20.623 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_features": "61", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_format": "2", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_map_options": "", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_order": "22", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_pool": "rbd", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_stripe_count": "0", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_stripe_unit": "0", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_disable_zero_copy_writes": "true", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_discard_granularity_bytes": "65536", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_enable_alloc_hint": "true", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-02-22T19:31:20.624 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_io_scheduler": "simple", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_commit_age": "5.000000", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_interval": "0", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_order": "24", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_pool": "", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_splay_width": "4", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_localize_parent_reads": "false", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_localize_snap_reads": "false", 2024-02-22T19:31:20.625 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_type": "simple", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_autotune": "true", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_base": "805306368", 2024-02-22T19:31:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_target": "4294967296", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_delete_delay": "0", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_replay_delay": "0", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_to_trash_on_remove": "false", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-02-22T19:31:20.627 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mtime_update_interval": "60", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_non_blocking_aio": "true", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_op_thread_timeout": "60", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_op_threads": "1", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_parent_cache_enabled": "false", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_mode": "disabled", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_path": "/tmp", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_size": "1073741824", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_plugins": "", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_burst": "0", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_limit": "0", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_burst": "0", 2024-02-22T19:31:20.628 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_limit": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_burst": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_limit": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_burst": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_limit": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_schedule_tick_min": "50", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_burst": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_limit": "0", 2024-02-22T19:31:20.629 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_burst": "0", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_limit": "0", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_quiesce_notification_attempts": "10", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_read_from_replica_policy": "default", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_max_bytes": "524288", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_trigger_requests": "10", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_request_timed_out_seconds": "30", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_skip_partial_discard": "true", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_tracing": "false", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_validate_names": "true", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_validate_pool": "true", 2024-02-22T19:31:20.630 INFO:teuthology.orchestra.run.smithi053.stdout: "restapi_base_url": "", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "restapi_log_level": "", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_acl_grants_max_num": "100", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_admin_entry": "admin", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_barbican_url": "", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_index_max_aio": "128", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_quota_ttl": "600", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_enabled": "true", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_expiry_interval": "900", 2024-02-22T19:31:20.631 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_lru_size": "10000", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_content_length_compat": "false", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_copy_obj_progress": "true", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cors_rules_max_num": "100", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cross_domain_policy": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_default_encryption_key": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_addr": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_ca_path": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_client_cert": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_client_key": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_password": "", 2024-02-22T19:31:20.632 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_username": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_require_ssl": "true", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_suppress_logs": "true", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_addr": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_auth": "token", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_namespace": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_prefix": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-02-22T19:31:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_token_file": "", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_buffersize": "524288", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_low_speed_limit": "1024", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_low_speed_time": "300", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_changes_size": "1000", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_num_shards": "128", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_window": "30", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_notify_interval_msec": "200", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_data_log_backing": "fifo", 2024-02-22T19:31:20.634 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_region_info_oid": "default.region", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_defer_to_bucket_acls": "", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_res": "500.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-02-22T19:31:20.635 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dns_name": "", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dns_s3website_name": "", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dynamic_resharding": "true", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_gc_threads": "true", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_lc_threads": "true", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_ops_log": "false", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_quota_threads": "true", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_static_website": "false", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_usage_log": "false", 2024-02-22T19:31:20.636 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enforce_swift_acls": "true", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_exit_timeout_secs": "120", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_expose_bucket": "false", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_extended_http_attrs": "", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_fcgi_socket_backlog": "1024", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_frontends": "beast port=7480", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_concurrent_io": "10", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_deferred": "50", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_objs": "32", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_queue_size": "134213632", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_trim_chunk": "16", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_obj_min_wait": "7200", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_processor_max_time": "3600", 2024-02-22T19:31:20.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_processor_period": "3600", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_get_obj_window_size": "16777216", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_healthcheck_disabling_path": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_host": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ignore_get_invalid_range": "false", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_init_timeout": "300", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_domain": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_password": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_password_path": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_project": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_tenant": "", 2024-02-22T19:31:20.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_token": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_token_path": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_user": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_api_version": "2", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_domain": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_password": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_project": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_tenant": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_user": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_implicit_tenants": "false", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_token_cache_size": "10000", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_url": "", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_verify_ssl": "true", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_debug_interval": "-1", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_lock_max_time": "90", 2024-02-22T19:31:20.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_objs": "32", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_rules": "1000", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_worker": "3", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_wp_worker": "3", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_thread_delay": "0", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_dnattr": "uid", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_searchfilter": "", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_uri": "ldaps://", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-02-22T19:31:20.640 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_http_headers": "", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_nonexistent_bucket": "false", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_object_name_utc": "false", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attr_name_len": "0", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attr_size": "0", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attrs_num_in_req": "0", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_chunk_size": "4194304", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_concurrent_requests": "1024", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_dynamic_shards": "1999", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_listing_results": "1000", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_notify_retries": "10", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_objs_per_shard": "100000", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_put_param_size": "1048576", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_put_size": "5368709120", 2024-02-22T19:31:20.641 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_slo_entries": "1000", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_md_log_max_shards": "64", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_md_notify_interval_msec": "200", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_mp_lock_max_time": "600", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multipart_min_part_size": "5242880", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_fhcache_size": "2017", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_lru_lanes": "5", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_max_gc": "300", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-02-22T19:31:20.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_gc_threads": "false", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_lc_threads": "false", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_quota_threads": "false", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_sync_thread": "false", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_async_rados_threads": "32", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_control_oids": "8", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_rados_handles": "1", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_numa_node": "-1", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_obj_stripe_size": "4194304", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_chunk_size": "100", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_gc_interval": "600", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_hints_num_shards": "127", 2024-02-22T19:31:20.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_thread_timeout": "600", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_tracing": "false", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_token": "", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_url": "", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_verify_ssl": "true", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_file_path": "", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_rados": "true", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_socket_path": "", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-02-22T19:31:20.644 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_push_interval": "2.000000", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_push_interval_max": "30.000000", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_root_pool": ".rgw.root", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_port": "", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_print_continue": "true", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_print_prohibited_content_length": "false", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_tracing": "false", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm": "", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_region": "", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_region_root_pool": ".rgw.root", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_relaxed_region_enforcement": "false", 2024-02-22T19:31:20.645 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_request_uri": "", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_batch_size": "64", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_max_aio": "128", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_num_logs": "16", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_thread_interval": "600", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_resolve_cname": "false", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rest_getusage_op_compat": "false", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_run_sync_thread": "true", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_keystone": "false", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_ldap": "false", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_rados": "true", 2024-02-22T19:31:20.646 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_sts": "false", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_success_create_obj_status": "0", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_scheduler_type": "throttler", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_script_uri": "", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_service_provider_name": "", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_socket_path": "", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_client_id": "", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_client_secret": "", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_entry": "sts", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_key": "sts", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_max_session_duration": "43200", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_min_session_duration": "900", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_token_introspection_url": "", 2024-02-22T19:31:20.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_account_in_url": "false", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_auth_entry": "auth", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_auth_url": "", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_custom_header": "", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_enforce_content_length": "false", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_need_stats": "true", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_tenant_name": "", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_token_expiration": "86400", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_url": "", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_url_prefix": "swift", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_versioning_enabled": "false", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_lease_period": "120", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-02-22T19:31:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_interval": "1200", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_obj_etag_verify": "false", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_history_size": "4096", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_thread_pool_size": "512", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_comment": "", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_createby": "", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_encoding": "", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_flag": "false", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_origin": "", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_sha_unit": "524288", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_tracker": "", 2024-02-22T19:31:20.649 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_trust_forwarded_https": "false", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_log_tick_interval": "30", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_max_shards": "32", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_max_user_shards": "1", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_use_opa_authz": "false", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_default_quota_max_size": "-1", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_max_buckets": "1000", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_interval": "86400", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_unique_email": "true", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_verify_ssl": "true", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_website_routing_rules_max_num": "50", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone": "", 2024-02-22T19:31:20.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup": "", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_block_size": "4096", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_shard_bits": "4", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_size": "536870912", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_type": "binned_lru", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion": "false", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-02-22T19:31:20.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_compaction_stats": "false", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_extended_stats": "false", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_memory_stats": "false", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_index_type": "binary_search", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_log_to_ceph_log": "true", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_metadata_block_size": "4096", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_partition_filters": "false", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_perf": "false", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rotating_keys_renewal_timeout": "10", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "run_dir": "/var/run/ceph", 2024-02-22T19:31:20.652 INFO:teuthology.orchestra.run.smithi053.stdout: "setgroup": "ceph", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "setuser": "ceph", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "setuser_match_path": "", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_misplaced_ratio": "0.050000", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "thp": "false", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "threadpool_default_timeout": "60", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "threadpool_empty_queue_max_wait": "2", 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout: "throttler_perf_counter": "true" 2024-02-22T19:31:20.653 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:31:20.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:19.207933+0000 mgr.y (mgr.14535) 97 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:21.242 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-02-22T19:31:21.522 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:21.208418+0000 mgr.y (mgr.14535) 98 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:22.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:21.208418+0000 mgr.y (mgr.14535) 98 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:22.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:21.208418+0000 mgr.y (mgr.14535) 98 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a: {} 2024-02-22T19:31:23.279 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b: {} 2024-02-22T19:31:23.288 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c: {} 2024-02-22T19:31:23.289 INFO:teuthology.orchestra.run.smithi053.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-22T19:31:23.289 INFO:teuthology.orchestra.run.smithi053.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-22T19:31:23.290 INFO:teuthology.orchestra.run.smithi053.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-22T19:31:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:23.259543+0000 mon.c (mon.2) 30 : audit [DBG] from='client.? 172.21.15.53:0/1892148167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:31:23.732 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:23.259543+0000 mon.c (mon.2) 30 : audit [DBG] from='client.? 172.21.15.53:0/1892148167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:31:23.732 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:23.259543+0000 mon.c (mon.2) 30 : audit [DBG] from='client.? 172.21.15.53:0/1892148167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:31:24.063 INFO:teuthology.run_tasks:Running task full_sequential... 2024-02-22T19:31:24.074 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-02-22T19:31:24.075 INFO:tasks.radosbench:Beginning radosbench... 2024-02-22T19:31:24.076 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-02-22T19:31:24.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool create unique_pool_0 16 2024-02-22T19:31:24.076 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-02-22T19:31:24.076 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:31:24.076 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:31:24.077 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-02-22T19:31:24.077 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:31:24.077 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:31:24.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-02-22T19:31:24.077 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-02-22T19:31:24.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:31:24.078 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-02-22T19:31:24.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:31:24.079 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-02-22T19:31:24.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:31:24.079 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-02-22T19:31:24.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:31:24.122 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:31:24.162 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:24.162 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:31:24.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:31:24.290 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:24.314 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:31:24.317 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:24.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:31:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:31:24.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:24.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:31:24.444 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:23.006080+0000 osd.7 (osd.7) 47 : cluster [DBG] 3.15 deep-scrub starts 2024-02-22T19:31:24.444 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:23.009017+0000 osd.7 (osd.7) 48 : cluster [DBG] 3.15 deep-scrub ok 2024-02-22T19:31:24.444 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:23.208850+0000 mgr.y (mgr.14535) 99 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:24.444 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:23.681566+0000 osd.5 (osd.5) 45 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:31:24.444 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:23.683216+0000 osd.5 (osd.5) 46 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:31:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:24.644 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:24.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:24.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:23.006080+0000 osd.7 (osd.7) 47 : cluster [DBG] 3.15 deep-scrub starts 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:23.009017+0000 osd.7 (osd.7) 48 : cluster [DBG] 3.15 deep-scrub ok 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:23.208850+0000 mgr.y (mgr.14535) 99 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:23.681566+0000 osd.5 (osd.5) 45 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:23.683216+0000 osd.5 (osd.5) 46 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:23.006080+0000 osd.7 (osd.7) 47 : cluster [DBG] 3.15 deep-scrub starts 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:23.009017+0000 osd.7 (osd.7) 48 : cluster [DBG] 3.15 deep-scrub ok 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:23.208850+0000 mgr.y (mgr.14535) 99 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:24.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:23.681566+0000 osd.5 (osd.5) 45 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:31:24.721 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:23.683216+0000 osd.5 (osd.5) 46 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:31:24.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:31:24.775 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:31:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:31:24.777 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:24.850 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:31:24.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:24.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:31:24.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:31:24.982 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:25.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:24.303367+0000 mgr.y (mgr.14535) 100 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:25.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:24.505018+0000 osd.3 (osd.3) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:25.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:24.505067+0000 osd.3 (osd.3) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:25.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:24.303367+0000 mgr.y (mgr.14535) 100 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:25.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:24.505018+0000 osd.3 (osd.3) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:25.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:24.505067+0000 osd.3 (osd.3) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:24.303367+0000 mgr.y (mgr.14535) 100 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:24.505018+0000 osd.3 (osd.3) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:24.505067+0000 osd.3 (osd.3) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:31:25.934 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:25.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:31:26.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:26.067 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:26.173 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:26.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:26.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:26.189 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:31:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-22T19:31:26.189 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:31:26.277 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:26.310 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:26.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:25.209681+0000 mgr.y (mgr.14535) 101 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:31:26.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:25.303889+0000 mon.a (mon.0) 750 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-22T19:31:26.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:25.681670+0000 osd.5 (osd.5) 47 : cluster [DBG] 6.d scrub starts 2024-02-22T19:31:26.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:25.682644+0000 osd.5 (osd.5) 48 : cluster [DBG] 6.d scrub ok 2024-02-22T19:31:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:25.209681+0000 mgr.y (mgr.14535) 101 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:31:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:25.303889+0000 mon.a (mon.0) 750 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-22T19:31:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:25.681670+0000 osd.5 (osd.5) 47 : cluster [DBG] 6.d scrub starts 2024-02-22T19:31:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:25.682644+0000 osd.5 (osd.5) 48 : cluster [DBG] 6.d scrub ok 2024-02-22T19:31:26.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:25.209681+0000 mgr.y (mgr.14535) 101 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-22T19:31:26.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:25.303889+0000 mon.a (mon.0) 750 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-22T19:31:26.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:25.681670+0000 osd.5 (osd.5) 47 : cluster [DBG] 6.d scrub starts 2024-02-22T19:31:26.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:25.682644+0000 osd.5 (osd.5) 48 : cluster [DBG] 6.d scrub ok 2024-02-22T19:31:27.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:26.187725+0000 osd.4 (osd.4) 51 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:31:27.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:26.190622+0000 osd.4 (osd.4) 52 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:31:27.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:26.453152+0000 osd.0 (osd.0) 53 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:31:27.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:26.454149+0000 osd.0 (osd.0) 54 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:31:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:26.187725+0000 osd.4 (osd.4) 51 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:31:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:26.190622+0000 osd.4 (osd.4) 52 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:31:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:26.453152+0000 osd.0 (osd.0) 53 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:31:27.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:26.454149+0000 osd.0 (osd.0) 54 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:31:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:26.187725+0000 osd.4 (osd.4) 51 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:31:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:26.190622+0000 osd.4 (osd.4) 52 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:31:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:26.453152+0000 osd.0 (osd.0) 53 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:31:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:26.454149+0000 osd.0 (osd.0) 54 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:31:27.661 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:27.664 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:27.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:27.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:31:27.803 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:31:28.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:28.009 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:31:28.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:28.140 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:28.338 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:28.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:28.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:28.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:31:28.478 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:28.500 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:31:28.502 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:28.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:31:28.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:28.634 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:28.709 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:28.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:28.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:27.210253+0000 mgr.y (mgr.14535) 102 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:31:29.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:27.210253+0000 mgr.y (mgr.14535) 102 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:31:29.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:27.210253+0000 mgr.y (mgr.14535) 102 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:31:29.470 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:29] "GET /metrics HTTP/1.1" 200 172109 "" "Prometheus/2.33.4" 2024-02-22T19:31:29.624 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:31:29.641 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:31:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:31:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-02-22T19:31:29.641 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:31:29.745 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:28.886286+0000 mon.a (mon.0) 751 : audit [INF] from='client.? 172.21.15.53:0/1249271429' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:31:29.745 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:28.997609+0000 osd.7 (osd.7) 49 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:31:29.745 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:29.001092+0000 osd.7 (osd.7) 50 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:31:29.745 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:28.886286+0000 mon.a (mon.0) 751 : audit [INF] from='client.? 172.21.15.53:0/1249271429' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:31:29.745 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:28.997609+0000 osd.7 (osd.7) 49 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:31:29.745 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:29.001092+0000 osd.7 (osd.7) 50 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:31:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:28.886286+0000 mon.a (mon.0) 751 : audit [INF] from='client.? 172.21.15.53:0/1249271429' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:31:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:28.997609+0000 osd.7 (osd.7) 49 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:31:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:29.001092+0000 osd.7 (osd.7) 50 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:31:30.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:31:30.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:29.210791+0000 mgr.y (mgr.14535) 103 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:30.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:29.613640+0000 mon.a (mon.0) 752 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:30.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:29.620901+0000 mon.a (mon.0) 753 : audit [INF] from='client.? 172.21.15.53:0/1249271429' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:31:30.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:29.620949+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-22T19:31:30.728 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:29.677351+0000 osd.5 (osd.5) 49 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:29.679178+0000 osd.5 (osd.5) 50 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:29.210791+0000 mgr.y (mgr.14535) 103 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:29.613640+0000 mon.a (mon.0) 752 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:29.620901+0000 mon.a (mon.0) 753 : audit [INF] from='client.? 172.21.15.53:0/1249271429' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:29.620949+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:29.677351+0000 osd.5 (osd.5) 49 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:31:30.729 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:29.679178+0000 osd.5 (osd.5) 50 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:31:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:29.210791+0000 mgr.y (mgr.14535) 103 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:31:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:29.613640+0000 mon.a (mon.0) 752 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:29.620901+0000 mon.a (mon.0) 753 : audit [INF] from='client.? 172.21.15.53:0/1249271429' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:31:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:29.620949+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-22T19:31:31.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:29.677351+0000 osd.5 (osd.5) 49 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:31:31.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:29.679178+0000 osd.5 (osd.5) 50 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:31:31.823 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:31:31.823 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:31:31.823 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-02-22T19:31:31.823 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:31:31.823 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:31:31.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-02-22T19:31:31.946 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:31.949 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:31.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:32.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:31:32.058 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_0' created 2024-02-22T19:31:32.081 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:31:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:31:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:31.233694+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.53:0/2240136377' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:32.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:31.233694+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.53:0/2240136377' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:31.233694+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.53:0/2240136377' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:32.731 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:31:32.734 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:32.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:31:32.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:31:32.864 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:33.137 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:31.211435+0000 mgr.y (mgr.14535) 104 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:31.513215+0000 mon.a (mon.0) 755 : audit [INF] from='client.? 172.21.15.53:0/672376293' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:32.002547+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.53:0/672376293' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:32.002616+0000 mon.a (mon.0) 757 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:31.211435+0000 mgr.y (mgr.14535) 104 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:31.513215+0000 mon.a (mon.0) 755 : audit [INF] from='client.? 172.21.15.53:0/672376293' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:32.002547+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.53:0/672376293' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-02-22T19:31:33.138 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:32.002616+0000 mon.a (mon.0) 757 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-22T19:31:33.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-02-22T19:31:33.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:31.211435+0000 mgr.y (mgr.14535) 104 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:31:33.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:31.513215+0000 mon.a (mon.0) 755 : audit [INF] from='client.? 172.21.15.53:0/672376293' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-02-22T19:31:33.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:32.002547+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.53:0/672376293' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-02-22T19:31:33.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:32.002616+0000 mon.a (mon.0) 757 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-22T19:31:33.492 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:33.513 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:31:33.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:33.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:31:33.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:31:33.649 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:33.722 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:31:33.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:33.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:31:33.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:31:33.836 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:31:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-22T19:31:33.836 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:31:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:31:33.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:33.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:31:33.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:31:33.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:34.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:33.003140+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-22T19:31:34.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:33.211759+0000 mgr.y (mgr.14535) 105 : cluster [DBG] pgmap v97: 148 pgs: 16 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:31:34.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:33.003140+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-22T19:31:34.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:33.211759+0000 mgr.y (mgr.14535) 105 : cluster [DBG] pgmap v97: 148 pgs: 16 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:31:34.029 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:34.237 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:33.003140+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-22T19:31:34.237 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:33.211759+0000 mgr.y (mgr.14535) 105 : cluster [DBG] pgmap v97: 148 pgs: 16 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:31:34.283 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:34.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:34.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:34.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:34.418 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:34.517 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:31:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:31:34.532 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:34.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:34.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:34.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:31:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:34.695 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:31:34.699 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:34.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:31:34.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:34.827 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:34.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:31:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:31:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:34.010924+0000 mon.a (mon.0) 759 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-22T19:31:35.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:34.314204+0000 mgr.y (mgr.14535) 106 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:34.010924+0000 mon.a (mon.0) 759 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-22T19:31:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:34.314204+0000 mgr.y (mgr.14535) 106 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:34.010924+0000 mon.a (mon.0) 759 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-22T19:31:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:34.314204+0000 mgr.y (mgr.14535) 106 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:35.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:31:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:35.212661+0000 mgr.y (mgr.14535) 107 : cluster [DBG] pgmap v99: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-02-22T19:31:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:35.212661+0000 mgr.y (mgr.14535) 107 : cluster [DBG] pgmap v99: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-02-22T19:31:36.306 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:36.309 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:36.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:35.212661+0000 mgr.y (mgr.14535) 107 : cluster [DBG] pgmap v99: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-02-22T19:31:36.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:37.858 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:31:37.858 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:31:38.278 INFO:teuthology.orchestra.run.smithi053.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-02-22T19:31:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:37.212994+0000 mgr.y (mgr.14535) 108 : cluster [DBG] pgmap v100: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:31:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:37.473181+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.53:0/1864142213' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:31:38.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:37.473501+0000 mon.a (mon.0) 760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:31:38.690 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:38.692 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:37.212994+0000 mgr.y (mgr.14535) 108 : cluster [DBG] pgmap v100: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:31:38.692 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:37.473181+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.53:0/1864142213' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:31:38.692 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:37.473501+0000 mon.a (mon.0) 760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:31:38.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:37.212994+0000 mgr.y (mgr.14535) 108 : cluster [DBG] pgmap v100: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:31:38.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:37.473181+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.53:0/1864142213' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:31:38.693 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:37.473501+0000 mon.a (mon.0) 760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:31:38.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:38.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:38.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:31:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:38.899 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:31:38.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:38.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:31:39.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:39.037 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:39.300 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:38.097689+0000 osd.4 (osd.4) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:39.300 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:38.097750+0000 osd.4 (osd.4) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:39.300 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:38.273707+0000 mon.a (mon.0) 761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:38.273814+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:38.332915+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:38.537446+0000 mon.a (mon.0) 764 : audit [INF] from='client.? 172.21.15.53:0/2791570139' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:38.097689+0000 osd.4 (osd.4) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:38.097750+0000 osd.4 (osd.4) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:38.273707+0000 mon.a (mon.0) 761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:38.273814+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:38.332915+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:39.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:38.537446+0000 mon.a (mon.0) 764 : audit [INF] from='client.? 172.21.15.53:0/2791570139' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:31:39.342 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:31:39.537 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:31:39.538 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:31:39.538 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:31:39.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 5 2024-02-22T19:31:39.563 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:39] "GET /metrics HTTP/1.1" 200 174598 "" "Prometheus/2.33.4" 2024-02-22T19:31:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:38.097689+0000 osd.4 (osd.4) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:38.097750+0000 osd.4 (osd.4) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:38.273707+0000 mon.a (mon.0) 761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:31:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:38.273814+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-22T19:31:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:38.332915+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:38.537446+0000 mon.a (mon.0) 764 : audit [INF] from='client.? 172.21.15.53:0/2791570139' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:31:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:39.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:39.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:39.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:31:39.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:39.820 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:39.918 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:39.918 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:39.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:39.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:40.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:40.048 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:40.176 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:40.181 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:40.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:40.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:31:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:31:40.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:40.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:31:40.358 DEBUG:teuthology.orchestra.run.smithi053:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-02-22T19:31:40.361 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-02-22T19:31:40.361 INFO:teuthology.orchestra.run:waiting for 3000 2024-02-22T19:31:40.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:40.475 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:40.496 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:40.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:40.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:39.213846+0000 mgr.y (mgr.14535) 109 : cluster [DBG] pgmap v102: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-22T19:31:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:39.331501+0000 mon.a (mon.0) 765 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:40.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:39.338396+0000 mon.a (mon.0) 766 : audit [INF] from='client.? 172.21.15.53:0/2791570139' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:31:40.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:39.338458+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-22T19:31:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:39.213846+0000 mgr.y (mgr.14535) 109 : cluster [DBG] pgmap v102: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-22T19:31:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:39.331501+0000 mon.a (mon.0) 765 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:39.338396+0000 mon.a (mon.0) 766 : audit [INF] from='client.? 172.21.15.53:0/2791570139' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:31:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:39.338458+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-22T19:31:40.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:39.213846+0000 mgr.y (mgr.14535) 109 : cluster [DBG] pgmap v102: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-22T19:31:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:39.331501+0000 mon.a (mon.0) 765 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:39.338396+0000 mon.a (mon.0) 766 : audit [INF] from='client.? 172.21.15.53:0/2791570139' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:31:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:39.338458+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-22T19:31:40.631 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:40.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:40.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:40.657 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:hints = 1 2024-02-22T19:31:40.657 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-02-22T19:31:40.657 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object prefix: benchmark_data_smithi053_107660 2024-02-22T19:31:40.658 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:31:40.658 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 0 6 6 0 0 0 - 0 2024-02-22T19:31:40.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:31:40.796 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:31:40.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:40.828 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:31:40.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:31:40.968 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:31:40.998 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:40.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:31:41.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:31:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:31:41.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:41.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:31:41.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:41.317 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:41.658 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 1 16 3250 3234 201.986 202.125 0.00435288 0.00493335 2024-02-22T19:31:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:41.688 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:41.689 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:41.689 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:41.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:31:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:41.214260+0000 mgr.y (mgr.14535) 110 : cluster [DBG] pgmap v104: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:31:42.658 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 2 16 6077 6061 189.334 176.688 0.00635987 0.00526901 2024-02-22T19:31:42.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:31:42.722 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:41.214260+0000 mgr.y (mgr.14535) 110 : cluster [DBG] pgmap v104: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:31:42.725 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:41.214260+0000 mgr.y (mgr.14535) 110 : cluster [DBG] pgmap v104: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:31:43.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:42.351068+0000 osd.2 (osd.2) 31 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:43.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:42.351122+0000 osd.2 (osd.2) 32 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:43.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:42.351068+0000 osd.2 (osd.2) 31 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:43.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:42.351122+0000 osd.2 (osd.2) 32 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:43.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:42.351068+0000 osd.2 (osd.2) 31 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:31:43.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:42.351122+0000 osd.2 (osd.2) 32 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:31:43.659 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 3 16 8814 8798 183.218 171.062 0.0102121 0.00544702 2024-02-22T19:31:44.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:31:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:31:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:43.214737+0000 mgr.y (mgr.14535) 111 : cluster [DBG] pgmap v105: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:43.296558+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-22T19:31:44.660 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 4 16 11408 11392 177.904 162.125 0.00512631 0.00561258 2024-02-22T19:31:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:31:44.717 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:43.214737+0000 mgr.y (mgr.14535) 111 : cluster [DBG] pgmap v105: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:44.717 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:43.296558+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-22T19:31:44.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:43.214737+0000 mgr.y (mgr.14535) 111 : cluster [DBG] pgmap v105: 148 pgs: 148 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:31:44.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:43.296558+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-22T19:31:44.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:44.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:31:44.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:31:44.868 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:44.897 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:31:44.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:44.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:31:45.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:45.050 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:31:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:31:45.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:44.324348+0000 mgr.y (mgr.14535) 112 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:44.324348+0000 mgr.y (mgr.14535) 112 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:45.660 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 5 16 13931 13915 173.857 157.688 0.00670802 0.0057441 2024-02-22T19:31:45.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:44.324348+0000 mgr.y (mgr.14535) 112 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:45.215594+0000 mgr.y (mgr.14535) 113 : cluster [DBG] pgmap v107: 148 pgs: 148 active+clean; 640 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:31:46.660 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 6 15 16133 16118 167.828 137.688 0.00427769 0.0059515 2024-02-22T19:31:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:45.215594+0000 mgr.y (mgr.14535) 113 : cluster [DBG] pgmap v107: 148 pgs: 148 active+clean; 640 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:31:46.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:45.215594+0000 mgr.y (mgr.14535) 113 : cluster [DBG] pgmap v107: 148 pgs: 148 active+clean; 640 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:31:47.424 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:47.426 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:47.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:47.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:47.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:47.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:47.660 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 7 16 18319 18303 163.36 136.562 0.0069379 0.00611461 2024-02-22T19:31:47.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:47.729 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:47.758 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:47.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:47.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:31:47.915 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:47.932 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:31:47.948 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:31:47.952 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:31:47.968 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:47.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:31:47.980 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:31:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:31:48.048 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:31:48.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:48.082 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:31:48.115 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:31:48.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:47.216200+0000 mgr.y (mgr.14535) 114 : cluster [DBG] pgmap v108: 148 pgs: 148 active+clean; 640 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-02-22T19:31:48.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:47.863867+0000 mon.c (mon.2) 33 : audit [DBG] from='client.? 172.21.15.53:0/3908577378' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:48.129 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:47.216200+0000 mgr.y (mgr.14535) 114 : cluster [DBG] pgmap v108: 148 pgs: 148 active+clean; 640 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-02-22T19:31:48.129 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:47.863867+0000 mon.c (mon.2) 33 : audit [DBG] from='client.? 172.21.15.53:0/3908577378' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:48.130 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:48.154 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:31:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:31:48.226 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:48.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:48.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:48.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:47.216200+0000 mgr.y (mgr.14535) 114 : cluster [DBG] pgmap v108: 148 pgs: 148 active+clean; 640 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-02-22T19:31:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:47.863867+0000 mon.c (mon.2) 33 : audit [DBG] from='client.? 172.21.15.53:0/3908577378' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:48.386 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:48.455 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:48.462 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:48.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:48.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:31:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:48.663 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 8 16 20102 20086 156.827 111.438 0.00540073 0.00636933 2024-02-22T19:31:48.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:31:48.607+0000 7fe2d794e700 -1 mon.a@0(leader).osd e98 definitely_dead 0 2024-02-22T19:31:49.007 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.5. 2024-02-22T19:31:49.046 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:48.607733+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.53:0/3079546430' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:31:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:48.608204+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:31:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:48.608294+0000 mon.a (mon.0) 770 : cluster [INF] Client client.admin marked osd.5 out, while it was still marked up 2024-02-22T19:31:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:48.607733+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.53:0/3079546430' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:31:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:48.608204+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:31:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:48.608294+0000 mon.a (mon.0) 770 : cluster [INF] Client client.admin marked osd.5 out, while it was still marked up 2024-02-22T19:31:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:48.607733+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.53:0/3079546430' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:31:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:48.608204+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:31:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:48.608294+0000 mon.a (mon.0) 770 : cluster [INF] Client client.admin marked osd.5 out, while it was still marked up 2024-02-22T19:31:49.587 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:49] "GET /metrics HTTP/1.1" 200 174598 "" "Prometheus/2.33.4" 2024-02-22T19:31:49.662 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 9 16 20889 20873 144.871 49.1875 0.00248643 0.00640649 2024-02-22T19:31:50.176 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:48.983426+0000 mon.a (mon.0) 771 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["5"]}]': finished 2024-02-22T19:31:50.176 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:48.983478+0000 mon.a (mon.0) 772 : cluster [DBG] osdmap e99: 8 total, 8 up, 7 in 2024-02-22T19:31:50.176 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:49.216985+0000 mgr.y (mgr.14535) 115 : cluster [DBG] pgmap v110: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 511 B/s rd, 163 MiB/s wr, 2.61k op/s 2024-02-22T19:31:50.176 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:48.983426+0000 mon.a (mon.0) 771 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["5"]}]': finished 2024-02-22T19:31:50.176 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:48.983478+0000 mon.a (mon.0) 772 : cluster [DBG] osdmap e99: 8 total, 8 up, 7 in 2024-02-22T19:31:50.176 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:49.216985+0000 mgr.y (mgr.14535) 115 : cluster [DBG] pgmap v110: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 511 B/s rd, 163 MiB/s wr, 2.61k op/s 2024-02-22T19:31:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:48.983426+0000 mon.a (mon.0) 771 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["5"]}]': finished 2024-02-22T19:31:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:48.983478+0000 mon.a (mon.0) 772 : cluster [DBG] osdmap e99: 8 total, 8 up, 7 in 2024-02-22T19:31:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:49.216985+0000 mgr.y (mgr.14535) 115 : cluster [DBG] pgmap v110: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 511 B/s rd, 163 MiB/s wr, 2.61k op/s 2024-02-22T19:31:50.663 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 10 16 20889 20873 130.39 0 - 0.00640649 2024-02-22T19:31:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:31:50.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:50.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:31:50.779 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:50.809 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:51.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:31:51.233 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:18825 7.9 7:90040ba0:::benchmark_data_smithi053_107660_object18824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:47.937661+0000", 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.5533081009999998, 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014177323, 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18825 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937661+0000", 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937661+0000", 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937659+0000", 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5950000000000001e-05 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937685+0000", 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1545e-05 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937697+0000", 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8810000000000004e-06 2024-02-22T19:31:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937701+0000", 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9265999999999997e-05 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937771+0000", 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016851600000000001 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.937939+0000", 2024-02-22T19:31:51.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025449199999999999 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.938194+0000", 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020822449999999999 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.940276+0000", 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6657000000000006e-05 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.940373+0000", 2024-02-22T19:31:51.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011370651000000001 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.951743+0000", 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9020000000000002e-05 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.951792+0000", 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5633000000000002e-05 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.951838+0000", 2024-02-22T19:31:51.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014177323 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:18862 7.6 7:6b7c3816:::benchmark_data_smithi053_107660_object18861:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:47.959887+0000", 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.5310815899999999, 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016477602000000001, 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18862 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.959887+0000", 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.959887+0000", 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.959887+0000", 2024-02-22T19:31:51.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4344e-05 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.959901+0000", 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4829999999999999e-06 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.959905+0000", 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1100000000000001e-06 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.959907+0000", 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002330225 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.962237+0000", 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013833000000000001 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.962375+0000", 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016571700000000001 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:31:51.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.962541+0000", 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019135599999999999 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.964455+0000", 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094943999999999996 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.965404+0000", 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010885695000000001 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.976290+0000", 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8101e-05 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.976328+0000", 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7015999999999999e-05 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:47.976365+0000", 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016477602000000001 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19336 7.f 7:f24c6360:::benchmark_data_smithi053_107660_object19335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.242049+0000", 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.2489192679999999, 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017962116, 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19336 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242049+0000", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242049+0000", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242048+0000", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8617000000000001e-05 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242076+0000", 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3555999999999999e-05 2024-02-22T19:31:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242100+0000", 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.443e-06 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242106+0000", 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6490999999999999e-05 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242153+0000", 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014332400000000001 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242296+0000", 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016319699999999999 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.242459+0000", 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089621500000000001 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.243356+0000", 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001153158 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.244509+0000", 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015436055000000001 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.259945+0000", 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2663000000000001e-05 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.259988+0000", 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.406e-05 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.260012+0000", 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017962116 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:19397 7.7 e98/93 7:e11aad75:::benchmark_data_smithi053_107660_object19396:head v 98'1219, mlcod=98'1217)", 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.281873+0000", 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.209095558, 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015440837000000001, 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.281873+0000", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.281873+0000", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.281873+0000", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090866199999999997 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.282781+0000", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9639999999999998e-06 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.282791+0000", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0850000000000002e-06 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.282797+0000", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060147680000000002 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.288812+0000", 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7493e-05 2024-02-22T19:31:51.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.288830+0000", 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0084598039999999996 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.297289+0000", 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4615000000000001e-05 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.297314+0000", 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015440837000000001 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19408 7.6 7:6b5fa1ad:::benchmark_data_smithi053_107660_object19407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.290502+0000", 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.2004666359999998, 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016508668000000001, 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19408 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290502+0000", 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290502+0000", 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290501+0000", 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2932999999999999e-05 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290524+0000", 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0855e-05 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290534+0000", 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290542+0000", 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041764719999999996 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.294718+0000", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012286999999999999 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.294841+0000", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013992099999999999 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.294981+0000", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047740730000000002 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.299755+0000", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052908299999999998 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.305046+0000", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018875560000000001 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.306933+0000", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2262999999999998e-05 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.306976+0000", 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5045000000000001e-05 2024-02-22T19:31:51.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.307011+0000", 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016508668000000001 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19415 7.6 7:6b4ee7ee:::benchmark_data_smithi053_107660_object19414:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.290591+0000", 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.200377338, 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016524647999999999, 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19415 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290591+0000", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290591+0000", 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290591+0000", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8623000000000001e-05 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290609+0000", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3670000000000002e-06 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290616+0000", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1760000000000003e-06 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.290620+0000", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005079961 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.295700+0000", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1383000000000004e-05 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.295771+0000", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100987 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.295872+0000", 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039173150000000002 2024-02-22T19:31:51.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.299790+0000", 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052616709999999999 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.305051+0000", 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019914199999999998 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.307043+0000", 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4607999999999998e-05 2024-02-22T19:31:51.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.307077+0000", 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8760999999999997e-05 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.307116+0000", 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016524647999999999 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:19413 7.e e98/93 7:76049252:::benchmark_data_smithi053_107660_object19412:head v 98'1174, mlcod=98'1172)", 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.291197+0000", 2024-02-22T19:31:51.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.199772249, 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013839018, 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.291197+0000", 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.291197+0000", 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.291196+0000", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.639e-05 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.291222+0000", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7799999999999995e-06 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.291232+0000", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7129999999999999e-06 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.291239+0000", 2024-02-22T19:31:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002742207 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.293981+0000", 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9326000000000001e-05 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.294000+0000", 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011003693 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.305004+0000", 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1769e-05 2024-02-22T19:31:51.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.305036+0000", 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013839018 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19594 7.9 7:92c1dedd:::benchmark_data_smithi053_107660_object19593:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.393316+0000", 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.0976523340000002, 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014570296, 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19594 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393316+0000", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393316+0000", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393315+0000", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5602999999999999e-05 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393340+0000", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3553000000000001e-05 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393354+0000", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1580000000000004e-06 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393359+0000", 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2688000000000002e-05 2024-02-22T19:31:51.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393402+0000", 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015368399999999999 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393556+0000", 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000185682 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393741+0000", 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029007460000000001 2024-02-22T19:31:51.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.396642+0000", 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018003959999999999 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.398442+0000", 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.009354523 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.407797+0000", 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6811999999999998e-05 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.407844+0000", 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2978000000000001e-05 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.407887+0000", 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014570296 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19595 7.9 7:97b6972b:::benchmark_data_smithi053_107660_object19594:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.393373+0000", 2024-02-22T19:31:51.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.0975961120000002, 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014645276, 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19595 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393373+0000", 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393373+0000", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393372+0000", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1591e-05 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393393+0000", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8789999999999997e-06 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393400+0000", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1300000000000003e-06 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393404+0000", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001146532 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.394551+0000", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112528 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.394663+0000", 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148307 2024-02-22T19:31:51.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.394812+0000", 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003622517 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.398434+0000", 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6767999999999999e-05 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.398451+0000", 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0094893459999999992 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.407940+0000", 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9614999999999999e-05 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.407980+0000", 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7867999999999997e-05 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408018+0000", 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014645276 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19597 7.9 7:9244af62:::benchmark_data_smithi053_107660_object19596:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.393842+0000", 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.0971267330000001, 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014281719, 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19597 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393842+0000", 2024-02-22T19:31:51.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393842+0000", 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393841+0000", 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3031e-05 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393864+0000", 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9190000000000005e-06 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393873+0000", 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393879+0000", 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017886270000000001 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.395668+0000", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109719 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.395777+0000", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016390900000000001 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.395941+0000", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025147989999999999 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.398456+0000", 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030187299999999997 2024-02-22T19:31:51.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.398758+0000", 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0092900759999999995 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408048+0000", 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3801999999999998e-05 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408092+0000", 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1798000000000003e-05 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408124+0000", 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014281719 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19603 7.9 7:9257e860:::benchmark_data_smithi053_107660_object19602:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.393998+0000", 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.0969709660000002, 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014250008, 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19603 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393998+0000", 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393998+0000", 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.393997+0000", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1610000000000001e-05 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.394018+0000", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2023e-05 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.394030+0000", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.394036+0000", 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002627267 2024-02-22T19:31:51.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.396663+0000", 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000119034 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.396782+0000", 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000153084 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.396935+0000", 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015259329999999999 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.398461+0000", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00073765199999999999 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.399199+0000", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0089729900000000001 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408172+0000", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2936999999999999e-05 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408215+0000", 2024-02-22T19:31:51.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3132000000000001e-05 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.408248+0000", 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014250008 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:19651 7.6 7:6e18b308:::benchmark_data_smithi053_107660_object19650:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.425250+0000", 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.0657185760000001, 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019140305999999999, 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19651 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.425250+0000", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.425250+0000", 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.425250+0000", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3770999999999997e-05 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.425303+0000", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1020000000000002e-06 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.425310+0000", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.177e-06 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.425314+0000", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011748570000000001 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.426489+0000", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4061999999999997e-05 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.426563+0000", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000120926 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.426684+0000", 2024-02-22T19:31:51.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031907860000000001 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.429874+0000", 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018903299999999999 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.430063+0000", 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01427665 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.444340+0000", 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6746000000000001e-05 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.444367+0000", 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3662000000000001e-05 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.444390+0000", 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019140305999999999 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20823 7.4 7:2840abb0:::benchmark_data_smithi053_107660_object20822:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.979760+0000", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.5112092339999998, 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021889668000000001, 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20823 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979760+0000", 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979760+0000", 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979758+0000", 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1299e-05 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979790+0000", 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8989999999999999e-05 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979809+0000", 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7909999999999999e-06 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979816+0000", 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7646999999999998e-05 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.979883+0000", 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014352399999999999 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980027+0000", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019997099999999999 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980227+0000", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011845569999999999 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981411+0000", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046444360000000001 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986056+0000", 2024-02-22T19:31:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015499260000000001 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.001555+0000", 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2828000000000003e-05 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.001608+0000", 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1446000000000003e-05 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.001649+0000", 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021889668000000001 2024-02-22T19:31:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:20825 7.e e98/93 7:74b6997e:::benchmark_data_smithi053_107660_object20824:head v 98'1254, mlcod=98'1251)", 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.985078+0000", 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.5058903610000001, 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01796561, 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985078+0000", 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985078+0000", 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985078+0000", 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1072999999999999e-05 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985109+0000", 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2660000000000004e-06 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985116+0000", 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985121+0000", 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080829700000000003 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985929+0000", 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8949000000000001e-05 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985948+0000", 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017062226999999999 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003011+0000", 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3348e-05 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003044+0000", 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01796561 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20826 7.9 7:903c5892:::benchmark_data_smithi053_107660_object20825:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.985169+0000", 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.505799978, 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01787267, 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20826 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985169+0000", 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985169+0000", 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985167+0000", 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.012e-05 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985197+0000", 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0784e-05 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985208+0000", 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.985214+0000", 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088725999999999996 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986101+0000", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9247999999999996e-05 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986200+0000", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011291299999999999 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986313+0000", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039389589999999997 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990252+0000", 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011448560999999999 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.001701+0000", 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012897340000000001 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.002991+0000", 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9207e-05 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003020+0000", 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1664000000000001e-05 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003041+0000", 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01787267 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:20827 7.d e98/93 7:bcd56fa1:::benchmark_data_smithi053_107660_object20826:head v 98'1372, mlcod=98'1369)", 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.986440+0000", 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.5045283, 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016810286000000001, 2024-02-22T19:31:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986440+0000", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986440+0000", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986440+0000", 2024-02-22T19:31:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8660999999999997e-05 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986528+0000", 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3147e-05 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986541+0000", 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2740000000000004e-06 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986548+0000", 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010618799999999999 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986654+0000", 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0259e-05 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986674+0000", 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016555149000000002 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003229+0000", 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1404999999999999e-05 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003251+0000", 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016810286000000001 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:20829 7.d e98/93 7:b35e7bd4:::benchmark_data_smithi053_107660_object20828:head v 98'1373, mlcod=98'1369)", 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.986560+0000", 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.504408558, 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017380486000000001, 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986560+0000", 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986560+0000", 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986560+0000", 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3578999999999998e-05 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986593+0000", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4169999999999997e-06 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986600+0000", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.34e-06 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.986604+0000", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091555599999999999 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987520+0000", 2024-02-22T19:31:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8895000000000001e-05 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987539+0000", 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016354372999999998 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003893+0000", 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7790999999999997e-05 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003941+0000", 2024-02-22T19:31:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017380486000000001 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:20833 7.d e98/93 7:b4811d57:::benchmark_data_smithi053_107660_object20832:head v 98'1374, mlcod=98'1369)", 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.988162+0000", 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.502806718, 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015789971999999999, 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.988162+0000", 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.988162+0000", 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.988162+0000", 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2888e-05 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.988204+0000", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7479999999999993e-06 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.988212+0000", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.553e-06 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.988218+0000", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020930609999999998 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990311+0000", 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1270000000000001e-05 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990332+0000", 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013610283000000001 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003942+0000", 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6970000000000005e-06 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.003952+0000", 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015789971999999999 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:689 5.3 5:c609908c:::notify.5:head [watch ping cookie 94129597101696] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.3366330310000001, 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86272942500000005, 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 689 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-02-22T19:31:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154333+0000", 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4004e-05 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154347+0000", 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3070000000000001e-05 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154370+0000", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2280000000000004e-06 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154379+0000", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108694 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154488+0000", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3376e-05 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154501+0000", 2024-02-22T19:31:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85773138400000004 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012232+0000", 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055974900000000001 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012792+0000", 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040998229999999998 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.016892+0000", 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6339000000000002e-05 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.016968+0000", 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6751999999999998e-05 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017065+0000", 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86272942500000005 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:698 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94129597101696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:31:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.3362373439999999, 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87369409200000003, 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 698 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4479999999999996e-06 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154735+0000", 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0008e-05 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154745+0000", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6600000000000003e-06 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154750+0000", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7904000000000001e-05 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154798+0000", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1430000000000001e-05 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154809+0000", 2024-02-22T19:31:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85813308200000005 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012942+0000", 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0219000000000001e-05 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013002+0000", 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004174891 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017177+0000", 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7285000000000004e-05 2024-02-22T19:31:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017245+0000", 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011180936000000001 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.028426+0000", 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87369409200000003 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:51.334 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:31:51.335 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:49.979769+0000 mon.a (mon.0) 773 : cluster [WRN] Health check failed: Reduced data availability: 13 pgs inactive, 16 pgs peering (PG_AVAILABILITY) 2024-02-22T19:31:51.335 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:49.988133+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e100: 8 total, 8 up, 7 in 2024-02-22T19:31:51.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:49.979769+0000 mon.a (mon.0) 773 : cluster [WRN] Health check failed: Reduced data availability: 13 pgs inactive, 16 pgs peering (PG_AVAILABILITY) 2024-02-22T19:31:51.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:49.988133+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e100: 8 total, 8 up, 7 in 2024-02-22T19:31:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:49.979769+0000 mon.a (mon.0) 773 : cluster [WRN] Health check failed: Reduced data availability: 13 pgs inactive, 16 pgs peering (PG_AVAILABILITY) 2024-02-22T19:31:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:49.988133+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e100: 8 total, 8 up, 7 in 2024-02-22T19:31:51.664 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 11 16 21497 21481 121.981 19 0.0126508 0.00818802 2024-02-22T19:31:52.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:31:52.333 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:50.991068+0000 mon.a (mon.0) 775 : cluster [DBG] osdmap e101: 8 total, 8 up, 7 in 2024-02-22T19:31:52.333 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:51.217549+0000 mgr.y (mgr.14535) 116 : cluster [DBG] pgmap v113: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 111 MiB/s wr, 1.77k op/s 2024-02-22T19:31:52.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:50.991068+0000 mon.a (mon.0) 775 : cluster [DBG] osdmap e101: 8 total, 8 up, 7 in 2024-02-22T19:31:52.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:51.217549+0000 mgr.y (mgr.14535) 116 : cluster [DBG] pgmap v113: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 111 MiB/s wr, 1.77k op/s 2024-02-22T19:31:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:50.991068+0000 mon.a (mon.0) 775 : cluster [DBG] osdmap e101: 8 total, 8 up, 7 in 2024-02-22T19:31:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:51.217549+0000 mgr.y (mgr.14535) 116 : cluster [DBG] pgmap v113: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 111 MiB/s wr, 1.77k op/s 2024-02-22T19:31:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:52.396 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:52.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:52.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:52.593 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:52.635 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:52.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:52.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:52.664 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 12 16 22503 22487 117.057 62.875 0.0606749 0.00852574 2024-02-22T19:31:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:52.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:52.801 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:52.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:52.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:31:52.930 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:52.951 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:31:52.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:52.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:31:53.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:53.085 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:53.110 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:53.113 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:53.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:53.127 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:31:53.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:31:53.243 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:31:53.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:53.268 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:31:53.307 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:53.033124+0000 mon.a (mon.0) 776 : audit [INF] from='client.? 172.21.15.53:0/3754636226' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:31:53.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:53.033124+0000 mon.a (mon.0) 776 : audit [INF] from='client.? 172.21.15.53:0/3754636226' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:31:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:53.033124+0000 mon.a (mon.0) 776 : audit [INF] from='client.? 172.21.15.53:0/3754636226' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:31:53.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:53.418 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:53.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:53.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:53.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:31:53.592 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:53.667 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 13 16 23435 23419 112.525 58.25 0.0136972 0.00887969 2024-02-22T19:31:54.498 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:53.087341+0000 mon.a (mon.0) 777 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:53.107225+0000 mon.a (mon.0) 778 : audit [INF] from='client.? 172.21.15.53:0/3754636226' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:53.107276+0000 mon.a (mon.0) 779 : cluster [DBG] osdmap e102: 8 total, 8 up, 7 in 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:53.218157+0000 mgr.y (mgr.14535) 117 : cluster [DBG] pgmap v115: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 157 MiB/s wr, 2.51k op/s 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:53.335028+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:53.087341+0000 mon.a (mon.0) 777 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:53.107225+0000 mon.a (mon.0) 778 : audit [INF] from='client.? 172.21.15.53:0/3754636226' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:53.107276+0000 mon.a (mon.0) 779 : cluster [DBG] osdmap e102: 8 total, 8 up, 7 in 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:53.218157+0000 mgr.y (mgr.14535) 117 : cluster [DBG] pgmap v115: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 157 MiB/s wr, 2.51k op/s 2024-02-22T19:31:54.499 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:53.335028+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:54.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:31:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:53.087341+0000 mon.a (mon.0) 777 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:31:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:53.107225+0000 mon.a (mon.0) 778 : audit [INF] from='client.? 172.21.15.53:0/3754636226' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:31:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:53.107276+0000 mon.a (mon.0) 779 : cluster [DBG] osdmap e102: 8 total, 8 up, 7 in 2024-02-22T19:31:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:53.218157+0000 mgr.y (mgr.14535) 117 : cluster [DBG] pgmap v115: 148 pgs: 51 peering, 97 active+clean; 1.3 GiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 157 MiB/s wr, 2.51k op/s 2024-02-22T19:31:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:53.335028+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:31:54.665 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 14 16 24544 24528 109.44 69.3125 0.0280763 0.00912782 2024-02-22T19:31:54.932 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:31:54.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 0 1 2024-02-22T19:31:55.020 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:31:55.028 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:55.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:31:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:31:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:31:55.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:31:55.158 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:55.180 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:31:55.183 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:55.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:31:55.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:54.335418+0000 mgr.y (mgr.14535) 118 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:55.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:54.335418+0000 mgr.y (mgr.14535) 118 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:55.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:31:55.354 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:54.335418+0000 mgr.y (mgr.14535) 118 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:31:55.665 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 15 16 25552 25536 106.345 63 0.0128179 0.00939498 2024-02-22T19:31:55.850 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:31:55.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:55.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:31:55.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:31:56.037 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:56.137 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:31:56.143 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:56.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:31:56.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:31:56.324 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:56.431 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:55.219843+0000 mgr.y (mgr.14535) 119 : cluster [DBG] pgmap v116: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.5 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 33 MiB/s wr, 522 op/s; 4602/72597 objects misplaced (6.339%); 5.3 MiB/s, 93 objects/s recovering 2024-02-22T19:31:56.432 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:55.219843+0000 mgr.y (mgr.14535) 119 : cluster [DBG] pgmap v116: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.5 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 33 MiB/s wr, 522 op/s; 4602/72597 objects misplaced (6.339%); 5.3 MiB/s, 93 objects/s recovering 2024-02-22T19:31:56.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:55.219843+0000 mgr.y (mgr.14535) 119 : cluster [DBG] pgmap v116: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.5 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 33 MiB/s wr, 522 op/s; 4602/72597 objects misplaced (6.339%); 5.3 MiB/s, 93 objects/s recovering 2024-02-22T19:31:56.667 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 16 16 26517 26501 103.459 60.3125 0.0194791 0.00965693 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:26541 7.4 e102/93 7:2f3b6bdf:::benchmark_data_smithi053_107660_object26540:head v 102'1593, mlcod=102'1591)", 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:56.716468+0000", 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.012343168999999999, 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012373967, 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:56.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716468+0000", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716468+0000", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716467+0000", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5086999999999999e-05 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716492+0000", 2024-02-22T19:31:56.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5321000000000001e-05 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716507+0000", 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716514+0000", 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0338999999999999e-05 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716574+0000", 2024-02-22T19:31:56.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9712999999999999e-05 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.716594+0000", 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012595999999999999 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:26546 7.b 7:df0889a4:::benchmark_data_smithi053_107660_object26545:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e102)", 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:56.717664+0000", 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.011147413, 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011259101000000001, 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:31:56.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26546 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717664+0000", 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717664+0000", 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:31:56.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717662+0000", 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2629000000000001e-05 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717705+0000", 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5779999999999999e-06 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717713+0000", 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9289999999999997e-06 2024-02-22T19:31:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717718+0000", 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021223090000000002 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.719840+0000", 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013667799999999999 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.719977+0000", 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000185331 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.720162+0000", 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0068680429999999999 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.727030+0000", 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044e-05 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.727041+0000", 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0093771319999999998 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:26550 7.0 7:038a194a:::benchmark_data_smithi053_107660_object26549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e102)", 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:56.717729+0000", 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.011082585000000001, 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011288886, 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26550 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:31:56.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717729+0000", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717729+0000", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717728+0000", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8209000000000001e-05 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717746+0000", 2024-02-22T19:31:56.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6409999999999999e-06 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717752+0000", 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0249999999999996e-06 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.717756+0000", 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085638100000000003 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.718612+0000", 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151528 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.718764+0000", 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019299700000000001 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.718957+0000", 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033805160000000001 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.722337+0000", 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046087680000000001 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:31:56.755 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:31:56.755 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 3 2024-02-22T19:31:56.755 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:31:56.765 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:56.880 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:31:56.896 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:31:56.943 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:31:56.977 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:31:57.022 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:31:57.060 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:31:57.098 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:31:57.141 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:31:57.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:56.182366+0000 mon.a (mon.0) 781 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 13 pgs inactive, 16 pgs peering) 2024-02-22T19:31:57.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:31:56.813229+0000 mon.a (mon.0) 782 : audit [DBG] from='client.? 172.21.15.53:0/3444444032' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:57.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:56.182366+0000 mon.a (mon.0) 781 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 13 pgs inactive, 16 pgs peering) 2024-02-22T19:31:57.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:31:56.813229+0000 mon.a (mon.0) 782 : audit [DBG] from='client.? 172.21.15.53:0/3444444032' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:57.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:56.182366+0000 mon.a (mon.0) 781 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 13 pgs inactive, 16 pgs peering) 2024-02-22T19:31:57.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:31:56.813229+0000 mon.a (mon.0) 782 : audit [DBG] from='client.? 172.21.15.53:0/3444444032' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:31:57.667 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 17 16 27553 27537 101.183 64.75 0.0168333 0.00987545 2024-02-22T19:31:57.894 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:31:58.216 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:31:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:57.221247+0000 mgr.y (mgr.14535) 120 : cluster [DBG] pgmap v117: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.5 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 707 B/s rd, 27 MiB/s wr, 433 op/s; 4602/72597 objects misplaced (6.339%); 4.4 MiB/s, 77 objects/s recovering 2024-02-22T19:31:58.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:31:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:57.221247+0000 mgr.y (mgr.14535) 120 : cluster [DBG] pgmap v117: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.5 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 707 B/s rd, 27 MiB/s wr, 433 op/s; 4602/72597 objects misplaced (6.339%); 4.4 MiB/s, 77 objects/s recovering 2024-02-22T19:31:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:31:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:57.221247+0000 mgr.y (mgr.14535) 120 : cluster [DBG] pgmap v117: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.5 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 707 B/s rd, 27 MiB/s wr, 433 op/s; 4602/72597 objects misplaced (6.339%); 4.4 MiB/s, 77 objects/s recovering 2024-02-22T19:31:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:31:58.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:58.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:31:58.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:31:58.668 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 18 16 28606 28590 99.2123 65.8125 0.0147511 0.0100727 2024-02-22T19:31:58.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:31:58.765 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:58.785 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:31:58.787 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:58.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:31:58.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:31:58.992 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:59.035 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:31:59.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:59.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:31:59.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:31:59.172 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:59.194 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:31:59.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:59.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:31:59.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:31:59.390 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:31:59.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:59.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:31:59.630 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:31:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:31:59] "GET /metrics HTTP/1.1" 200 175234 "" "Prometheus/2.33.4" 2024-02-22T19:31:59.670 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 19 16 29595 29579 97.2398 61.8125 0.0143595 0.0102753 2024-02-22T19:31:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:31:59.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:31:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:59.779 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:31:59.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:59.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:31:59.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:31:59.911 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:31:59.933 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:31:59.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:31:59.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:00.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:00.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:32:00.161 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:00.166 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:00.171 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:00.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:00.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:00.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:31:59.222375+0000 mgr.y (mgr.14535) 121 : cluster [DBG] pgmap v118: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 998 op/s; 3688/87825 objects misplaced (4.199%); 11 MiB/s, 178 objects/s recovering 2024-02-22T19:32:00.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:31:59.222375+0000 mgr.y (mgr.14535) 121 : cluster [DBG] pgmap v118: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 998 op/s; 3688/87825 objects misplaced (4.199%); 11 MiB/s, 178 objects/s recovering 2024-02-22T19:32:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:00.586 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:00.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:00.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:31:59.222375+0000 mgr.y (mgr.14535) 121 : cluster [DBG] pgmap v118: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 998 op/s; 3688/87825 objects misplaced (4.199%); 11 MiB/s, 178 objects/s recovering 2024-02-22T19:32:00.671 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:32:00.670295+0000 min lat: 0.00116596 max lat: 2.0693 avg lat: 0.0104901 2024-02-22T19:32:00.671 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:32:00.671 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 20 16 30512 30496 95.2437 57.3125 0.0108659 0.0104901 2024-02-22T19:32:00.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:00.775 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:00.864 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:00.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:00.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:01.046 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:01.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:01.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:01.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:01.273 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:01.296 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:01.299 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:01.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:01.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:01.455 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:01.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:01.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:01.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:01.615 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:01.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:01.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:01.670 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 21 16 31552 31536 93.8023 65 0.00840372 0.0106532 2024-02-22T19:32:01.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:01.791 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:02.307 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:01.222838+0000 mgr.y (mgr.14535) 122 : cluster [DBG] pgmap v119: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 51 MiB/s wr, 821 op/s; 3688/87825 objects misplaced (4.199%); 8.9 MiB/s, 147 objects/s recovering 2024-02-22T19:32:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:02.585 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:02.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:01.222838+0000 mgr.y (mgr.14535) 122 : cluster [DBG] pgmap v119: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 51 MiB/s wr, 821 op/s; 3688/87825 objects misplaced (4.199%); 8.9 MiB/s, 147 objects/s recovering 2024-02-22T19:32:02.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:01.222838+0000 mgr.y (mgr.14535) 122 : cluster [DBG] pgmap v119: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 51 MiB/s wr, 821 op/s; 3688/87825 objects misplaced (4.199%); 8.9 MiB/s, 147 objects/s recovering 2024-02-22T19:32:02.670 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 22 16 32630 32614 92.6014 67.375 0.00366199 0.0107893 2024-02-22T19:32:02.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:02.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:02.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:02.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:02.925 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:03.197 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:03.200 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:03.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:03.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:03.331 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:03.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:03.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:03.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:03.547 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:03.670 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 23 16 33658 33642 91.3682 64.25 0.00521839 0.0109368 2024-02-22T19:32:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:03.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:03.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:04.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:04.059 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:04.090 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:04.092 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:04.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:04.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:04.334 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:04.396 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:32:04.396 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:32:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-02-22T19:32:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:32:04.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:03.223273+0000 mgr.y (mgr.14535) 123 : cluster [DBG] pgmap v120: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1012 B/s rd, 51 MiB/s wr, 812 op/s; 3688/87825 objects misplaced (4.199%); 8.8 MiB/s, 145 objects/s recovering 2024-02-22T19:32:04.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:03.432674+0000 mon.a (mon.0) 783 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:04.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.242555+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.53:0/2996835471' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:32:04.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.245502+0000 mon.a (mon.0) 784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:32:04.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.347481+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.53:0/252979379' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:04.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.349943+0000 mon.a (mon.0) 785 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:04.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:03.223273+0000 mgr.y (mgr.14535) 123 : cluster [DBG] pgmap v120: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1012 B/s rd, 51 MiB/s wr, 812 op/s; 3688/87825 objects misplaced (4.199%); 8.8 MiB/s, 145 objects/s recovering 2024-02-22T19:32:04.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:03.432674+0000 mon.a (mon.0) 783 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:04.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.242555+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.53:0/2996835471' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:32:04.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.245502+0000 mon.a (mon.0) 784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:32:04.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.347481+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.53:0/252979379' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:04.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.349943+0000 mon.a (mon.0) 785 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:04.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:32:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:03.223273+0000 mgr.y (mgr.14535) 123 : cluster [DBG] pgmap v120: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 1.8 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1012 B/s rd, 51 MiB/s wr, 812 op/s; 3688/87825 objects misplaced (4.199%); 8.8 MiB/s, 145 objects/s recovering 2024-02-22T19:32:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:03.432674+0000 mon.a (mon.0) 783 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.242555+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.53:0/2996835471' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:32:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.245502+0000 mon.a (mon.0) 784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:32:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.347481+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.53:0/252979379' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.349943+0000 mon.a (mon.0) 785 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:04.670 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 24 16 34462 34446 89.6555 50.25 0.0230827 0.0111493 2024-02-22T19:32:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:32:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:32:05.670 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 25 15 35562 35547 88.8219 68.8125 0.0147436 0.0112512 2024-02-22T19:32:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.347532+0000 mgr.y (mgr.14535) 124 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:04.430639+0000 mon.a (mon.0) 786 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:32:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:04.430670+0000 mon.a (mon.0) 787 : cluster [INF] Cluster is now healthy 2024-02-22T19:32:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.438882+0000 mon.a (mon.0) 788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:32:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:04.438951+0000 mon.a (mon.0) 789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:32:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:04.438997+0000 mon.a (mon.0) 790 : cluster [DBG] osdmap e103: 8 total, 8 up, 7 in 2024-02-22T19:32:05.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.347532+0000 mgr.y (mgr.14535) 124 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:05.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:04.430639+0000 mon.a (mon.0) 786 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:32:05.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:04.430670+0000 mon.a (mon.0) 787 : cluster [INF] Cluster is now healthy 2024-02-22T19:32:05.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.438882+0000 mon.a (mon.0) 788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:32:05.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:04.438951+0000 mon.a (mon.0) 789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:32:05.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:04.438997+0000 mon.a (mon.0) 790 : cluster [DBG] osdmap e103: 8 total, 8 up, 7 in 2024-02-22T19:32:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.347532+0000 mgr.y (mgr.14535) 124 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:04.430639+0000 mon.a (mon.0) 786 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:32:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:04.430670+0000 mon.a (mon.0) 787 : cluster [INF] Cluster is now healthy 2024-02-22T19:32:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.438882+0000 mon.a (mon.0) 788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:32:05.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:04.438951+0000 mon.a (mon.0) 789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:32:05.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:04.438997+0000 mon.a (mon.0) 790 : cluster [DBG] osdmap e103: 8 total, 8 up, 7 in 2024-02-22T19:32:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:05.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:05.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:05.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:32:06.042 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:06.071 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:06.093 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:06.096 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:06.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:06.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:06.321 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:06.326 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:06.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:06.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:06.458 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:06.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:06.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:06.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:06.617 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:06.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:06.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:06.672 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 26 16 36573 36557 87.8277 63.125 0.00347806 0.0113605 2024-02-22T19:32:06.736 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:05.224903+0000 mgr.y (mgr.14535) 125 : cluster [DBG] pgmap v122: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s; 2716/102792 objects misplaced (2.642%); 12 MiB/s, 188 objects/s recovering 2024-02-22T19:32:06.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:05.224903+0000 mgr.y (mgr.14535) 125 : cluster [DBG] pgmap v122: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s; 2716/102792 objects misplaced (2.642%); 12 MiB/s, 188 objects/s recovering 2024-02-22T19:32:06.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:05.224903+0000 mgr.y (mgr.14535) 125 : cluster [DBG] pgmap v122: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s; 2716/102792 objects misplaced (2.642%); 12 MiB/s, 188 objects/s recovering 2024-02-22T19:32:07.672 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 27 16 36955 36939 85.46 23.875 0.00284617 0.011455 2024-02-22T19:32:07.731 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 1 1 2024-02-22T19:32:07.750 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:07.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:07.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:07.776 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:06.514171+0000 mon.a (mon.0) 791 : cluster [DBG] osdmap e104: 8 total, 8 up, 7 in 2024-02-22T19:32:07.776 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:06.963248+0000 mon.a (mon.0) 792 : cluster [DBG] osdmap e105: 8 total, 8 up, 7 in 2024-02-22T19:32:07.777 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:06.514171+0000 mon.a (mon.0) 791 : cluster [DBG] osdmap e104: 8 total, 8 up, 7 in 2024-02-22T19:32:07.777 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:06.963248+0000 mon.a (mon.0) 792 : cluster [DBG] osdmap e105: 8 total, 8 up, 7 in 2024-02-22T19:32:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:06.514171+0000 mon.a (mon.0) 791 : cluster [DBG] osdmap e104: 8 total, 8 up, 7 in 2024-02-22T19:32:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:06.963248+0000 mon.a (mon.0) 792 : cluster [DBG] osdmap e105: 8 total, 8 up, 7 in 2024-02-22T19:32:07.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:07.882 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:07.916 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:07.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:07.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:08.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:08.080 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:08.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:08.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:08.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:08.260 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:08.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:07.225263+0000 mgr.y (mgr.14535) 126 : cluster [DBG] pgmap v125: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 52 MiB/s wr, 831 op/s; 2716/102792 objects misplaced (2.642%); 10 MiB/s, 161 objects/s recovering 2024-02-22T19:32:08.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:07.963878+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e106: 8 total, 8 up, 7 in 2024-02-22T19:32:08.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:08.346866+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:08.672 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 28 16 37715 37699 84.1048 47.5 0.00920996 0.0118842 2024-02-22T19:32:08.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:32:08.801 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:07.225263+0000 mgr.y (mgr.14535) 126 : cluster [DBG] pgmap v125: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 52 MiB/s wr, 831 op/s; 2716/102792 objects misplaced (2.642%); 10 MiB/s, 161 objects/s recovering 2024-02-22T19:32:08.801 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:07.963878+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e106: 8 total, 8 up, 7 in 2024-02-22T19:32:08.801 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:08.346866+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:07.225263+0000 mgr.y (mgr.14535) 126 : cluster [DBG] pgmap v125: 148 pgs: 4 active+recovering+undersized+remapped, 144 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 52 MiB/s wr, 831 op/s; 2716/102792 objects misplaced (2.642%); 10 MiB/s, 161 objects/s recovering 2024-02-22T19:32:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:07.963878+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e106: 8 total, 8 up, 7 in 2024-02-22T19:32:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:08.346866+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:09.680 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 29 16 38731 38715 83.3723 63.5 0.00541898 0.0119882 2024-02-22T19:32:09.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:09.122771+0000 mon.a (mon.0) 795 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:32:09.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:09.122771+0000 mon.a (mon.0) 795 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:32:09.756 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:32:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:09] "GET /metrics HTTP/1.1" 200 175374 "" "Prometheus/2.33.4" 2024-02-22T19:32:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:09.122771+0000 mon.a (mon.0) 795 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:32:10.680 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 30 15 39842 39827 82.9098 69.5 0.00889484 0.0120545 2024-02-22T19:32:10.754 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:10.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:10.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:10.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:09.226819+0000 mgr.y (mgr.14535) 127 : cluster [DBG] pgmap v127: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 1.0 KiB/s rd, 54 MiB/s wr, 871 op/s; 2102/115287 objects misplaced (1.823%); 8.0 MiB/s, 128 objects/s recovering 2024-02-22T19:32:10.848 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:09.226819+0000 mgr.y (mgr.14535) 127 : cluster [DBG] pgmap v127: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 1.0 KiB/s rd, 54 MiB/s wr, 871 op/s; 2102/115287 objects misplaced (1.823%); 8.0 MiB/s, 128 objects/s recovering 2024-02-22T19:32:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:09.226819+0000 mgr.y (mgr.14535) 127 : cluster [DBG] pgmap v127: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 1.0 KiB/s rd, 54 MiB/s wr, 871 op/s; 2102/115287 objects misplaced (1.823%); 8.0 MiB/s, 128 objects/s recovering 2024-02-22T19:32:10.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:10.905 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:10.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:10.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:11.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:11.123 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:11.159 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:11.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:11.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:11.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:32:11.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:11.304 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:32:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:11.335 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:32:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:11.346 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:11.347 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:11.372 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:32:11.408 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:32:11.441 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:32:11.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:11.479 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:32:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:11.519 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.530 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:32:11.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:10.960590+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:32:11.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:10.961806+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:32:11.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:10.969421+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:11.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:11.230048+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? 172.21.15.53:0/348808822' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:11.681 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 31 16 40703 40687 81.9678 53.75 0.016714 0.0121933 2024-02-22T19:32:12.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:10.960590+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:32:12.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:10.961806+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:32:12.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:10.969421+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:12.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:11.230048+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? 172.21.15.53:0/348808822' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:12.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:10.960590+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:32:12.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:10.961806+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:32:12.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:10.969421+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:12.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:11.230048+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? 172.21.15.53:0/348808822' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:12.681 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 32 16 41765 41749 81.4806 66.375 0.0279747 0.0122669 2024-02-22T19:32:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:11.227535+0000 mgr.y (mgr.14535) 128 : cluster [DBG] pgmap v128: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 43 MiB/s wr, 694 op/s; 2102/115287 objects misplaced (1.823%); 6.4 MiB/s, 102 objects/s recovering 2024-02-22T19:32:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:12.039013+0000 osd.6 (osd.6) 23 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:12.039050+0000 osd.6 (osd.6) 24 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:11.227535+0000 mgr.y (mgr.14535) 128 : cluster [DBG] pgmap v128: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 43 MiB/s wr, 694 op/s; 2102/115287 objects misplaced (1.823%); 6.4 MiB/s, 102 objects/s recovering 2024-02-22T19:32:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:12.039013+0000 osd.6 (osd.6) 23 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:12.039050+0000 osd.6 (osd.6) 24 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:11.227535+0000 mgr.y (mgr.14535) 128 : cluster [DBG] pgmap v128: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 43 MiB/s wr, 694 op/s; 2102/115287 objects misplaced (1.823%); 6.4 MiB/s, 102 objects/s recovering 2024-02-22T19:32:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:12.039013+0000 osd.6 (osd.6) 23 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:12.039050+0000 osd.6 (osd.6) 24 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:12.948 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:12.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:12.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:13.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:13.083 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:13.106 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:13.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:13.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:13.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:13.290 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:13.685 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 33 16 42879 42863 81.115 69.625 0.0053188 0.0123226 2024-02-22T19:32:14.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:12.980442+0000 mon.a (mon.0) 799 : cluster [DBG] osdmap e107: 8 total, 8 up, 7 in 2024-02-22T19:32:14.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:13.231406+0000 mgr.y (mgr.14535) 129 : cluster [DBG] pgmap v130: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 816 B/s rd, 42 MiB/s wr, 665 op/s; 2102/115287 objects misplaced (1.823%); 6.1 MiB/s, 97 objects/s recovering 2024-02-22T19:32:14.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:12.980442+0000 mon.a (mon.0) 799 : cluster [DBG] osdmap e107: 8 total, 8 up, 7 in 2024-02-22T19:32:14.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:13.231406+0000 mgr.y (mgr.14535) 129 : cluster [DBG] pgmap v130: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 816 B/s rd, 42 MiB/s wr, 665 op/s; 2102/115287 objects misplaced (1.823%); 6.1 MiB/s, 97 objects/s recovering 2024-02-22T19:32:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:12.980442+0000 mon.a (mon.0) 799 : cluster [DBG] osdmap e107: 8 total, 8 up, 7 in 2024-02-22T19:32:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:13.231406+0000 mgr.y (mgr.14535) 129 : cluster [DBG] pgmap v130: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.3 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 816 B/s rd, 42 MiB/s wr, 665 op/s; 2102/115287 objects misplaced (1.823%); 6.1 MiB/s, 97 objects/s recovering 2024-02-22T19:32:14.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:32:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:32:14.660 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:32:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:32:14.684 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 34 16 43989 43973 80.7691 69.375 0.00704855 0.0123755 2024-02-22T19:32:14.976 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:14.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:14.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:14.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:32:15.000 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:14.359704+0000 mgr.y (mgr.14535) 130 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:15.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:15.113 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:15.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:15.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:15.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:15.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:14.359704+0000 mgr.y (mgr.14535) 130 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:15.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:14.359704+0000 mgr.y (mgr.14535) 130 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:15.342 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:15.420 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:15.425 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:15.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:15.526 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:15.688 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 35 16 44894 44878 80.0692 56.5625 0.022366 0.0124832 2024-02-22T19:32:15.906 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:15.909 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:15.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:16.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:16.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:15.232315+0000 mgr.y (mgr.14535) 131 : cluster [DBG] pgmap v131: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.7 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 1986/130851 objects misplaced (1.518%); 5.7 MiB/s, 91 objects/s recovering 2024-02-22T19:32:16.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:15.232315+0000 mgr.y (mgr.14535) 131 : cluster [DBG] pgmap v131: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.7 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 1986/130851 objects misplaced (1.518%); 5.7 MiB/s, 91 objects/s recovering 2024-02-22T19:32:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:15.232315+0000 mgr.y (mgr.14535) 131 : cluster [DBG] pgmap v131: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.7 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 1986/130851 objects misplaced (1.518%); 5.7 MiB/s, 91 objects/s recovering 2024-02-22T19:32:16.571 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:16.594 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:16.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:16.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:16.693 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 36 16 45787 45771 79.392 55.8125 0.0129975 0.0125899 2024-02-22T19:32:16.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:16.728 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:16.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:16.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:16.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:16.903 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:16.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:16.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:17.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-02-22T19:32:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:32:17.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:16.060719+0000 mon.a (mon.0) 800 : audit [INF] from='client.? 172.21.15.53:0/2540165278' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:32:17.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:16.183766+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.53:0/3942380152' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:32:17.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:16.184401+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:32:17.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:16.964577+0000 mon.a (mon.0) 802 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:17.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:16.970010+0000 mon.a (mon.0) 803 : audit [INF] from='client.? 172.21.15.53:0/2540165278' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:32:17.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:16.970074+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:32:17.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:16.970137+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e108: 8 total, 8 up, 7 in 2024-02-22T19:32:17.069 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:17.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:16.060719+0000 mon.a (mon.0) 800 : audit [INF] from='client.? 172.21.15.53:0/2540165278' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:32:17.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:16.183766+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.53:0/3942380152' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:32:17.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:16.184401+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:32:17.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:16.964577+0000 mon.a (mon.0) 802 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:16.970010+0000 mon.a (mon.0) 803 : audit [INF] from='client.? 172.21.15.53:0/2540165278' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:16.970074+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:16.970137+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e108: 8 total, 8 up, 7 in 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:16.060719+0000 mon.a (mon.0) 800 : audit [INF] from='client.? 172.21.15.53:0/2540165278' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:16.183766+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.53:0/3942380152' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:16.184401+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:16.964577+0000 mon.a (mon.0) 802 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:16.970010+0000 mon.a (mon.0) 803 : audit [INF] from='client.? 172.21.15.53:0/2540165278' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:16.970074+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:32:17.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:16.970137+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e108: 8 total, 8 up, 7 in 2024-02-22T19:32:17.346 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:17.350 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:17.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:17.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:17.503 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:17.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:17.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:17.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:17.638 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:17.665 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:17.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:17.692 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 37 16 46756 46740 78.8787 60.5625 0.0119096 0.0126716 2024-02-22T19:32:17.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:18.004 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:18.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:18.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:18.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:18.181 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:17.233872+0000 mgr.y (mgr.14535) 132 : cluster [DBG] pgmap v133: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.7 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 639 B/s rd, 40 MiB/s wr, 648 op/s; 1986/130851 objects misplaced (1.518%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:32:18.182 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:17.233872+0000 mgr.y (mgr.14535) 132 : cluster [DBG] pgmap v133: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.7 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 639 B/s rd, 40 MiB/s wr, 648 op/s; 1986/130851 objects misplaced (1.518%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:32:18.185 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20843 7.0 7:0b664233:::benchmark_data_smithi053_107660_object20842:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.194929145, 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472188600000001, 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20843 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005334+0000", 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4664999999999998e-05 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005359+0000", 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1853e-05 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005371+0000", 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005378+0000", 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016187800000000001 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005539+0000", 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1446e-05 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005551+0000", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98239731299999999 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987948+0000", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4382000000000001e-05 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987963+0000", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.054996391 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.042959+0000", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041492100000000001 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.043374+0000", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016716699999999999 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.043541+0000", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003835649 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047377+0000", 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011540490000000001 2024-02-22T19:32:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048531+0000", 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033136839999999999 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051844+0000", 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1493999999999999e-05 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051896+0000", 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065935400000000002 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052555+0000", 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472188600000001 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20848 7.0 7:00506f5c:::benchmark_data_smithi053_107660_object20847:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.182612138, 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0355339840000002, 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20848 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017651+0000", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6801999999999999e-05 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017678+0000", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1832e-05 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017690+0000", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9899999999999997e-06 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017698+0000", 2024-02-22T19:32:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0209e-05 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017748+0000", 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017755+0000", 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97023272699999996 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987988+0000", 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0849999999999997e-06 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987996+0000", 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0570709490000001 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045067+0000", 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016401100000000001 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045231+0000", 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158806 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045390+0000", 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031485200000000001 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048538+0000", 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040701499999999998 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052608+0000", 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2032e-05 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052650+0000", 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2859000000000003e-05 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052703+0000", 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000484162 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053187+0000", 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0355339840000002 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20867 7.b 7:d5ccad63:::benchmark_data_smithi053_107660_object20866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.175514302, 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0358723360000002, 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20867 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024749+0000", 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8314e-05 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024777+0000", 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953000000000001e-05 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024792+0000", 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8079999999999994e-06 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024801+0000", 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9221000000000001e-05 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024860+0000", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0496000000000001e-05 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024871+0000", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.964611467 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989482+0000", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0557000000000001e-05 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989493+0000", 2024-02-22T19:32:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06140933 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050902+0000", 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015795699999999999 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051060+0000", 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000234997 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051295+0000", 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029240099999999999 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054219+0000", 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001188099 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055407+0000", 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042592480000000002 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059666+0000", 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0272000000000002e-05 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059727+0000", 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089686099999999999 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060624+0000", 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0358723360000002 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20879 7.b 7:d83f7aa7:::benchmark_data_smithi053_107660_object20878:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.169348785, 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0357336730000002, 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20879 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030915+0000", 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4876e-05 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030940+0000", 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7564e-05 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030957+0000", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5020000000000003e-06 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030965+0000", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2354999999999997e-05 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.031007+0000", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5809999999999999e-06 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.031015+0000", 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.958500719 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989516+0000", 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5569999999999998e-06 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989523+0000", 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635326220000001 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053056+0000", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014994699999999999 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053206+0000", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169927 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053376+0000", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049563699999999999 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058332+0000", 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079005799999999995 2024-02-22T19:32:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059122+0000", 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066417830000000001 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.065764+0000", 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0451999999999997e-05 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.065814+0000", 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083617100000000003 2024-02-22T19:32:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066650+0000", 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0357336730000002 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20885 7.0 7:074624f9:::benchmark_data_smithi053_107660_object20884:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.164348824000001, 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0216042769999998, 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20885 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035915+0000", 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7532999999999999e-05 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035942+0000", 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6866000000000001e-05 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035959+0000", 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3299999999999999e-06 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035968+0000", 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0296000000000006e-05 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036038+0000", 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2227e-05 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036050+0000", 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95196719699999999 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988017+0000", 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5079999999999999e-06 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988025+0000", 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0581795979999999 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046204+0000", 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137909 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046342+0000", 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017299200000000001 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046515+0000", 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046217719999999997 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051137+0000", 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003247869 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054385+0000", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022844919999999999 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.056669+0000", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0172e-05 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.056720+0000", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080141399999999997 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057521+0000", 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0216042769999998 2024-02-22T19:32:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20888 7.0 7:0390592f:::benchmark_data_smithi053_107660_object20887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.159625104, 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.017606019, 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20888 2024-02-22T19:32:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040639+0000", 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7110999999999998e-05 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040666+0000", 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4090999999999999e-05 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040680+0000", 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2790000000000004e-06 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040686+0000", 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4023999999999996e-05 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040750+0000", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2262000000000001e-05 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040763+0000", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.9472855 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988048+0000", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8350000000000004e-06 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988056+0000", 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.059385043 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047441+0000", 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012125600000000001 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047562+0000", 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017584799999999999 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047738+0000", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054622200000000003 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053200+0000", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001243353 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054444+0000", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031954219999999998 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057639+0000", 2024-02-22T19:32:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4193000000000001e-05 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057693+0000", 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055305899999999995 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058246+0000", 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.017606019 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20889 7.b 7:deffbca2:::benchmark_data_smithi053_107660_object20888:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.159567011, 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0267241290000002, 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20889 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:32:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040698+0000", 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7917000000000001e-05 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040716+0000", 2024-02-22T19:32:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6410000000000001e-06 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040721+0000", 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0340000000000001e-06 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040724+0000", 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3059000000000001e-05 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040777+0000", 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3380000000000001e-05 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040790+0000", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94875468299999999 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989545+0000", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.374e-06 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989552+0000", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0647073950000001 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054259+0000", 2024-02-22T19:32:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142843 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054402+0000", 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018314000000000001 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054585+0000", 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037587060000000001 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058344+0000", 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002519851 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060864+0000", 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058517530000000003 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066716+0000", 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0093999999999998e-05 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066776+0000", 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064683000000000004 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.067423+0000", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0267241290000002 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:701 5.15 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.045455608000001, 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86739943600000002, 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:32:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 701 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154809+0000", 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6779999999999999e-05 2024-02-22T19:32:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154825+0000", 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3336000000000002e-05 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154849+0000", 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6969999999999998e-06 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154854+0000", 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3915999999999998e-05 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154898+0000", 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.413e-06 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154906+0000", 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85827278699999998 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013179+0000", 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030207999999999998 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013481+0000", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003549445 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017030+0000", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3808999999999997e-05 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017114+0000", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005095502 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.022209+0000", 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86739943600000002 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20979 7.b 7:d41b82f5:::benchmark_data_smithi053_107660_object20978:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:51.163277+0000", 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.036988870999998, 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.034943746999999997, 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20979 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163277+0000", 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163277+0000", 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163275+0000", 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4820000000000002e-05 2024-02-22T19:32:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163309+0000", 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0244000000000001e-05 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163330+0000", 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7670000000000003e-06 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163337+0000", 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2967000000000003e-05 2024-02-22T19:32:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163390+0000", 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015336399999999999 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163544+0000", 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022447799999999999 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.163768+0000", 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051745089999999999 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.168943+0000", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7681999999999993e-05 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.169010+0000", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028287987000000001 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.197298+0000", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9689999999999999e-05 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.197358+0000", 2024-02-22T19:32:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086224499999999998 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.198220+0000", 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.034943746999999997 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22246 7.b 7:d5d85893:::benchmark_data_smithi053_107660_object22245:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:52.355803+0000", 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.844462590999999, 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033211605999999998, 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22246 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.355803+0000", 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.355803+0000", 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.355801+0000", 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0182000000000002e-05 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.355851+0000", 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0020000000000001e-05 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.355861+0000", 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8189999999999997e-06 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.355867+0000", 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001073809 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.356941+0000", 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010935299999999999 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.357050+0000", 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000174272 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.357225+0000", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091133499999999999 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.358136+0000", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016824347999999999 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.374960+0000", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01322046 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.388181+0000", 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9694000000000001e-05 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.388250+0000", 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076404999999999995 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.389015+0000", 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033211605999999998 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22251 7.0 7:0a9fdfc6:::benchmark_data_smithi053_107660_object22250:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:52.363817+0000", 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.836448266000001, 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032959663, 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22251 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.363817+0000", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.363817+0000", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.363816+0000", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1042999999999999e-05 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.363867+0000", 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5470000000000002e-06 2024-02-22T19:32:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.363876+0000", 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6269999999999997e-06 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.363882+0000", 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093216499999999999 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.364814+0000", 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017610400000000001 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.364990+0000", 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023969700000000001 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.365230+0000", 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010763319999999999 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.366306+0000", 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095666770000000009 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.375873+0000", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020322294000000001 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.396195+0000", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1849999999999999e-05 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.396257+0000", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052003699999999999 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.396777+0000", 2024-02-22T19:32:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032959663 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22458 7.0 7:0ef0dd72:::benchmark_data_smithi053_107660_object22457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:52.600209+0000", 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.600056786, 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045517037000000003, 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22458 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600209+0000", 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600209+0000", 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600207+0000", 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4549000000000002e-05 2024-02-22T19:32:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600231+0000", 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2116999999999999e-05 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600244+0000", 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.011e-06 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600252+0000", 2024-02-22T19:32:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036691229999999998 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.603921+0000", 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017633300000000001 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.604097+0000", 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000192058 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.604289+0000", 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021584540000000002 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.606448+0000", 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078427310000000004 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.614290+0000", 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030662571999999999 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.644953+0000", 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9848e-05 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.645013+0000", 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071306099999999997 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.645726+0000", 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045517037000000003 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22477 7.0 7:0bbe32c5:::benchmark_data_smithi053_107660_object22476:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:52.617844+0000", 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.582421822000001, 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044229362000000001, 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22477 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.617844+0000", 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.617844+0000", 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.617843+0000", 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2410000000000003e-05 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.617875+0000", 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7630000000000003e-06 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.617882+0000", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0359999999999997e-06 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.617887+0000", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014599722000000001 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.632487+0000", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158104 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.632645+0000", 2024-02-22T19:32:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022026899999999999 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.632865+0000", 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012914444000000001 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.645780+0000", 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015521584999999999 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.661301+0000", 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280000000000003e-06 2024-02-22T19:32:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.661310+0000", 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7958999999999999e-05 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.661368+0000", 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070524100000000001 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.662073+0000", 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044229362000000001 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22525 7.0 7:00018fad:::benchmark_data_smithi053_107660_object22524:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:52.712761+0000", 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.487504610999999, 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036303581000000001, 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22525 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.712761+0000", 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.712761+0000", 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.712759+0000", 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7268e-05 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.712786+0000", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5158e-05 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.712801+0000", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.832e-06 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.712808+0000", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039171809999999996 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.716725+0000", 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013788200000000001 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.716863+0000", 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018898299999999999 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.717052+0000", 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030720600000000002 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.720124+0000", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003111712 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.723236+0000", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.025290417999999999 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.748527+0000", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5014e-05 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.748572+0000", 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049294200000000001 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.749065+0000", 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036303581000000001 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22859 7.0 7:06438809:::benchmark_data_smithi053_107660_object22858:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:53.105849+0000", 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.094416703, 2024-02-22T19:32:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035583828999999997, 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22859 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105849+0000", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105849+0000", 2024-02-22T19:32:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105847+0000", 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7382e-05 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105874+0000", 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4626e-05 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105889+0000", 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6919999999999996e-06 2024-02-22T19:32:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105895+0000", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6938000000000003e-05 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.105952+0000", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015161499999999999 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.106104+0000", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000187128 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.106291+0000", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028562556999999999 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.134854+0000", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8000000000000002e-05 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.134892+0000", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058193450000000001 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.140711+0000", 2024-02-22T19:32:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7027999999999997e-05 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.140768+0000", 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066469599999999999 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.141433+0000", 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035583828999999997 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:23360 7.0 7:078e5c3d:::benchmark_data_smithi053_107660_object23359:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e102)", 2024-02-22T19:32:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:53.603345+0000", 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.596920888, 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036069274999999998, 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 23360 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603345+0000", 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603345+0000", 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603344+0000", 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9004e-05 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603363+0000", 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3040000000000001e-06 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603369+0000", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5170000000000001e-06 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603374+0000", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045205100000000003 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603826+0000", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137267 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.603963+0000", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018786700000000001 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.604151+0000", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060985730000000004 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.610250+0000", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0068044070000000002 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.617054+0000", 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021705347 2024-02-22T19:32:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.638759+0000", 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2895000000000001e-05 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.638812+0000", 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00060165800000000003 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:53.639414+0000", 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036069274999999998 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:25008 7.b 7:d212800a:::benchmark_data_smithi053_107660_object25007:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e102)", 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:55.066740+0000", 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.133525355, 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032767927000000002, 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 25008 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.066740+0000", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.066740+0000", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.066738+0000", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3215999999999999e-05 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.066762+0000", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8949999999999998e-06 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.066770+0000", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.066774+0000", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013563 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.068130+0000", 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013430199999999999 2024-02-22T19:32:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.068265+0000", 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000165358 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.068430+0000", 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0089301769999999992 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.077360+0000", 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6209999999999993e-06 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.077369+0000", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021519147999999998 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.098888+0000", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4262000000000002e-05 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.098952+0000", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055587599999999996 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:55.099508+0000", 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032767927000000002 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:26204 7.0 7:095cc512:::benchmark_data_smithi053_107660_object26203:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e102)", 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:56.314794+0000", 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.885471868, 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033376029000000002, 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26204 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.314794+0000", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.314794+0000", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.314792+0000", 2024-02-22T19:32:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8993000000000003e-05 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.314851+0000", 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1943000000000001e-05 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.314863+0000", 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5839999999999997e-06 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.314870+0000", 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011313666 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.326184+0000", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134707 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.326319+0000", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021309500000000001 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.326532+0000", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001028564 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.327560+0000", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019914502000000001 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.347475+0000", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7110000000000002e-05 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.347512+0000", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2075000000000005e-05 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.347574+0000", 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00059555899999999995 2024-02-22T19:32:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:56.348170+0000", 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033376029000000002 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:44552 7.0 7:05f3f967:::benchmark_data_smithi053_107660_object44551:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:15.258640+0000", 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.941626018, 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032424557999999999, 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 44552 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.258640+0000", 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.258640+0000", 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.258639+0000", 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5664999999999999e-05 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.258665+0000", 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2450000000000003e-06 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.258671+0000", 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2359999999999998e-06 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.258675+0000", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001752586 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.260428+0000", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102533 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.260530+0000", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000128818 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.260659+0000", 2024-02-22T19:32:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048998770000000004 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.265559+0000", 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087432899999999997 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.266433+0000", 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024548687 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.290982+0000", 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8398e-05 2024-02-22T19:32:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.291030+0000", 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3831000000000002e-05 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:15.291064+0000", 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032424557999999999 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:47138 7.b 7:de552459:::benchmark_data_smithi053_107660_object47137:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:18.070947+0000", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.12931806200000001, 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041551007000000001, 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47138 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.070947+0000", 2024-02-22T19:32:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.070947+0000", 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.070947+0000", 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2079000000000001e-05 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.070999+0000", 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6989999999999997e-06 2024-02-22T19:32:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.071006+0000", 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8960000000000002e-06 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.071011+0000", 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000572818 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.071583+0000", 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139517 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.071723+0000", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017044599999999999 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.071893+0000", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002072969 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.073966+0000", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016607620000000001 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.075627+0000", 2024-02-22T19:32:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036778038999999998 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.112405+0000", 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3018e-05 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.112458+0000", 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0373999999999997e-05 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.112498+0000", 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041551007000000001 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:18.322 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:32:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:17.233872+0000 mgr.y (mgr.14535) 132 : cluster [DBG] pgmap v133: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.7 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 639 B/s rd, 40 MiB/s wr, 648 op/s; 1986/130851 objects misplaced (1.518%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:32:18.369 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:18.386 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:18.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:18.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:18.567 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:18.692 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 38 16 47735 47719 78.4133 61.1875 0.0099175 0.0127477 2024-02-22T19:32:19.170 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:19.179 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:19.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:19.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 2 1 2024-02-22T19:32:19.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:19.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:17.427355+0000 osd.5 (osd.5) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:19.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:17.427409+0000 osd.5 (osd.5) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:19.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:18.059639+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e109: 8 total, 8 up, 7 in 2024-02-22T19:32:19.310 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:19.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:19.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:19.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:19.509 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:19] "GET /metrics HTTP/1.1" 200 175374 "" "Prometheus/2.33.4" 2024-02-22T19:32:19.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:17.427355+0000 osd.5 (osd.5) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:19.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:17.427409+0000 osd.5 (osd.5) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:19.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:18.059639+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e109: 8 total, 8 up, 7 in 2024-02-22T19:32:19.513 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:17.427355+0000 osd.5 (osd.5) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:19.513 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:17.427409+0000 osd.5 (osd.5) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:19.513 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:18.059639+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e109: 8 total, 8 up, 7 in 2024-02-22T19:32:19.560 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:19.692 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 39 16 48724 48708 77.9879 61.8125 0.0110695 0.012815 2024-02-22T19:32:19.707 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:19.719 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:19.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:19.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:32:19.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:19.870 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:19.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:19.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:19.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:20.005 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:20.027 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:20.030 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:20.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:19.236160+0000 mgr.y (mgr.14535) 133 : cluster [DBG] pgmap v135: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.60k op/s; 1870/145359 objects misplaced (1.286%); 2.3 MiB/s, 37 objects/s recovering 2024-02-22T19:32:20.131 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:20.161 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:20.162 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:19.236160+0000 mgr.y (mgr.14535) 133 : cluster [DBG] pgmap v135: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.60k op/s; 1870/145359 objects misplaced (1.286%); 2.3 MiB/s, 37 objects/s recovering 2024-02-22T19:32:20.163 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:19.236160+0000 mgr.y (mgr.14535) 133 : cluster [DBG] pgmap v135: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.60k op/s; 1870/145359 objects misplaced (1.286%); 2.3 MiB/s, 37 objects/s recovering 2024-02-22T19:32:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:20.186 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:20.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:20.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:20.697 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:32:20.697422+0000 min lat: 0.00116596 max lat: 2.0693 avg lat: 0.0128937 2024-02-22T19:32:20.697 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:32:20.697 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 40 16 49664 49648 77.4996 58.75 0.0160864 0.0128937 2024-02-22T19:32:20.723 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:20.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:20.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:20.802 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:20.818 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:20.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:20.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:20.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:20.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:21.022 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:21.048 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:21.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:21.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:21.177 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:20.445203+0000 osd.7 (osd.7) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:21.177 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:20.445245+0000 osd.7 (osd.7) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:21.179 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:21.199 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:21.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:21.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:21.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:21.332 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:21.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:21.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:21.458 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:21.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:21.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:21.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:20.445203+0000 osd.7 (osd.7) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:21.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:20.445245+0000 osd.7 (osd.7) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:21.517 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:20.445203+0000 osd.7 (osd.7) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:21.517 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:20.445245+0000 osd.7 (osd.7) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:21.517 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:32:21.568 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:32:21.608 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:32:21.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:21.657 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:32:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:32:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:21.698 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 41 16 50535 50519 76.9353 54.4375 0.0169191 0.0129906 2024-02-22T19:32:21.725 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:32:21.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:21.744 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:21.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:21.761 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:32:21.813 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:32:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:32:21.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:21.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:21.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:22.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:22.049 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:22.129 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:22.130 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:22.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:22.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:32:22.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:21.093985+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e110: 8 total, 8 up, 7 in 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:21.239347+0000 mgr.y (mgr.14535) 134 : cluster [DBG] pgmap v137: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 50 MiB/s wr, 805 op/s; 1870/145359 objects misplaced (1.286%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:21.472110+0000 mon.a (mon.0) 808 : audit [DBG] from='client.? 172.21.15.53:0/1326608602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:21.976383+0000 mon.a (mon.0) 809 : cluster [DBG] osdmap e111: 8 total, 8 up, 7 in 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:21.093985+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e110: 8 total, 8 up, 7 in 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:21.239347+0000 mgr.y (mgr.14535) 134 : cluster [DBG] pgmap v137: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 50 MiB/s wr, 805 op/s; 1870/145359 objects misplaced (1.286%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:21.472110+0000 mon.a (mon.0) 808 : audit [DBG] from='client.? 172.21.15.53:0/1326608602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:22.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:21.976383+0000 mon.a (mon.0) 809 : cluster [DBG] osdmap e111: 8 total, 8 up, 7 in 2024-02-22T19:32:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:21.093985+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e110: 8 total, 8 up, 7 in 2024-02-22T19:32:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:21.239347+0000 mgr.y (mgr.14535) 134 : cluster [DBG] pgmap v137: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 50 MiB/s wr, 805 op/s; 1870/145359 objects misplaced (1.286%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:21.472110+0000 mon.a (mon.0) 808 : audit [DBG] from='client.? 172.21.15.53:0/1326608602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:21.976383+0000 mon.a (mon.0) 809 : cluster [DBG] osdmap e111: 8 total, 8 up, 7 in 2024-02-22T19:32:22.552 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:22.555 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:22.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:22.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:22.701 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 42 16 51379 51363 76.3541 52.75 0.0312889 0.0130917 2024-02-22T19:32:22.755 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:22.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:22.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:23.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:23.130 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:23.150 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:23.153 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:23.154 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:23.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:23.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:21.702719+0000 osd.0 (osd.0) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:23.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:21.702776+0000 osd.0 (osd.0) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:23.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:21.702719+0000 osd.0 (osd.0) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:23.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:21.702776+0000 osd.0 (osd.0) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:23.334 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:21.702719+0000 osd.0 (osd.0) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:21.702776+0000 osd.0 (osd.0) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:23.704 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 43 16 52334 52318 75.9668 59.6875 0.0139633 0.013158 2024-02-22T19:32:24.311 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:24.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:24.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:24.417 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:24.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:32:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:32:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:23.239753+0000 mgr.y (mgr.14535) 135 : cluster [DBG] pgmap v139: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 50 MiB/s wr, 806 op/s; 1870/145359 objects misplaced (1.286%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:23.337477+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:23.428036+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]: dispatch 2024-02-22T19:32:24.701 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 44 16 53283 53267 75.5885 59.3125 0.0154994 0.0132253 2024-02-22T19:32:24.741 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:23.239753+0000 mgr.y (mgr.14535) 135 : cluster [DBG] pgmap v139: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 50 MiB/s wr, 806 op/s; 1870/145359 objects misplaced (1.286%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:24.741 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:23.337477+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:24.741 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:23.428036+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]: dispatch 2024-02-22T19:32:24.742 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:23.239753+0000 mgr.y (mgr.14535) 135 : cluster [DBG] pgmap v139: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 50 MiB/s wr, 806 op/s; 1870/145359 objects misplaced (1.286%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:24.742 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:23.337477+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:24.742 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:23.428036+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]: dispatch 2024-02-22T19:32:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:32:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:32:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:32:25.280 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:32:25.310 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:25.701 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 45 16 54091 54075 75.0312 50.5 0.0199151 0.0133216 2024-02-22T19:32:25.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:24.351613+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]': finished 2024-02-22T19:32:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:24.351674+0000 mon.a (mon.0) 813 : cluster [DBG] osdmap e112: 8 total, 8 up, 7 in 2024-02-22T19:32:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:24.370760+0000 mgr.y (mgr.14535) 136 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:25.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:24.351613+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]': finished 2024-02-22T19:32:25.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:24.351674+0000 mon.a (mon.0) 813 : cluster [DBG] osdmap e112: 8 total, 8 up, 7 in 2024-02-22T19:32:25.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:24.370760+0000 mgr.y (mgr.14535) 136 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:24.351613+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]': finished 2024-02-22T19:32:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:24.351674+0000 mon.a (mon.0) 813 : cluster [DBG] osdmap e112: 8 total, 8 up, 7 in 2024-02-22T19:32:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:24.370760+0000 mgr.y (mgr.14535) 136 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:26.218 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:26.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:26.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:26.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:32:26.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:26.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:32:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:26.377 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:26.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:26.408 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:25.244573+0000 mgr.y (mgr.14535) 137 : cluster [DBG] pgmap v141: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.2 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 47 MiB/s wr, 760 op/s; 1754/159054 objects misplaced (1.103%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:25.369851+0000 mon.a (mon.0) 814 : cluster [DBG] osdmap e113: 8 total, 8 up, 7 in 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:25.488377+0000 mon.b (mon.1) 32 : audit [INF] from='client.? 172.21.15.53:0/1014081990' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:25.489692+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:25.244573+0000 mgr.y (mgr.14535) 137 : cluster [DBG] pgmap v141: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.2 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 47 MiB/s wr, 760 op/s; 1754/159054 objects misplaced (1.103%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:25.369851+0000 mon.a (mon.0) 814 : cluster [DBG] osdmap e113: 8 total, 8 up, 7 in 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:25.488377+0000 mon.b (mon.1) 32 : audit [INF] from='client.? 172.21.15.53:0/1014081990' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:32:26.467 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:25.489692+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:32:26.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:26.529 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:26.532 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:26.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:25.244573+0000 mgr.y (mgr.14535) 137 : cluster [DBG] pgmap v141: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.2 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 47 MiB/s wr, 760 op/s; 1754/159054 objects misplaced (1.103%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:25.369851+0000 mon.a (mon.0) 814 : cluster [DBG] osdmap e113: 8 total, 8 up, 7 in 2024-02-22T19:32:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:25.488377+0000 mon.b (mon.1) 32 : audit [INF] from='client.? 172.21.15.53:0/1014081990' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:32:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:25.489692+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:32:26.633 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:26.705 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 46 16 54941 54925 74.5509 53.125 0.00968351 0.0134068 2024-02-22T19:32:26.739 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:26.789 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:26.793 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:26.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:26.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:26.990 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:27.150 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:27.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:27.252 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:27.304 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:27.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:27.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:27.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:27.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:27.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:27.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 3 1 2024-02-22T19:32:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:26.376305+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:32:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:26.376370+0000 mon.a (mon.0) 817 : cluster [DBG] osdmap e114: 8 total, 8 up, 7 in 2024-02-22T19:32:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:26.987625+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e115: 8 total, 8 up, 7 in 2024-02-22T19:32:27.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:27.652 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:26.376305+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:32:27.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:26.376370+0000 mon.a (mon.0) 817 : cluster [DBG] osdmap e114: 8 total, 8 up, 7 in 2024-02-22T19:32:27.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:26.987625+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e115: 8 total, 8 up, 7 in 2024-02-22T19:32:27.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:26.376305+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:32:27.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:26.376370+0000 mon.a (mon.0) 817 : cluster [DBG] osdmap e114: 8 total, 8 up, 7 in 2024-02-22T19:32:27.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:26.987625+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e115: 8 total, 8 up, 7 in 2024-02-22T19:32:27.684 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:27.706 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 47 16 55792 55776 74.0942 53.1875 0.0134147 0.0134902 2024-02-22T19:32:27.706 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:27.709 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:27.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:27.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:27.870 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:27.912 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:27.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:28.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:28.097 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:28.405 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:28.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:28.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:28.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:28.553 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:28.575 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:28.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:28.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:28.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:28.696 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:27.245182+0000 mgr.y (mgr.14535) 138 : cluster [DBG] pgmap v145: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.2 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1.2 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 1754/159054 objects misplaced (1.103%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:32:28.697 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:27.245182+0000 mgr.y (mgr.14535) 138 : cluster [DBG] pgmap v145: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.2 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1.2 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 1754/159054 objects misplaced (1.103%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:32:28.706 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 48 16 56635 56619 73.6481 52.6875 0.0140534 0.0135715 2024-02-22T19:32:28.779 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:27.245182+0000 mgr.y (mgr.14535) 138 : cluster [DBG] pgmap v145: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.2 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1.2 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 1754/159054 objects misplaced (1.103%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:32:28.924 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:28.929 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:28.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:29.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:29.154 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:29.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:29.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:29.289 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:29.310 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:29.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:29.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:29.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:29.446 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:29.468 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:29.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:29.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:29.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:29.585 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:29.605 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:29.626 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:29.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:29.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:29.706 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 49 16 57441 57425 73.1734 50.375 0.0143856 0.0136603 2024-02-22T19:32:29.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:29.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:32:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:29] "GET /metrics HTTP/1.1" 200 175405 "" "Prometheus/2.33.4" 2024-02-22T19:32:29.759 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:29.787 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:29.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:29.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:29.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:29.947 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:30.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:30.211 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:30.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:30.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:30.223 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:30.316 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:30.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:30.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:30.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:30.488 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:32:30.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:29.245971+0000 mgr.y (mgr.14535) 139 : cluster [DBG] pgmap v146: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 1.0 KiB/s rd, 53 MiB/s wr, 855 op/s; 1638/171597 objects misplaced (0.955%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:30.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:29.530894+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.53:0/2360364117' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:30.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:29.533917+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:30.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:29.245971+0000 mgr.y (mgr.14535) 139 : cluster [DBG] pgmap v146: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 1.0 KiB/s rd, 53 MiB/s wr, 855 op/s; 1638/171597 objects misplaced (0.955%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:30.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:29.530894+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.53:0/2360364117' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:30.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:29.533917+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:29.245971+0000 mgr.y (mgr.14535) 139 : cluster [DBG] pgmap v146: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 1.0 KiB/s rd, 53 MiB/s wr, 855 op/s; 1638/171597 objects misplaced (0.955%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:29.530894+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.53:0/2360364117' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:29.533917+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:32:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:30.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:30.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:30.707 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 50 16 58263 58247 72.7366 51.375 0.0138158 0.0137417 2024-02-22T19:32:30.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:31.709 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 51 16 59137 59121 72.379 54.625 0.0224178 0.0138101 2024-02-22T19:32:31.750 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:30.412766+0000 mon.a (mon.0) 820 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:31.750 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:30.436315+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:32:31.750 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:30.436387+0000 mon.a (mon.0) 822 : cluster [DBG] osdmap e116: 8 total, 8 up, 7 in 2024-02-22T19:32:31.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:30.412766+0000 mon.a (mon.0) 820 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:31.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:30.436315+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:32:31.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:30.436387+0000 mon.a (mon.0) 822 : cluster [DBG] osdmap e116: 8 total, 8 up, 7 in 2024-02-22T19:32:31.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:30.412766+0000 mon.a (mon.0) 820 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:31.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:30.436315+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:32:31.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:30.436387+0000 mon.a (mon.0) 822 : cluster [DBG] osdmap e116: 8 total, 8 up, 7 in 2024-02-22T19:32:32.711 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 52 16 60043 60027 72.0745 56.625 0.0226339 0.0138693 2024-02-22T19:32:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:31.247231+0000 mgr.y (mgr.14535) 140 : cluster [DBG] pgmap v148: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 870 B/s rd, 44 MiB/s wr, 711 op/s; 1638/171597 objects misplaced (0.955%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:31.983297+0000 mon.a (mon.0) 823 : cluster [DBG] osdmap e117: 8 total, 8 up, 7 in 2024-02-22T19:32:32.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:31.247231+0000 mgr.y (mgr.14535) 140 : cluster [DBG] pgmap v148: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 870 B/s rd, 44 MiB/s wr, 711 op/s; 1638/171597 objects misplaced (0.955%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:32.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:31.983297+0000 mon.a (mon.0) 823 : cluster [DBG] osdmap e117: 8 total, 8 up, 7 in 2024-02-22T19:32:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:31.247231+0000 mgr.y (mgr.14535) 140 : cluster [DBG] pgmap v148: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 870 B/s rd, 44 MiB/s wr, 711 op/s; 1638/171597 objects misplaced (0.955%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:31.983297+0000 mon.a (mon.0) 823 : cluster [DBG] osdmap e117: 8 total, 8 up, 7 in 2024-02-22T19:32:33.650 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:32:33.650 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:32:33.710 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 53 16 60980 60964 71.8197 58.5625 0.0166517 0.0139178 2024-02-22T19:32:33.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:31.726147+0000 osd.1 (osd.1) 39 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:33.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:31.726205+0000 osd.1 (osd.1) 40 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:33.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:31.726147+0000 osd.1 (osd.1) 39 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:33.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:31.726205+0000 osd.1 (osd.1) 40 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:33.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:31.726147+0000 osd.1 (osd.1) 39 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:33.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:31.726205+0000 osd.1 (osd.1) 40 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:34.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:32:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:32:34.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:33.250391+0000 mgr.y (mgr.14535) 141 : cluster [DBG] pgmap v150: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 817 B/s rd, 42 MiB/s wr, 668 op/s; 1638/171597 objects misplaced (0.955%); 1.2 MiB/s, 18 objects/s recovering 2024-02-22T19:32:34.660 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:32:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:32:34.714 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 54 16 61864 61848 71.509 55.25 0.0165883 0.0139792 2024-02-22T19:32:34.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:33.250391+0000 mgr.y (mgr.14535) 141 : cluster [DBG] pgmap v150: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 817 B/s rd, 42 MiB/s wr, 668 op/s; 1638/171597 objects misplaced (0.955%); 1.2 MiB/s, 18 objects/s recovering 2024-02-22T19:32:34.840 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:33.250391+0000 mgr.y (mgr.14535) 141 : cluster [DBG] pgmap v150: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.5 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 817 B/s rd, 42 MiB/s wr, 668 op/s; 1638/171597 objects misplaced (0.955%); 1.2 MiB/s, 18 objects/s recovering 2024-02-22T19:32:35.717 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 55 16 62759 62743 71.2242 55.9375 0.017624 0.014034 2024-02-22T19:32:35.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:34.381414+0000 mgr.y (mgr.14535) 142 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:35.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:34.381414+0000 mgr.y (mgr.14535) 142 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:35.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:34.381414+0000 mgr.y (mgr.14535) 142 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:36.715 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 56 15 63668 63653 70.9682 56.875 0.00622617 0.0140843 2024-02-22T19:32:36.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:35.255661+0000 mgr.y (mgr.14535) 143 : cluster [DBG] pgmap v151: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 67 MiB/s wr, 1.07k op/s; 1520/184719 objects misplaced (0.823%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:32:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:35.255661+0000 mgr.y (mgr.14535) 143 : cluster [DBG] pgmap v151: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 67 MiB/s wr, 1.07k op/s; 1520/184719 objects misplaced (0.823%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:32:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:35.255661+0000 mgr.y (mgr.14535) 143 : cluster [DBG] pgmap v151: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 67 MiB/s wr, 1.07k op/s; 1520/184719 objects misplaced (0.823%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:32:37.715 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 57 16 64532 64516 70.6696 53.9375 0.0129174 0.014146 2024-02-22T19:32:38.722 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 58 16 65376 65360 70.3534 52.75 0.0145814 0.0142085 2024-02-22T19:32:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:37.256098+0000 mgr.y (mgr.14535) 144 : cluster [DBG] pgmap v152: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 34 MiB/s wr, 546 op/s; 1520/184719 objects misplaced (0.823%); 943 KiB/s, 14 objects/s recovering 2024-02-22T19:32:39.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:37.256098+0000 mgr.y (mgr.14535) 144 : cluster [DBG] pgmap v152: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 34 MiB/s wr, 546 op/s; 1520/184719 objects misplaced (0.823%); 943 KiB/s, 14 objects/s recovering 2024-02-22T19:32:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:37.256098+0000 mgr.y (mgr.14535) 144 : cluster [DBG] pgmap v152: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 34 MiB/s wr, 546 op/s; 1520/184719 objects misplaced (0.823%); 943 KiB/s, 14 objects/s recovering 2024-02-22T19:32:39.722 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 59 16 66221 66205 70.0563 52.8125 0.0220387 0.0142679 2024-02-22T19:32:39.758 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:32:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:39] "GET /metrics HTTP/1.1" 200 175432 "" "Prometheus/2.33.4" 2024-02-22T19:32:40.722 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:32:40.722665+0000 min lat: 0.00116596 max lat: 2.0693 avg lat: 0.0143205 2024-02-22T19:32:40.722 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:32:40.722 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 60 16 67102 67086 69.8066 55.0625 0.0150545 0.0143205 2024-02-22T19:32:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:39.258726+0000 mgr.y (mgr.14535) 145 : cluster [DBG] pgmap v153: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 62 MiB/s wr, 997 op/s; 1404/197964 objects misplaced (0.709%); 1.7 MiB/s, 26 objects/s recovering 2024-02-22T19:32:41.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:39.258726+0000 mgr.y (mgr.14535) 145 : cluster [DBG] pgmap v153: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 62 MiB/s wr, 997 op/s; 1404/197964 objects misplaced (0.709%); 1.7 MiB/s, 26 objects/s recovering 2024-02-22T19:32:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:39.258726+0000 mgr.y (mgr.14535) 145 : cluster [DBG] pgmap v153: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 62 MiB/s wr, 997 op/s; 1404/197964 objects misplaced (0.709%); 1.7 MiB/s, 26 objects/s recovering 2024-02-22T19:32:41.724 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 61 16 67986 67970 69.5654 55.25 0.0285577 0.0143689 2024-02-22T19:32:42.331 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:42.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:42.337 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:42.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:42.492 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:42.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:42.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:42.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:42.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:41.259162+0000 mgr.y (mgr.14535) 146 : cluster [DBG] pgmap v154: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1022 B/s rd, 55 MiB/s wr, 878 op/s; 1404/197964 objects misplaced (0.709%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:42.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:41.259162+0000 mgr.y (mgr.14535) 146 : cluster [DBG] pgmap v154: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1022 B/s rd, 55 MiB/s wr, 878 op/s; 1404/197964 objects misplaced (0.709%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:42.658 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:42.724 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 62 16 68786 68770 69.2501 50 0.0177741 0.0144356 2024-02-22T19:32:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:41.259162+0000 mgr.y (mgr.14535) 146 : cluster [DBG] pgmap v154: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1022 B/s rd, 55 MiB/s wr, 878 op/s; 1404/197964 objects misplaced (0.709%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:43.724 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 63 16 69609 69593 68.9676 51.4375 0.0182632 0.0144942 2024-02-22T19:32:44.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:32:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:32:44.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:43.260457+0000 mgr.y (mgr.14535) 147 : cluster [DBG] pgmap v155: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 908 B/s rd, 49 MiB/s wr, 780 op/s; 1404/197964 objects misplaced (0.709%); 1.3 MiB/s, 20 objects/s recovering 2024-02-22T19:32:44.660 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:32:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:32:44.724 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 64 16 70461 70445 68.7221 53.25 0.0197793 0.0145459 2024-02-22T19:32:44.761 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:44.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:44.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:44.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:32:44.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:44.897 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:44.907 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:43.260457+0000 mgr.y (mgr.14535) 147 : cluster [DBG] pgmap v155: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 908 B/s rd, 49 MiB/s wr, 780 op/s; 1404/197964 objects misplaced (0.709%); 1.3 MiB/s, 20 objects/s recovering 2024-02-22T19:32:44.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:43.260457+0000 mgr.y (mgr.14535) 147 : cluster [DBG] pgmap v155: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 908 B/s rd, 49 MiB/s wr, 780 op/s; 1404/197964 objects misplaced (0.709%); 1.3 MiB/s, 20 objects/s recovering 2024-02-22T19:32:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:44.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:44.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:45.022 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:45.155 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:45.726 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 65 16 71156 71140 68.3313 43.4375 0.020279 0.0146283 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:32:45.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:32:45.760 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:32:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:45.775 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:45.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:45.803 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:32:45.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:45.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:44.391301+0000 mgr.y (mgr.14535) 148 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:45.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:45.448641+0000 mon.b (mon.1) 34 : audit [DBG] from='client.? 172.21.15.53:0/1858005526' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:45.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:44.391301+0000 mgr.y (mgr.14535) 148 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:45.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:45.448641+0000 mon.b (mon.1) 34 : audit [DBG] from='client.? 172.21.15.53:0/1858005526' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:45.964 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:44.391301+0000 mgr.y (mgr.14535) 148 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:45.448641+0000 mon.b (mon.1) 34 : audit [DBG] from='client.? 172.21.15.53:0/1858005526' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:46.182 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:46.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:46.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:46.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:32:46.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:46.315 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:46.337 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:46.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:46.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:46.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:46.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:46.500 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:46.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:46.651 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:46.654 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:46.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:46.731 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 66 16 71901 71885 67.9991 46.5625 0.0248127 0.0147004 2024-02-22T19:32:46.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:46.784 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:46.813 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:46.817 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:46.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:46.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:47.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:45.261786+0000 mgr.y (mgr.14535) 149 : cluster [DBG] pgmap v156: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 1288/210636 objects misplaced (0.611%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:32:47.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:45.261786+0000 mgr.y (mgr.14535) 149 : cluster [DBG] pgmap v156: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 1288/210636 objects misplaced (0.611%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:32:47.010 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:47.090 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:47.090 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:47.097 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:47.097 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:45.261786+0000 mgr.y (mgr.14535) 149 : cluster [DBG] pgmap v156: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 1288/210636 objects misplaced (0.611%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:32:47.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:47.265 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:47.387 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:47.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:47.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:47.516 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:47.538 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:47.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:47.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:47.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:47.674 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:47.696 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:47.700 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:47.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:47.736 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 67 16 72606 72590 67.6357 44.0625 0.0311513 0.01478 2024-02-22T19:32:47.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:47.830 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:47.851 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:47.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:47.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:47.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:47.968 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:48.023 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.3 to 1 (10000) 2024-02-22T19:32:48.050 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:47.126193+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.53:0/957838277' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:32:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:47.131671+0000 mon.a (mon.0) 824 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:32:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:47.264216+0000 mgr.y (mgr.14535) 150 : cluster [DBG] pgmap v157: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 720 op/s; 1288/210636 objects misplaced (0.611%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:48.104 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:48.125 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:48.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:48.226 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:48.304 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:47.126193+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.53:0/957838277' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:32:48.304 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:47.131671+0000 mon.a (mon.0) 824 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:32:48.305 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:47.264216+0000 mgr.y (mgr.14535) 150 : cluster [DBG] pgmap v157: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 720 op/s; 1288/210636 objects misplaced (0.611%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:48.305 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:47.126193+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.53:0/957838277' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:32:48.305 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:47.131671+0000 mon.a (mon.0) 824 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:32:48.305 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:47.264216+0000 mgr.y (mgr.14535) 150 : cluster [DBG] pgmap v157: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 720 op/s; 1288/210636 objects misplaced (0.611%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:48.315 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:48.708 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:48.719 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:48.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:48.737 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 68 16 73374 73358 67.3457 48 0.0257667 0.0148431 2024-02-22T19:32:48.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:48.885 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:48.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:48.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:49.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:32:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:32:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:49.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:32:49.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:32:49.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:49.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:48.000472+0000 mon.a (mon.0) 825 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:32:49.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:48.000535+0000 mon.a (mon.0) 826 : cluster [DBG] osdmap e118: 8 total, 8 up, 7 in 2024-02-22T19:32:49.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:48.000472+0000 mon.a (mon.0) 825 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:32:49.214 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:48.000535+0000 mon.a (mon.0) 826 : cluster [DBG] osdmap e118: 8 total, 8 up, 7 in 2024-02-22T19:32:49.263 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:48.000472+0000 mon.a (mon.0) 825 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:32:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:48.000535+0000 mon.a (mon.0) 826 : cluster [DBG] osdmap e118: 8 total, 8 up, 7 in 2024-02-22T19:32:49.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 4 1 2024-02-22T19:32:49.506 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:32:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:49] "GET /metrics HTTP/1.1" 200 175432 "" "Prometheus/2.33.4" 2024-02-22T19:32:49.739 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 69 16 74047 74031 66.9777 42.0625 0.0305717 0.0149243 2024-02-22T19:32:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:49.265877+0000 mgr.y (mgr.14535) 151 : cluster [DBG] pgmap v159: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 49 MiB/s wr, 792 op/s; 1171/221730 objects misplaced (0.528%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:50.500 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:49.265877+0000 mgr.y (mgr.14535) 151 : cluster [DBG] pgmap v159: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 49 MiB/s wr, 792 op/s; 1171/221730 objects misplaced (0.528%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:50.500 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:49.265877+0000 mgr.y (mgr.14535) 151 : cluster [DBG] pgmap v159: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 49 MiB/s wr, 792 op/s; 1171/221730 objects misplaced (0.528%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:50.743 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 70 16 74783 74767 66.6747 46 0.0251369 0.0149929 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "MOSDPGPush(4.7 84/83 [PushOp(4:e47c9322:gc::gc.29:head, version: 64'7, data_included: [0~110], data_size: 110, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(4:e47c9322:gc::gc.29:head@64'7, size: 110, copy_subset: [0~110], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:110, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-02-22T19:32:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:30:24.670257+0000", 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 145.79556322400001, 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086187427999999996, 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.670257+0000", 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.670257+0000", 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.670256+0000", 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.647e-05 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.670273+0000", 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9147000000000002e-05 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.670292+0000", 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1902000000000001e-05 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.670304+0000", 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085768288999999998 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.756072+0000", 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2880000000000004e-06 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.756079+0000", 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036553200000000001 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:30:24.756445+0000", 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086187427999999996 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20820 7.e 7:75479808:::benchmark_data_smithi053_107660_object20819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:32:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.485008510999997, 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0656461249999998, 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20820 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980810+0000", 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.211e-05 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980833+0000", 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8699999999999992e-06 2024-02-22T19:32:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980840+0000", 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9620000000000001e-06 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980845+0000", 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084919699999999999 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981695+0000", 2024-02-22T19:32:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000152538 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981847+0000", 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117994 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5", 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981965+0000", 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001797442 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.983763+0000", 2024-02-22T19:32:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007165561 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990928+0000", 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6160000000000008e-06 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990938+0000", 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004499088 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995437+0000", 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2568000000000001e-05 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995449+0000", 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509275339999999 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046377+0000", 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1163999999999997e-05 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046458+0000", 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0656461249999998 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20825 7.e 7:74b6997e:::benchmark_data_smithi053_107660_object20824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.484041396000002, 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0648078949999999, 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20825 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:32:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981778+0000", 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6081000000000006e-05 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981844+0000", 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7260000000000002e-06 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981852+0000", 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0959999999999997e-06 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981858+0000", 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088905199999999998 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.982747+0000", 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118694 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.982866+0000", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014208600000000001 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.983008+0000", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00131347 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.984321+0000", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006647231 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990968+0000", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.109e-06 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990976+0000", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004509871 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995485+0000", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4160000000000004e-06 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995494+0000", 2024-02-22T19:32:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0510047179999999 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046499+0000", 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8375999999999995e-05 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046587+0000", 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0648078949999999 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20834 7.e 7:73899197:::benchmark_data_smithi053_107660_object20833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:32:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.478043491000001, 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0674701959999999, 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20834 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987775+0000", 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0616999999999999e-05 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987796+0000", 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7879999999999995e-06 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987805+0000", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9380000000000003e-06 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987812+0000", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043811340000000001 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.992193+0000", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9510000000000001e-06 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.992203+0000", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003316238 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995519+0000", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0299999999999994e-06 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995527+0000", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0510881539999999 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046615+0000", 2024-02-22T19:32:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012597899999999999 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046741+0000", 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014336500000000001 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046885+0000", 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073188910000000001 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054204+0000", 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1601999999999997e-05 2024-02-22T19:32:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054265+0000", 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010688699999999999 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054372+0000", 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2679000000000001e-05 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054435+0000", 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081248500000000005 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055247+0000", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0674701959999999 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20860 7.e 7:7d18546b:::benchmark_data_smithi053_107660_object20859:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.441573157999997, 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.034585796, 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20860 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024245+0000", 2024-02-22T19:32:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7005e-05 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024272+0000", 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3974999999999999e-05 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024286+0000", 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6139999999999997e-06 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024293+0000", 2024-02-22T19:32:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6604000000000002e-05 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024349+0000", 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6550000000000004e-06 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024357+0000", 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.971194102 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995551+0000", 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3649999999999998e-06 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995558+0000", 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521218619999999 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047680+0000", 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114424 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047795+0000", 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210046 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048005+0000", 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062518670000000004 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054257+0000", 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5546e-05 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054272+0000", 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003673805 2024-02-22T19:32:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057946+0000", 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8497000000000005e-05 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058014+0000", 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081864399999999999 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058833+0000", 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.034585796 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20873 7.e 7:70011a90:::benchmark_data_smithi053_107660_object20872:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.437854190000003, 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0315724149999999, 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20873 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027965+0000", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3580000000000001e-05 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027988+0000", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.577e-05 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028004+0000", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028011+0000", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6941999999999999e-05 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028058+0000", 2024-02-22T19:32:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3840000000000002e-06 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028064+0000", 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96751664000000004 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995581+0000", 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3509999999999996e-06 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995588+0000", 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0532674879999999 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048856+0000", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015597000000000001 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049012+0000", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015027700000000001 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049162+0000", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051174610000000002 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054279+0000", 2024-02-22T19:32:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.541e-05 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054335+0000", 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045385269999999997 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058873+0000", 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5123000000000001e-05 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058919+0000", 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062019400000000002 2024-02-22T19:32:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059539+0000", 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0315724149999999 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20878 7.e 7:7891943c:::benchmark_data_smithi053_107660_object20877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.43514038, 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0297337099999999, 2024-02-22T19:32:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20878 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030678+0000", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2857000000000002e-05 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030700+0000", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3713000000000001e-05 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030714+0000", 2024-02-22T19:32:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1090000000000003e-06 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030720+0000", 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9595000000000002e-05 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030770+0000", 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1749999999999994e-06 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030779+0000", 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96483299300000003 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995612+0000", 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5809999999999999e-06 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995620+0000", 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0543208209999999 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049940+0000", 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7372000000000003e-05 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050028+0000", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013467700000000001 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050162+0000", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041241170000000001 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054287+0000", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010961230000000001 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055383+0000", 2024-02-22T19:32:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042013659999999998 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059584+0000", 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8198000000000001e-05 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059642+0000", 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077155300000000002 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060414+0000", 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0297337099999999 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20883 7.e 7:7c7f3c6f:::benchmark_data_smithi053_107660_object20882:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.431190190000002, 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265122409999998, 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20883 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034628+0000", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6835e-05 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034655+0000", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8521000000000001e-05 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034674+0000", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2910000000000002e-06 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034681+0000", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9109999999999994e-05 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034750+0000", 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0144e-05 2024-02-22T19:32:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034760+0000", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96088248899999995 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995643+0000", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995650+0000", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.055232344 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050883+0000", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016314899999999999 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051046+0000", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000208643 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051255+0000", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030399609999999999 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054295+0000", 2024-02-22T19:32:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061688639999999996 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060463+0000", 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0683000000000002e-05 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060524+0000", 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1838e-05 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060586+0000", 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055658699999999999 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061143+0000", 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265122409999998 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20886 7.e 7:75e1811d:::benchmark_data_smithi053_107660_object20885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.429846675999997, 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0257961170000001, 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20886 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035972+0000", 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7784e-05 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036000+0000", 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8953e-05 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036019+0000", 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1119999999999996e-06 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036027+0000", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106254 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036133+0000", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2412000000000001e-05 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036146+0000", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95952591799999998 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995672+0000", 2024-02-22T19:32:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5299999999999999e-06 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995679+0000", 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0562192079999999 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051898+0000", 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000153024 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052051+0000", 2024-02-22T19:32:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195556 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052247+0000", 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049309180000000003 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057178+0000", 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040298340000000004 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061208+0000", 2024-02-22T19:32:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6218999999999995e-05 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061304+0000", 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3900000000000002e-05 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061358+0000", 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041208899999999998 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061770+0000", 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0257961170000001 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:691 5.d 5:b4812045:::notify.4:head [watch ping cookie 94129597100544] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154335+0000", 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.31148503, 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86412915000000001, 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:32:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 691 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154335+0000", 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154335+0000", 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154333+0000", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3249e-05 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154347+0000", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1719999999999999e-05 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154368+0000", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.078e-05 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154379+0000", 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111048 2024-02-22T19:32:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154490+0000", 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2326e-05 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154502+0000", 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.860437224 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.014940+0000", 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034966700000000001 2024-02-22T19:32:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.015289+0000", 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029997380000000001 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.018289+0000", 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0181999999999999e-05 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.018369+0000", 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5346999999999994e-05 2024-02-22T19:32:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.018465+0000", 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86412915000000001 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:699 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94129597100544 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154758+0000", 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.311062864, 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86851007199999997, 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:32:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 699 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154758+0000", 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154758+0000", 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154756+0000", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7130000000000005e-06 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154764+0000", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3845e-05 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154778+0000", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154785+0000", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5505000000000004e-05 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154850+0000", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2819999999999996e-06 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154859+0000", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86045618700000004 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.015315+0000", 2024-02-22T19:32:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3312000000000005e-05 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.015398+0000", 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031180069999999999 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.018516+0000", 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3959e-05 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.018600+0000", 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046674519999999999 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.023268+0000", 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86851007199999997 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:47832 7.e 7:7132670d:::benchmark_data_smithi053_107660_object47831:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:18.799239+0000", 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.666581823000001, 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042347182999999997, 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47832 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.799239+0000", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.799239+0000", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.799238+0000", 2024-02-22T19:32:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3659000000000002e-05 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.799262+0000", 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8329999999999996e-06 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.799269+0000", 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9239999999999998e-06 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.799273+0000", 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022928619000000001 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.822202+0000", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013889300000000001 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.822341+0000", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022916399999999999 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.822570+0000", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0092153640000000002 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.831786+0000", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020693640000000002 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.833855+0000", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069304939999999997 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.840785+0000", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6756999999999998e-05 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.840852+0000", 2024-02-22T19:32:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00073371600000000003 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:18.841586+0000", 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042347182999999997 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:49129 7.e 7:778d9991:::benchmark_data_smithi053_107660_object49128:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:20.142131+0000", 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.323688988000001, 2024-02-22T19:32:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045063854, 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 49129 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142131+0000", 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142131+0000", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142130+0000", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5003999999999999e-05 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142165+0000", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0500999999999999e-05 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142185+0000", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5110000000000003e-06 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142194+0000", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144146 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142338+0000", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000164706 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142502+0000", 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020327 2024-02-22T19:32:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142706+0000", 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020592340000000001 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.144765+0000", 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012840956000000001 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.157606+0000", 2024-02-22T19:32:50.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028822381000000001 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.186428+0000", 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2978999999999995e-05 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.186491+0000", 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070408199999999997 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.187195+0000", 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045063854 2024-02-22T19:32:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:49134 7.e 7:7c086599:::benchmark_data_smithi053_107660_object49133:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:20.142264+0000", 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.323556867000001, 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045270871999999997, 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 49134 2024-02-22T19:32:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142264+0000", 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142264+0000", 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142263+0000", 2024-02-22T19:32:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7841e-05 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142291+0000", 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.472e-06 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142297+0000", 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.164e-06 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.142302+0000", 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0088840550000000001 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.151187+0000", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000196646 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.151383+0000", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083760099999999999 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.152221+0000", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044918650000000003 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.156713+0000", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095373220000000009 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.166250+0000", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020975066000000001 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.187225+0000", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1183000000000003e-05 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.187256+0000", 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000278249 2024-02-22T19:32:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:20.187534+0000", 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045270871999999997 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:72067 7.e 7:72bfb43a:::benchmark_data_smithi053_107660_object72066:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.962016+0000", 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5038042379999998, 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048095003999999997, 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 72067 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962016+0000", 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962016+0000", 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962014+0000", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9430000000000001e-05 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962044+0000", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0930999999999999e-05 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962055+0000", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.691e-06 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962061+0000", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000204928 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962266+0000", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013878299999999999 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962405+0000", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017322600000000001 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.962578+0000", 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0076023769999999996 2024-02-22T19:32:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.970181+0000", 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021477320000000001 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.991658+0000", 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017849199999999999 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.009507+0000", 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0958999999999997e-05 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.009568+0000", 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054295500000000002 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.010111+0000", 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048095003999999997 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:72097 7.e 7:7ca49aa8:::benchmark_data_smithi053_107660_object72096:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:47.017356+0000", 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.4484646350000001, 2024-02-22T19:32:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047282049999999999, 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 72097 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.017356+0000", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.017356+0000", 2024-02-22T19:32:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.017353+0000", 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0120999999999998e-05 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.017393+0000", 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6979999999999999e-05 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.017420+0000", 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1297e-05 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.017431+0000", 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001407922 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.018839+0000", 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015442900000000001 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.018994+0000", 2024-02-22T19:32:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020867600000000001 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.019202+0000", 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044058609999999996 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.023608+0000", 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015632665 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.039241+0000", 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024919486000000001 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.064160+0000", 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8473000000000001e-05 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.064219+0000", 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041909899999999998 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.064638+0000", 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047282049999999999 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:73047 7.e 7:7902a045:::benchmark_data_smithi053_107660_object73046:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:48.317261+0000", 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.148559208, 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048095207000000001, 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73047 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.317261+0000", 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.317261+0000", 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.317261+0000", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2804999999999999e-05 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.317284+0000", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5600000000000001e-06 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.317289+0000", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9960000000000004e-06 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.317293+0000", 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077562220000000001 2024-02-22T19:32:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.325049+0000", 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019274200000000001 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.325242+0000", 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021379 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.325456+0000", 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074626650000000003 2024-02-22T19:32:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.332919+0000", 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050517180000000002 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.337970+0000", 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026944155000000001 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.364914+0000", 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8627e-05 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.364963+0000", 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039340499999999999 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.365356+0000", 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048095207000000001 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:73742 7.e 7:71c1b3dc:::benchmark_data_smithi053_107660_object73741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:49.253489+0000", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.212331713, 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048138954999999997, 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73742 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.253489+0000", 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.253489+0000", 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.253488+0000", 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1523999999999999e-05 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.253510+0000", 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8000000000000001e-06 2024-02-22T19:32:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.253516+0000", 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1869999999999999e-06 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.253521+0000", 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055042390000000002 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.259025+0000", 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019203199999999999 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.259217+0000", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021540599999999999 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.259432+0000", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010183019999999999 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.269615+0000", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014782809999999999 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.271094+0000", 2024-02-22T19:32:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029951543000000001 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.301045+0000", 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8800000000000004e-05 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.301124+0000", 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000503773 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.301628+0000", 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048138954999999997 2024-02-22T19:32:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:73878 7.e 7:7ca2ad95:::benchmark_data_smithi053_107660_object73877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:49.484333+0000", 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.98148779100000005, 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050972302999999997, 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73878 2024-02-22T19:32:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.484333+0000", 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.484333+0000", 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.484331+0000", 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9070000000000001e-05 2024-02-22T19:32:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.484360+0000", 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7855000000000001e-05 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.484378+0000", 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1040000000000007e-06 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.484386+0000", 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012500581 2024-02-22T19:32:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.496887+0000", 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015619599999999999 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.497043+0000", 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022173200000000001 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.497265+0000", 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049457939999999999 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.502210+0000", 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0087284400000000005 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.510939+0000", 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023914667000000001 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.534853+0000", 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0605999999999997e-05 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.534894+0000", 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041096600000000002 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.535305+0000", 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050972302999999997 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:74033 7.e 7:7f07b824:::benchmark_data_smithi053_107660_object74032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:49.717026+0000", 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.748793979, 2024-02-22T19:32:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041000855000000003, 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74033 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.717026+0000", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.717026+0000", 2024-02-22T19:32:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.717026+0000", 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3540000000000002e-05 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.717049+0000", 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0590000000000003e-06 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.717057+0000", 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9320000000000004e-06 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.717062+0000", 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021187369999999999 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.719181+0000", 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017887900000000001 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.719360+0000", 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023036700000000001 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.719590+0000", 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022081480000000001 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.721798+0000", 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0056270879999999997 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.727426+0000", 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030202343 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.757628+0000", 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0381999999999999e-05 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.757688+0000", 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033905300000000002 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.758027+0000", 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041000855000000003 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:32:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:32:50.871 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:32:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:51.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:51.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:51.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:51.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:51.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:51.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:51.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:51.322414+0000 mon.a (mon.0) 827 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-02-22T19:32:51.512 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:51.322414+0000 mon.a (mon.0) 827 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-02-22T19:32:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:51.322414+0000 mon.a (mon.0) 827 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-02-22T19:32:51.736 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:51.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:51.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:51.746 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 71 15 75537 75522 66.3978 47.1875 0.025244 0.015053 2024-02-22T19:32:51.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:32:51.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:51.903 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:51.914 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:51.980 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:32:52.468 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:52.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:52.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:52.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:52.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:51.266306+0000 mgr.y (mgr.14535) 152 : cluster [DBG] pgmap v160: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 49 MiB/s wr, 792 op/s; 1171/221730 objects misplaced (0.528%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:52.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:51.931489+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.53:0/4248000535' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:32:52.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:51.932467+0000 mon.a (mon.0) 828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:32:52.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:51.987002+0000 mon.a (mon.0) 829 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:52.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:52.003578+0000 mon.a (mon.0) 830 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:32:52.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:52.003675+0000 mon.a (mon.0) 831 : cluster [DBG] osdmap e119: 8 total, 8 up, 7 in 2024-02-22T19:32:52.655 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:51.266306+0000 mgr.y (mgr.14535) 152 : cluster [DBG] pgmap v160: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 49 MiB/s wr, 792 op/s; 1171/221730 objects misplaced (0.528%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:52.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:51.931489+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.53:0/4248000535' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:32:52.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:51.932467+0000 mon.a (mon.0) 828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:32:52.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:51.987002+0000 mon.a (mon.0) 829 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:52.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:52.003578+0000 mon.a (mon.0) 830 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:32:52.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:52.003675+0000 mon.a (mon.0) 831 : cluster [DBG] osdmap e119: 8 total, 8 up, 7 in 2024-02-22T19:32:52.656 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:52.749 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 72 16 76215 76199 66.0605 42.3125 0.010409 0.0151299 2024-02-22T19:32:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:51.266306+0000 mgr.y (mgr.14535) 152 : cluster [DBG] pgmap v160: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 49 MiB/s wr, 792 op/s; 1171/221730 objects misplaced (0.528%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:32:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:51.931489+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.53:0/4248000535' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:32:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:51.932467+0000 mon.a (mon.0) 828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:32:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:51.987002+0000 mon.a (mon.0) 829 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:32:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:52.003578+0000 mon.a (mon.0) 830 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:32:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:52.003675+0000 mon.a (mon.0) 831 : cluster [DBG] osdmap e119: 8 total, 8 up, 7 in 2024-02-22T19:32:53.678 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:53.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:53.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:53.756 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 73 16 77004 76988 65.8255 49.3125 0.0142471 0.015186 2024-02-22T19:32:53.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:54.025 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:54.033 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:54.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:54.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:54.189 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:54.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:53.266732+0000 mgr.y (mgr.14535) 153 : cluster [DBG] pgmap v162: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 639 B/s rd, 29 MiB/s wr, 462 op/s; 1171/221730 objects misplaced (0.528%); 935 KiB/s, 14 objects/s recovering 2024-02-22T19:32:54.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:53.337468+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:54.458 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:32:54.461 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:54.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:32:54.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:32:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:53.266732+0000 mgr.y (mgr.14535) 153 : cluster [DBG] pgmap v162: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 639 B/s rd, 29 MiB/s wr, 462 op/s; 1171/221730 objects misplaced (0.528%); 935 KiB/s, 14 objects/s recovering 2024-02-22T19:32:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:53.337468+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:54.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:32:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:32:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:53.266732+0000 mgr.y (mgr.14535) 153 : cluster [DBG] pgmap v162: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 639 B/s rd, 29 MiB/s wr, 462 op/s; 1171/221730 objects misplaced (0.528%); 935 KiB/s, 14 objects/s recovering 2024-02-22T19:32:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:53.337468+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:32:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:54.758 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 74 16 77759 77743 65.5721 47.1875 0.0109303 0.0152437 2024-02-22T19:32:54.992 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:32:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:32:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:32:55.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:32:55.500 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:54.121939+0000 osd.4 (osd.4) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:55.500 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:54.121990+0000 osd.4 (osd.4) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:55.500 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:54.403882+0000 mgr.y (mgr.14535) 154 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:55.500 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:54.414743+0000 mon.a (mon.0) 833 : cluster [DBG] osdmap e120: 8 total, 8 up, 7 in 2024-02-22T19:32:55.500 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:54.121939+0000 osd.4 (osd.4) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:55.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:54.121990+0000 osd.4 (osd.4) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:55.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:54.403882+0000 mgr.y (mgr.14535) 154 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:55.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:54.414743+0000 mon.a (mon.0) 833 : cluster [DBG] osdmap e120: 8 total, 8 up, 7 in 2024-02-22T19:32:55.769 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 75 16 78464 78448 65.2844 44.0625 0.0227511 0.0153116 2024-02-22T19:32:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:54.121939+0000 osd.4 (osd.4) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:54.121990+0000 osd.4 (osd.4) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:54.403882+0000 mgr.y (mgr.14535) 154 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:32:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:54.414743+0000 mon.a (mon.0) 833 : cluster [DBG] osdmap e120: 8 total, 8 up, 7 in 2024-02-22T19:32:55.895 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:32:55.904 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:32:55.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:55.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:32:55.939 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:32:55.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:32:56.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:56.039 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:32:56.043 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:56.065 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:56.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:56.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:56.072 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:32:56.112 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:32:56.143 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:32:56.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:56.182 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:32:56.207 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.207 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:32:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:56.361 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:56.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:56.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:56.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:32:56.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:54.760741+0000 osd.3 (osd.3) 59 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:56.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:54.760808+0000 osd.3 (osd.3) 60 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:56.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:55.268191+0000 mgr.y (mgr.14535) 155 : cluster [DBG] pgmap v164: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 1055/232638 objects misplaced (0.453%); 2.0 MiB/s, 32 objects/s recovering 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:55.446549+0000 mon.a (mon.0) 834 : cluster [DBG] osdmap e121: 8 total, 8 up, 7 in 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:55.826363+0000 mon.a (mon.0) 835 : audit [DBG] from='client.? 172.21.15.53:0/2314868189' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:54.760741+0000 osd.3 (osd.3) 59 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:54.760808+0000 osd.3 (osd.3) 60 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:55.268191+0000 mgr.y (mgr.14535) 155 : cluster [DBG] pgmap v164: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 1055/232638 objects misplaced (0.453%); 2.0 MiB/s, 32 objects/s recovering 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:55.446549+0000 mon.a (mon.0) 834 : cluster [DBG] osdmap e121: 8 total, 8 up, 7 in 2024-02-22T19:32:56.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:55.826363+0000 mon.a (mon.0) 835 : audit [DBG] from='client.? 172.21.15.53:0/2314868189' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:56.556 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:56.760 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 76 16 79193 79177 65.0251 45.5625 0.00990831 0.0153733 2024-02-22T19:32:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:54.760741+0000 osd.3 (osd.3) 59 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:32:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:54.760808+0000 osd.3 (osd.3) 60 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:32:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:55.268191+0000 mgr.y (mgr.14535) 155 : cluster [DBG] pgmap v164: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 1055/232638 objects misplaced (0.453%); 2.0 MiB/s, 32 objects/s recovering 2024-02-22T19:32:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:55.446549+0000 mon.a (mon.0) 834 : cluster [DBG] osdmap e121: 8 total, 8 up, 7 in 2024-02-22T19:32:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:55.826363+0000 mon.a (mon.0) 835 : audit [DBG] from='client.? 172.21.15.53:0/2314868189' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:32:57.163 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:32:57.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:57.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:32:57.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:57.336 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:57.364 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:57.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:57.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:57.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:57.524 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:57.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:57.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:57.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:32:57.751 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:57.760 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 77 16 79903 79887 64.7568 44.375 0.0257058 0.0154349 2024-02-22T19:32:58.059 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-02-22T19:32:58.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:57.269349+0000 mgr.y (mgr.14535) 156 : cluster [DBG] pgmap v166: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 38 MiB/s wr, 606 op/s; 1055/232638 objects misplaced (0.453%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:58.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:57.897561+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.53:0/954364493' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:32:58.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:57.898101+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:32:58.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:57.269349+0000 mgr.y (mgr.14535) 156 : cluster [DBG] pgmap v166: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 38 MiB/s wr, 606 op/s; 1055/232638 objects misplaced (0.453%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:58.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:57.897561+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.53:0/954364493' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:32:58.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:57.898101+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:32:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:57.269349+0000 mgr.y (mgr.14535) 156 : cluster [DBG] pgmap v166: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 38 MiB/s wr, 606 op/s; 1055/232638 objects misplaced (0.453%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:32:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:57.897561+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.53:0/954364493' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:32:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:57.898101+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:32:58.435 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:32:58.435 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:32:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:32:58.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:58.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:32:58.763 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 78 16 80654 80638 64.5261 46.9375 0.0127167 0.0154924 2024-02-22T19:32:58.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:32:58.807 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:58.830 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:32:58.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:58.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:32:58.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:32:58.966 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:32:58.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:58.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:32:59.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:32:58.023931+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:32:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:32:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:58.023990+0000 mon.a (mon.0) 838 : cluster [DBG] osdmap e122: 8 total, 8 up, 7 in 2024-02-22T19:32:59.120 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:59.146 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:59.148 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:59.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:59.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:32:59.278 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:59.302 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:32:58.023931+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:32:59.302 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:32:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:58.023990+0000 mon.a (mon.0) 838 : cluster [DBG] osdmap e122: 8 total, 8 up, 7 in 2024-02-22T19:32:59.303 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:32:58.023931+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:32:59.303 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:32:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:58.023990+0000 mon.a (mon.0) 838 : cluster [DBG] osdmap e122: 8 total, 8 up, 7 in 2024-02-22T19:32:59.306 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:32:59.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:32:59.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:32:59.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:32:59.490 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:32:59.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:32:59.566 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:32:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:32:59] "GET /metrics HTTP/1.1" 200 175533 "" "Prometheus/2.33.4" 2024-02-22T19:32:59.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 79 16 81403 81387 64.302 46.8125 0.00794839 0.0155455 2024-02-22T19:33:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:00.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:00.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:00.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 6 1 2024-02-22T19:33:00.164 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:00.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:00.189 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:00.219 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:00.244 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:00.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:00.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:32:59.270169+0000 mgr.y (mgr.14535) 157 : cluster [DBG] pgmap v168: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.7 KiB/s rd, 76 MiB/s wr, 1.22k op/s; 938/243642 objects misplaced (0.385%); 2.4 MiB/s, 38 objects/s recovering 2024-02-22T19:33:00.375 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:00.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:33:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:00.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:00.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:00.460 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:32:59.270169+0000 mgr.y (mgr.14535) 157 : cluster [DBG] pgmap v168: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.7 KiB/s rd, 76 MiB/s wr, 1.22k op/s; 938/243642 objects misplaced (0.385%); 2.4 MiB/s, 38 objects/s recovering 2024-02-22T19:33:00.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:32:59.270169+0000 mgr.y (mgr.14535) 157 : cluster [DBG] pgmap v168: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.7 KiB/s rd, 76 MiB/s wr, 1.22k op/s; 938/243642 objects misplaced (0.385%); 2.4 MiB/s, 38 objects/s recovering 2024-02-22T19:33:00.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:00.615 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:00.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:33:00.764734+0000 min lat: 0.00116596 max lat: 2.0693 avg lat: 0.0156177 2024-02-22T19:33:00.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:33:00.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 80 16 82053 82037 64.0064 40.625 0.0227181 0.0156177 2024-02-22T19:33:00.794 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:00.800 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:00.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:00.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:01.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 81 16 82802 82786 63.7943 46.8125 0.0142516 0.0156706 2024-02-22T19:33:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:01.271338+0000 mgr.y (mgr.14535) 158 : cluster [DBG] pgmap v169: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 746 B/s rd, 33 MiB/s wr, 535 op/s; 938/243642 objects misplaced (0.385%); 1.1 MiB/s, 17 objects/s recovering 2024-02-22T19:33:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:01.271338+0000 mgr.y (mgr.14535) 158 : cluster [DBG] pgmap v169: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 746 B/s rd, 33 MiB/s wr, 535 op/s; 938/243642 objects misplaced (0.385%); 1.1 MiB/s, 17 objects/s recovering 2024-02-22T19:33:02.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:01.271338+0000 mgr.y (mgr.14535) 158 : cluster [DBG] pgmap v169: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 746 B/s rd, 33 MiB/s wr, 535 op/s; 938/243642 objects misplaced (0.385%); 1.1 MiB/s, 17 objects/s recovering 2024-02-22T19:33:02.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 82 16 83592 83576 63.6186 49.375 0.0100675 0.0157136 2024-02-22T19:33:02.952 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:02.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:02.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:03.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:03.446 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:03.446 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:03.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 83 16 84291 84275 63.3788 43.6875 0.02424 0.0157721 2024-02-22T19:33:04.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:33:04.392 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:04.410 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:04.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:04.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:04.428 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:33:04.439 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:04.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:04.628 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:03.143796+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:04.628 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:03.143859+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:04.629 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:03.273067+0000 mgr.y (mgr.14535) 159 : cluster [DBG] pgmap v170: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 29 MiB/s wr, 458 op/s; 938/243642 objects misplaced (0.385%); 935 KiB/s, 14 objects/s recovering 2024-02-22T19:33:04.629 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:03.362942+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e123: 8 total, 8 up, 7 in 2024-02-22T19:33:04.629 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:03.995585+0000 mon.a (mon.0) 840 : audit [INF] from='client.? 172.21.15.53:0/2980768247' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:04.629 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:03.143796+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:04.629 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:03.143859+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:04.629 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:03.273067+0000 mgr.y (mgr.14535) 159 : cluster [DBG] pgmap v170: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 29 MiB/s wr, 458 op/s; 938/243642 objects misplaced (0.385%); 935 KiB/s, 14 objects/s recovering 2024-02-22T19:33:04.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:03.362942+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e123: 8 total, 8 up, 7 in 2024-02-22T19:33:04.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:03.995585+0000 mon.a (mon.0) 840 : audit [INF] from='client.? 172.21.15.53:0/2980768247' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:04.639 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:04.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:33:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:03.143796+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:03.143859+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:03.273067+0000 mgr.y (mgr.14535) 159 : cluster [DBG] pgmap v170: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 29 MiB/s wr, 458 op/s; 938/243642 objects misplaced (0.385%); 935 KiB/s, 14 objects/s recovering 2024-02-22T19:33:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:03.362942+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e123: 8 total, 8 up, 7 in 2024-02-22T19:33:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:03.995585+0000 mon.a (mon.0) 840 : audit [INF] from='client.? 172.21.15.53:0/2980768247' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:04.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 84 16 84947 84931 63.1126 41 0.0190303 0.0158375 2024-02-22T19:33:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:33:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:33:05.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:04.359199+0000 mon.a (mon.0) 841 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:33:05.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:04.407258+0000 mon.a (mon.0) 842 : audit [INF] from='client.? 172.21.15.53:0/2980768247' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:05.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:04.407314+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e124: 8 total, 8 up, 7 in 2024-02-22T19:33:05.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:04.417421+0000 mgr.y (mgr.14535) 160 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:04.359199+0000 mon.a (mon.0) 841 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:33:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:04.407258+0000 mon.a (mon.0) 842 : audit [INF] from='client.? 172.21.15.53:0/2980768247' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:04.407314+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e124: 8 total, 8 up, 7 in 2024-02-22T19:33:05.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:04.417421+0000 mgr.y (mgr.14535) 160 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:05.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 85 16 85708 85692 62.9298 47.5625 0.0250032 0.0158847 2024-02-22T19:33:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:04.359199+0000 mon.a (mon.0) 841 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:33:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:04.407258+0000 mon.a (mon.0) 842 : audit [INF] from='client.? 172.21.15.53:0/2980768247' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:04.407314+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e124: 8 total, 8 up, 7 in 2024-02-22T19:33:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:04.417421+0000 mgr.y (mgr.14535) 160 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:06.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:05.273523+0000 mgr.y (mgr.14535) 161 : cluster [DBG] pgmap v173: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.00k op/s; 822/254436 objects misplaced (0.323%); 2.0 MiB/s, 32 objects/s recovering 2024-02-22T19:33:06.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:05.273523+0000 mgr.y (mgr.14535) 161 : cluster [DBG] pgmap v173: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.00k op/s; 822/254436 objects misplaced (0.323%); 2.0 MiB/s, 32 objects/s recovering 2024-02-22T19:33:06.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 86 16 86515 86499 62.7847 50.4375 0.0182682 0.015922 2024-02-22T19:33:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:05.273523+0000 mgr.y (mgr.14535) 161 : cluster [DBG] pgmap v173: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.00k op/s; 822/254436 objects misplaced (0.323%); 2.0 MiB/s, 32 objects/s recovering 2024-02-22T19:33:07.170 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:07.176 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:07.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:07.277 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:07.384 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:07.701 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:07.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:07.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:07.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 87 16 87266 87250 62.6027 46.9375 0.0109064 0.0159666 2024-02-22T19:33:07.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:07.861 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:07.864 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:07.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:07.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:08.024 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:08.057 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:07.274768+0000 mgr.y (mgr.14535) 162 : cluster [DBG] pgmap v174: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 28 MiB/s wr, 450 op/s; 822/254436 objects misplaced (0.323%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:33:08.258 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:33:08.287 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:07.274768+0000 mgr.y (mgr.14535) 162 : cluster [DBG] pgmap v174: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 28 MiB/s wr, 450 op/s; 822/254436 objects misplaced (0.323%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:33:08.291 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:07.274768+0000 mgr.y (mgr.14535) 162 : cluster [DBG] pgmap v174: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 28 MiB/s wr, 450 op/s; 822/254436 objects misplaced (0.323%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:33:08.480 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:08.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:08.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:08.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:08.662 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:08.701 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:08.704 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:08.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:08.746 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:33:08.765 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 88 16 87876 87860 62.3241 38.125 0.0398206 0.0160372 2024-02-22T19:33:08.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:08.816 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:33:08.833 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:08.857 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:08.858 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:08.866 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:33:08.919 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:33:08.952 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:33:08.958 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:08.987 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:33:09.035 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:09.041 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:33:09.066 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:09.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:09.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:09.075 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:33:09.088 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.088 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.088 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:33:09.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:09.201 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:09.224 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:09.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:09.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:09.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:09.358 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:09.373 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.6 to 1 (10000) 2024-02-22T19:33:09.381 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:09.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:09.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:09.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:09.538 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:08.343859+0000 mon.a (mon.0) 844 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:09.538 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:08.671161+0000 mon.b (mon.1) 37 : audit [DBG] from='client.? 172.21.15.53:0/2266378360' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:09.538 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:08.893233+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.53:0/3928092302' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:33:09.538 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:08.893758+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:33:09.539 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:09] "GET /metrics HTTP/1.1" 200 175548 "" "Prometheus/2.33.4" 2024-02-22T19:33:09.539 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:08.343859+0000 mon.a (mon.0) 844 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:09.539 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:08.671161+0000 mon.b (mon.1) 37 : audit [DBG] from='client.? 172.21.15.53:0/2266378360' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:09.539 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:08.893233+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.53:0/3928092302' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:33:09.539 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:08.893758+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:33:09.540 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:08.343859+0000 mon.a (mon.0) 844 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:08.671161+0000 mon.b (mon.1) 37 : audit [DBG] from='client.? 172.21.15.53:0/2266378360' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:08.893233+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.53:0/3928092302' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:33:09.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:08.893758+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:33:09.696 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:09.707 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:09.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:09.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 89 16 88532 88516 62.0836 41 0.0193574 0.0161018 2024-02-22T19:33:09.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:10.023 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:10.024 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:10.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:10.125 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:10.270 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:10.282 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:10.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:10.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:10.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:09.275526+0000 mgr.y (mgr.14535) 163 : cluster [DBG] pgmap v175: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 56 MiB/s wr, 892 op/s; 706/265035 objects misplaced (0.266%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:33:10.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:09.348030+0000 mon.a (mon.0) 846 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:33:10.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:09.348113+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e125: 8 total, 8 up, 7 in 2024-02-22T19:33:10.448 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:09.275526+0000 mgr.y (mgr.14535) 163 : cluster [DBG] pgmap v175: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 56 MiB/s wr, 892 op/s; 706/265035 objects misplaced (0.266%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:33:10.448 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:09.348030+0000 mon.a (mon.0) 846 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:33:10.448 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:09.348113+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e125: 8 total, 8 up, 7 in 2024-02-22T19:33:10.449 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:10.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 7 1 2024-02-22T19:33:10.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total time run: 90.0107 2024-02-22T19:33:10.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total writes made: 89204 2024-02-22T19:33:10.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Write size: 65536 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object size: 65536 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Bandwidth (MB/sec): 61.9398 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Bandwidth: 32.8131 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max bandwidth (MB/sec): 202.125 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min bandwidth (MB/sec): 0 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average IOPS: 991 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev IOPS: 525.009 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max IOPS: 3234 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min IOPS: 0 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average Latency(s): 0.0161412 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Latency(s): 0.0314214 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max latency(s): 2.0693 2024-02-22T19:33:10.768 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min latency(s): 0.00116596 2024-02-22T19:33:10.772 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Cleaning up (deleting benchmark objects) 2024-02-22T19:33:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:09.275526+0000 mgr.y (mgr.14535) 163 : cluster [DBG] pgmap v175: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 56 MiB/s wr, 892 op/s; 706/265035 objects misplaced (0.266%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:33:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:09.348030+0000 mon.a (mon.0) 846 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:33:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:09.348113+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e125: 8 total, 8 up, 7 in 2024-02-22T19:33:11.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:10.979298+0000 mon.a (mon.0) 848 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:33:11.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:10.979298+0000 mon.a (mon.0) 848 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:33:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:10.979298+0000 mon.a (mon.0) 848 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:33:12.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:11.278062+0000 mgr.y (mgr.14535) 164 : cluster [DBG] pgmap v177: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 902 op/s; 706/265035 objects misplaced (0.266%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:33:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:11.278062+0000 mgr.y (mgr.14535) 164 : cluster [DBG] pgmap v177: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 902 op/s; 706/265035 objects misplaced (0.266%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:33:12.782 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:11.278062+0000 mgr.y (mgr.14535) 164 : cluster [DBG] pgmap v177: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 902 op/s; 706/265035 objects misplaced (0.266%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:33:12.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:12.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:13.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:13.065 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:13.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:13.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:13.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:13.259 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:13.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:13.148327+0000 mon.a (mon.0) 849 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:33:13.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:13.149537+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:33:13.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:13.155895+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:13.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:13.148327+0000 mon.a (mon.0) 849 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:33:13.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:13.149537+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:33:13.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:13.155895+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:13.148327+0000 mon.a (mon.0) 849 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:33:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:13.149537+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:33:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:13.155895+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:14.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:13.280387+0000 mgr.y (mgr.14535) 165 : cluster [DBG] pgmap v178: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 577 B/s rd, 25 MiB/s wr, 398 op/s; 706/265035 objects misplaced (0.266%); 837 KiB/s, 13 objects/s recovering 2024-02-22T19:33:14.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:13.280387+0000 mgr.y (mgr.14535) 165 : cluster [DBG] pgmap v178: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 577 B/s rd, 25 MiB/s wr, 398 op/s; 706/265035 objects misplaced (0.266%); 837 KiB/s, 13 objects/s recovering 2024-02-22T19:33:14.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:33:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:33:14.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:13.280387+0000 mgr.y (mgr.14535) 165 : cluster [DBG] pgmap v178: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 577 B/s rd, 25 MiB/s wr, 398 op/s; 706/265035 objects misplaced (0.266%); 837 KiB/s, 13 objects/s recovering 2024-02-22T19:33:14.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:33:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:33:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:14.431249+0000 mgr.y (mgr.14535) 166 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:15.767 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:14.431249+0000 mgr.y (mgr.14535) 166 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:14.431249+0000 mgr.y (mgr.14535) 166 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:16.357 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:16.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:16.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:16.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:16.540 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:16.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:15.281225+0000 mgr.y (mgr.14535) 167 : cluster [DBG] pgmap v179: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 29 MiB/s wr, 739 op/s; 589/259767 objects misplaced (0.227%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:16.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:15.281225+0000 mgr.y (mgr.14535) 167 : cluster [DBG] pgmap v179: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 29 MiB/s wr, 739 op/s; 589/259767 objects misplaced (0.227%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:15.281225+0000 mgr.y (mgr.14535) 167 : cluster [DBG] pgmap v179: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 29 MiB/s wr, 739 op/s; 589/259767 objects misplaced (0.227%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:17.115 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:17.122 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:17.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:17.223 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:17.281 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:17.284 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:17.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:17.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:17.482 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:17.719 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:17.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:17.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:17.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:17.889 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:17.974 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:17.978 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:17.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:18.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:18.166 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:18.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:33:18.336 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:18.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:18.337 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:18.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:18.467 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:18.493 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:18.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:18.564 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20875 7.a 7:5368db82:::benchmark_data_smithi053_107660_object20874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.136867410999997, 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001, 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20875 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030654+0000", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5442000000000001e-05 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1599e-05 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030691+0000", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8590000000000001e-06 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030697+0000", 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5996999999999998e-05 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030763+0000", 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1030000000000002e-06 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030771+0000", 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96976369299999998 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000535+0000", 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3981999999999999e-05 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000549+0000", 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449743229999999 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045523+0000", 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015328299999999999 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045676+0000", 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167651 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045844+0000", 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003101946 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048946+0000", 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029887199999999998 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051935+0000", 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057814199999999998 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057716+0000", 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5281000000000001e-05 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057771+0000", 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092475900000000004 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058696+0000", 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:42970 7.a 7:5bea72bb:::benchmark_data_smithi053_107660_object42969:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:13.775147+0000", 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.392375896000004, 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048519745000000003, 2024-02-22T19:33:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 42970 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.775147+0000", 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.775147+0000", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.775145+0000", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9266999999999999e-05 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.775174+0000", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3101e-05 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.775187+0000", 2024-02-22T19:33:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8079999999999999e-06 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.775194+0000", 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029956019 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.805150+0000", 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199958 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.805350+0000", 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020968399999999999 2024-02-22T19:33:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.805560+0000", 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0086460089999999996 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.814206+0000", 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029637700000000001 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.817170+0000", 2024-02-22T19:33:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005686628 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.822856+0000", 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9786e-05 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.822916+0000", 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075079900000000002 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:13.823667+0000", 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048519745000000003 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71776 7.a 7:5bc69d53:::benchmark_data_smithi053_107660_object71775:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.547697+0000", 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.619825784, 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049414262, 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71776 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547697+0000", 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547697+0000", 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547697+0000", 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4151999999999998e-05 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547721+0000", 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2370000000000003e-06 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547728+0000", 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.724e-06 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547733+0000", 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010372222 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.558105+0000", 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000165938 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.558271+0000", 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020483599999999999 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.558476+0000", 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078108680000000003 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.566287+0000", 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010282082 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.576569+0000", 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019861969 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.596431+0000", 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8337999999999995e-05 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.596499+0000", 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061255800000000003 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.597111+0000", 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049414262 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71777 7.2 7:49dcf613:::benchmark_data_smithi053_107660_object71776:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.547738+0000", 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.619784684999999, 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052361877000000001, 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71777 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547738+0000", 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547738+0000", 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547738+0000", 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2113e-05 2024-02-22T19:33:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547760+0000", 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547767+0000", 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3329999999999998e-06 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547771+0000", 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015917817000000001 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.563689+0000", 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013538499999999999 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.563824+0000", 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017499400000000001 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.563999+0000", 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010422055 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.574421+0000", 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022704572999999999 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.597126+0000", 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028628260000000002 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.599989+0000", 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.19e-05 2024-02-22T19:33:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600050+0000", 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.973e-05 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600100+0000", 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052361877000000001 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71779 7.a 7:5ab405e7:::benchmark_data_smithi053_107660_object71778:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.547776+0000", 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.619746695, 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.061219121000000001, 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71779 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547776+0000", 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547776+0000", 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547776+0000", 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4621999999999999e-05 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547800+0000", 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-02-22T19:33:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547808+0000", 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8189999999999997e-06 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.547814+0000", 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018498318999999999 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.566312+0000", 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013353700000000001 2024-02-22T19:33:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.566445+0000", 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018348600000000001 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.566629+0000", 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.009902882 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.576532+0000", 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023615585000000001 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600147+0000", 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0081672289999999998 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.608315+0000", 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4165000000000006e-05 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.608389+0000", 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00060664499999999995 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.608995+0000", 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.061219121000000001 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71785 7.a 7:50a0082d:::benchmark_data_smithi053_107660_object71784:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.562790+0000", 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.604732666, 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049682042000000003, 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71785 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562790+0000", 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562790+0000", 2024-02-22T19:33:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562788+0000", 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9682000000000001e-05 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562818+0000", 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2237e-05 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562830+0000", 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9990000000000002e-06 2024-02-22T19:33:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562837+0000", 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010939657 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.573777+0000", 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000177684 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.573955+0000", 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024801499999999998 2024-02-22T19:33:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.574203+0000", 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023980920000000001 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.576601+0000", 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023610415999999999 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600211+0000", 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882625000000001 2024-02-22T19:33:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612094+0000", 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0574000000000001e-05 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612144+0000", 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032809099999999997 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612472+0000", 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049682042000000003 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71791 7.2 7:4926b02d:::benchmark_data_smithi053_107660_object71790:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.562846+0000", 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.604676679000001, 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049192147999999998, 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71791 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562846+0000", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562846+0000", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562846+0000", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1758000000000001e-05 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562867+0000", 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2099999999999998e-06 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562874+0000", 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.437e-06 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.562878+0000", 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060982679999999996 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.568976+0000", 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144327 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.569121+0000", 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000170287 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.569291+0000", 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051580999999999997 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.574449+0000", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.025727257999999999 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600176+0000", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011774056 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.611950+0000", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0753e-05 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612001+0000", 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.731e-05 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612038+0000", 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049192147999999998 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71798 7.a 7:53cd1414:::benchmark_data_smithi053_107660_object71797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.577464+0000", 2024-02-22T19:33:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.590058514999999, 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.055904978000000001, 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71798 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.577464+0000", 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.577464+0000", 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.577464+0000", 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4910000000000003e-05 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.577499+0000", 2024-02-22T19:33:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0359e-05 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.577509+0000", 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8830000000000003e-06 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.577515+0000", 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022721209999999999 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600236+0000", 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018144299999999999 2024-02-22T19:33:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600417+0000", 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000218882 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.600636+0000", 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011442588 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612079+0000", 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079601599999999998 2024-02-22T19:33:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612875+0000", 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019841564999999999 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.632716+0000", 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5069999999999999e-05 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.632782+0000", 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058794000000000001 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.633369+0000", 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.055904978000000001 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71809 7.2 7:4fe96e83:::benchmark_data_smithi053_107660_object71808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.589714+0000", 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.577808948000001, 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052131028000000003, 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71809 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.589714+0000", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.589714+0000", 2024-02-22T19:33:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.589713+0000", 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2568000000000001e-05 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.589735+0000", 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.077e-05 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.589746+0000", 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.032e-06 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.589752+0000", 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013350503999999999 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.603102+0000", 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012218299999999999 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.603225+0000", 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159063 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.603384+0000", 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0094774030000000006 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612861+0000", 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8068e-05 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612879+0000", 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028876457000000001 2024-02-22T19:33:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.641756+0000", 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9493000000000002e-05 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.641805+0000", 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0003000000000003e-05 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.641845+0000", 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052131028000000003 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:71812 7.a 7:59195be1:::benchmark_data_smithi053_107660_object71811:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.592792+0000", 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.574730589000001, 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048846739, 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71812 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.592792+0000", 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.592792+0000", 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.592791+0000", 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.317e-05 2024-02-22T19:33:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.592814+0000", 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8810000000000002e-06 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.592823+0000", 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7249999999999997e-06 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.592828+0000", 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012915309999999999 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.605743+0000", 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016102799999999999 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.605904+0000", 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020921199999999999 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.606113+0000", 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067710390000000004 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612884+0000", 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2526e-05 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.612917+0000", 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028024232 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.640941+0000", 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8269000000000006e-05 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.641019+0000", 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061981999999999998 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.641639+0000", 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048846739 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:72807 7.a 7:5df14ba7:::benchmark_data_smithi053_107660_object72806:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:47.986000+0000", 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.181522789999999, 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047914327, 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 72807 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.986000+0000", 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.986000+0000", 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.986000+0000", 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2506000000000001e-05 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.986022+0000", 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7769999999999997e-06 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.986029+0000", 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.986033+0000", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0068003739999999997 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.992834+0000", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142688 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.992976+0000", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149264 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.993126+0000", 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069041770000000001 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.000030+0000", 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.025944287 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.025974+0000", 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007481026 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.033455+0000", 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8369999999999999e-05 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.033504+0000", 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041094800000000001 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:48.033915+0000", 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047914327 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:73898 7.3 7:c1878801:::benchmark_data_smithi053_107660_object73897:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:49.507986+0000", 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.659536547999998, 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051668852000000001, 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73898 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.507986+0000", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.507986+0000", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.507986+0000", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6207000000000002e-05 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.508012+0000", 2024-02-22T19:33:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0049e-05 2024-02-22T19:33:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.508022+0000", 2024-02-22T19:33:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2360000000000001e-06 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.508028+0000", 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027209869999999998 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.510749+0000", 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013064000000000001 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.510880+0000", 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016886200000000001 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.511049+0000", 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0129747 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.524024+0000", 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048740839999999999 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.528898+0000", 2024-02-22T19:33:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030671430999999999 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.559569+0000", 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6430000000000001e-05 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.559615+0000", 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9867999999999998e-05 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:49.559655+0000", 2024-02-22T19:33:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051668852000000001 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:75537 7.a 7:555d9ffb:::benchmark_data_smithi053_107660_object75536:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:51.741155+0000", 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.426367629000001, 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052316280999999999, 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 75537 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.741155+0000", 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.741155+0000", 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.741154+0000", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3266999999999999e-05 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.741178+0000", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2265e-05 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.741190+0000", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.674e-06 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.741197+0000", 2024-02-22T19:33:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022231629999999998 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.743420+0000", 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018614799999999999 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.743606+0000", 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022566200000000001 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.743832+0000", 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011867399000000001 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.755699+0000", 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0089079220000000004 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.764607+0000", 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028201058000000001 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.792808+0000", 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9495999999999999e-05 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.792887+0000", 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0005842 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.793472+0000", 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052316280999999999 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:76146 7.a 7:55c6761a:::benchmark_data_smithi053_107660_object76145:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e119)", 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:52.628557+0000", 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.538965657999999, 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050122055999999998, 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 76146 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.628557+0000", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.628557+0000", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.628555+0000", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4097999999999999e-05 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.628579+0000", 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3997e-05 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.628593+0000", 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0389999999999999e-06 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.628600+0000", 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062606010000000002 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.634861+0000", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110426 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.634971+0000", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014185700000000001 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.635113+0000", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00380985 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.638923+0000", 2024-02-22T19:33:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044979879999999996 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.643421+0000", 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.034643274000000002 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.678064+0000", 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5226e-05 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.678150+0000", 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052973099999999997 2024-02-22T19:33:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.678679+0000", 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050122055999999998 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:88014 7.a 7:50cfe586:::benchmark_data_smithi053_107660_object88013:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e124)", 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:08.999301+0000", 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.1682219230000008, 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051400833999999999, 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 88014 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:08.999301+0000", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:08.999301+0000", 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:33:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:08.999299+0000", 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2962000000000001e-05 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:08.999331+0000", 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0210000000000001e-05 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:08.999352+0000", 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-02-22T19:33:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:08.999362+0000", 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029126308 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.028488+0000", 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010573000000000001 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.028594+0000", 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137281 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.028731+0000", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049511210000000002 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.033682+0000", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016038719 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.049721+0000", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065518 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.050376+0000", 2024-02-22T19:33:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7789000000000001e-05 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.050404+0000", 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029822600000000001 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:09.050702+0000", 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051400833999999999 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:88970 7.a 7:527f32c4:::benchmark_data_smithi053_107660_object88969:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:33:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:10.351575+0000", 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.8159480720000003, 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048164959, 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 88970 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.351575+0000", 2024-02-22T19:33:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.351575+0000", 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.351574+0000", 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2311e-05 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.351596+0000", 2024-02-22T19:33:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4618e-05 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.351611+0000", 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7340000000000002e-06 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.351617+0000", 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017573835999999999 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.369191+0000", 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015668799999999999 2024-02-22T19:33:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.369348+0000", 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020598799999999999 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.369554+0000", 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005627572 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.375181+0000", 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010837263 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.386019+0000", 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013256181000000001 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.399275+0000", 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2343999999999998e-05 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.399337+0000", 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040267599999999998 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:10.399740+0000", 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.048164959 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:91268 7.a 7:5086e72c:::benchmark_data_smithi053_107660_object2061:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:13.332197+0000", 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8353255849999996, 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052758105999999999, 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 91268 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332197+0000", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332197+0000", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332197+0000", 2024-02-22T19:33:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.937e-06 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332200+0000", 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5249999999999996e-06 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332205+0000", 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.946e-06 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332209+0000", 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9869000000000001e-05 2024-02-22T19:33:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332249+0000", 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115058 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332364+0000", 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014997099999999999 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.332514+0000", 2024-02-22T19:33:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069266400000000004 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.339441+0000", 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005803257 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.345244+0000", 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062766710000000002 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.351521+0000", 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7340000000000001e-05 2024-02-22T19:33:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.351608+0000", 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347388999999998 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.384956+0000", 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.052758105999999999 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:91269 7.a 7:503e1c6f:::benchmark_data_smithi053_107660_object2062:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:33:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:13.335200+0000", 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8323229190000001, 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050104518000000001, 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 91269 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335200+0000", 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335200+0000", 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335199+0000", 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6809999999999998e-06 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335206+0000", 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0047999999999999e-05 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335216+0000", 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335223+0000", 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8830000000000003e-05 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335292+0000", 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013626900000000001 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335428+0000", 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158963 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.335587+0000", 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003864868 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.339452+0000", 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058217360000000001 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.345274+0000", 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.039742693000000003 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.385016+0000", 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8242999999999998e-05 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.385074+0000", 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023013600000000001 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.385305+0000", 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050104518000000001 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92128 7.2 7:428443eb:::benchmark_data_smithi053_107660_object2921:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.367183+0000", 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8003397359999997, 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054294532999999999, 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92128 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367183+0000", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367183+0000", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367181+0000", 2024-02-22T19:33:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9889999999999998e-06 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367190+0000", 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.216e-05 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367202+0000", 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1840000000000004e-06 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367209+0000", 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049419269999999996 2024-02-22T19:33:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.372151+0000", 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106381 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.372257+0000", 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000119098 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.372376+0000", 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060303800000000001 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.378406+0000", 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017271490000000001 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.380134+0000", 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041248439999999997 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421382+0000", 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4293999999999999e-05 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421436+0000", 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1511999999999998e-05 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421478+0000", 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054294532999999999 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92141 7.a 7:59736f41:::benchmark_data_smithi053_107660_object2934:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.374485+0000", 2024-02-22T19:33:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7930379570000001, 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047967857000000003, 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92141 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374485+0000", 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374485+0000", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374485+0000", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0340000000000001e-06 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374488+0000", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9729999999999997e-06 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:33:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374493+0000", 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.67e-06 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374496+0000", 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019281739999999999 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376424+0000", 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124313 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:33:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376549+0000", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146176 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376695+0000", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00329854 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.379993+0000", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001045437 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.381039+0000", 2024-02-22T19:33:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041144172 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.422183+0000", 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9663000000000001e-05 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.422243+0000", 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021018299999999999 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.422453+0000", 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047967857000000003 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:33:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:33:18.683 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:33:18.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:18.711 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:17.282442+0000 mgr.y (mgr.14535) 168 : cluster [DBG] pgmap v180: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 29 MiB/s wr, 739 op/s; 589/259767 objects misplaced (0.227%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:18.712 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:18.738 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:18.738 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:18.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:17.282442+0000 mgr.y (mgr.14535) 168 : cluster [DBG] pgmap v180: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 29 MiB/s wr, 739 op/s; 589/259767 objects misplaced (0.227%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:18.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:17.282442+0000 mgr.y (mgr.14535) 168 : cluster [DBG] pgmap v180: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 29 MiB/s wr, 739 op/s; 589/259767 objects misplaced (0.227%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:18.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:18.891 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:19.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:19.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:19.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:33:19.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:19.428 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:19.452 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:19.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:19.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:19.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:19.615 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:33:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:19] "GET /metrics HTTP/1.1" 200 175548 "" "Prometheus/2.33.4" 2024-02-22T19:33:19.615 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:19.642 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:19.673 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:19.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:19.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:19.779 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:19.930 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:19.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:19.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:20.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:20.132 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:19.283271+0000 mgr.y (mgr.14535) 169 : cluster [DBG] pgmap v181: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 6.6 MiB/s wr, 778 op/s; 472/248004 objects misplaced (0.190%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:20.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:19.283271+0000 mgr.y (mgr.14535) 169 : cluster [DBG] pgmap v181: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 6.6 MiB/s wr, 778 op/s; 472/248004 objects misplaced (0.190%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:19.283271+0000 mgr.y (mgr.14535) 169 : cluster [DBG] pgmap v181: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 6.6 MiB/s wr, 778 op/s; 472/248004 objects misplaced (0.190%); 1.5 MiB/s, 23 objects/s recovering 2024-02-22T19:33:20.739 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:20.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:20.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:20.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:20.884 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:21.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:21.443 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:21.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:20.872338+0000 mon.a (mon.0) 852 : audit [INF] from='client.? 172.21.15.53:0/2359650106' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:33:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:33:21.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:21.599 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:21.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:21.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:21.739 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:20.872338+0000 mon.a (mon.0) 852 : audit [INF] from='client.? 172.21.15.53:0/2359650106' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:33:21.739 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:33:21.739 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:33:21.739 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:33:21.740 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:33:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:21.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:21.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:21.871 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:21.982 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:21.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:21.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:22.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:20.872338+0000 mon.a (mon.0) 852 : audit [INF] from='client.? 172.21.15.53:0/2359650106' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:33:22.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:22.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:33:22.472 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:22.475 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:22.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:22.541 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.7 to 1 (10000) 2024-02-22T19:33:22.576 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:22.610 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:22.632 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:22.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:22.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:22.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:22.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:21.284457+0000 mgr.y (mgr.14535) 170 : cluster [DBG] pgmap v182: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 943 B/s rd, 5.5 MiB/s wr, 652 op/s; 472/248004 objects misplaced (0.190%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:22.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:21.477873+0000 mon.a (mon.0) 853 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:22.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:21.506830+0000 mon.a (mon.0) 854 : audit [INF] from='client.? 172.21.15.53:0/2359650106' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:33:22.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:21.506883+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e126: 8 total, 8 up, 7 in 2024-02-22T19:33:22.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:22.290639+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.53:0/1328234104' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:33:22.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:22.295905+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:33:22.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:21.284457+0000 mgr.y (mgr.14535) 170 : cluster [DBG] pgmap v182: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 943 B/s rd, 5.5 MiB/s wr, 652 op/s; 472/248004 objects misplaced (0.190%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:22.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:21.477873+0000 mon.a (mon.0) 853 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:22.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:21.506830+0000 mon.a (mon.0) 854 : audit [INF] from='client.? 172.21.15.53:0/2359650106' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:33:22.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:21.506883+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e126: 8 total, 8 up, 7 in 2024-02-22T19:33:22.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:22.290639+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.53:0/1328234104' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:33:22.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:22.295905+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:33:22.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:21.284457+0000 mgr.y (mgr.14535) 170 : cluster [DBG] pgmap v182: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 943 B/s rd, 5.5 MiB/s wr, 652 op/s; 472/248004 objects misplaced (0.190%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:22.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:21.477873+0000 mon.a (mon.0) 853 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:22.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:21.506830+0000 mon.a (mon.0) 854 : audit [INF] from='client.? 172.21.15.53:0/2359650106' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:33:22.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:21.506883+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e126: 8 total, 8 up, 7 in 2024-02-22T19:33:22.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:22.290639+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.53:0/1328234104' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:33:22.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:22.295905+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:33:22.771 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:22.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:22.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:22.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:22.970 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:23.095 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:23.111 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:23.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:23.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:23.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:23.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:23.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:23.481 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:23.499 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-22T19:33:23.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:33:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:22.516646+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:33:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:22.516711+0000 mon.a (mon.0) 858 : cluster [DBG] osdmap e127: 8 total, 8 up, 7 in 2024-02-22T19:33:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:23.339722+0000 mon.a (mon.0) 859 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:23.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:22.516646+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:33:23.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:22.516711+0000 mon.a (mon.0) 858 : cluster [DBG] osdmap e127: 8 total, 8 up, 7 in 2024-02-22T19:33:23.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:23.339722+0000 mon.a (mon.0) 859 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:23.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:22.516646+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:33:23.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:22.516711+0000 mon.a (mon.0) 858 : cluster [DBG] osdmap e127: 8 total, 8 up, 7 in 2024-02-22T19:33:23.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:23.339722+0000 mon.a (mon.0) 859 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:23.875 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:23.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:23.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:23.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:24.048 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:24.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:24.150 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:24.232 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:24.342 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:24.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:24.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:24.429 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:24.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:24.527 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:24.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:33:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:33:24.549 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:23.284898+0000 mgr.y (mgr.14535) 171 : cluster [DBG] pgmap v185: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 490 op/s; 472/248004 objects misplaced (0.190%); 936 KiB/s, 14 objects/s recovering 2024-02-22T19:33:24.577 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:33:24.577 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:33:24.577 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:33:24.577 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:33:24.577 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:33:24.696 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:24.712 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:24.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:24.781 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:23.284898+0000 mgr.y (mgr.14535) 171 : cluster [DBG] pgmap v185: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 490 op/s; 472/248004 objects misplaced (0.190%); 936 KiB/s, 14 objects/s recovering 2024-02-22T19:33:24.783 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:23.284898+0000 mgr.y (mgr.14535) 171 : cluster [DBG] pgmap v185: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 490 op/s; 472/248004 objects misplaced (0.190%); 936 KiB/s, 14 objects/s recovering 2024-02-22T19:33:24.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:24.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:33:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:33:24.872 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:24.954 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:24.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:24.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:25.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:25.140 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:25.300 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:33:25.313 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:25.317 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:25.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:25.356 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:33:25.408 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:33:25.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:25.453 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:33:25.489 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:25.505 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:33:25.543 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:33:25.578 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:33:25.627 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:33:25.639 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:33:25.676 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:25.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:25.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:25.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:24.441908+0000 mgr.y (mgr.14535) 172 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:25.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:25.249929+0000 mon.b (mon.1) 40 : audit [DBG] from='client.? 172.21.15.53:0/3824385890' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:24.441908+0000 mgr.y (mgr.14535) 172 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:25.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:25.249929+0000 mon.b (mon.1) 40 : audit [DBG] from='client.? 172.21.15.53:0/3824385890' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:25.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:24.441908+0000 mgr.y (mgr.14535) 172 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:25.249929+0000 mon.b (mon.1) 40 : audit [DBG] from='client.? 172.21.15.53:0/3824385890' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:25.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:33:25.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:33:25.945 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:25.948 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:25.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:26.049 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:26.186 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:26.215 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:26.382 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:26.391 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:26.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:26.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:26.556 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:26.559 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:26.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:26.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:26.691 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:26.715 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:26.720 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:26.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:26.735 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:25.287098+0000 mgr.y (mgr.14535) 173 : cluster [DBG] pgmap v186: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 961 op/s; 356/236697 objects misplaced (0.150%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:33:26.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:25.287098+0000 mgr.y (mgr.14535) 173 : cluster [DBG] pgmap v186: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 961 op/s; 356/236697 objects misplaced (0.150%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:33:26.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:25.287098+0000 mgr.y (mgr.14535) 173 : cluster [DBG] pgmap v186: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 961 op/s; 356/236697 objects misplaced (0.150%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:33:26.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:26.917 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:27.184 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:27.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:27.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:27.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:27.322 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:27.533 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:27.675 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:27.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:27.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:27.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:33:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:33:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:27.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:27.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:28.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:28.075 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:28.098 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:28.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:28.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:28.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:28.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:27.293538+0000 mgr.y (mgr.14535) 174 : cluster [DBG] pgmap v187: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 471 op/s; 356/236697 objects misplaced (0.150%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:33:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:28.249 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:28.249 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:28.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:27.293538+0000 mgr.y (mgr.14535) 174 : cluster [DBG] pgmap v187: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 471 op/s; 356/236697 objects misplaced (0.150%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:33:28.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:27.293538+0000 mgr.y (mgr.14535) 174 : cluster [DBG] pgmap v187: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 471 op/s; 356/236697 objects misplaced (0.150%); 927 KiB/s, 14 objects/s recovering 2024-02-22T19:33:28.265 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:28.270 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:28.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:28.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:28.433 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:28.673 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:28.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:28.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:28.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:28.858 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:28.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:33:29.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:33:29.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:27.869874+0000 mgr.y (mgr.14535) 175 : audit [DBG] from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:29.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:28.201608+0000 osd.6 (osd.6) 25 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:29.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:28.201649+0000 osd.6 (osd.6) 26 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:29.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:27.869874+0000 mgr.y (mgr.14535) 175 : audit [DBG] from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:29.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:28.201608+0000 osd.6 (osd.6) 25 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:29.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:28.201649+0000 osd.6 (osd.6) 26 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:29.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:29.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:29.335 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:27.869874+0000 mgr.y (mgr.14535) 175 : audit [DBG] from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:28.201608+0000 osd.6 (osd.6) 25 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:28.201649+0000 osd.6 (osd.6) 26 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:29.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:29.416 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:29.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:33:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:29] "GET /metrics HTTP/1.1" 200 175568 "" "Prometheus/2.33.4" 2024-02-22T19:33:29.584 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:29.598 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:29.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:29.612 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-02-22T19:33:27.293267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5158082236,"num_objects":78899,"num_object_clones":0,"num_object_copies":236697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":356,"num_objects_unfound":0,"num_objects_dirty":78899,"num_whiteouts":0,"num_read":971,"num_read_kb":723,"num_write":100131,"num_write_kb":5709098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4821,"num_bytes_recovered":311822555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78204,"ondisk_log_size":78204,"up":444,"acting":442,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":17281948,"kb_used_data":15156132,"kb_used_omap":0,"kb_used_meta":2125568,"kb_avail":638934116,"statfs":{"total":671965249536,"available":654268534784,"internally_reserved":0,"allocated":15519879168,"data_stored":15517078232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2176581632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2,4,5,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-247005184,"num_objects":-3769,"num_object_clones":0,"num_object_copies":-11307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-116,"num_objects_unfound":0,"num_objects_dirty":-3769,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3769,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7602176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.010675"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511797+0000","last_change":"2024-02-22T19:31:50.004483+0000","last_active":"2024-02-22T19:33:21.511797+0000","last_peered":"2024-02-22T19:33:21.511797+0000","last_clean":"2024-02-22T19:33:21.511797+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:33:21.511797+0000","last_undegraded":"2024-02-22T19:33:21.511797+0000","last_fullsized":"2024-02-22T19:33:21.511797+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":89,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525452+0000","last_change":"2024-02-22T19:31:50.003253+0000","last_active":"2024-02-22T19:33:21.525452+0000","last_peered":"2024-02-22T19:33:21.525452+0000","last_clean":"2024-02-22T19:33:21.525452+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:33:21.525452+0000","last_undegraded":"2024-02-22T19:33:21.525452+0000","last_fullsized":"2024-02-22T19:33:21.525452+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.522502+0000","last_change":"2024-02-22T19:31:50.013054+0000","last_active":"2024-02-22T19:33:21.522502+0000","last_peered":"2024-02-22T19:33:21.522502+0000","last_clean":"2024-02-22T19:33:21.522502+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:33:21.522502+0000","last_undegraded":"2024-02-22T19:33:21.522502+0000","last_fullsized":"2024-02-22T19:33:21.522502+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:12.445206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:12.445206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511484+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:33:21.511484+0000","last_peered":"2024-02-22T19:33:21.511484+0000","last_clean":"2024-02-22T19:33:21.511484+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:33:21.511484+0000","last_undegraded":"2024-02-22T19:33:21.511484+0000","last_fullsized":"2024-02-22T19:33:21.511484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":75,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.528413+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:33:21.528413+0000","last_peered":"2024-02-22T19:33:21.528413+0000","last_clean":"2024-02-22T19:33:21.528413+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:33:21.528413+0000","last_undegraded":"2024-02-22T19:33:21.528413+0000","last_fullsized":"2024-02-22T19:33:21.528413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511560+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:33:21.511560+0000","last_peered":"2024-02-22T19:33:21.511560+0000","last_clean":"2024-02-22T19:33:21.511560+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:33:21.511560+0000","last_undegraded":"2024-02-22T19:33:21.511560+0000","last_fullsized":"2024-02-22T19:33:21.511560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":81,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526048+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:33:21.526048+0000","last_peered":"2024-02-22T19:33:21.526048+0000","last_clean":"2024-02-22T19:33:21.526048+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:33:21.526048+0000","last_undegraded":"2024-02-22T19:33:21.526048+0000","last_fullsized":"2024-02-22T19:33:21.526048+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526606+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:33:21.526606+0000","last_peered":"2024-02-22T19:33:21.526606+0000","last_clean":"2024-02-22T19:33:21.526606+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:33:21.526606+0000","last_undegraded":"2024-02-22T19:33:21.526606+0000","last_fullsized":"2024-02-22T19:33:21.526606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.521337+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:33:21.521337+0000","last_peered":"2024-02-22T19:33:21.521337+0000","last_clean":"2024-02-22T19:33:21.521337+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:33:21.521337+0000","last_undegraded":"2024-02-22T19:33:21.521337+0000","last_fullsized":"2024-02-22T19:33:21.521337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520101+0000","last_change":"2024-02-22T19:31:49.999104+0000","last_active":"2024-02-22T19:33:21.520101+0000","last_peered":"2024-02-22T19:33:21.520101+0000","last_clean":"2024-02-22T19:33:21.520101+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:33:21.520101+0000","last_undegraded":"2024-02-22T19:33:21.520101+0000","last_fullsized":"2024-02-22T19:33:21.520101+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.527965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.527965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.534467+0000","last_change":"2024-02-22T19:32:25.385315+0000","last_active":"2024-02-22T19:33:21.534467+0000","last_peered":"2024-02-22T19:33:21.534467+0000","last_clean":"2024-02-22T19:33:21.534467+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:33:21.534467+0000","last_undegraded":"2024-02-22T19:33:21.534467+0000","last_fullsized":"2024-02-22T19:33:21.534467+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.440058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.440058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512140+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:33:21.512140+0000","last_peered":"2024-02-22T19:33:21.512140+0000","last_clean":"2024-02-22T19:33:21.512140+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:33:21.512140+0000","last_undegraded":"2024-02-22T19:33:21.512140+0000","last_fullsized":"2024-02-22T19:33:21.512140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.521331+0000","last_change":"2024-02-22T19:31:50.005065+0000","last_active":"2024-02-22T19:33:21.521331+0000","last_peered":"2024-02-22T19:33:21.521331+0000","last_clean":"2024-02-22T19:33:21.521331+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:33:21.521331+0000","last_undegraded":"2024-02-22T19:33:21.521331+0000","last_fullsized":"2024-02-22T19:33:21.521331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.523990+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:33:21.523990+0000","last_peered":"2024-02-22T19:33:21.523990+0000","last_clean":"2024-02-22T19:33:21.523990+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:33:21.523990+0000","last_undegraded":"2024-02-22T19:33:21.523990+0000","last_fullsized":"2024-02-22T19:33:21.523990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":51,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511730+0000","last_change":"2024-02-22T19:31:50.439339+0000","last_active":"2024-02-22T19:33:21.511730+0000","last_peered":"2024-02-22T19:33:21.511730+0000","last_clean":"2024-02-22T19:33:21.511730+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:33:21.511730+0000","last_undegraded":"2024-02-22T19:33:21.511730+0000","last_fullsized":"2024-02-22T19:33:21.511730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:31:23.683223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.683223+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":92,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512222+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:33:21.512222+0000","last_peered":"2024-02-22T19:33:21.512222+0000","last_clean":"2024-02-22T19:33:21.512222+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:33:21.512222+0000","last_undegraded":"2024-02-22T19:33:21.512222+0000","last_fullsized":"2024-02-22T19:33:21.512222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512303+0000","last_change":"2024-02-22T19:31:50.009328+0000","last_active":"2024-02-22T19:33:21.512303+0000","last_peered":"2024-02-22T19:33:21.512303+0000","last_clean":"2024-02-22T19:33:21.512303+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:33:21.512303+0000","last_undegraded":"2024-02-22T19:33:21.512303+0000","last_fullsized":"2024-02-22T19:33:21.512303+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527769+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:33:21.527769+0000","last_peered":"2024-02-22T19:33:21.527769+0000","last_clean":"2024-02-22T19:33:21.527769+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:33:21.527769+0000","last_undegraded":"2024-02-22T19:33:21.527769+0000","last_fullsized":"2024-02-22T19:33:21.527769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520418+0000","last_change":"2024-02-22T19:31:26.454221+0000","last_active":"2024-02-22T19:33:21.520418+0000","last_peered":"2024-02-22T19:33:21.520418+0000","last_clean":"2024-02-22T19:33:21.520418+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:33:21.520418+0000","last_undegraded":"2024-02-22T19:33:21.520418+0000","last_fullsized":"2024-02-22T19:33:21.520418+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.454155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.454155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":107,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.534117+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:33:21.534117+0000","last_peered":"2024-02-22T19:33:21.534117+0000","last_clean":"2024-02-22T19:33:21.534117+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:33:21.534117+0000","last_undegraded":"2024-02-22T19:33:21.534117+0000","last_fullsized":"2024-02-22T19:33:21.534117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.529690+0000","last_change":"2024-02-22T19:31:50.014548+0000","last_active":"2024-02-22T19:33:21.529690+0000","last_peered":"2024-02-22T19:33:21.529690+0000","last_clean":"2024-02-22T19:33:21.529690+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:33:21.529690+0000","last_undegraded":"2024-02-22T19:33:21.529690+0000","last_fullsized":"2024-02-22T19:33:21.529690+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.823363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.823363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512056+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:33:21.512056+0000","last_peered":"2024-02-22T19:33:21.512056+0000","last_clean":"2024-02-22T19:33:21.512056+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:33:21.512056+0000","last_undegraded":"2024-02-22T19:33:21.512056+0000","last_fullsized":"2024-02-22T19:33:21.512056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512056+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:33:21.512056+0000","last_peered":"2024-02-22T19:33:21.512056+0000","last_clean":"2024-02-22T19:33:21.512056+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:33:21.512056+0000","last_undegraded":"2024-02-22T19:33:21.512056+0000","last_fullsized":"2024-02-22T19:33:21.512056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":107,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524805+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:33:21.524805+0000","last_peered":"2024-02-22T19:33:21.524805+0000","last_clean":"2024-02-22T19:33:21.524805+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:33:21.524805+0000","last_undegraded":"2024-02-22T19:33:21.524805+0000","last_fullsized":"2024-02-22T19:33:21.524805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":90,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524548+0000","last_change":"2024-02-22T19:31:50.095022+0000","last_active":"2024-02-22T19:33:21.524548+0000","last_peered":"2024-02-22T19:33:21.524548+0000","last_clean":"2024-02-22T19:33:21.524548+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:33:21.524548+0000","last_undegraded":"2024-02-22T19:33:21.524548+0000","last_fullsized":"2024-02-22T19:33:21.524548+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":144,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.519081+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:33:22.519081+0000","last_peered":"2024-02-22T19:33:22.519081+0000","last_clean":"2024-02-22T19:33:22.519081+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:33:22.519081+0000","last_undegraded":"2024-02-22T19:33:22.519081+0000","last_fullsized":"2024-02-22T19:33:22.519081+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525533+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:33:21.525533+0000","last_peered":"2024-02-22T19:33:21.525533+0000","last_clean":"2024-02-22T19:33:21.525533+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:33:21.525533+0000","last_undegraded":"2024-02-22T19:33:21.525533+0000","last_fullsized":"2024-02-22T19:33:21.525533+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":95,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520489+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:33:21.520489+0000","last_peered":"2024-02-22T19:33:21.520489+0000","last_clean":"2024-02-22T19:33:21.520489+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:33:21.520489+0000","last_undegraded":"2024-02-22T19:33:21.520489+0000","last_fullsized":"2024-02-22T19:33:21.520489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":89,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527990+0000","last_change":"2024-02-22T19:31:50.009891+0000","last_active":"2024-02-22T19:33:21.527990+0000","last_peered":"2024-02-22T19:33:21.527990+0000","last_clean":"2024-02-22T19:33:21.527990+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:33:21.527990+0000","last_undegraded":"2024-02-22T19:33:21.527990+0000","last_fullsized":"2024-02-22T19:33:21.527990+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520802+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:33:21.520802+0000","last_peered":"2024-02-22T19:33:21.520802+0000","last_clean":"2024-02-22T19:33:21.520802+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:33:21.520802+0000","last_undegraded":"2024-02-22T19:33:21.520802+0000","last_fullsized":"2024-02-22T19:33:21.520802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":81,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524825+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:33:21.524825+0000","last_peered":"2024-02-22T19:33:21.524825+0000","last_clean":"2024-02-22T19:33:21.524825+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:33:21.524825+0000","last_undegraded":"2024-02-22T19:33:21.524825+0000","last_fullsized":"2024-02-22T19:33:21.524825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":151,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520613+0000","last_change":"2024-02-22T19:33:20.849568+0000","last_active":"2024-02-22T19:33:21.520613+0000","last_peered":"2024-02-22T19:33:21.520613+0000","last_clean":"2024-02-22T19:33:21.520613+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:33:21.520613+0000","last_undegraded":"2024-02-22T19:33:21.520613+0000","last_fullsized":"2024-02-22T19:33:21.520613+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.607011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.607011+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.523982+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:33:21.523982+0000","last_peered":"2024-02-22T19:33:21.523982+0000","last_clean":"2024-02-22T19:33:21.523982+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:33:21.523982+0000","last_undegraded":"2024-02-22T19:33:21.523982+0000","last_fullsized":"2024-02-22T19:33:21.523982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520525+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:33:21.520525+0000","last_peered":"2024-02-22T19:33:21.520525+0000","last_clean":"2024-02-22T19:33:21.520525+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:33:21.520525+0000","last_undegraded":"2024-02-22T19:33:21.520525+0000","last_fullsized":"2024-02-22T19:33:21.520525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":91,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.522718+0000","last_change":"2024-02-22T19:31:50.010848+0000","last_active":"2024-02-22T19:33:21.522718+0000","last_peered":"2024-02-22T19:33:21.522718+0000","last_clean":"2024-02-22T19:33:21.522718+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:33:21.522718+0000","last_undegraded":"2024-02-22T19:33:21.522718+0000","last_fullsized":"2024-02-22T19:33:21.522718+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:08.671238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:08.671238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":85,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524940+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:33:21.524940+0000","last_peered":"2024-02-22T19:33:21.524940+0000","last_clean":"2024-02-22T19:33:21.524940+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:33:21.524940+0000","last_undegraded":"2024-02-22T19:33:21.524940+0000","last_fullsized":"2024-02-22T19:33:21.524940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":75,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.528541+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:33:21.528541+0000","last_peered":"2024-02-22T19:33:21.528541+0000","last_clean":"2024-02-22T19:33:21.528541+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:33:21.528541+0000","last_undegraded":"2024-02-22T19:33:21.528541+0000","last_fullsized":"2024-02-22T19:33:21.528541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525042+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:33:21.525042+0000","last_peered":"2024-02-22T19:33:21.525042+0000","last_clean":"2024-02-22T19:33:21.525042+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:33:21.525042+0000","last_undegraded":"2024-02-22T19:33:21.525042+0000","last_fullsized":"2024-02-22T19:33:21.525042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":83,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524628+0000","last_change":"2024-02-22T19:33:19.711116+0000","last_active":"2024-02-22T19:33:21.524628+0000","last_peered":"2024-02-22T19:33:21.524628+0000","last_clean":"2024-02-22T19:33:21.524628+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:33:21.524628+0000","last_undegraded":"2024-02-22T19:33:21.524628+0000","last_fullsized":"2024-02-22T19:33:21.524628+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.463122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.463122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526541+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:33:21.526541+0000","last_peered":"2024-02-22T19:33:21.526541+0000","last_clean":"2024-02-22T19:33:21.526541+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:33:21.526541+0000","last_undegraded":"2024-02-22T19:33:21.526541+0000","last_fullsized":"2024-02-22T19:33:21.526541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":85,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520999+0000","last_change":"2024-02-22T19:31:50.014237+0000","last_active":"2024-02-22T19:33:21.520999+0000","last_peered":"2024-02-22T19:33:21.520999+0000","last_clean":"2024-02-22T19:33:21.520999+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:33:21.520999+0000","last_undegraded":"2024-02-22T19:33:21.520999+0000","last_fullsized":"2024-02-22T19:33:21.520999+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.528339+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:33:21.528339+0000","last_peered":"2024-02-22T19:33:21.528339+0000","last_clean":"2024-02-22T19:33:21.528339+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:33:21.528339+0000","last_undegraded":"2024-02-22T19:33:21.528339+0000","last_fullsized":"2024-02-22T19:33:21.528339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525050+0000","last_change":"2024-02-22T19:31:23.009097+0000","last_active":"2024-02-22T19:33:21.525050+0000","last_peered":"2024-02-22T19:33:21.525050+0000","last_clean":"2024-02-22T19:33:21.525050+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:33:21.525050+0000","last_undegraded":"2024-02-22T19:33:21.525050+0000","last_fullsized":"2024-02-22T19:33:21.525050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.009021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527571+0000","last_change":"2024-02-22T19:30:14.042137+0000","last_active":"2024-02-22T19:33:21.527571+0000","last_peered":"2024-02-22T19:33:21.527571+0000","last_clean":"2024-02-22T19:33:21.527571+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:33:21.527571+0000","last_undegraded":"2024-02-22T19:33:21.527571+0000","last_fullsized":"2024-02-22T19:33:21.527571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.042059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.042059+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526356+0000","last_change":"2024-02-22T19:31:50.007895+0000","last_active":"2024-02-22T19:33:21.526356+0000","last_peered":"2024-02-22T19:33:21.526356+0000","last_clean":"2024-02-22T19:33:21.526356+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:33:21.526356+0000","last_undegraded":"2024-02-22T19:33:21.526356+0000","last_fullsized":"2024-02-22T19:33:21.526356+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.709691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.709691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":89,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527339+0000","last_change":"2024-02-22T19:31:50.012381+0000","last_active":"2024-02-22T19:33:21.527339+0000","last_peered":"2024-02-22T19:33:21.527339+0000","last_clean":"2024-02-22T19:33:21.527339+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:33:21.527339+0000","last_undegraded":"2024-02-22T19:33:21.527339+0000","last_fullsized":"2024-02-22T19:33:21.527339+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511836+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:33:21.511836+0000","last_peered":"2024-02-22T19:33:21.511836+0000","last_clean":"2024-02-22T19:33:21.511836+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:33:21.511836+0000","last_undegraded":"2024-02-22T19:33:21.511836+0000","last_fullsized":"2024-02-22T19:33:21.511836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":101,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512055+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:33:21.512055+0000","last_peered":"2024-02-22T19:33:21.512055+0000","last_clean":"2024-02-22T19:33:21.512055+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:33:21.512055+0000","last_undegraded":"2024-02-22T19:33:21.512055+0000","last_fullsized":"2024-02-22T19:33:21.512055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520067+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:33:21.520067+0000","last_peered":"2024-02-22T19:33:21.520067+0000","last_clean":"2024-02-22T19:33:21.520067+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:33:21.520067+0000","last_undegraded":"2024-02-22T19:33:21.520067+0000","last_fullsized":"2024-02-22T19:33:21.520067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":82,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527837+0000","last_change":"2024-02-22T19:31:50.105130+0000","last_active":"2024-02-22T19:33:22.527837+0000","last_peered":"2024-02-22T19:33:22.527837+0000","last_clean":"2024-02-22T19:33:22.527837+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:33:22.527837+0000","last_undegraded":"2024-02-22T19:33:22.527837+0000","last_fullsized":"2024-02-22T19:33:22.527837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:29.679183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.679183+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":82,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524083+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:33:21.524083+0000","last_peered":"2024-02-22T19:33:21.524083+0000","last_clean":"2024-02-22T19:33:21.524083+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:33:21.524083+0000","last_undegraded":"2024-02-22T19:33:21.524083+0000","last_fullsized":"2024-02-22T19:33:21.524083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525494+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:33:21.525494+0000","last_peered":"2024-02-22T19:33:21.525494+0000","last_clean":"2024-02-22T19:33:21.525494+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:33:21.525494+0000","last_undegraded":"2024-02-22T19:33:21.525494+0000","last_fullsized":"2024-02-22T19:33:21.525494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512105+0000","last_change":"2024-02-22T19:31:50.016234+0000","last_active":"2024-02-22T19:33:21.512105+0000","last_peered":"2024-02-22T19:33:21.512105+0000","last_clean":"2024-02-22T19:33:21.512105+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:33:21.512105+0000","last_undegraded":"2024-02-22T19:33:21.512105+0000","last_fullsized":"2024-02-22T19:33:21.512105+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534010+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:33:22.534010+0000","last_peered":"2024-02-22T19:33:22.534010+0000","last_clean":"2024-02-22T19:33:22.534010+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:33:22.534010+0000","last_undegraded":"2024-02-22T19:33:22.534010+0000","last_fullsized":"2024-02-22T19:33:22.534010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520115+0000","last_change":"2024-02-22T19:31:11.415809+0000","last_active":"2024-02-22T19:33:21.520115+0000","last_peered":"2024-02-22T19:33:21.520115+0000","last_clean":"2024-02-22T19:33:21.520115+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:33:21.520115+0000","last_undegraded":"2024-02-22T19:33:21.520115+0000","last_fullsized":"2024-02-22T19:33:21.520115+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.415718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":107,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525235+0000","last_change":"2024-02-22T19:31:50.352892+0000","last_active":"2024-02-22T19:33:21.525235+0000","last_peered":"2024-02-22T19:33:21.525235+0000","last_clean":"2024-02-22T19:33:21.525235+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:33:21.525235+0000","last_undegraded":"2024-02-22T19:33:21.525235+0000","last_fullsized":"2024-02-22T19:33:21.525235+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.685295+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.534341+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:33:21.534341+0000","last_peered":"2024-02-22T19:33:21.534341+0000","last_clean":"2024-02-22T19:33:21.534341+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:33:21.534341+0000","last_undegraded":"2024-02-22T19:33:21.534341+0000","last_fullsized":"2024-02-22T19:33:21.534341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527178+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:33:21.527178+0000","last_peered":"2024-02-22T19:33:21.527178+0000","last_clean":"2024-02-22T19:33:21.527178+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:33:21.527178+0000","last_undegraded":"2024-02-22T19:33:21.527178+0000","last_fullsized":"2024-02-22T19:33:21.527178+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":82,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524087+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:33:21.524087+0000","last_peered":"2024-02-22T19:33:21.524087+0000","last_clean":"2024-02-22T19:33:21.524087+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:33:21.524087+0000","last_undegraded":"2024-02-22T19:33:21.524087+0000","last_fullsized":"2024-02-22T19:33:21.524087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520667+0000","last_change":"2024-02-22T19:30:13.766624+0000","last_active":"2024-02-22T19:33:21.520667+0000","last_peered":"2024-02-22T19:33:21.520667+0000","last_clean":"2024-02-22T19:33:21.520667+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:33:21.520667+0000","last_undegraded":"2024-02-22T19:33:21.520667+0000","last_fullsized":"2024-02-22T19:33:21.520667+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:13.766546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.766546+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524965+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:33:21.524965+0000","last_peered":"2024-02-22T19:33:21.524965+0000","last_clean":"2024-02-22T19:33:21.524965+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:33:21.524965+0000","last_undegraded":"2024-02-22T19:33:21.524965+0000","last_fullsized":"2024-02-22T19:33:21.524965+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526325+0000","last_change":"2024-02-22T19:31:50.013893+0000","last_active":"2024-02-22T19:33:21.526325+0000","last_peered":"2024-02-22T19:33:21.526325+0000","last_clean":"2024-02-22T19:33:21.526325+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:33:21.526325+0000","last_undegraded":"2024-02-22T19:33:21.526325+0000","last_fullsized":"2024-02-22T19:33:21.526325+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":92,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.519185+0000","last_change":"2024-02-22T19:31:50.006289+0000","last_active":"2024-02-22T19:33:21.519185+0000","last_peered":"2024-02-22T19:33:21.519185+0000","last_clean":"2024-02-22T19:33:21.519185+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:33:21.519185+0000","last_undegraded":"2024-02-22T19:33:21.519185+0000","last_fullsized":"2024-02-22T19:33:21.519185+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":88,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.530648+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:33:21.530648+0000","last_peered":"2024-02-22T19:33:21.530648+0000","last_clean":"2024-02-22T19:33:21.530648+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:33:21.530648+0000","last_undegraded":"2024-02-22T19:33:21.530648+0000","last_fullsized":"2024-02-22T19:33:21.530648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"127'6246","reported_seq":8856,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.198194+0000","last_change":"2024-02-22T19:32:06.969062+0000","last_active":"2024-02-22T19:33:24.198194+0000","last_peered":"2024-02-22T19:33:24.198194+0000","last_clean":"2024-02-22T19:33:24.198194+0000","last_became_active":"2024-02-22T19:32:06.966453+0000","last_became_peered":"2024-02-22T19:32:06.966453+0000","last_unstale":"2024-02-22T19:33:24.198194+0000","last_undegraded":"2024-02-22T19:33:24.198194+0000","last_fullsized":"2024-02-22T19:33:24.198194+0000","mapping_epoch":104,"log_start":"102'1700","ondisk_log_start":"102'1700","created":93,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4546,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":322306048,"num_objects":4918,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6246,"num_write_kb":357248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.519325+0000","last_change":"2024-02-22T19:31:50.006727+0000","last_active":"2024-02-22T19:33:21.519325+0000","last_peered":"2024-02-22T19:33:21.519325+0000","last_clean":"2024-02-22T19:33:21.519325+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:33:21.519325+0000","last_undegraded":"2024-02-22T19:33:21.519325+0000","last_fullsized":"2024-02-22T19:33:21.519325+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524803+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:33:21.524803+0000","last_peered":"2024-02-22T19:33:21.524803+0000","last_clean":"2024-02-22T19:33:21.524803+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:33:21.524803+0000","last_undegraded":"2024-02-22T19:33:21.524803+0000","last_fullsized":"2024-02-22T19:33:21.524803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":92,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524972+0000","last_change":"2024-02-22T19:31:50.611447+0000","last_active":"2024-02-22T19:33:21.524972+0000","last_peered":"2024-02-22T19:33:21.524972+0000","last_clean":"2024-02-22T19:33:21.524972+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:33:21.524972+0000","last_undegraded":"2024-02-22T19:33:21.524972+0000","last_fullsized":"2024-02-22T19:33:21.524972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":139,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524371+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:33:22.524371+0000","last_peered":"2024-02-22T19:33:22.524371+0000","last_clean":"2024-02-22T19:33:22.524371+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:33:22.524371+0000","last_undegraded":"2024-02-22T19:33:22.524371+0000","last_fullsized":"2024-02-22T19:33:22.524371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"127'6203","reported_seq":8424,"reported_epoch":127,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:33:24.233767+0000","last_change":"2024-02-22T19:31:51.068957+0000","last_active":"2024-02-22T19:33:24.233767+0000","last_peered":"2024-02-22T19:33:24.233767+0000","last_clean":"2024-02-22T19:31:48.970281+0000","last_became_active":"2024-02-22T19:31:51.043104+0000","last_became_peered":"2024-02-22T19:31:51.043104+0000","last_unstale":"2024-02-22T19:33:24.233767+0000","last_undegraded":"2024-02-22T19:33:24.233767+0000","last_fullsized":"2024-02-22T19:31:50.992947+0000","mapping_epoch":100,"log_start":"102'2000","ondisk_log_start":"102'2000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4203,"ondisk_log_size":4203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":321323008,"num_objects":4903,"num_object_clones":0,"num_object_copies":14709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":187,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":355392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1085,"num_bytes_recovered":71106560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2","5"],"object_location_counts":[{"shards":"2,3","objects":4716},{"shards":"2,3,5","objects":187}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":75,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.528455+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:33:21.528455+0000","last_peered":"2024-02-22T19:33:21.528455+0000","last_clean":"2024-02-22T19:33:21.528455+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:33:21.528455+0000","last_undegraded":"2024-02-22T19:33:21.528455+0000","last_fullsized":"2024-02-22T19:33:21.528455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":82,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524702+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:33:21.524702+0000","last_peered":"2024-02-22T19:33:21.524702+0000","last_clean":"2024-02-22T19:33:21.524702+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:33:21.524702+0000","last_undegraded":"2024-02-22T19:33:21.524702+0000","last_fullsized":"2024-02-22T19:33:21.524702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511847+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:33:21.511847+0000","last_peered":"2024-02-22T19:33:21.511847+0000","last_clean":"2024-02-22T19:33:21.511847+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:33:21.511847+0000","last_undegraded":"2024-02-22T19:33:21.511847+0000","last_fullsized":"2024-02-22T19:33:21.511847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.533418+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:33:21.533418+0000","last_peered":"2024-02-22T19:33:21.533418+0000","last_clean":"2024-02-22T19:33:21.533418+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:33:21.533418+0000","last_undegraded":"2024-02-22T19:33:21.533418+0000","last_fullsized":"2024-02-22T19:33:21.533418+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"127'6316","reported_seq":6357,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.262980+0000","last_change":"2024-02-22T19:31:33.014127+0000","last_active":"2024-02-22T19:33:24.262980+0000","last_peered":"2024-02-22T19:33:24.262980+0000","last_clean":"2024-02-22T19:33:24.262980+0000","last_became_active":"2024-02-22T19:31:33.013760+0000","last_became_peered":"2024-02-22T19:31:33.013760+0000","last_unstale":"2024-02-22T19:33:24.262980+0000","last_undegraded":"2024-02-22T19:33:24.262980+0000","last_fullsized":"2024-02-22T19:33:24.262980+0000","mapping_epoch":93,"log_start":"101'1400","ondisk_log_start":"101'1400","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4916,"ondisk_log_size":4916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":323485696,"num_objects":4936,"num_object_clones":0,"num_object_copies":14808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6316,"num_write_kb":360064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524768+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:33:21.524768+0000","last_peered":"2024-02-22T19:33:21.524768+0000","last_clean":"2024-02-22T19:33:21.524768+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:33:21.524768+0000","last_undegraded":"2024-02-22T19:33:21.524768+0000","last_fullsized":"2024-02-22T19:33:21.524768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":89,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.537653+0000","last_change":"2024-02-22T19:31:50.008803+0000","last_active":"2024-02-22T19:33:21.537653+0000","last_peered":"2024-02-22T19:33:21.537653+0000","last_clean":"2024-02-22T19:33:21.537653+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:33:21.537653+0000","last_undegraded":"2024-02-22T19:33:21.537653+0000","last_fullsized":"2024-02-22T19:33:21.537653+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.110513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.110513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525190+0000","last_change":"2024-02-22T19:31:49.999189+0000","last_active":"2024-02-22T19:33:21.525190+0000","last_peered":"2024-02-22T19:33:21.525190+0000","last_clean":"2024-02-22T19:33:21.525190+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:33:21.525190+0000","last_undegraded":"2024-02-22T19:33:21.525190+0000","last_fullsized":"2024-02-22T19:33:21.525190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:15.882992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.882992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.519477+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:33:21.519477+0000","last_peered":"2024-02-22T19:33:21.519477+0000","last_clean":"2024-02-22T19:33:21.519477+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:33:21.519477+0000","last_undegraded":"2024-02-22T19:33:21.519477+0000","last_fullsized":"2024-02-22T19:33:21.519477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"127'6156","reported_seq":6198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.238666+0000","last_change":"2024-02-22T19:31:33.015434+0000","last_active":"2024-02-22T19:33:24.238666+0000","last_peered":"2024-02-22T19:33:24.238666+0000","last_clean":"2024-02-22T19:33:24.238666+0000","last_became_active":"2024-02-22T19:31:33.015132+0000","last_became_peered":"2024-02-22T19:31:33.015132+0000","last_unstale":"2024-02-22T19:33:24.238666+0000","last_undegraded":"2024-02-22T19:33:24.238666+0000","last_fullsized":"2024-02-22T19:33:24.238666+0000","mapping_epoch":93,"log_start":"98'800","ondisk_log_start":"98'800","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5356,"ondisk_log_size":5356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":320077824,"num_objects":4884,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.521838+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:33:21.521838+0000","last_peered":"2024-02-22T19:33:21.521838+0000","last_clean":"2024-02-22T19:33:21.521838+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:33:21.521838+0000","last_undegraded":"2024-02-22T19:33:21.521838+0000","last_fullsized":"2024-02-22T19:33:21.521838+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.542505+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:33:21.542505+0000","last_peered":"2024-02-22T19:33:21.542505+0000","last_clean":"2024-02-22T19:33:21.542505+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:33:21.542505+0000","last_undegraded":"2024-02-22T19:33:21.542505+0000","last_fullsized":"2024-02-22T19:33:21.542505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525607+0000","last_change":"2024-02-22T19:31:50.014265+0000","last_active":"2024-02-22T19:33:21.525607+0000","last_peered":"2024-02-22T19:33:21.525607+0000","last_clean":"2024-02-22T19:33:21.525607+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:33:21.525607+0000","last_undegraded":"2024-02-22T19:33:21.525607+0000","last_fullsized":"2024-02-22T19:33:21.525607+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":115,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520492+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:33:21.520492+0000","last_peered":"2024-02-22T19:33:21.520492+0000","last_clean":"2024-02-22T19:33:21.520492+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:33:21.520492+0000","last_undegraded":"2024-02-22T19:33:21.520492+0000","last_fullsized":"2024-02-22T19:33:21.520492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"127'6154","reported_seq":6196,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.183811+0000","last_change":"2024-02-22T19:31:33.014009+0000","last_active":"2024-02-22T19:33:24.183811+0000","last_peered":"2024-02-22T19:33:24.183811+0000","last_clean":"2024-02-22T19:33:24.183811+0000","last_became_active":"2024-02-22T19:31:33.013715+0000","last_became_peered":"2024-02-22T19:31:33.013715+0000","last_unstale":"2024-02-22T19:33:24.183811+0000","last_undegraded":"2024-02-22T19:33:24.183811+0000","last_fullsized":"2024-02-22T19:33:24.183811+0000","mapping_epoch":93,"log_start":"98'700","ondisk_log_start":"98'700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5454,"ondisk_log_size":5454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":320208896,"num_objects":4886,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6154,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":75,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526535+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:33:21.526535+0000","last_peered":"2024-02-22T19:33:21.526535+0000","last_clean":"2024-02-22T19:33:21.526535+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:33:21.526535+0000","last_undegraded":"2024-02-22T19:33:21.526535+0000","last_fullsized":"2024-02-22T19:33:21.526535+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527135+0000","last_change":"2024-02-22T19:31:50.010144+0000","last_active":"2024-02-22T19:33:21.527135+0000","last_peered":"2024-02-22T19:33:21.527135+0000","last_clean":"2024-02-22T19:33:21.527135+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:33:21.527135+0000","last_undegraded":"2024-02-22T19:33:21.527135+0000","last_fullsized":"2024-02-22T19:33:21.527135+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.692694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511833+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:33:21.511833+0000","last_peered":"2024-02-22T19:33:21.511833+0000","last_clean":"2024-02-22T19:33:21.511833+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:33:21.511833+0000","last_undegraded":"2024-02-22T19:33:21.511833+0000","last_fullsized":"2024-02-22T19:33:21.511833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526911+0000","last_change":"2024-02-22T19:30:07.992740+0000","last_active":"2024-02-22T19:33:22.526911+0000","last_peered":"2024-02-22T19:33:22.526911+0000","last_clean":"2024-02-22T19:33:22.526911+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:33:22.526911+0000","last_undegraded":"2024-02-22T19:33:22.526911+0000","last_fullsized":"2024-02-22T19:33:22.526911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:30:07.992671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.992671+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.521616+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:33:21.521616+0000","last_peered":"2024-02-22T19:33:21.521616+0000","last_clean":"2024-02-22T19:33:21.521616+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:33:21.521616+0000","last_undegraded":"2024-02-22T19:33:21.521616+0000","last_fullsized":"2024-02-22T19:33:21.521616+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"127'6139","reported_seq":6181,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.235757+0000","last_change":"2024-02-22T19:31:33.008987+0000","last_active":"2024-02-22T19:33:24.235757+0000","last_peered":"2024-02-22T19:33:24.235757+0000","last_clean":"2024-02-22T19:33:24.235757+0000","last_became_active":"2024-02-22T19:31:33.008726+0000","last_became_peered":"2024-02-22T19:31:33.008726+0000","last_unstale":"2024-02-22T19:33:24.235757+0000","last_undegraded":"2024-02-22T19:33:24.235757+0000","last_fullsized":"2024-02-22T19:33:24.235757+0000","mapping_epoch":93,"log_start":"101'1300","ondisk_log_start":"101'1300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4839,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":320929792,"num_objects":4897,"num_object_clones":0,"num_object_copies":14691,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6139,"num_write_kb":353152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525278+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:33:21.525278+0000","last_peered":"2024-02-22T19:33:21.525278+0000","last_clean":"2024-02-22T19:33:21.525278+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:33:21.525278+0000","last_undegraded":"2024-02-22T19:33:21.525278+0000","last_fullsized":"2024-02-22T19:33:21.525278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.522120+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:33:21.522120+0000","last_peered":"2024-02-22T19:33:21.522120+0000","last_clean":"2024-02-22T19:33:21.522120+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:33:21.522120+0000","last_undegraded":"2024-02-22T19:33:21.522120+0000","last_fullsized":"2024-02-22T19:33:21.522120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.533556+0000","last_change":"2024-02-22T19:30:18.117213+0000","last_active":"2024-02-22T19:33:21.533556+0000","last_peered":"2024-02-22T19:33:21.533556+0000","last_clean":"2024-02-22T19:33:21.533556+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:33:21.533556+0000","last_undegraded":"2024-02-22T19:33:21.533556+0000","last_fullsized":"2024-02-22T19:33:21.533556+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.117111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":91,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.534973+0000","last_change":"2024-02-22T19:31:50.458765+0000","last_active":"2024-02-22T19:33:21.534973+0000","last_peered":"2024-02-22T19:33:21.534973+0000","last_clean":"2024-02-22T19:33:21.534973+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:33:21.534973+0000","last_undegraded":"2024-02-22T19:33:21.534973+0000","last_fullsized":"2024-02-22T19:33:21.534973+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":89,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.535341+0000","last_change":"2024-02-22T19:31:50.006204+0000","last_active":"2024-02-22T19:33:21.535341+0000","last_peered":"2024-02-22T19:33:21.535341+0000","last_clean":"2024-02-22T19:33:21.535341+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:33:21.535341+0000","last_undegraded":"2024-02-22T19:33:21.535341+0000","last_fullsized":"2024-02-22T19:33:21.535341+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.102065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.102065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"127'6209","reported_seq":6251,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.240059+0000","last_change":"2024-02-22T19:31:33.014815+0000","last_active":"2024-02-22T19:33:24.240059+0000","last_peered":"2024-02-22T19:33:24.240059+0000","last_clean":"2024-02-22T19:33:24.240059+0000","last_became_active":"2024-02-22T19:31:33.014489+0000","last_became_peered":"2024-02-22T19:31:33.014489+0000","last_unstale":"2024-02-22T19:33:24.240059+0000","last_undegraded":"2024-02-22T19:33:24.240059+0000","last_fullsized":"2024-02-22T19:33:24.240059+0000","mapping_epoch":93,"log_start":"101'1300","ondisk_log_start":"101'1300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4909,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324468736,"num_objects":4951,"num_object_clones":0,"num_object_copies":14853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6209,"num_write_kb":357120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527102+0000","last_change":"2024-02-22T19:31:50.014877+0000","last_active":"2024-02-22T19:33:21.527102+0000","last_peered":"2024-02-22T19:33:21.527102+0000","last_clean":"2024-02-22T19:33:21.527102+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:33:21.527102+0000","last_undegraded":"2024-02-22T19:33:21.527102+0000","last_fullsized":"2024-02-22T19:33:21.527102+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524865+0000","last_change":"2024-02-22T19:31:11.906662+0000","last_active":"2024-02-22T19:33:21.524865+0000","last_peered":"2024-02-22T19:33:21.524865+0000","last_clean":"2024-02-22T19:33:21.524865+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:33:21.524865+0000","last_undegraded":"2024-02-22T19:33:21.524865+0000","last_fullsized":"2024-02-22T19:33:21.524865+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.906561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.906561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":90,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524680+0000","last_change":"2024-02-22T19:31:50.009641+0000","last_active":"2024-02-22T19:33:21.524680+0000","last_peered":"2024-02-22T19:33:21.524680+0000","last_clean":"2024-02-22T19:33:21.524680+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:33:21.524680+0000","last_undegraded":"2024-02-22T19:33:21.524680+0000","last_fullsized":"2024-02-22T19:33:21.524680+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":108,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.519926+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:33:21.519926+0000","last_peered":"2024-02-22T19:33:21.519926+0000","last_clean":"2024-02-22T19:33:21.519926+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:33:21.519926+0000","last_undegraded":"2024-02-22T19:33:21.519926+0000","last_fullsized":"2024-02-22T19:33:21.519926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524553+0000","last_change":"2024-02-22T19:31:49.995726+0000","last_active":"2024-02-22T19:33:21.524553+0000","last_peered":"2024-02-22T19:33:21.524553+0000","last_clean":"2024-02-22T19:33:21.524553+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:33:21.524553+0000","last_undegraded":"2024-02-22T19:33:21.524553+0000","last_fullsized":"2024-02-22T19:33:21.524553+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"127'6260","reported_seq":6302,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.162198+0000","last_change":"2024-02-22T19:31:33.010431+0000","last_active":"2024-02-22T19:33:24.162198+0000","last_peered":"2024-02-22T19:33:24.162198+0000","last_clean":"2024-02-22T19:33:24.162198+0000","last_became_active":"2024-02-22T19:31:33.010187+0000","last_became_peered":"2024-02-22T19:31:33.010187+0000","last_unstale":"2024-02-22T19:33:24.162198+0000","last_undegraded":"2024-02-22T19:33:24.162198+0000","last_fullsized":"2024-02-22T19:33:24.162198+0000","mapping_epoch":93,"log_start":"101'1400","ondisk_log_start":"101'1400","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4860,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":319946752,"num_objects":4882,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":87,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525848+0000","last_change":"2024-02-22T19:31:50.012369+0000","last_active":"2024-02-22T19:33:21.525848+0000","last_peered":"2024-02-22T19:33:21.525848+0000","last_clean":"2024-02-22T19:33:21.525848+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:33:21.525848+0000","last_undegraded":"2024-02-22T19:33:21.525848+0000","last_fullsized":"2024-02-22T19:33:21.525848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":82,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.521565+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:33:21.521565+0000","last_peered":"2024-02-22T19:33:21.521565+0000","last_clean":"2024-02-22T19:33:21.521565+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:33:21.521565+0000","last_undegraded":"2024-02-22T19:33:21.521565+0000","last_fullsized":"2024-02-22T19:33:21.521565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":347,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.544274+0000","last_change":"2024-02-22T19:31:29.001184+0000","last_active":"2024-02-22T19:33:22.544274+0000","last_peered":"2024-02-22T19:33:22.544274+0000","last_clean":"2024-02-22T19:33:22.544274+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:33:22.544274+0000","last_undegraded":"2024-02-22T19:33:22.544274+0000","last_fullsized":"2024-02-22T19:33:22.544274+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:31:29.001098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.001098+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":257,"num_read_kb":252,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520199+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:33:21.520199+0000","last_peered":"2024-02-22T19:33:21.520199+0000","last_clean":"2024-02-22T19:33:21.520199+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:33:21.520199+0000","last_undegraded":"2024-02-22T19:33:21.520199+0000","last_fullsized":"2024-02-22T19:33:21.520199+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.529207+0000","last_change":"2024-02-22T19:31:50.010171+0000","last_active":"2024-02-22T19:33:21.529207+0000","last_peered":"2024-02-22T19:33:21.529207+0000","last_clean":"2024-02-22T19:33:21.529207+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:33:21.529207+0000","last_undegraded":"2024-02-22T19:33:21.529207+0000","last_fullsized":"2024-02-22T19:33:21.529207+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"127'6238","reported_seq":8923,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.214337+0000","last_change":"2024-02-22T19:32:07.977104+0000","last_active":"2024-02-22T19:33:24.214337+0000","last_peered":"2024-02-22T19:33:24.214337+0000","last_clean":"2024-02-22T19:33:24.214337+0000","last_became_active":"2024-02-22T19:32:07.973466+0000","last_became_peered":"2024-02-22T19:32:07.973466+0000","last_unstale":"2024-02-22T19:33:24.214337+0000","last_undegraded":"2024-02-22T19:33:24.214337+0000","last_fullsized":"2024-02-22T19:33:24.214337+0000","mapping_epoch":105,"log_start":"102'1600","ondisk_log_start":"102'1600","created":93,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4638,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":321388544,"num_objects":4904,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6238,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":86048768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525329+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:33:21.525329+0000","last_peered":"2024-02-22T19:33:21.525329+0000","last_clean":"2024-02-22T19:33:21.525329+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:33:21.525329+0000","last_undegraded":"2024-02-22T19:33:21.525329+0000","last_fullsized":"2024-02-22T19:33:21.525329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526067+0000","last_change":"2024-02-22T19:31:50.010143+0000","last_active":"2024-02-22T19:33:21.526067+0000","last_peered":"2024-02-22T19:33:21.526067+0000","last_clean":"2024-02-22T19:33:21.526067+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:33:21.526067+0000","last_undegraded":"2024-02-22T19:33:21.526067+0000","last_fullsized":"2024-02-22T19:33:21.526067+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520373+0000","last_change":"2024-02-22T19:31:50.036817+0000","last_active":"2024-02-22T19:33:21.520373+0000","last_peered":"2024-02-22T19:33:21.520373+0000","last_clean":"2024-02-22T19:33:21.520373+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:33:21.520373+0000","last_undegraded":"2024-02-22T19:33:21.520373+0000","last_fullsized":"2024-02-22T19:33:21.520373+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":88,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.519585+0000","last_change":"2024-02-22T19:31:50.006906+0000","last_active":"2024-02-22T19:33:21.519585+0000","last_peered":"2024-02-22T19:33:21.519585+0000","last_clean":"2024-02-22T19:33:21.519585+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:33:21.519585+0000","last_undegraded":"2024-02-22T19:33:21.519585+0000","last_fullsized":"2024-02-22T19:33:21.519585+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"127'6176","reported_seq":6218,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.182774+0000","last_change":"2024-02-22T19:31:33.012064+0000","last_active":"2024-02-22T19:33:24.182774+0000","last_peered":"2024-02-22T19:33:24.182774+0000","last_clean":"2024-02-22T19:33:24.182774+0000","last_became_active":"2024-02-22T19:31:33.011750+0000","last_became_peered":"2024-02-22T19:31:33.011750+0000","last_unstale":"2024-02-22T19:33:24.182774+0000","last_undegraded":"2024-02-22T19:33:24.182774+0000","last_fullsized":"2024-02-22T19:33:24.182774+0000","mapping_epoch":93,"log_start":"102'1600","ondisk_log_start":"102'1600","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4576,"ondisk_log_size":4576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":319946752,"num_objects":4882,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":353856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.521077+0000","last_change":"2024-02-22T19:30:18.719372+0000","last_active":"2024-02-22T19:33:21.521077+0000","last_peered":"2024-02-22T19:33:21.521077+0000","last_clean":"2024-02-22T19:33:21.521077+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:33:21.521077+0000","last_undegraded":"2024-02-22T19:33:21.521077+0000","last_fullsized":"2024-02-22T19:33:21.521077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.719310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524657+0000","last_change":"2024-02-22T19:31:50.006662+0000","last_active":"2024-02-22T19:33:21.524657+0000","last_peered":"2024-02-22T19:33:21.524657+0000","last_clean":"2024-02-22T19:33:21.524657+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:33:21.524657+0000","last_undegraded":"2024-02-22T19:33:21.524657+0000","last_fullsized":"2024-02-22T19:33:21.524657+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":118,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527425+0000","last_change":"2024-02-22T19:31:50.027001+0000","last_active":"2024-02-22T19:33:21.527425+0000","last_peered":"2024-02-22T19:33:21.527425+0000","last_clean":"2024-02-22T19:33:21.527425+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:33:21.527425+0000","last_undegraded":"2024-02-22T19:33:21.527425+0000","last_fullsized":"2024-02-22T19:33:21.527425+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:12.059797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:12.059797+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":152,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525186+0000","last_change":"2024-02-22T19:31:50.440080+0000","last_active":"2024-02-22T19:33:22.525186+0000","last_peered":"2024-02-22T19:33:22.525186+0000","last_clean":"2024-02-22T19:33:22.525186+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:33:22.525186+0000","last_undegraded":"2024-02-22T19:33:22.525186+0000","last_fullsized":"2024-02-22T19:33:22.525186+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"127'6238","reported_seq":6279,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.224884+0000","last_change":"2024-02-22T19:31:33.012817+0000","last_active":"2024-02-22T19:33:24.224884+0000","last_peered":"2024-02-22T19:33:24.224884+0000","last_clean":"2024-02-22T19:33:24.224884+0000","last_became_active":"2024-02-22T19:31:33.012470+0000","last_became_peered":"2024-02-22T19:31:33.012470+0000","last_unstale":"2024-02-22T19:33:24.224884+0000","last_undegraded":"2024-02-22T19:33:24.224884+0000","last_fullsized":"2024-02-22T19:33:24.224884+0000","mapping_epoch":93,"log_start":"102'2100","ondisk_log_start":"102'2100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4138,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":322699264,"num_objects":4924,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6238,"num_write_kb":357184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":76,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.524341+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:33:21.524341+0000","last_peered":"2024-02-22T19:33:21.524341+0000","last_clean":"2024-02-22T19:33:21.524341+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:33:21.524341+0000","last_undegraded":"2024-02-22T19:33:21.524341+0000","last_fullsized":"2024-02-22T19:33:21.524341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":82,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.520308+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:33:21.520308+0000","last_peered":"2024-02-22T19:33:21.520308+0000","last_clean":"2024-02-22T19:33:21.520308+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:33:21.520308+0000","last_undegraded":"2024-02-22T19:33:21.520308+0000","last_fullsized":"2024-02-22T19:33:21.520308+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":135,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511726+0000","last_change":"2024-02-22T19:31:50.644540+0000","last_active":"2024-02-22T19:33:21.511726+0000","last_peered":"2024-02-22T19:33:21.511726+0000","last_clean":"2024-02-22T19:33:21.511726+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:33:21.511726+0000","last_undegraded":"2024-02-22T19:33:21.511726+0000","last_fullsized":"2024-02-22T19:33:21.511726+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528075+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:33:22.528075+0000","last_peered":"2024-02-22T19:33:22.528075+0000","last_clean":"2024-02-22T19:33:22.528075+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:33:22.528075+0000","last_undegraded":"2024-02-22T19:33:22.528075+0000","last_fullsized":"2024-02-22T19:33:22.528075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"127'6362","reported_seq":6404,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.267648+0000","last_change":"2024-02-22T19:31:33.013118+0000","last_active":"2024-02-22T19:33:24.267648+0000","last_peered":"2024-02-22T19:33:24.267648+0000","last_clean":"2024-02-22T19:33:24.267648+0000","last_became_active":"2024-02-22T19:31:33.012002+0000","last_became_peered":"2024-02-22T19:31:33.012002+0000","last_unstale":"2024-02-22T19:33:24.267648+0000","last_undegraded":"2024-02-22T19:33:24.267648+0000","last_fullsized":"2024-02-22T19:33:24.267648+0000","mapping_epoch":93,"log_start":"103'2200","ondisk_log_start":"103'2200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4162,"ondisk_log_size":4162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":328859648,"num_objects":5018,"num_object_clones":0,"num_object_copies":15054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6362,"num_write_kb":364160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511746+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:33:21.511746+0000","last_peered":"2024-02-22T19:33:21.511746+0000","last_clean":"2024-02-22T19:33:21.511746+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:33:21.511746+0000","last_undegraded":"2024-02-22T19:33:21.511746+0000","last_fullsized":"2024-02-22T19:33:21.511746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":81,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526985+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:33:21.526985+0000","last_peered":"2024-02-22T19:33:21.526985+0000","last_clean":"2024-02-22T19:33:21.526985+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:33:21.526985+0000","last_undegraded":"2024-02-22T19:33:21.526985+0000","last_fullsized":"2024-02-22T19:33:21.526985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":111,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525127+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:33:21.525127+0000","last_peered":"2024-02-22T19:33:21.525127+0000","last_clean":"2024-02-22T19:33:21.525127+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:33:21.525127+0000","last_undegraded":"2024-02-22T19:33:21.525127+0000","last_fullsized":"2024-02-22T19:33:21.525127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":85,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511551+0000","last_change":"2024-02-22T19:31:26.190739+0000","last_active":"2024-02-22T19:33:21.511551+0000","last_peered":"2024-02-22T19:33:21.511551+0000","last_clean":"2024-02-22T19:33:21.511551+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:33:21.511551+0000","last_undegraded":"2024-02-22T19:33:21.511551+0000","last_fullsized":"2024-02-22T19:33:21.511551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.190632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.190632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"127'6318","reported_seq":6360,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.094997+0000","last_change":"2024-02-22T19:31:33.012589+0000","last_active":"2024-02-22T19:33:24.094997+0000","last_peered":"2024-02-22T19:33:24.094997+0000","last_clean":"2024-02-22T19:33:24.094997+0000","last_became_active":"2024-02-22T19:31:33.012188+0000","last_became_peered":"2024-02-22T19:31:33.012188+0000","last_unstale":"2024-02-22T19:33:24.094997+0000","last_undegraded":"2024-02-22T19:33:24.094997+0000","last_fullsized":"2024-02-22T19:33:24.094997+0000","mapping_epoch":93,"log_start":"98'900","ondisk_log_start":"98'900","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5418,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":326500352,"num_objects":4982,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318,"num_write_kb":361600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.527443+0000","last_change":"2024-02-22T19:31:50.009247+0000","last_active":"2024-02-22T19:33:21.527443+0000","last_peered":"2024-02-22T19:33:21.527443+0000","last_clean":"2024-02-22T19:33:21.527443+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:33:21.527443+0000","last_undegraded":"2024-02-22T19:33:21.527443+0000","last_fullsized":"2024-02-22T19:33:21.527443+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:25.682652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:25.682652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536482+0000","last_change":"2024-02-22T19:31:50.016299+0000","last_active":"2024-02-22T19:33:22.536482+0000","last_peered":"2024-02-22T19:33:22.536482+0000","last_clean":"2024-02-22T19:33:22.536482+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:33:22.536482+0000","last_undegraded":"2024-02-22T19:33:22.536482+0000","last_fullsized":"2024-02-22T19:33:22.536482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":86,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525036+0000","last_change":"2024-02-22T19:31:14.073980+0000","last_active":"2024-02-22T19:33:21.525036+0000","last_peered":"2024-02-22T19:33:21.525036+0000","last_clean":"2024-02-22T19:33:21.525036+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:33:21.525036+0000","last_undegraded":"2024-02-22T19:33:21.525036+0000","last_fullsized":"2024-02-22T19:33:21.525036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.073908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.073908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":74,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.534868+0000","last_change":"2024-02-22T19:30:27.174847+0000","last_active":"2024-02-22T19:33:21.534868+0000","last_peered":"2024-02-22T19:33:21.534868+0000","last_clean":"2024-02-22T19:33:21.534868+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:33:21.534868+0000","last_undegraded":"2024-02-22T19:33:21.534868+0000","last_fullsized":"2024-02-22T19:33:21.534868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:27.174779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.174779+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"127'6404","reported_seq":6447,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.125277+0000","last_change":"2024-02-22T19:31:33.011737+0000","last_active":"2024-02-22T19:33:24.125277+0000","last_peered":"2024-02-22T19:33:24.125277+0000","last_clean":"2024-02-22T19:33:24.125277+0000","last_became_active":"2024-02-22T19:31:33.011438+0000","last_became_peered":"2024-02-22T19:31:33.011438+0000","last_unstale":"2024-02-22T19:33:24.125277+0000","last_undegraded":"2024-02-22T19:33:24.125277+0000","last_fullsized":"2024-02-22T19:33:24.125277+0000","mapping_epoch":93,"log_start":"98'1000","ondisk_log_start":"98'1000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5404,"ondisk_log_size":5404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":329580568,"num_objects":5030,"num_object_clones":0,"num_object_copies":15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5030,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6404,"num_write_kb":365825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.525411+0000","last_change":"2024-02-22T19:31:50.006646+0000","last_active":"2024-02-22T19:33:21.525411+0000","last_peered":"2024-02-22T19:33:21.525411+0000","last_clean":"2024-02-22T19:33:21.525411+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:33:21.525411+0000","last_undegraded":"2024-02-22T19:33:21.525411+0000","last_fullsized":"2024-02-22T19:33:21.525411+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512061+0000","last_change":"2024-02-22T19:31:50.016416+0000","last_active":"2024-02-22T19:33:21.512061+0000","last_peered":"2024-02-22T19:33:21.512061+0000","last_clean":"2024-02-22T19:33:21.512061+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:33:21.512061+0000","last_undegraded":"2024-02-22T19:33:21.512061+0000","last_fullsized":"2024-02-22T19:33:21.512061+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.511635+0000","last_change":"2024-02-22T19:33:10.664920+0000","last_active":"2024-02-22T19:33:21.511635+0000","last_peered":"2024-02-22T19:33:21.511635+0000","last_clean":"2024-02-22T19:33:21.511635+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:33:21.511635+0000","last_undegraded":"2024-02-22T19:33:21.511635+0000","last_fullsized":"2024-02-22T19:33:21.511635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:19.102765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:19.102765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":101,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512240+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:33:21.512240+0000","last_peered":"2024-02-22T19:33:21.512240+0000","last_clean":"2024-02-22T19:33:21.512240+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:33:21.512240+0000","last_undegraded":"2024-02-22T19:33:21.512240+0000","last_fullsized":"2024-02-22T19:33:21.512240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"127'6054","reported_seq":8273,"reported_epoch":127,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:33:24.240214+0000","last_change":"2024-02-22T19:31:51.077193+0000","last_active":"2024-02-22T19:33:24.240214+0000","last_peered":"2024-02-22T19:33:24.240214+0000","last_clean":"2024-02-22T19:31:48.945369+0000","last_became_active":"2024-02-22T19:31:51.043861+0000","last_became_peered":"2024-02-22T19:31:51.043861+0000","last_unstale":"2024-02-22T19:33:24.240214+0000","last_undegraded":"2024-02-22T19:33:24.240214+0000","last_fullsized":"2024-02-22T19:31:50.995661+0000","mapping_epoch":100,"log_start":"98'700","ondisk_log_start":"98'700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5354,"ondisk_log_size":5354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":312475648,"num_objects":4768,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":169,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6054,"num_write_kb":346304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1085,"num_bytes_recovered":71106560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0],"avail_no_missing":["2","0","5"],"object_location_counts":[{"shards":"0,2","objects":4599},{"shards":"0,2,5","objects":169}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.535059+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:33:21.535059+0000","last_peered":"2024-02-22T19:33:21.535059+0000","last_clean":"2024-02-22T19:33:21.535059+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:33:21.535059+0000","last_undegraded":"2024-02-22T19:33:21.535059+0000","last_fullsized":"2024-02-22T19:33:21.535059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.526271+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:33:21.526271+0000","last_peered":"2024-02-22T19:33:21.526271+0000","last_clean":"2024-02-22T19:33:21.526271+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:33:21.526271+0000","last_undegraded":"2024-02-22T19:33:21.526271+0000","last_fullsized":"2024-02-22T19:33:21.526271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":81,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.519324+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:33:21.519324+0000","last_peered":"2024-02-22T19:33:21.519324+0000","last_clean":"2024-02-22T19:33:21.519324+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:33:21.519324+0000","last_undegraded":"2024-02-22T19:33:21.519324+0000","last_fullsized":"2024-02-22T19:33:21.519324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512102+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:33:21.512102+0000","last_peered":"2024-02-22T19:33:21.512102+0000","last_clean":"2024-02-22T19:33:21.512102+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:33:21.512102+0000","last_undegraded":"2024-02-22T19:33:21.512102+0000","last_fullsized":"2024-02-22T19:33:21.512102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"127'6230","reported_seq":6272,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:24.229319+0000","last_change":"2024-02-22T19:31:33.014534+0000","last_active":"2024-02-22T19:33:24.229319+0000","last_peered":"2024-02-22T19:33:24.229319+0000","last_clean":"2024-02-22T19:33:24.229319+0000","last_became_active":"2024-02-22T19:31:33.014231+0000","last_became_peered":"2024-02-22T19:31:33.014231+0000","last_unstale":"2024-02-22T19:33:24.229319+0000","last_undegraded":"2024-02-22T19:33:24.229319+0000","last_fullsized":"2024-02-22T19:33:24.229319+0000","mapping_epoch":93,"log_start":"98'1300","ondisk_log_start":"98'1300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4930,"ondisk_log_size":4930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":323878912,"num_objects":4942,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6230,"num_write_kb":357504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":80,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512404+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:33:21.512404+0000","last_peered":"2024-02-22T19:33:21.512404+0000","last_clean":"2024-02-22T19:33:21.512404+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:33:21.512404+0000","last_undegraded":"2024-02-22T19:33:21.512404+0000","last_fullsized":"2024-02-22T19:33:21.512404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.540838+0000","last_change":"2024-02-22T19:31:50.104329+0000","last_active":"2024-02-22T19:33:22.540838+0000","last_peered":"2024-02-22T19:33:22.540838+0000","last_clean":"2024-02-22T19:33:22.540838+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:33:22.540838+0000","last_undegraded":"2024-02-22T19:33:22.540838+0000","last_fullsized":"2024-02-22T19:33:22.540838+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:23.100177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.100177+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":90,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.528273+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:33:21.528273+0000","last_peered":"2024-02-22T19:33:21.528273+0000","last_clean":"2024-02-22T19:33:21.528273+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:33:21.528273+0000","last_undegraded":"2024-02-22T19:33:21.528273+0000","last_fullsized":"2024-02-22T19:33:21.528273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-22T19:33:21.512542+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:33:21.512542+0000","last_peered":"2024-02-22T19:33:21.512542+0000","last_clean":"2024-02-22T19:33:21.512542+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:33:21.512542+0000","last_undegraded":"2024-02-22T19:33:21.512542+0000","last_fullsized":"2024-02-22T19:33:21.512542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":5158076440,"num_objects":78707,"num_object_clones":0,"num_object_copies":236121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":356,"num_objects_unfound":0,"num_objects_dirty":78707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":99703,"num_write_kb":5709057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4758,"num_bytes_recovered":311820288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15615340544,"data_stored":15615328328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77703,"ondisk_log_size":77703,"up":48,"acting":46,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":257,"num_read_kb":252,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266744,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1891204,"kb_used_data":1585788,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91853948,"statfs":{"total":95995035648,"available":94058442752,"internally_reserved":0,"allocated":1623846912,"data_stored":1623446718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":5.0609999999999999,"5min":1.506,"15min":0.88400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":42.597999999999999,"5min":42.597999999999999,"15min":42.597999999999999},"last":3.3620000000000001},{"interface":"front","average":{"1min":3.5640000000000001,"5min":1.1499999999999999,"15min":0.73599999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":42.155999999999999,"5min":42.155999999999999,"15min":42.155999999999999},"last":2.9940000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.6139999999999999,"5min":1.1719999999999999,"15min":0.77600000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":38.188000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":0.93500000000000005},{"interface":"front","average":{"1min":4.2999999999999998,"5min":1.3089999999999999,"15min":0.79900000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":37.938000000000002,"5min":37.938000000000002,"15min":37.938000000000002},"last":3.6139999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":1.226,"15min":0.78200000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":41.640999999999998,"5min":41.640999999999998,"15min":41.640999999999998},"last":5.2220000000000004},{"interface":"front","average":{"1min":4.2530000000000001,"5min":1.3560000000000001,"15min":0.85199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":41.722000000000001,"5min":41.722000000000001,"15min":41.722000000000001},"last":2.032}]},{"osd":3,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1479999999999997,"5min":1.329,"15min":0.84499999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":41.457000000000001,"5min":41.457000000000001,"15min":41.457000000000001},"last":5.9649999999999999},{"interface":"front","average":{"1min":3.6549999999999998,"5min":1.204,"15min":0.81899999999999995},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":41.311999999999998,"5min":41.311999999999998,"15min":41.311999999999998},"last":3.6040000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":2.9620000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":3.4780000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":3.0499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":3.8519999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4889999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":3.226},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5539999999999998,"5min":12.067,"15min":12.067},"last":3.4430000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1887596,"kb_used_data":1586276,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91857556,"statfs":{"total":95995035648,"available":94062137344,"internally_reserved":0,"allocated":1624346624,"data_stored":1623970098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.472,"5min":0.71199999999999997,"15min":0.57999999999999996},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.6150000000000002,"5min":6.6150000000000002,"15min":6.6150000000000002},"last":4.9889999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.622,"15min":0.55600000000000005},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":7.415,"5min":7.415,"15min":7.415},"last":1.5469999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":0.76100000000000001,"15min":0.60999999999999999},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":12.859999999999999,"5min":12.859999999999999,"15min":12.859999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.63100000000000001,"15min":0.57799999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.0289999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":3.4780000000000002,"5min":1.125,"15min":0.73499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":38.762,"5min":38.762,"15min":38.762},"last":1.8140000000000001},{"interface":"front","average":{"1min":3.569,"5min":1.1579999999999999,"15min":0.75600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":35.612000000000002,"5min":35.612000000000002,"15min":35.612000000000002},"last":0.998}]},{"osd":4,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69399999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":1.583},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.45100000000000001,"15min":0.432},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.629,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45500000000000002,"15min":0.438},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65400000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.432},"min":{"1min":0.309,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68000000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.94699999999999995}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48999999999999999,"15min":0.51000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.5700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":1.232}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2508508,"kb_used_data":2211220,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":91236644,"statfs":{"total":95995035648,"available":93426323456,"internally_reserved":0,"allocated":2264289280,"data_stored":2263864978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":0.73799999999999999,"15min":0.46400000000000002},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":6.1950000000000003},{"interface":"front","average":{"1min":3.0569999999999999,"5min":0.93500000000000005,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":11.352,"5min":11.352,"15min":11.352},"last":4.0129999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":4.5469999999999997,"5min":1.3220000000000001,"15min":0.70199999999999996},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":15.683,"5min":15.683,"15min":15.683},"last":2.6520000000000001},{"interface":"front","average":{"1min":4.9130000000000003,"5min":1.3400000000000001,"15min":0.70799999999999996},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":13.461,"5min":13.461,"15min":13.461},"last":3.218}]},{"osd":3,"last update":"Thu Feb 22 19:32:32 2024","interfaces":[{"interface":"back","average":{"1min":3.7759999999999998,"5min":1.079,"15min":0.67100000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":25.145,"5min":25.145,"15min":25.145},"last":5.8380000000000001},{"interface":"front","average":{"1min":4.2690000000000001,"5min":1.2310000000000001,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.252,"15min":0.252},"max":{"1min":22.25,"5min":22.25,"15min":22.25},"last":6.2359999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:32:50 2024","interfaces":[{"interface":"back","average":{"1min":2.5350000000000001,"5min":0.95899999999999996,"15min":0.73499999999999999},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":15.542999999999999,"5min":15.542999999999999,"15min":15.542999999999999},"last":2.5489999999999999},{"interface":"front","average":{"1min":1.226,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.40400000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":2.4239999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:33:09 2024","interfaces":[{"interface":"back","average":{"1min":3.1429999999999998,"5min":1.1819999999999999,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":15.587999999999999,"5min":15.587999999999999,"15min":15.587999999999999},"last":2.6269999999999998},{"interface":"front","average":{"1min":2.694,"5min":1.0149999999999999,"15min":0.72299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.5839999999999996,"5min":7.5839999999999996,"15min":7.5839999999999996},"last":3.1890000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":0.67100000000000004,"15min":0.58599999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.375,"5min":5.375,"15min":5.375},"last":2.4569999999999999},{"interface":"front","average":{"1min":2.4670000000000001,"5min":0.96799999999999997,"15min":0.71699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.651999999999999,"5min":15.651999999999999,"15min":15.651999999999999},"last":2.601}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.5579999999999998,"5min":0.97299999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":3.1659999999999999},{"interface":"front","average":{"1min":1.706,"5min":0.82399999999999995,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":2.4780000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2824772,"kb_used_data":2519356,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":90920380,"statfs":{"total":95995035648,"available":93102469120,"internally_reserved":0,"allocated":2579820544,"data_stored":2579420724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:32:48 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":0.68899999999999995,"15min":0.46600000000000003},"min":{"1min":0.20599999999999999,"5min":0.191,"15min":0.191},"max":{"1min":9.8640000000000008,"5min":9.8640000000000008,"15min":9.8640000000000008},"last":4.1689999999999996},{"interface":"front","average":{"1min":2.819,"5min":0.874,"15min":0.52900000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":8.3670000000000009,"5min":8.3670000000000009,"15min":8.3670000000000009},"last":5.9219999999999997}]},{"osd":2,"last update":"Thu Feb 22 19:33:06 2024","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":1.335,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":19.927,"5min":19.927,"15min":19.927},"last":3.077},{"interface":"front","average":{"1min":4.2110000000000003,"5min":1.327,"15min":0.69099999999999995},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":17.907,"5min":17.907,"15min":17.907},"last":1.718}]},{"osd":3,"last update":"Thu Feb 22 19:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":0.68300000000000005,"15min":0.51800000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":7.585,"5min":7.585,"15min":7.585},"last":10.946},{"interface":"front","average":{"1min":2.4089999999999998,"5min":0.80700000000000005,"15min":0.56899999999999995},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":22.936,"5min":22.936,"15min":22.936},"last":3.2970000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.66900000000000004,"15min":0.60499999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":4.3129999999999997,"5min":4.3129999999999997,"15min":4.3129999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":2.7930000000000001,"5min":1.0109999999999999,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.085000000000001,"5min":12.085000000000001,"15min":12.085000000000001},"last":2.9830000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":1.26,"15min":0.79800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":13.903,"5min":13.903,"15min":13.903},"last":3.0409999999999999},{"interface":"front","average":{"1min":1.7270000000000001,"5min":0.81100000000000005,"15min":0.64900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.8920000000000003,"5min":6.8920000000000003,"15min":6.8920000000000003},"last":0.28699999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.30099999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":0.745,"15min":0.64400000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":5.1630000000000003,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":3.1960000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:33:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":0.84399999999999997,"15min":0.66400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9660000000000002,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.315},{"interface":"front","average":{"1min":2.4279999999999999,"5min":0.95299999999999996,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":6.0800000000000001,"5min":6.0800000000000001,"15min":6.0800000000000001},"last":3.2320000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2823544,"kb_used_data":2522224,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":90921608,"statfs":{"total":95995035648,"available":93103726592,"internally_reserved":0,"allocated":2582757376,"data_stored":2582368766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":1.1719999999999999,"15min":0.63600000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":11.898999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":5.3280000000000003},{"interface":"front","average":{"1min":3.6859999999999999,"5min":1.4239999999999999,"15min":0.71599999999999997},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":16.138999999999999,"5min":18.059999999999999,"15min":18.059999999999999},"last":2.093}]},{"osd":1,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":0.96199999999999997,"15min":0.57199999999999995},"min":{"1min":0.21199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.183,"5min":10.183,"15min":10.183},"last":4.1310000000000002},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.73799999999999999,"15min":0.51600000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":9.7189999999999994,"5min":9.7189999999999994,"15min":9.7189999999999994},"last":5.2889999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:32:31 2024","interfaces":[{"interface":"back","average":{"1min":2.9470000000000001,"5min":0.94499999999999995,"15min":0.624},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":17.431000000000001,"5min":17.431000000000001,"15min":17.431000000000001},"last":2.032},{"interface":"front","average":{"1min":3.4390000000000001,"5min":1.02,"15min":0.63800000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":18.099,"5min":18.099,"15min":18.099},"last":2.1150000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:32:53 2024","interfaces":[{"interface":"back","average":{"1min":2.1669999999999998,"5min":0.90400000000000003,"15min":0.68200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":6.0449999999999999,"5min":6.0449999999999999,"15min":6.0449999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.67500000000000004,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":1.984}]},{"osd":5,"last update":"Thu Feb 22 19:33:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":0.747,"15min":0.66500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":2.0070000000000001},{"interface":"front","average":{"1min":2.1669999999999998,"5min":0.97999999999999998,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0739999999999998,"5min":6.0739999999999998,"15min":6.0739999999999998},"last":0.97599999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.8580000000000001,"5min":0.85799999999999998,"15min":0.68799999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.0119999999999996,"5min":6.0119999999999996,"15min":6.0119999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.71199999999999997,"15min":0.65800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":9.5990000000000002,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":1.925}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":0.94799999999999995,"15min":0.70199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.1029999999999998,"5min":6.1029999999999998,"15min":6.1029999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.73699999999999999,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":9.5660000000000007,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":1.96}]}]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3457548,"kb_used_data":3147908,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":90287604,"statfs":{"total":95995035648,"available":92454506496,"internally_reserved":0,"allocated":3223457792,"data_stored":3223050112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:28 2024","interfaces":[{"interface":"back","average":{"1min":2.3660000000000001,"5min":0.78100000000000003,"15min":0.55400000000000005},"min":{"1min":0.22,"5min":0.193,"15min":0.193},"max":{"1min":7.9699999999999998,"5min":7.9699999999999998,"15min":7.9699999999999998},"last":0.22500000000000001},{"interface":"front","average":{"1min":2.7309999999999999,"5min":0.88200000000000001,"15min":0.61899999999999999},"min":{"1min":0.311,"5min":0.247,"15min":0.247},"max":{"1min":11.436,"5min":11.436,"15min":11.436},"last":3.7570000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:32:28 2024","interfaces":[{"interface":"back","average":{"1min":3.282,"5min":0.99099999999999999,"15min":0.64400000000000002},"min":{"1min":0.27000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":11.342000000000001,"5min":11.342000000000001,"15min":11.342000000000001},"last":8.0909999999999993},{"interface":"front","average":{"1min":2.6669999999999998,"5min":0.86199999999999999,"15min":0.59299999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":10.279,"5min":10.279,"15min":10.279},"last":3.0169999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:32:28 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":0.97399999999999998,"15min":0.65300000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.1109999999999998,"5min":7.1109999999999998,"15min":7.1109999999999998},"last":1.877},{"interface":"front","average":{"1min":3.069,"5min":0.95799999999999996,"15min":0.63800000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.5830000000000002,"5min":8.5830000000000002,"15min":8.5830000000000002},"last":0.498}]},{"osd":4,"last update":"Thu Feb 22 19:32:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3359999999999999,"5min":0.99099999999999999,"15min":0.747},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":11.241,"5min":11.241,"15min":11.241},"last":3.2290000000000001},{"interface":"front","average":{"1min":2.1680000000000001,"5min":0.95699999999999996,"15min":0.72299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.3700000000000001,"5min":6.7089999999999996,"15min":6.7089999999999996},"last":1.8460000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":1.125,"15min":0.76800000000000002},"min":{"1min":0.437,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.8550000000000004,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":3.3239999999999998,"5min":1.2689999999999999,"15min":0.81100000000000005},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":12.744999999999999,"5min":12.744999999999999,"15min":12.744999999999999},"last":3.7919999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:32:41 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":0.83899999999999997,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":2.5,"5min":0.98199999999999998,"15min":0.76100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":11.308,"5min":11.308,"15min":11.308},"last":4.0359999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:32:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":1.0780000000000001,"15min":0.74299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":10.055,"5min":10.055,"15min":10.055},"last":1.9199999999999999},{"interface":"front","average":{"1min":2.3319999999999999,"5min":1.022,"15min":0.746},"min":{"1min":0.45700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.8200000000000003,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.59599999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1888776,"kb_used_data":1583360,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91856376,"statfs":{"total":95995035648,"available":94060929024,"internally_reserved":0,"allocated":1621360640,"data_stored":1620956836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":2.3029999999999999,"5min":0.88100000000000001,"15min":0.63800000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":24.009,"5min":24.009,"15min":24.009},"last":1.3100000000000001},{"interface":"front","average":{"1min":2.7599999999999998,"5min":0.98499999999999999,"15min":0.69199999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":24.263000000000002,"5min":24.263000000000002,"15min":24.263000000000002},"last":2.6600000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":0.68100000000000005,"15min":0.59399999999999997},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":2.1549999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.59999999999999998,"15min":0.53500000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.097}]},{"osd":2,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1709999999999998,"5min":0.86399999999999999,"15min":0.66900000000000004},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":23.082000000000001,"5min":23.082000000000001,"15min":23.082000000000001},"last":2.802},{"interface":"front","average":{"1min":2.6110000000000002,"5min":0.95899999999999996,"15min":0.68100000000000005},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":23.138999999999999,"5min":23.138999999999999,"15min":23.138999999999999},"last":2.1499999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":0.749,"15min":0.61899999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":5.3460000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":0.73499999999999999,"15min":0.59899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.0149999999999997,"5min":7.0149999999999997,"15min":7.0149999999999997},"last":8.9909999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.46500000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.48099999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.35999999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.54400000000000004}]}]},{"osd":5,"up_from":0,"seq":167503724610,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2574520320,"data_stored":2574516248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2249723904,"data_stored":2249719832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2568814592,"data_stored":2568814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3209101312,"data_stored":3209101312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1613955072,"data_stored":1613955072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":165543936,"data_stored":165543936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1618415616,"data_stored":1618411544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1615265792,"data_stored":1615265792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:33:29.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:33:29.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:29.733 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:30.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:29.077137+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e128: 8 total, 8 up, 7 in 2024-02-22T19:33:30.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:29.294581+0000 mgr.y (mgr.14535) 176 : cluster [DBG] pgmap v189: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 958 op/s; 240/225639 objects misplaced (0.106%); 1.9 MiB/s, 29 objects/s recovering 2024-02-22T19:33:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:29.077137+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e128: 8 total, 8 up, 7 in 2024-02-22T19:33:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:29.294581+0000 mgr.y (mgr.14535) 176 : cluster [DBG] pgmap v189: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 958 op/s; 240/225639 objects misplaced (0.106%); 1.9 MiB/s, 29 objects/s recovering 2024-02-22T19:33:30.317 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:30.356 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:29.077137+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e128: 8 total, 8 up, 7 in 2024-02-22T19:33:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:29.294581+0000 mgr.y (mgr.14535) 176 : cluster [DBG] pgmap v189: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 958 op/s; 240/225639 objects misplaced (0.106%); 1.9 MiB/s, 29 objects/s recovering 2024-02-22T19:33:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:30.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:30.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:30.793 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:30.828 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:30.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:30.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:31.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:31.125 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:31.194 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:33:31.217 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:31.217 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:31.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:29.993162+0000 osd.0 (osd.0) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:31.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:29.993213+0000 osd.0 (osd.0) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:31.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:30.131521+0000 mon.a (mon.0) 861 : cluster [DBG] osdmap e129: 8 total, 8 up, 7 in 2024-02-22T19:33:31.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:30.669457+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.53:0/2325124084' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:31.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:30.674514+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:31.477 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:29.993162+0000 osd.0 (osd.0) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:31.477 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:29.993213+0000 osd.0 (osd.0) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:30.131521+0000 mon.a (mon.0) 861 : cluster [DBG] osdmap e129: 8 total, 8 up, 7 in 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:30.669457+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.53:0/2325124084' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:30.674514+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:29.993162+0000 osd.0 (osd.0) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:29.993213+0000 osd.0 (osd.0) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:30.131521+0000 mon.a (mon.0) 861 : cluster [DBG] osdmap e129: 8 total, 8 up, 7 in 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:30.669457+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.53:0/2325124084' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:31.478 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:30.674514+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:31.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:33:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:32.128 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:32.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:32.158 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:31.130813+0000 mon.a (mon.0) 863 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:32.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:31.151806+0000 mon.a (mon.0) 864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:32.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:31.151852+0000 mon.a (mon.0) 865 : cluster [DBG] osdmap e130: 8 total, 8 up, 7 in 2024-02-22T19:33:32.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:31.295016+0000 mgr.y (mgr.14535) 177 : cluster [DBG] pgmap v192: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 614 op/s; 240/225639 objects misplaced (0.106%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:32.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:32.264 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:32.293 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:32.296 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:32.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:32.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:32.457 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:31.130813+0000 mon.a (mon.0) 863 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:31.151806+0000 mon.a (mon.0) 864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:31.151852+0000 mon.a (mon.0) 865 : cluster [DBG] osdmap e130: 8 total, 8 up, 7 in 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:31.295016+0000 mgr.y (mgr.14535) 177 : cluster [DBG] pgmap v192: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 614 op/s; 240/225639 objects misplaced (0.106%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:31.130813+0000 mon.a (mon.0) 863 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:31.151806+0000 mon.a (mon.0) 864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:31.151852+0000 mon.a (mon.0) 865 : cluster [DBG] osdmap e130: 8 total, 8 up, 7 in 2024-02-22T19:33:32.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:31.295016+0000 mgr.y (mgr.14535) 177 : cluster [DBG] pgmap v192: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 614 op/s; 240/225639 objects misplaced (0.106%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:34.148 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:33:34.150 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:34.150 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:34.174 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:33:34.233 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:33:34.279 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:34.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:34.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:34.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:33:34.341 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:33:34.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:34.406 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:33:34.455 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:33.296332+0000 mgr.y (mgr.14535) 178 : cluster [DBG] pgmap v193: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 614 op/s; 240/225639 objects misplaced (0.106%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:34.455 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:34.123994+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? 172.21.15.53:0/2152630116' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:34.456 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:33:34.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:33.296332+0000 mgr.y (mgr.14535) 178 : cluster [DBG] pgmap v193: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 614 op/s; 240/225639 objects misplaced (0.106%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:34.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:34.123994+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? 172.21.15.53:0/2152630116' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:34.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:33.296332+0000 mgr.y (mgr.14535) 178 : cluster [DBG] pgmap v193: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 614 op/s; 240/225639 objects misplaced (0.106%); 1.2 MiB/s, 19 objects/s recovering 2024-02-22T19:33:34.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:34.123994+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? 172.21.15.53:0/2152630116' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:34.503 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:34.503 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:33:34.531 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:34.537 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:33:34.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:34.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.550 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:33:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:33:34.635 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:33:34.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:34.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:33:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:33:34.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:33:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:33:34.931 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:34.934 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:34.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:34.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:33:35.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:35.105 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:35.189 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:35.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:35.210 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:35.304 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:35.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:35.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:34.454473+0000 mgr.y (mgr.14535) 179 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:35.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:34.557357+0000 mgr.y (mgr.14535) 180 : audit [DBG] from='client.24787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:35.406 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:34.454473+0000 mgr.y (mgr.14535) 179 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:35.407 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:34.557357+0000 mgr.y (mgr.14535) 180 : audit [DBG] from='client.24787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:34.454473+0000 mgr.y (mgr.14535) 179 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:34.557357+0000 mgr.y (mgr.14535) 180 : audit [DBG] from='client.24787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:35.978 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-02-22T19:33:33.296044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4916516540,"num_objects":75213,"num_object_clones":0,"num_object_copies":225639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":240,"num_objects_unfound":0,"num_objects_dirty":75213,"num_whiteouts":0,"num_read":976,"num_read_kb":728,"num_write":103817,"num_write_kb":5709098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4937,"num_bytes_recovered":319424731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77690,"ondisk_log_size":77690,"up":444,"acting":442,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":16594408,"kb_used_data":14448048,"kb_used_omap":0,"kb_used_meta":2146048,"kb_avail":639621656,"statfs":{"total":671965249536,"available":654972575744,"internally_reserved":0,"allocated":14794801152,"data_stored":14791994765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2197553152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-241565696,"num_objects":-3686,"num_object_clones":0,"num_object_copies":-11058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-116,"num_objects_unfound":0,"num_objects_dirty":-3686,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3686,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7602176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002777"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534433+0000","last_change":"2024-02-22T19:31:50.004483+0000","last_active":"2024-02-22T19:33:22.534433+0000","last_peered":"2024-02-22T19:33:22.534433+0000","last_clean":"2024-02-22T19:33:22.534433+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:33:22.534433+0000","last_undegraded":"2024-02-22T19:33:22.534433+0000","last_fullsized":"2024-02-22T19:33:22.534433+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":90,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.522207+0000","last_change":"2024-02-22T19:31:50.003253+0000","last_active":"2024-02-22T19:33:22.522207+0000","last_peered":"2024-02-22T19:33:22.522207+0000","last_clean":"2024-02-22T19:33:22.522207+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:33:22.522207+0000","last_undegraded":"2024-02-22T19:33:22.522207+0000","last_fullsized":"2024-02-22T19:33:22.522207+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":97,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527019+0000","last_change":"2024-02-22T19:31:50.013054+0000","last_active":"2024-02-22T19:33:22.527019+0000","last_peered":"2024-02-22T19:33:22.527019+0000","last_clean":"2024-02-22T19:33:22.527019+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:33:22.527019+0000","last_undegraded":"2024-02-22T19:33:22.527019+0000","last_fullsized":"2024-02-22T19:33:22.527019+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:12.445206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:12.445206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536202+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:33:22.536202+0000","last_peered":"2024-02-22T19:33:22.536202+0000","last_clean":"2024-02-22T19:33:22.536202+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:33:22.536202+0000","last_undegraded":"2024-02-22T19:33:22.536202+0000","last_fullsized":"2024-02-22T19:33:22.536202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":76,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528706+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:33:22.528706+0000","last_peered":"2024-02-22T19:33:22.528706+0000","last_clean":"2024-02-22T19:33:22.528706+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:33:22.528706+0000","last_undegraded":"2024-02-22T19:33:22.528706+0000","last_fullsized":"2024-02-22T19:33:22.528706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536353+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:33:22.536353+0000","last_peered":"2024-02-22T19:33:22.536353+0000","last_clean":"2024-02-22T19:33:22.536353+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:33:22.536353+0000","last_undegraded":"2024-02-22T19:33:22.536353+0000","last_fullsized":"2024-02-22T19:33:22.536353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":82,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534431+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:33:22.534431+0000","last_peered":"2024-02-22T19:33:22.534431+0000","last_clean":"2024-02-22T19:33:22.534431+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:33:22.534431+0000","last_undegraded":"2024-02-22T19:33:22.534431+0000","last_fullsized":"2024-02-22T19:33:22.534431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":105,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528076+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:33:22.528076+0000","last_peered":"2024-02-22T19:33:22.528076+0000","last_clean":"2024-02-22T19:33:22.528076+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:33:22.528076+0000","last_undegraded":"2024-02-22T19:33:22.528076+0000","last_fullsized":"2024-02-22T19:33:22.528076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524108+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:33:22.524108+0000","last_peered":"2024-02-22T19:33:22.524108+0000","last_clean":"2024-02-22T19:33:22.524108+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:33:22.524108+0000","last_undegraded":"2024-02-22T19:33:22.524108+0000","last_fullsized":"2024-02-22T19:33:22.524108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524706+0000","last_change":"2024-02-22T19:31:49.999104+0000","last_active":"2024-02-22T19:33:22.524706+0000","last_peered":"2024-02-22T19:33:22.524706+0000","last_clean":"2024-02-22T19:33:22.524706+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:33:22.524706+0000","last_undegraded":"2024-02-22T19:33:22.524706+0000","last_fullsized":"2024-02-22T19:33:22.524706+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.527965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.527965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.539481+0000","last_change":"2024-02-22T19:32:25.385315+0000","last_active":"2024-02-22T19:33:22.539481+0000","last_peered":"2024-02-22T19:33:22.539481+0000","last_clean":"2024-02-22T19:33:22.539481+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:33:22.539481+0000","last_undegraded":"2024-02-22T19:33:22.539481+0000","last_fullsized":"2024-02-22T19:33:22.539481+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:10.440058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:10.440058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534956+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:33:22.534956+0000","last_peered":"2024-02-22T19:33:22.534956+0000","last_clean":"2024-02-22T19:33:22.534956+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:33:22.534956+0000","last_undegraded":"2024-02-22T19:33:22.534956+0000","last_fullsized":"2024-02-22T19:33:22.534956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.520933+0000","last_change":"2024-02-22T19:31:50.005065+0000","last_active":"2024-02-22T19:33:22.520933+0000","last_peered":"2024-02-22T19:33:22.520933+0000","last_clean":"2024-02-22T19:33:22.520933+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:33:22.520933+0000","last_undegraded":"2024-02-22T19:33:22.520933+0000","last_fullsized":"2024-02-22T19:33:22.520933+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525183+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:33:22.525183+0000","last_peered":"2024-02-22T19:33:22.525183+0000","last_clean":"2024-02-22T19:33:22.525183+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:33:22.525183+0000","last_undegraded":"2024-02-22T19:33:22.525183+0000","last_fullsized":"2024-02-22T19:33:22.525183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":52,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534750+0000","last_change":"2024-02-22T19:31:50.439339+0000","last_active":"2024-02-22T19:33:22.534750+0000","last_peered":"2024-02-22T19:33:22.534750+0000","last_clean":"2024-02-22T19:33:22.534750+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:33:22.534750+0000","last_undegraded":"2024-02-22T19:33:22.534750+0000","last_fullsized":"2024-02-22T19:33:22.534750+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:31:23.683223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.683223+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":93,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536042+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:33:22.536042+0000","last_peered":"2024-02-22T19:33:22.536042+0000","last_clean":"2024-02-22T19:33:22.536042+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:33:22.536042+0000","last_undegraded":"2024-02-22T19:33:22.536042+0000","last_fullsized":"2024-02-22T19:33:22.536042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536404+0000","last_change":"2024-02-22T19:31:50.009328+0000","last_active":"2024-02-22T19:33:22.536404+0000","last_peered":"2024-02-22T19:33:22.536404+0000","last_clean":"2024-02-22T19:33:22.536404+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:33:22.536404+0000","last_undegraded":"2024-02-22T19:33:22.536404+0000","last_fullsized":"2024-02-22T19:33:22.536404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527969+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:33:22.527969+0000","last_peered":"2024-02-22T19:33:22.527969+0000","last_clean":"2024-02-22T19:33:22.527969+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:33:22.527969+0000","last_undegraded":"2024-02-22T19:33:22.527969+0000","last_fullsized":"2024-02-22T19:33:22.527969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524976+0000","last_change":"2024-02-22T19:31:26.454221+0000","last_active":"2024-02-22T19:33:22.524976+0000","last_peered":"2024-02-22T19:33:22.524976+0000","last_clean":"2024-02-22T19:33:22.524976+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:33:22.524976+0000","last_undegraded":"2024-02-22T19:33:22.524976+0000","last_fullsized":"2024-02-22T19:33:22.524976+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.454155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.454155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":108,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.539198+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:33:22.539198+0000","last_peered":"2024-02-22T19:33:22.539198+0000","last_clean":"2024-02-22T19:33:22.539198+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:33:22.539198+0000","last_undegraded":"2024-02-22T19:33:22.539198+0000","last_fullsized":"2024-02-22T19:33:22.539198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534898+0000","last_change":"2024-02-22T19:31:50.014548+0000","last_active":"2024-02-22T19:33:22.534898+0000","last_peered":"2024-02-22T19:33:22.534898+0000","last_clean":"2024-02-22T19:33:22.534898+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:33:22.534898+0000","last_undegraded":"2024-02-22T19:33:22.534898+0000","last_fullsized":"2024-02-22T19:33:22.534898+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:26.823363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:26.823363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536031+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:33:22.536031+0000","last_peered":"2024-02-22T19:33:22.536031+0000","last_clean":"2024-02-22T19:33:22.536031+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:33:22.536031+0000","last_undegraded":"2024-02-22T19:33:22.536031+0000","last_fullsized":"2024-02-22T19:33:22.536031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.535064+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:33:22.535064+0000","last_peered":"2024-02-22T19:33:22.535064+0000","last_clean":"2024-02-22T19:33:22.535064+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:33:22.535064+0000","last_undegraded":"2024-02-22T19:33:22.535064+0000","last_fullsized":"2024-02-22T19:33:22.535064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":108,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.533424+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:33:22.533424+0000","last_peered":"2024-02-22T19:33:22.533424+0000","last_clean":"2024-02-22T19:33:22.533424+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:33:22.533424+0000","last_undegraded":"2024-02-22T19:33:22.533424+0000","last_fullsized":"2024-02-22T19:33:22.533424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":91,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525083+0000","last_change":"2024-02-22T19:31:50.095022+0000","last_active":"2024-02-22T19:33:22.525083+0000","last_peered":"2024-02-22T19:33:22.525083+0000","last_clean":"2024-02-22T19:33:22.525083+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:33:22.525083+0000","last_undegraded":"2024-02-22T19:33:22.525083+0000","last_fullsized":"2024-02-22T19:33:22.525083+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":146,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.086325+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:33:29.086325+0000","last_peered":"2024-02-22T19:33:29.086325+0000","last_clean":"2024-02-22T19:33:29.086325+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:33:29.086325+0000","last_undegraded":"2024-02-22T19:33:29.086325+0000","last_fullsized":"2024-02-22T19:33:29.086325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.522276+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:33:22.522276+0000","last_peered":"2024-02-22T19:33:22.522276+0000","last_clean":"2024-02-22T19:33:22.522276+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:33:22.522276+0000","last_undegraded":"2024-02-22T19:33:22.522276+0000","last_fullsized":"2024-02-22T19:33:22.522276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":96,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525847+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:33:22.525847+0000","last_peered":"2024-02-22T19:33:22.525847+0000","last_clean":"2024-02-22T19:33:22.525847+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:33:22.525847+0000","last_undegraded":"2024-02-22T19:33:22.525847+0000","last_fullsized":"2024-02-22T19:33:22.525847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":90,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.530933+0000","last_change":"2024-02-22T19:31:50.009891+0000","last_active":"2024-02-22T19:33:22.530933+0000","last_peered":"2024-02-22T19:33:22.530933+0000","last_clean":"2024-02-22T19:33:22.530933+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:33:22.530933+0000","last_undegraded":"2024-02-22T19:33:22.530933+0000","last_fullsized":"2024-02-22T19:33:22.530933+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526141+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:33:22.526141+0000","last_peered":"2024-02-22T19:33:22.526141+0000","last_clean":"2024-02-22T19:33:22.526141+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:33:22.526141+0000","last_undegraded":"2024-02-22T19:33:22.526141+0000","last_fullsized":"2024-02-22T19:33:22.526141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":82,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527731+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:33:22.527731+0000","last_peered":"2024-02-22T19:33:22.527731+0000","last_clean":"2024-02-22T19:33:22.527731+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:33:22.527731+0000","last_undegraded":"2024-02-22T19:33:22.527731+0000","last_fullsized":"2024-02-22T19:33:22.527731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":152,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.519081+0000","last_change":"2024-02-22T19:33:20.849568+0000","last_active":"2024-02-22T19:33:22.519081+0000","last_peered":"2024-02-22T19:33:22.519081+0000","last_clean":"2024-02-22T19:33:22.519081+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:33:22.519081+0000","last_undegraded":"2024-02-22T19:33:22.519081+0000","last_fullsized":"2024-02-22T19:33:22.519081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.607011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.607011+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525107+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:33:22.525107+0000","last_peered":"2024-02-22T19:33:22.525107+0000","last_clean":"2024-02-22T19:33:22.525107+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:33:22.525107+0000","last_undegraded":"2024-02-22T19:33:22.525107+0000","last_fullsized":"2024-02-22T19:33:22.525107+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":78,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.518257+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:33:22.518257+0000","last_peered":"2024-02-22T19:33:22.518257+0000","last_clean":"2024-02-22T19:33:22.518257+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:33:22.518257+0000","last_undegraded":"2024-02-22T19:33:22.518257+0000","last_fullsized":"2024-02-22T19:33:22.518257+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":92,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527141+0000","last_change":"2024-02-22T19:31:50.010848+0000","last_active":"2024-02-22T19:33:22.527141+0000","last_peered":"2024-02-22T19:33:22.527141+0000","last_clean":"2024-02-22T19:33:22.527141+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:33:22.527141+0000","last_undegraded":"2024-02-22T19:33:22.527141+0000","last_fullsized":"2024-02-22T19:33:22.527141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:08.671238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:08.671238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":86,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527823+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:33:22.527823+0000","last_peered":"2024-02-22T19:33:22.527823+0000","last_clean":"2024-02-22T19:33:22.527823+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:33:22.527823+0000","last_undegraded":"2024-02-22T19:33:22.527823+0000","last_fullsized":"2024-02-22T19:33:22.527823+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":76,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528934+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:33:22.528934+0000","last_peered":"2024-02-22T19:33:22.528934+0000","last_clean":"2024-02-22T19:33:22.528934+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:33:22.528934+0000","last_undegraded":"2024-02-22T19:33:22.528934+0000","last_fullsized":"2024-02-22T19:33:22.528934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524669+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:33:22.524669+0000","last_peered":"2024-02-22T19:33:22.524669+0000","last_clean":"2024-02-22T19:33:22.524669+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:33:22.524669+0000","last_undegraded":"2024-02-22T19:33:22.524669+0000","last_fullsized":"2024-02-22T19:33:22.524669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":84,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525137+0000","last_change":"2024-02-22T19:33:19.711116+0000","last_active":"2024-02-22T19:33:22.525137+0000","last_peered":"2024-02-22T19:33:22.525137+0000","last_clean":"2024-02-22T19:33:22.525137+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:33:22.525137+0000","last_undegraded":"2024-02-22T19:33:22.525137+0000","last_fullsized":"2024-02-22T19:33:22.525137+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:07.463122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.463122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527993+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:33:22.527993+0000","last_peered":"2024-02-22T19:33:22.527993+0000","last_clean":"2024-02-22T19:33:22.527993+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:33:22.527993+0000","last_undegraded":"2024-02-22T19:33:22.527993+0000","last_fullsized":"2024-02-22T19:33:22.527993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":86,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.523730+0000","last_change":"2024-02-22T19:31:50.014237+0000","last_active":"2024-02-22T19:33:22.523730+0000","last_peered":"2024-02-22T19:33:22.523730+0000","last_clean":"2024-02-22T19:33:22.523730+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:33:22.523730+0000","last_undegraded":"2024-02-22T19:33:22.523730+0000","last_fullsized":"2024-02-22T19:33:22.523730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":78,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528618+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:33:22.528618+0000","last_peered":"2024-02-22T19:33:22.528618+0000","last_clean":"2024-02-22T19:33:22.528618+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:33:22.528618+0000","last_undegraded":"2024-02-22T19:33:22.528618+0000","last_fullsized":"2024-02-22T19:33:22.528618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524776+0000","last_change":"2024-02-22T19:31:23.009097+0000","last_active":"2024-02-22T19:33:22.524776+0000","last_peered":"2024-02-22T19:33:22.524776+0000","last_clean":"2024-02-22T19:33:22.524776+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:33:22.524776+0000","last_undegraded":"2024-02-22T19:33:22.524776+0000","last_fullsized":"2024-02-22T19:33:22.524776+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.009021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.521853+0000","last_change":"2024-02-22T19:30:14.042137+0000","last_active":"2024-02-22T19:33:22.521853+0000","last_peered":"2024-02-22T19:33:22.521853+0000","last_clean":"2024-02-22T19:33:22.521853+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:33:22.521853+0000","last_undegraded":"2024-02-22T19:33:22.521853+0000","last_fullsized":"2024-02-22T19:33:22.521853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.042059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.042059+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527707+0000","last_change":"2024-02-22T19:31:50.007895+0000","last_active":"2024-02-22T19:33:22.527707+0000","last_peered":"2024-02-22T19:33:22.527707+0000","last_clean":"2024-02-22T19:33:22.527707+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:33:22.527707+0000","last_undegraded":"2024-02-22T19:33:22.527707+0000","last_fullsized":"2024-02-22T19:33:22.527707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.709691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.709691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":90,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.521640+0000","last_change":"2024-02-22T19:31:50.012381+0000","last_active":"2024-02-22T19:33:22.521640+0000","last_peered":"2024-02-22T19:33:22.521640+0000","last_clean":"2024-02-22T19:33:22.521640+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:33:22.521640+0000","last_undegraded":"2024-02-22T19:33:22.521640+0000","last_fullsized":"2024-02-22T19:33:22.521640+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.535049+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:33:22.535049+0000","last_peered":"2024-02-22T19:33:22.535049+0000","last_clean":"2024-02-22T19:33:22.535049+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:33:22.535049+0000","last_undegraded":"2024-02-22T19:33:22.535049+0000","last_fullsized":"2024-02-22T19:33:22.535049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.535204+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:33:22.535204+0000","last_peered":"2024-02-22T19:33:22.535204+0000","last_clean":"2024-02-22T19:33:22.535204+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:33:22.535204+0000","last_undegraded":"2024-02-22T19:33:22.535204+0000","last_fullsized":"2024-02-22T19:33:22.535204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525550+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:33:22.525550+0000","last_peered":"2024-02-22T19:33:22.525550+0000","last_clean":"2024-02-22T19:33:22.525550+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:33:22.525550+0000","last_undegraded":"2024-02-22T19:33:22.525550+0000","last_fullsized":"2024-02-22T19:33:22.525550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":84,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.088723+0000","last_change":"2024-02-22T19:31:50.105130+0000","last_active":"2024-02-22T19:33:29.088723+0000","last_peered":"2024-02-22T19:33:29.088723+0000","last_clean":"2024-02-22T19:33:29.088723+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:33:29.088723+0000","last_undegraded":"2024-02-22T19:33:29.088723+0000","last_fullsized":"2024-02-22T19:33:29.088723+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:29.679183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.679183+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":83,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524845+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:33:22.524845+0000","last_peered":"2024-02-22T19:33:22.524845+0000","last_clean":"2024-02-22T19:33:22.524845+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:33:22.524845+0000","last_undegraded":"2024-02-22T19:33:22.524845+0000","last_fullsized":"2024-02-22T19:33:22.524845+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":100,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.533835+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:33:22.533835+0000","last_peered":"2024-02-22T19:33:22.533835+0000","last_clean":"2024-02-22T19:33:22.533835+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:33:22.533835+0000","last_undegraded":"2024-02-22T19:33:22.533835+0000","last_fullsized":"2024-02-22T19:33:22.533835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536589+0000","last_change":"2024-02-22T19:31:50.016234+0000","last_active":"2024-02-22T19:33:22.536589+0000","last_peered":"2024-02-22T19:33:22.536589+0000","last_clean":"2024-02-22T19:33:22.536589+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:33:22.536589+0000","last_undegraded":"2024-02-22T19:33:22.536589+0000","last_fullsized":"2024-02-22T19:33:22.536589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":140,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.098540+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:33:29.098540+0000","last_peered":"2024-02-22T19:33:29.098540+0000","last_clean":"2024-02-22T19:33:29.098540+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:33:29.098540+0000","last_undegraded":"2024-02-22T19:33:29.098540+0000","last_fullsized":"2024-02-22T19:33:29.098540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524520+0000","last_change":"2024-02-22T19:31:11.415809+0000","last_active":"2024-02-22T19:33:22.524520+0000","last_peered":"2024-02-22T19:33:22.524520+0000","last_clean":"2024-02-22T19:33:22.524520+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:33:22.524520+0000","last_undegraded":"2024-02-22T19:33:22.524520+0000","last_fullsized":"2024-02-22T19:33:22.524520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.415718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":108,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524973+0000","last_change":"2024-02-22T19:31:50.352892+0000","last_active":"2024-02-22T19:33:22.524973+0000","last_peered":"2024-02-22T19:33:22.524973+0000","last_clean":"2024-02-22T19:33:22.524973+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:33:22.524973+0000","last_undegraded":"2024-02-22T19:33:22.524973+0000","last_fullsized":"2024-02-22T19:33:22.524973+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.685295+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.539480+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:33:22.539480+0000","last_peered":"2024-02-22T19:33:22.539480+0000","last_clean":"2024-02-22T19:33:22.539480+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:33:22.539480+0000","last_undegraded":"2024-02-22T19:33:22.539480+0000","last_fullsized":"2024-02-22T19:33:22.539480+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":78,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534704+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:33:22.534704+0000","last_peered":"2024-02-22T19:33:22.534704+0000","last_clean":"2024-02-22T19:33:22.534704+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:33:22.534704+0000","last_undegraded":"2024-02-22T19:33:22.534704+0000","last_fullsized":"2024-02-22T19:33:22.534704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":83,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524465+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:33:22.524465+0000","last_peered":"2024-02-22T19:33:22.524465+0000","last_clean":"2024-02-22T19:33:22.524465+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:33:22.524465+0000","last_undegraded":"2024-02-22T19:33:22.524465+0000","last_fullsized":"2024-02-22T19:33:22.524465+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":97,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526008+0000","last_change":"2024-02-22T19:30:13.766624+0000","last_active":"2024-02-22T19:33:22.526008+0000","last_peered":"2024-02-22T19:33:22.526008+0000","last_clean":"2024-02-22T19:33:22.526008+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:33:22.526008+0000","last_undegraded":"2024-02-22T19:33:22.526008+0000","last_fullsized":"2024-02-22T19:33:22.526008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:13.766546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.766546+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524696+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:33:22.524696+0000","last_peered":"2024-02-22T19:33:22.524696+0000","last_clean":"2024-02-22T19:33:22.524696+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:33:22.524696+0000","last_undegraded":"2024-02-22T19:33:22.524696+0000","last_fullsized":"2024-02-22T19:33:22.524696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534550+0000","last_change":"2024-02-22T19:31:50.013893+0000","last_active":"2024-02-22T19:33:22.534550+0000","last_peered":"2024-02-22T19:33:22.534550+0000","last_clean":"2024-02-22T19:33:22.534550+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:33:22.534550+0000","last_undegraded":"2024-02-22T19:33:22.534550+0000","last_fullsized":"2024-02-22T19:33:22.534550+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":93,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.518046+0000","last_change":"2024-02-22T19:31:50.006289+0000","last_active":"2024-02-22T19:33:22.518046+0000","last_peered":"2024-02-22T19:33:22.518046+0000","last_clean":"2024-02-22T19:33:22.518046+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:33:22.518046+0000","last_undegraded":"2024-02-22T19:33:22.518046+0000","last_fullsized":"2024-02-22T19:33:22.518046+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":89,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534977+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:33:22.534977+0000","last_peered":"2024-02-22T19:33:22.534977+0000","last_clean":"2024-02-22T19:33:22.534977+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:33:22.534977+0000","last_undegraded":"2024-02-22T19:33:22.534977+0000","last_fullsized":"2024-02-22T19:33:22.534977+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"128'6459","reported_seq":9070,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.226540+0000","last_change":"2024-02-22T19:32:06.969062+0000","last_active":"2024-02-22T19:33:29.226540+0000","last_peered":"2024-02-22T19:33:29.226540+0000","last_clean":"2024-02-22T19:33:29.226540+0000","last_became_active":"2024-02-22T19:32:06.966453+0000","last_became_peered":"2024-02-22T19:32:06.966453+0000","last_unstale":"2024-02-22T19:33:29.226540+0000","last_undegraded":"2024-02-22T19:33:29.226540+0000","last_fullsized":"2024-02-22T19:33:29.226540+0000","mapping_epoch":104,"log_start":"102'1900","ondisk_log_start":"102'1900","created":93,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4559,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":308346880,"num_objects":4705,"num_object_clones":0,"num_object_copies":14115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6459,"num_write_kb":357248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.518175+0000","last_change":"2024-02-22T19:31:50.006727+0000","last_active":"2024-02-22T19:33:22.518175+0000","last_peered":"2024-02-22T19:33:22.518175+0000","last_clean":"2024-02-22T19:33:22.518175+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:33:22.518175+0000","last_undegraded":"2024-02-22T19:33:22.518175+0000","last_fullsized":"2024-02-22T19:33:22.518175+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":98,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525422+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:33:22.525422+0000","last_peered":"2024-02-22T19:33:22.525422+0000","last_clean":"2024-02-22T19:33:22.525422+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:33:22.525422+0000","last_undegraded":"2024-02-22T19:33:22.525422+0000","last_fullsized":"2024-02-22T19:33:22.525422+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":93,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524577+0000","last_change":"2024-02-22T19:31:50.611447+0000","last_active":"2024-02-22T19:33:22.524577+0000","last_peered":"2024-02-22T19:33:22.524577+0000","last_clean":"2024-02-22T19:33:22.524577+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:33:22.524577+0000","last_undegraded":"2024-02-22T19:33:22.524577+0000","last_fullsized":"2024-02-22T19:33:22.524577+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":141,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.097460+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:33:29.097460+0000","last_peered":"2024-02-22T19:33:29.097460+0000","last_clean":"2024-02-22T19:33:29.097460+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:33:29.097460+0000","last_undegraded":"2024-02-22T19:33:29.097460+0000","last_fullsized":"2024-02-22T19:33:29.097460+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"128'6417","reported_seq":8755,"reported_epoch":128,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:33:29.214591+0000","last_change":"2024-02-22T19:31:51.068957+0000","last_active":"2024-02-22T19:33:29.214591+0000","last_peered":"2024-02-22T19:33:29.214591+0000","last_clean":"2024-02-22T19:31:48.970281+0000","last_became_active":"2024-02-22T19:31:51.043104+0000","last_became_peered":"2024-02-22T19:31:51.043104+0000","last_unstale":"2024-02-22T19:33:29.214591+0000","last_undegraded":"2024-02-22T19:33:29.214591+0000","last_fullsized":"2024-02-22T19:31:50.992947+0000","mapping_epoch":100,"log_start":"105'2300","ondisk_log_start":"105'2300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4117,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":307298304,"num_objects":4689,"num_object_clones":0,"num_object_copies":14067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":129,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6417,"num_write_kb":355392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1143,"num_bytes_recovered":74907648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2","5"],"object_location_counts":[{"shards":"2,3","objects":4560},{"shards":"2,3,5","objects":129}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":76,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528780+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:33:22.528780+0000","last_peered":"2024-02-22T19:33:22.528780+0000","last_clean":"2024-02-22T19:33:22.528780+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:33:22.528780+0000","last_undegraded":"2024-02-22T19:33:22.528780+0000","last_fullsized":"2024-02-22T19:33:22.528780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":83,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525241+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:33:22.525241+0000","last_peered":"2024-02-22T19:33:22.525241+0000","last_clean":"2024-02-22T19:33:22.525241+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:33:22.525241+0000","last_undegraded":"2024-02-22T19:33:22.525241+0000","last_fullsized":"2024-02-22T19:33:22.525241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":106,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534676+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:33:22.534676+0000","last_peered":"2024-02-22T19:33:22.534676+0000","last_clean":"2024-02-22T19:33:22.534676+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:33:22.534676+0000","last_undegraded":"2024-02-22T19:33:22.534676+0000","last_fullsized":"2024-02-22T19:33:22.534676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.533054+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:33:22.533054+0000","last_peered":"2024-02-22T19:33:22.533054+0000","last_clean":"2024-02-22T19:33:22.533054+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:33:22.533054+0000","last_undegraded":"2024-02-22T19:33:22.533054+0000","last_fullsized":"2024-02-22T19:33:22.533054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"128'6550","reported_seq":6593,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.255928+0000","last_change":"2024-02-22T19:31:33.014127+0000","last_active":"2024-02-22T19:33:29.255928+0000","last_peered":"2024-02-22T19:33:29.255928+0000","last_clean":"2024-02-22T19:33:29.255928+0000","last_became_active":"2024-02-22T19:31:33.013760+0000","last_became_peered":"2024-02-22T19:31:33.013760+0000","last_unstale":"2024-02-22T19:33:29.255928+0000","last_undegraded":"2024-02-22T19:33:29.255928+0000","last_fullsized":"2024-02-22T19:33:29.255928+0000","mapping_epoch":93,"log_start":"102'1700","ondisk_log_start":"102'1700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4850,"ondisk_log_size":4850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":308150272,"num_objects":4702,"num_object_clones":0,"num_object_copies":14106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":360064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524352+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:33:22.524352+0000","last_peered":"2024-02-22T19:33:22.524352+0000","last_clean":"2024-02-22T19:33:22.524352+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:33:22.524352+0000","last_undegraded":"2024-02-22T19:33:22.524352+0000","last_fullsized":"2024-02-22T19:33:22.524352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":90,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.540126+0000","last_change":"2024-02-22T19:31:50.008803+0000","last_active":"2024-02-22T19:33:22.540126+0000","last_peered":"2024-02-22T19:33:22.540126+0000","last_clean":"2024-02-22T19:33:22.540126+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:33:22.540126+0000","last_undegraded":"2024-02-22T19:33:22.540126+0000","last_fullsized":"2024-02-22T19:33:22.540126+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.110513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.110513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":97,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524886+0000","last_change":"2024-02-22T19:31:49.999189+0000","last_active":"2024-02-22T19:33:22.524886+0000","last_peered":"2024-02-22T19:33:22.524886+0000","last_clean":"2024-02-22T19:33:22.524886+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:33:22.524886+0000","last_undegraded":"2024-02-22T19:33:22.524886+0000","last_fullsized":"2024-02-22T19:33:22.524886+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:15.882992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.882992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":111,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.518314+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:33:22.518314+0000","last_peered":"2024-02-22T19:33:22.518314+0000","last_clean":"2024-02-22T19:33:22.518314+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:33:22.518314+0000","last_undegraded":"2024-02-22T19:33:22.518314+0000","last_fullsized":"2024-02-22T19:33:22.518314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"128'6390","reported_seq":6433,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.191247+0000","last_change":"2024-02-22T19:31:33.015434+0000","last_active":"2024-02-22T19:33:29.191247+0000","last_peered":"2024-02-22T19:33:29.191247+0000","last_clean":"2024-02-22T19:33:29.191247+0000","last_became_active":"2024-02-22T19:31:33.015132+0000","last_became_peered":"2024-02-22T19:31:33.015132+0000","last_unstale":"2024-02-22T19:33:29.191247+0000","last_undegraded":"2024-02-22T19:33:29.191247+0000","last_fullsized":"2024-02-22T19:33:29.191247+0000","mapping_epoch":93,"log_start":"98'1100","ondisk_log_start":"98'1100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5290,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":304742400,"num_objects":4650,"num_object_clones":0,"num_object_copies":13950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6390,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524214+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:33:22.524214+0000","last_peered":"2024-02-22T19:33:22.524214+0000","last_clean":"2024-02-22T19:33:22.524214+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:33:22.524214+0000","last_undegraded":"2024-02-22T19:33:22.524214+0000","last_fullsized":"2024-02-22T19:33:22.524214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.531623+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:33:22.531623+0000","last_peered":"2024-02-22T19:33:22.531623+0000","last_clean":"2024-02-22T19:33:22.531623+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:33:22.531623+0000","last_undegraded":"2024-02-22T19:33:22.531623+0000","last_fullsized":"2024-02-22T19:33:22.531623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.533878+0000","last_change":"2024-02-22T19:31:50.014265+0000","last_active":"2024-02-22T19:33:22.533878+0000","last_peered":"2024-02-22T19:33:22.533878+0000","last_clean":"2024-02-22T19:33:22.533878+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:33:22.533878+0000","last_undegraded":"2024-02-22T19:33:22.533878+0000","last_fullsized":"2024-02-22T19:33:22.533878+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":116,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525107+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:33:22.525107+0000","last_peered":"2024-02-22T19:33:22.525107+0000","last_clean":"2024-02-22T19:33:22.525107+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:33:22.525107+0000","last_undegraded":"2024-02-22T19:33:22.525107+0000","last_fullsized":"2024-02-22T19:33:22.525107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"128'6400","reported_seq":6443,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.129834+0000","last_change":"2024-02-22T19:31:33.014009+0000","last_active":"2024-02-22T19:33:29.129834+0000","last_peered":"2024-02-22T19:33:29.129834+0000","last_clean":"2024-02-22T19:33:29.129834+0000","last_became_active":"2024-02-22T19:31:33.013715+0000","last_became_peered":"2024-02-22T19:31:33.013715+0000","last_unstale":"2024-02-22T19:33:29.129834+0000","last_undegraded":"2024-02-22T19:33:29.129834+0000","last_fullsized":"2024-02-22T19:33:29.129834+0000","mapping_epoch":93,"log_start":"98'1000","ondisk_log_start":"98'1000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5400,"ondisk_log_size":5400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":304087040,"num_objects":4640,"num_object_clones":0,"num_object_copies":13920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":76,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534663+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:33:22.534663+0000","last_peered":"2024-02-22T19:33:22.534663+0000","last_clean":"2024-02-22T19:33:22.534663+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:33:22.534663+0000","last_undegraded":"2024-02-22T19:33:22.534663+0000","last_fullsized":"2024-02-22T19:33:22.534663+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.521451+0000","last_change":"2024-02-22T19:31:50.010144+0000","last_active":"2024-02-22T19:33:22.521451+0000","last_peered":"2024-02-22T19:33:22.521451+0000","last_clean":"2024-02-22T19:33:22.521451+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:33:22.521451+0000","last_undegraded":"2024-02-22T19:33:22.521451+0000","last_fullsized":"2024-02-22T19:33:22.521451+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.692694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.535945+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:33:22.535945+0000","last_peered":"2024-02-22T19:33:22.535945+0000","last_clean":"2024-02-22T19:33:22.535945+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:33:22.535945+0000","last_undegraded":"2024-02-22T19:33:22.535945+0000","last_fullsized":"2024-02-22T19:33:22.535945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":185,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.083485+0000","last_change":"2024-02-22T19:30:07.992740+0000","last_active":"2024-02-22T19:33:29.083485+0000","last_peered":"2024-02-22T19:33:29.083485+0000","last_clean":"2024-02-22T19:33:29.083485+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:33:29.083485+0000","last_undegraded":"2024-02-22T19:33:29.083485+0000","last_fullsized":"2024-02-22T19:33:29.083485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:30:07.992671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:07.992671+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526572+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:33:22.526572+0000","last_peered":"2024-02-22T19:33:22.526572+0000","last_clean":"2024-02-22T19:33:22.526572+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:33:22.526572+0000","last_undegraded":"2024-02-22T19:33:22.526572+0000","last_fullsized":"2024-02-22T19:33:22.526572+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"128'6345","reported_seq":6388,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.256160+0000","last_change":"2024-02-22T19:31:33.008987+0000","last_active":"2024-02-22T19:33:29.256160+0000","last_peered":"2024-02-22T19:33:29.256160+0000","last_clean":"2024-02-22T19:33:29.256160+0000","last_became_active":"2024-02-22T19:31:33.008726+0000","last_became_peered":"2024-02-22T19:31:33.008726+0000","last_unstale":"2024-02-22T19:33:29.256160+0000","last_undegraded":"2024-02-22T19:33:29.256160+0000","last_fullsized":"2024-02-22T19:33:29.256160+0000","mapping_epoch":93,"log_start":"102'1500","ondisk_log_start":"102'1500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4845,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":307429376,"num_objects":4691,"num_object_clones":0,"num_object_copies":14073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6345,"num_write_kb":353152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524989+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:33:22.524989+0000","last_peered":"2024-02-22T19:33:22.524989+0000","last_clean":"2024-02-22T19:33:22.524989+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:33:22.524989+0000","last_undegraded":"2024-02-22T19:33:22.524989+0000","last_fullsized":"2024-02-22T19:33:22.524989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525005+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:33:22.525005+0000","last_peered":"2024-02-22T19:33:22.525005+0000","last_clean":"2024-02-22T19:33:22.525005+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:33:22.525005+0000","last_undegraded":"2024-02-22T19:33:22.525005+0000","last_fullsized":"2024-02-22T19:33:22.525005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.529950+0000","last_change":"2024-02-22T19:30:18.117213+0000","last_active":"2024-02-22T19:33:22.529950+0000","last_peered":"2024-02-22T19:33:22.529950+0000","last_clean":"2024-02-22T19:33:22.529950+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:33:22.529950+0000","last_undegraded":"2024-02-22T19:33:22.529950+0000","last_fullsized":"2024-02-22T19:33:22.529950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.117111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":92,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.539825+0000","last_change":"2024-02-22T19:31:50.458765+0000","last_active":"2024-02-22T19:33:22.539825+0000","last_peered":"2024-02-22T19:33:22.539825+0000","last_clean":"2024-02-22T19:33:22.539825+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:33:22.539825+0000","last_undegraded":"2024-02-22T19:33:22.539825+0000","last_fullsized":"2024-02-22T19:33:22.539825+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":90,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.540455+0000","last_change":"2024-02-22T19:31:50.006204+0000","last_active":"2024-02-22T19:33:22.540455+0000","last_peered":"2024-02-22T19:33:22.540455+0000","last_clean":"2024-02-22T19:33:22.540455+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:33:22.540455+0000","last_undegraded":"2024-02-22T19:33:22.540455+0000","last_fullsized":"2024-02-22T19:33:22.540455+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.102065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.102065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"128'6438","reported_seq":6481,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.145754+0000","last_change":"2024-02-22T19:31:33.014815+0000","last_active":"2024-02-22T19:33:29.145754+0000","last_peered":"2024-02-22T19:33:29.145754+0000","last_clean":"2024-02-22T19:33:29.145754+0000","last_became_active":"2024-02-22T19:31:33.014489+0000","last_became_peered":"2024-02-22T19:31:33.014489+0000","last_unstale":"2024-02-22T19:33:29.145754+0000","last_undegraded":"2024-02-22T19:33:29.145754+0000","last_fullsized":"2024-02-22T19:33:29.145754+0000","mapping_epoch":93,"log_start":"102'1500","ondisk_log_start":"102'1500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4938,"ondisk_log_size":4938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":309460992,"num_objects":4722,"num_object_clones":0,"num_object_copies":14166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6438,"num_write_kb":357120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534628+0000","last_change":"2024-02-22T19:31:50.014877+0000","last_active":"2024-02-22T19:33:22.534628+0000","last_peered":"2024-02-22T19:33:22.534628+0000","last_clean":"2024-02-22T19:33:22.534628+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:33:22.534628+0000","last_undegraded":"2024-02-22T19:33:22.534628+0000","last_fullsized":"2024-02-22T19:33:22.534628+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524478+0000","last_change":"2024-02-22T19:31:11.906662+0000","last_active":"2024-02-22T19:33:22.524478+0000","last_peered":"2024-02-22T19:33:22.524478+0000","last_clean":"2024-02-22T19:33:22.524478+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:33:22.524478+0000","last_undegraded":"2024-02-22T19:33:22.524478+0000","last_fullsized":"2024-02-22T19:33:22.524478+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.906561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.906561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":91,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534778+0000","last_change":"2024-02-22T19:31:50.009641+0000","last_active":"2024-02-22T19:33:22.534778+0000","last_peered":"2024-02-22T19:33:22.534778+0000","last_clean":"2024-02-22T19:33:22.534778+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:33:22.534778+0000","last_undegraded":"2024-02-22T19:33:22.534778+0000","last_fullsized":"2024-02-22T19:33:22.534778+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":109,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.519100+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:33:22.519100+0000","last_peered":"2024-02-22T19:33:22.519100+0000","last_clean":"2024-02-22T19:33:22.519100+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:33:22.519100+0000","last_undegraded":"2024-02-22T19:33:22.519100+0000","last_fullsized":"2024-02-22T19:33:22.519100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525169+0000","last_change":"2024-02-22T19:31:49.995726+0000","last_active":"2024-02-22T19:33:22.525169+0000","last_peered":"2024-02-22T19:33:22.525169+0000","last_clean":"2024-02-22T19:33:22.525169+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:33:22.525169+0000","last_undegraded":"2024-02-22T19:33:22.525169+0000","last_fullsized":"2024-02-22T19:33:22.525169+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"128'6508","reported_seq":6551,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.141792+0000","last_change":"2024-02-22T19:31:33.010431+0000","last_active":"2024-02-22T19:33:29.141792+0000","last_peered":"2024-02-22T19:33:29.141792+0000","last_clean":"2024-02-22T19:33:29.141792+0000","last_became_active":"2024-02-22T19:31:33.010187+0000","last_became_peered":"2024-02-22T19:31:33.010187+0000","last_unstale":"2024-02-22T19:33:29.141792+0000","last_undegraded":"2024-02-22T19:33:29.141792+0000","last_fullsized":"2024-02-22T19:33:29.141792+0000","mapping_epoch":93,"log_start":"102'1700","ondisk_log_start":"102'1700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4808,"ondisk_log_size":4808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":303693824,"num_objects":4634,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":88,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526668+0000","last_change":"2024-02-22T19:31:50.012369+0000","last_active":"2024-02-22T19:33:22.526668+0000","last_peered":"2024-02-22T19:33:22.526668+0000","last_clean":"2024-02-22T19:33:22.526668+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:33:22.526668+0000","last_undegraded":"2024-02-22T19:33:22.526668+0000","last_fullsized":"2024-02-22T19:33:22.526668+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":83,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524018+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:33:22.524018+0000","last_peered":"2024-02-22T19:33:22.524018+0000","last_clean":"2024-02-22T19:33:22.524018+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:33:22.524018+0000","last_undegraded":"2024-02-22T19:33:22.524018+0000","last_fullsized":"2024-02-22T19:33:22.524018+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":353,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:28.553153+0000","last_change":"2024-02-22T19:31:29.001184+0000","last_active":"2024-02-22T19:33:28.553153+0000","last_peered":"2024-02-22T19:33:28.553153+0000","last_clean":"2024-02-22T19:33:28.553153+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:33:28.553153+0000","last_undegraded":"2024-02-22T19:33:28.553153+0000","last_fullsized":"2024-02-22T19:33:28.553153+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:31:29.001098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.001098+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":262,"num_read_kb":257,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525630+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:33:22.525630+0000","last_peered":"2024-02-22T19:33:22.525630+0000","last_clean":"2024-02-22T19:33:22.525630+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:33:22.525630+0000","last_undegraded":"2024-02-22T19:33:22.525630+0000","last_fullsized":"2024-02-22T19:33:22.525630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526207+0000","last_change":"2024-02-22T19:31:50.010171+0000","last_active":"2024-02-22T19:33:22.526207+0000","last_peered":"2024-02-22T19:33:22.526207+0000","last_clean":"2024-02-22T19:33:22.526207+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:33:22.526207+0000","last_undegraded":"2024-02-22T19:33:22.526207+0000","last_fullsized":"2024-02-22T19:33:22.526207+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"128'6480","reported_seq":9167,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.224597+0000","last_change":"2024-02-22T19:32:07.977104+0000","last_active":"2024-02-22T19:33:29.224597+0000","last_peered":"2024-02-22T19:33:29.224597+0000","last_clean":"2024-02-22T19:33:29.224597+0000","last_became_active":"2024-02-22T19:32:07.973466+0000","last_became_peered":"2024-02-22T19:32:07.973466+0000","last_unstale":"2024-02-22T19:33:29.224597+0000","last_undegraded":"2024-02-22T19:33:29.224597+0000","last_fullsized":"2024-02-22T19:33:29.224597+0000","mapping_epoch":105,"log_start":"102'1900","ondisk_log_start":"102'1900","created":93,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4580,"ondisk_log_size":4580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":305528832,"num_objects":4662,"num_object_clones":0,"num_object_copies":13986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6480,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":86048768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.521079+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:33:22.521079+0000","last_peered":"2024-02-22T19:33:22.521079+0000","last_clean":"2024-02-22T19:33:22.521079+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:33:22.521079+0000","last_undegraded":"2024-02-22T19:33:22.521079+0000","last_fullsized":"2024-02-22T19:33:22.521079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":94,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.526840+0000","last_change":"2024-02-22T19:31:50.010143+0000","last_active":"2024-02-22T19:33:22.526840+0000","last_peered":"2024-02-22T19:33:22.526840+0000","last_clean":"2024-02-22T19:33:22.526840+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:33:22.526840+0000","last_undegraded":"2024-02-22T19:33:22.526840+0000","last_fullsized":"2024-02-22T19:33:22.526840+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":144,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.524908+0000","last_change":"2024-02-22T19:31:50.036817+0000","last_active":"2024-02-22T19:33:22.524908+0000","last_peered":"2024-02-22T19:33:22.524908+0000","last_clean":"2024-02-22T19:33:22.524908+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:33:22.524908+0000","last_undegraded":"2024-02-22T19:33:22.524908+0000","last_fullsized":"2024-02-22T19:33:22.524908+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":89,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.518353+0000","last_change":"2024-02-22T19:31:50.006906+0000","last_active":"2024-02-22T19:33:22.518353+0000","last_peered":"2024-02-22T19:33:22.518353+0000","last_clean":"2024-02-22T19:33:22.518353+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:33:22.518353+0000","last_undegraded":"2024-02-22T19:33:22.518353+0000","last_fullsized":"2024-02-22T19:33:22.518353+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"128'6406","reported_seq":6449,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.128837+0000","last_change":"2024-02-22T19:31:33.012064+0000","last_active":"2024-02-22T19:33:29.128837+0000","last_peered":"2024-02-22T19:33:29.128837+0000","last_clean":"2024-02-22T19:33:29.128837+0000","last_became_active":"2024-02-22T19:31:33.011750+0000","last_became_peered":"2024-02-22T19:31:33.011750+0000","last_unstale":"2024-02-22T19:33:29.128837+0000","last_undegraded":"2024-02-22T19:33:29.128837+0000","last_fullsized":"2024-02-22T19:33:29.128837+0000","mapping_epoch":93,"log_start":"102'1900","ondisk_log_start":"102'1900","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4506,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":304873472,"num_objects":4652,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":353856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.523811+0000","last_change":"2024-02-22T19:30:18.719372+0000","last_active":"2024-02-22T19:33:22.523811+0000","last_peered":"2024-02-22T19:33:22.523811+0000","last_clean":"2024-02-22T19:33:22.523811+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:33:22.523811+0000","last_undegraded":"2024-02-22T19:33:22.523811+0000","last_fullsized":"2024-02-22T19:33:22.523811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.719310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.529140+0000","last_change":"2024-02-22T19:31:50.006662+0000","last_active":"2024-02-22T19:33:22.529140+0000","last_peered":"2024-02-22T19:33:22.529140+0000","last_clean":"2024-02-22T19:33:22.529140+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:33:22.529140+0000","last_undegraded":"2024-02-22T19:33:22.529140+0000","last_fullsized":"2024-02-22T19:33:22.529140+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":119,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.521719+0000","last_change":"2024-02-22T19:31:50.027001+0000","last_active":"2024-02-22T19:33:22.521719+0000","last_peered":"2024-02-22T19:33:22.521719+0000","last_clean":"2024-02-22T19:33:22.521719+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:33:22.521719+0000","last_undegraded":"2024-02-22T19:33:22.521719+0000","last_fullsized":"2024-02-22T19:33:22.521719+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:12.059797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:12.059797+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":154,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.095195+0000","last_change":"2024-02-22T19:31:50.440080+0000","last_active":"2024-02-22T19:33:29.095195+0000","last_peered":"2024-02-22T19:33:29.095195+0000","last_clean":"2024-02-22T19:33:29.095195+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:33:29.095195+0000","last_undegraded":"2024-02-22T19:33:29.095195+0000","last_fullsized":"2024-02-22T19:33:29.095195+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"128'6456","reported_seq":6497,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.279791+0000","last_change":"2024-02-22T19:31:33.012817+0000","last_active":"2024-02-22T19:33:29.279791+0000","last_peered":"2024-02-22T19:33:29.279791+0000","last_clean":"2024-02-22T19:33:29.279791+0000","last_became_active":"2024-02-22T19:31:33.012470+0000","last_became_peered":"2024-02-22T19:31:33.012470+0000","last_unstale":"2024-02-22T19:33:29.279791+0000","last_undegraded":"2024-02-22T19:33:29.279791+0000","last_fullsized":"2024-02-22T19:33:29.279791+0000","mapping_epoch":93,"log_start":"106'2300","ondisk_log_start":"106'2300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4156,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":308412416,"num_objects":4706,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":357184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":77,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525089+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:33:22.525089+0000","last_peered":"2024-02-22T19:33:22.525089+0000","last_clean":"2024-02-22T19:33:22.525089+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:33:22.525089+0000","last_undegraded":"2024-02-22T19:33:22.525089+0000","last_fullsized":"2024-02-22T19:33:22.525089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":83,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.525702+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:33:22.525702+0000","last_peered":"2024-02-22T19:33:22.525702+0000","last_clean":"2024-02-22T19:33:22.525702+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:33:22.525702+0000","last_undegraded":"2024-02-22T19:33:22.525702+0000","last_fullsized":"2024-02-22T19:33:22.525702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534571+0000","last_change":"2024-02-22T19:31:50.644540+0000","last_active":"2024-02-22T19:33:22.534571+0000","last_peered":"2024-02-22T19:33:22.534571+0000","last_clean":"2024-02-22T19:33:22.534571+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:33:22.534571+0000","last_undegraded":"2024-02-22T19:33:22.534571+0000","last_fullsized":"2024-02-22T19:33:22.534571+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":140,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.098812+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:33:29.098812+0000","last_peered":"2024-02-22T19:33:29.098812+0000","last_clean":"2024-02-22T19:33:29.098812+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:33:29.098812+0000","last_undegraded":"2024-02-22T19:33:29.098812+0000","last_fullsized":"2024-02-22T19:33:29.098812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"128'6624","reported_seq":6666,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.224723+0000","last_change":"2024-02-22T19:31:33.013118+0000","last_active":"2024-02-22T19:33:29.224723+0000","last_peered":"2024-02-22T19:33:29.224723+0000","last_clean":"2024-02-22T19:33:29.224723+0000","last_became_active":"2024-02-22T19:31:33.012002+0000","last_became_peered":"2024-02-22T19:31:33.012002+0000","last_unstale":"2024-02-22T19:33:29.224723+0000","last_undegraded":"2024-02-22T19:33:29.224723+0000","last_fullsized":"2024-02-22T19:33:29.224723+0000","mapping_epoch":93,"log_start":"106'2500","ondisk_log_start":"106'2500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4124,"ondisk_log_size":4124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":311689216,"num_objects":4756,"num_object_clones":0,"num_object_copies":14268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6624,"num_write_kb":364160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534455+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:33:22.534455+0000","last_peered":"2024-02-22T19:33:22.534455+0000","last_clean":"2024-02-22T19:33:22.534455+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:33:22.534455+0000","last_undegraded":"2024-02-22T19:33:22.534455+0000","last_fullsized":"2024-02-22T19:33:22.534455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":82,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534485+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:33:22.534485+0000","last_peered":"2024-02-22T19:33:22.534485+0000","last_clean":"2024-02-22T19:33:22.534485+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:33:22.534485+0000","last_undegraded":"2024-02-22T19:33:22.534485+0000","last_fullsized":"2024-02-22T19:33:22.534485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":112,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527989+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:33:22.527989+0000","last_peered":"2024-02-22T19:33:22.527989+0000","last_clean":"2024-02-22T19:33:22.527989+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:33:22.527989+0000","last_undegraded":"2024-02-22T19:33:22.527989+0000","last_fullsized":"2024-02-22T19:33:22.527989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":86,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534449+0000","last_change":"2024-02-22T19:31:26.190739+0000","last_active":"2024-02-22T19:33:22.534449+0000","last_peered":"2024-02-22T19:33:22.534449+0000","last_clean":"2024-02-22T19:33:22.534449+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:33:22.534449+0000","last_undegraded":"2024-02-22T19:33:22.534449+0000","last_fullsized":"2024-02-22T19:33:22.534449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.190632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.190632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"128'6547","reported_seq":6590,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.168216+0000","last_change":"2024-02-22T19:31:33.012589+0000","last_active":"2024-02-22T19:33:29.168216+0000","last_peered":"2024-02-22T19:33:29.168216+0000","last_clean":"2024-02-22T19:33:29.168216+0000","last_became_active":"2024-02-22T19:31:33.012188+0000","last_became_peered":"2024-02-22T19:31:33.012188+0000","last_unstale":"2024-02-22T19:33:29.168216+0000","last_undegraded":"2024-02-22T19:33:29.168216+0000","last_fullsized":"2024-02-22T19:33:29.168216+0000","mapping_epoch":93,"log_start":"98'1100","ondisk_log_start":"98'1100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5447,"ondisk_log_size":5447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":311492608,"num_objects":4753,"num_object_clones":0,"num_object_copies":14259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6547,"num_write_kb":361600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527684+0000","last_change":"2024-02-22T19:31:50.009247+0000","last_active":"2024-02-22T19:33:22.527684+0000","last_peered":"2024-02-22T19:33:22.527684+0000","last_clean":"2024-02-22T19:33:22.527684+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:33:22.527684+0000","last_undegraded":"2024-02-22T19:33:22.527684+0000","last_fullsized":"2024-02-22T19:33:22.527684+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:25.682652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:25.682652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":156,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.079665+0000","last_change":"2024-02-22T19:31:50.016299+0000","last_active":"2024-02-22T19:33:29.079665+0000","last_peered":"2024-02-22T19:33:29.079665+0000","last_clean":"2024-02-22T19:33:29.079665+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:33:29.079665+0000","last_undegraded":"2024-02-22T19:33:29.079665+0000","last_fullsized":"2024-02-22T19:33:29.079665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":87,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527906+0000","last_change":"2024-02-22T19:31:14.073980+0000","last_active":"2024-02-22T19:33:22.527906+0000","last_peered":"2024-02-22T19:33:22.527906+0000","last_clean":"2024-02-22T19:33:22.527906+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:33:22.527906+0000","last_undegraded":"2024-02-22T19:33:22.527906+0000","last_fullsized":"2024-02-22T19:33:22.527906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.073908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.073908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":75,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.539700+0000","last_change":"2024-02-22T19:30:27.174847+0000","last_active":"2024-02-22T19:33:22.539700+0000","last_peered":"2024-02-22T19:33:22.539700+0000","last_clean":"2024-02-22T19:33:22.539700+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:33:22.539700+0000","last_undegraded":"2024-02-22T19:33:22.539700+0000","last_fullsized":"2024-02-22T19:33:22.539700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:27.174779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.174779+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"128'6628","reported_seq":6672,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.169055+0000","last_change":"2024-02-22T19:31:33.011737+0000","last_active":"2024-02-22T19:33:29.169055+0000","last_peered":"2024-02-22T19:33:29.169055+0000","last_clean":"2024-02-22T19:33:29.169055+0000","last_became_active":"2024-02-22T19:31:33.011438+0000","last_became_peered":"2024-02-22T19:31:33.011438+0000","last_unstale":"2024-02-22T19:33:29.169055+0000","last_undegraded":"2024-02-22T19:33:29.169055+0000","last_fullsized":"2024-02-22T19:33:29.169055+0000","mapping_epoch":93,"log_start":"98'1200","ondisk_log_start":"98'1200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5428,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":314900504,"num_objects":4806,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6628,"num_write_kb":365825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":85,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.533790+0000","last_change":"2024-02-22T19:31:50.006646+0000","last_active":"2024-02-22T19:33:22.533790+0000","last_peered":"2024-02-22T19:33:22.533790+0000","last_clean":"2024-02-22T19:33:22.533790+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:33:22.533790+0000","last_undegraded":"2024-02-22T19:33:22.533790+0000","last_fullsized":"2024-02-22T19:33:22.533790+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":40,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536503+0000","last_change":"2024-02-22T19:31:50.016416+0000","last_active":"2024-02-22T19:33:22.536503+0000","last_peered":"2024-02-22T19:33:22.536503+0000","last_clean":"2024-02-22T19:33:22.536503+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:33:22.536503+0000","last_undegraded":"2024-02-22T19:33:22.536503+0000","last_fullsized":"2024-02-22T19:33:22.536503+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":97,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.534455+0000","last_change":"2024-02-22T19:33:10.664920+0000","last_active":"2024-02-22T19:33:22.534455+0000","last_peered":"2024-02-22T19:33:22.534455+0000","last_clean":"2024-02-22T19:33:22.534455+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:33:22.534455+0000","last_undegraded":"2024-02-22T19:33:22.534455+0000","last_fullsized":"2024-02-22T19:33:22.534455+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:19.102765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:19.102765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536332+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:33:22.536332+0000","last_peered":"2024-02-22T19:33:22.536332+0000","last_clean":"2024-02-22T19:33:22.536332+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:33:22.536332+0000","last_undegraded":"2024-02-22T19:33:22.536332+0000","last_fullsized":"2024-02-22T19:33:22.536332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"128'6266","reported_seq":8602,"reported_epoch":128,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:33:29.225169+0000","last_change":"2024-02-22T19:31:51.077193+0000","last_active":"2024-02-22T19:33:29.225169+0000","last_peered":"2024-02-22T19:33:29.225169+0000","last_clean":"2024-02-22T19:31:48.945369+0000","last_became_active":"2024-02-22T19:31:51.043861+0000","last_became_peered":"2024-02-22T19:31:51.043861+0000","last_unstale":"2024-02-22T19:33:29.225169+0000","last_undegraded":"2024-02-22T19:33:29.225169+0000","last_fullsized":"2024-02-22T19:31:50.995661+0000","mapping_epoch":100,"log_start":"98'1000","ondisk_log_start":"98'1000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5266,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":298582016,"num_objects":4556,"num_object_clones":0,"num_object_copies":13668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111,"num_objects_unfound":0,"num_objects_dirty":4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6266,"num_write_kb":346304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1143,"num_bytes_recovered":74907648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0],"avail_no_missing":["2","0","5"],"object_location_counts":[{"shards":"0,2","objects":4445},{"shards":"0,2,5","objects":111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":79,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.539997+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:33:22.539997+0000","last_peered":"2024-02-22T19:33:22.539997+0000","last_clean":"2024-02-22T19:33:22.539997+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:33:22.539997+0000","last_undegraded":"2024-02-22T19:33:22.539997+0000","last_fullsized":"2024-02-22T19:33:22.539997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.527050+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:33:22.527050+0000","last_peered":"2024-02-22T19:33:22.527050+0000","last_clean":"2024-02-22T19:33:22.527050+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:33:22.527050+0000","last_undegraded":"2024-02-22T19:33:22.527050+0000","last_fullsized":"2024-02-22T19:33:22.527050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":82,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.518240+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:33:22.518240+0000","last_peered":"2024-02-22T19:33:22.518240+0000","last_clean":"2024-02-22T19:33:22.518240+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:33:22.518240+0000","last_undegraded":"2024-02-22T19:33:22.518240+0000","last_fullsized":"2024-02-22T19:33:22.518240+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":111,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.535379+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:33:22.535379+0000","last_peered":"2024-02-22T19:33:22.535379+0000","last_clean":"2024-02-22T19:33:22.535379+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:33:22.535379+0000","last_undegraded":"2024-02-22T19:33:22.535379+0000","last_fullsized":"2024-02-22T19:33:22.535379+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"128'6475","reported_seq":6518,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.249361+0000","last_change":"2024-02-22T19:31:33.014534+0000","last_active":"2024-02-22T19:33:29.249361+0000","last_peered":"2024-02-22T19:33:29.249361+0000","last_clean":"2024-02-22T19:33:29.249361+0000","last_became_active":"2024-02-22T19:31:33.014231+0000","last_became_peered":"2024-02-22T19:31:33.014231+0000","last_unstale":"2024-02-22T19:33:29.249361+0000","last_undegraded":"2024-02-22T19:33:29.249361+0000","last_fullsized":"2024-02-22T19:33:29.249361+0000","mapping_epoch":93,"log_start":"102'1600","ondisk_log_start":"102'1600","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4875,"ondisk_log_size":4875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":307822592,"num_objects":4697,"num_object_clones":0,"num_object_copies":14091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6475,"num_write_kb":357504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":81,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.536562+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:33:22.536562+0000","last_peered":"2024-02-22T19:33:22.536562+0000","last_clean":"2024-02-22T19:33:22.536562+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:33:22.536562+0000","last_undegraded":"2024-02-22T19:33:22.536562+0000","last_fullsized":"2024-02-22T19:33:22.536562+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":156,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-22T19:33:29.102134+0000","last_change":"2024-02-22T19:31:50.104329+0000","last_active":"2024-02-22T19:33:29.102134+0000","last_peered":"2024-02-22T19:33:29.102134+0000","last_clean":"2024-02-22T19:33:29.102134+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:33:29.102134+0000","last_undegraded":"2024-02-22T19:33:29.102134+0000","last_fullsized":"2024-02-22T19:33:29.102134+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:23.100177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.100177+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":91,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.528533+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:33:22.528533+0000","last_peered":"2024-02-22T19:33:22.528533+0000","last_clean":"2024-02-22T19:33:22.528533+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:33:22.528533+0000","last_undegraded":"2024-02-22T19:33:22.528533+0000","last_fullsized":"2024-02-22T19:33:22.528533+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":98,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-22T19:33:22.535943+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:33:22.535943+0000","last_peered":"2024-02-22T19:33:22.535943+0000","last_clean":"2024-02-22T19:33:22.535943+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:33:22.535943+0000","last_undegraded":"2024-02-22T19:33:22.535943+0000","last_fullsized":"2024-02-22T19:33:22.535943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":4916510744,"num_objects":75021,"num_object_clones":0,"num_object_copies":225063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":240,"num_objects_unfound":0,"num_objects_dirty":75021,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":103389,"num_write_kb":5709057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4874,"num_bytes_recovered":319422464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14898704384,"data_stored":14898692168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77189,"ondisk_log_size":77189,"up":48,"acting":46,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":262,"num_read_kb":257,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266745,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1818180,"kb_used_data":1508668,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91926972,"statfs":{"total":95995035648,"available":94133219328,"internally_reserved":0,"allocated":1544876032,"data_stored":1544475838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":5.0609999999999999,"5min":1.506,"15min":0.88400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":42.597999999999999,"5min":42.597999999999999,"15min":42.597999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":3.5640000000000001,"5min":1.1499999999999999,"15min":0.73599999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":42.155999999999999,"5min":42.155999999999999,"15min":42.155999999999999},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.6139999999999999,"5min":1.1719999999999999,"15min":0.77600000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":38.188000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":4.2999999999999998,"5min":1.3089999999999999,"15min":0.79900000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":37.938000000000002,"5min":37.938000000000002,"15min":37.938000000000002},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":1.226,"15min":0.78200000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":41.640999999999998,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":4.2530000000000001,"5min":1.3560000000000001,"15min":0.85199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":41.722000000000001,"5min":41.722000000000001,"15min":41.722000000000001},"last":2.1749999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1479999999999997,"5min":1.329,"15min":0.84499999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":41.457000000000001,"5min":41.457000000000001,"15min":41.457000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":3.6549999999999998,"5min":1.204,"15min":0.81899999999999995},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":41.311999999999998,"5min":41.311999999999998,"15min":41.311999999999998},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4889999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5539999999999998,"5min":12.067,"15min":12.067},"last":0.48599999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1822252,"kb_used_data":1512740,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91922900,"statfs":{"total":95995035648,"available":94129049600,"internally_reserved":0,"allocated":1549045760,"data_stored":1548669234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.472,"5min":0.71199999999999997,"15min":0.57999999999999996},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.6150000000000002,"5min":6.6150000000000002,"15min":6.6150000000000002},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.375},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.622,"15min":0.55600000000000005},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":7.415,"5min":7.415,"15min":7.415},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":0.76100000000000001,"15min":0.60999999999999999},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":12.859999999999999,"5min":12.859999999999999,"15min":12.859999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.63100000000000001,"15min":0.57799999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.307}]},{"osd":3,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":3.4780000000000002,"5min":1.125,"15min":0.73499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":38.762,"5min":38.762,"15min":38.762},"last":0.38300000000000001},{"interface":"front","average":{"1min":3.569,"5min":1.1579999999999999,"15min":0.75600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":35.612000000000002,"5min":35.612000000000002,"15min":35.612000000000002},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69399999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.45100000000000001,"15min":0.432},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.629,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45500000000000002,"15min":0.438},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65400000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.432},"min":{"1min":0.309,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68000000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34999999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48999999999999999,"15min":0.51000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.42099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2414620,"kb_used_data":2113172,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91330532,"statfs":{"total":95995035648,"available":93522464768,"internally_reserved":0,"allocated":2163888128,"data_stored":2163463826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":0.73799999999999999,"15min":0.46400000000000002},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":3.2450000000000001},{"interface":"front","average":{"1min":3.0569999999999999,"5min":0.93500000000000005,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":11.352,"5min":11.352,"15min":11.352},"last":1.522}]},{"osd":2,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":4.5469999999999997,"5min":1.3220000000000001,"15min":0.70199999999999996},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":15.683,"5min":15.683,"15min":15.683},"last":3.2730000000000001},{"interface":"front","average":{"1min":4.9130000000000003,"5min":1.3400000000000001,"15min":0.70799999999999996},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":13.461,"5min":13.461,"15min":13.461},"last":3.9169999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:32:32 2024","interfaces":[{"interface":"back","average":{"1min":3.7759999999999998,"5min":1.079,"15min":0.67100000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":25.145,"5min":25.145,"15min":25.145},"last":10.028},{"interface":"front","average":{"1min":4.2690000000000001,"5min":1.2310000000000001,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.252,"15min":0.252},"max":{"1min":22.25,"5min":22.25,"15min":22.25},"last":6.7969999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:32:50 2024","interfaces":[{"interface":"back","average":{"1min":2.5350000000000001,"5min":0.95899999999999996,"15min":0.73499999999999999},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":15.542999999999999,"5min":15.542999999999999,"15min":15.542999999999999},"last":3.343},{"interface":"front","average":{"1min":1.226,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.40400000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":2.1520000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:33:09 2024","interfaces":[{"interface":"back","average":{"1min":3.1429999999999998,"5min":1.1819999999999999,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":15.587999999999999,"5min":15.587999999999999,"15min":15.587999999999999},"last":3.371},{"interface":"front","average":{"1min":2.694,"5min":1.0149999999999999,"15min":0.72299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.5839999999999996,"5min":7.5839999999999996,"15min":7.5839999999999996},"last":1.5429999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":0.67100000000000004,"15min":0.58599999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.375,"5min":5.375,"15min":5.375},"last":2.1219999999999999},{"interface":"front","average":{"1min":2.4670000000000001,"5min":0.96799999999999997,"15min":0.71699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.651999999999999,"5min":15.651999999999999,"15min":15.651999999999999},"last":3.3149999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.5579999999999998,"5min":0.97299999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":1.494},{"interface":"front","average":{"1min":1.706,"5min":0.82399999999999995,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":2.1720000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2709584,"kb_used_data":2400008,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91035568,"statfs":{"total":95995035648,"available":93220421632,"internally_reserved":0,"allocated":2457608192,"data_stored":2457202921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:32:48 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":0.68899999999999995,"15min":0.46600000000000003},"min":{"1min":0.20599999999999999,"5min":0.191,"15min":0.191},"max":{"1min":9.8640000000000008,"5min":9.8640000000000008,"15min":9.8640000000000008},"last":0.19800000000000001},{"interface":"front","average":{"1min":2.819,"5min":0.874,"15min":0.52900000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":8.3670000000000009,"5min":8.3670000000000009,"15min":8.3670000000000009},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:33:06 2024","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":1.335,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":19.927,"5min":19.927,"15min":19.927},"last":0.44400000000000001},{"interface":"front","average":{"1min":4.2110000000000003,"5min":1.327,"15min":0.69099999999999995},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":17.907,"5min":17.907,"15min":17.907},"last":5.3680000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:33:27 2024","interfaces":[{"interface":"back","average":{"1min":3.5150000000000001,"5min":1.2989999999999999,"15min":0.72299999999999998},"min":{"1min":0.22900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":13.06,"5min":13.06,"15min":13.06},"last":0.36399999999999999},{"interface":"front","average":{"1min":4.0919999999999996,"5min":1.5349999999999999,"15min":0.81200000000000006},"min":{"1min":0.39000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":13.837999999999999,"5min":22.936,"15min":22.936},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.66900000000000004,"15min":0.60499999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":4.3129999999999997,"5min":4.3129999999999997,"15min":4.3129999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":2.7930000000000001,"5min":1.0109999999999999,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.085000000000001,"5min":12.085000000000001,"15min":12.085000000000001},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":1.26,"15min":0.79800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":13.903,"5min":13.903,"15min":13.903},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.7270000000000001,"5min":0.81100000000000005,"15min":0.64900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.8920000000000003,"5min":6.8920000000000003,"15min":6.8920000000000003},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.3500000000000001,"5min":0.745,"15min":0.64400000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":5.1630000000000003,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":0.621}]},{"osd":7,"last update":"Thu Feb 22 19:33:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":0.84399999999999997,"15min":0.66400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9660000000000002,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":2.4279999999999999,"5min":0.95299999999999996,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":6.0800000000000001,"5min":6.0800000000000001,"15min":6.0800000000000001},"last":0.65300000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2712952,"kb_used_data":2403440,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91032200,"statfs":{"total":95995035648,"available":93216972800,"internally_reserved":0,"allocated":2461122560,"data_stored":2460733950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":1.1719999999999999,"15min":0.63600000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":11.898999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":1.482},{"interface":"front","average":{"1min":3.6859999999999999,"5min":1.4239999999999999,"15min":0.71599999999999997},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":16.138999999999999,"5min":18.059999999999999,"15min":18.059999999999999},"last":3.9969999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":0.96199999999999997,"15min":0.57199999999999995},"min":{"1min":0.21199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.183,"5min":10.183,"15min":10.183},"last":4.4779999999999998},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.73799999999999999,"15min":0.51600000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":9.7189999999999994,"5min":9.7189999999999994,"15min":9.7189999999999994},"last":1.5069999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:32:31 2024","interfaces":[{"interface":"back","average":{"1min":2.9470000000000001,"5min":0.94499999999999995,"15min":0.624},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":17.431000000000001,"5min":17.431000000000001,"15min":17.431000000000001},"last":1.528},{"interface":"front","average":{"1min":3.4390000000000001,"5min":1.02,"15min":0.63800000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":18.099,"5min":18.099,"15min":18.099},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:32:53 2024","interfaces":[{"interface":"back","average":{"1min":2.1669999999999998,"5min":0.90400000000000003,"15min":0.68200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":6.0449999999999999,"5min":6.0449999999999999,"15min":6.0449999999999999},"last":4.3630000000000004},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.67500000000000004,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:33:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":0.747,"15min":0.66500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.76600000000000001},{"interface":"front","average":{"1min":2.1669999999999998,"5min":0.97999999999999998,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0739999999999998,"5min":6.0739999999999998,"15min":6.0739999999999998},"last":4.4299999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.8580000000000001,"5min":0.85799999999999998,"15min":0.68799999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.0119999999999996,"5min":6.0119999999999996,"15min":6.0119999999999996},"last":4.4020000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.71199999999999997,"15min":0.65800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":9.5990000000000002,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":1.028}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":0.94799999999999995,"15min":0.70199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.1029999999999998,"5min":6.1029999999999998,"15min":6.1029999999999998},"last":4.4539999999999997},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.73699999999999999,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":9.5660000000000007,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.84499999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3298252,"kb_used_data":3000964,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":90446900,"statfs":{"total":95995035648,"available":92617625600,"internally_reserved":0,"allocated":3072987136,"data_stored":3072579456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:28 2024","interfaces":[{"interface":"back","average":{"1min":2.3660000000000001,"5min":0.78100000000000003,"15min":0.55400000000000005},"min":{"1min":0.22,"5min":0.193,"15min":0.193},"max":{"1min":7.9699999999999998,"5min":7.9699999999999998,"15min":7.9699999999999998},"last":12.929},{"interface":"front","average":{"1min":2.7309999999999999,"5min":0.88200000000000001,"15min":0.61899999999999999},"min":{"1min":0.311,"5min":0.247,"15min":0.247},"max":{"1min":11.436,"5min":11.436,"15min":11.436},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:32:28 2024","interfaces":[{"interface":"back","average":{"1min":3.282,"5min":0.99099999999999999,"15min":0.64400000000000002},"min":{"1min":0.27000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":11.342000000000001,"5min":11.342000000000001,"15min":11.342000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":2.6669999999999998,"5min":0.86199999999999999,"15min":0.59299999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":10.279,"5min":10.279,"15min":10.279},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:32:28 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":0.97399999999999998,"15min":0.65300000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.1109999999999998,"5min":7.1109999999999998,"15min":7.1109999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":3.069,"5min":0.95799999999999996,"15min":0.63800000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.5830000000000002,"5min":8.5830000000000002,"15min":8.5830000000000002},"last":13.007999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:32:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3359999999999999,"5min":0.99099999999999999,"15min":0.747},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":11.241,"5min":11.241,"15min":11.241},"last":0.57999999999999996},{"interface":"front","average":{"1min":2.1680000000000001,"5min":0.95699999999999996,"15min":0.72299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.3700000000000001,"5min":6.7089999999999996,"15min":6.7089999999999996},"last":0.434}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":1.125,"15min":0.76800000000000002},"min":{"1min":0.437,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.8550000000000004,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":12.879},{"interface":"front","average":{"1min":3.3239999999999998,"5min":1.2689999999999999,"15min":0.81100000000000005},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":12.744999999999999,"5min":12.744999999999999,"15min":12.744999999999999},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:32:41 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":0.83899999999999997,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":12.898999999999999},{"interface":"front","average":{"1min":2.5,"5min":0.98199999999999998,"15min":0.76100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":11.308,"5min":11.308,"15min":11.308},"last":0.625}]},{"osd":7,"last update":"Thu Feb 22 19:32:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":1.0780000000000001,"15min":0.74299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":10.055,"5min":10.055,"15min":10.055},"last":0.64400000000000002},{"interface":"front","average":{"1min":2.3319999999999999,"5min":1.022,"15min":0.746},"min":{"1min":0.45700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.8200000000000003,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":12.846}]}]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1818568,"kb_used_data":1509056,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91926584,"statfs":{"total":95995035648,"available":94132822016,"internally_reserved":0,"allocated":1545273344,"data_stored":1544869540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":2.3029999999999999,"5min":0.88100000000000001,"15min":0.63800000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":24.009,"5min":24.009,"15min":24.009},"last":5.9790000000000001},{"interface":"front","average":{"1min":2.7599999999999998,"5min":0.98499999999999999,"15min":0.69199999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":24.263000000000002,"5min":24.263000000000002,"15min":24.263000000000002},"last":3.4169999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":0.68100000000000005,"15min":0.59399999999999997},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":5.4420000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.59999999999999998,"15min":0.53500000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.518}]},{"osd":2,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1709999999999998,"5min":0.86399999999999999,"15min":0.66900000000000004},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":23.082000000000001,"5min":23.082000000000001,"15min":23.082000000000001},"last":3.6789999999999998},{"interface":"front","average":{"1min":2.6110000000000002,"5min":0.95899999999999996,"15min":0.68100000000000005},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":23.138999999999999,"5min":23.138999999999999,"15min":23.138999999999999},"last":1.673}]},{"osd":3,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":0.749,"15min":0.61899999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":0.73499999999999999,"15min":0.59899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.0149999999999997,"5min":7.0149999999999997,"15min":7.0149999999999997},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.212}]},{"osd":6,"last update":"Thu Feb 22 19:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.46500000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.48099999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.432}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.48499999999999999}]}]},{"osd":5,"up_from":0,"seq":167503724611,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2453409792,"data_stored":2453405720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2152075264,"data_stored":2152071192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2448097280,"data_stored":2448097280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3061186560,"data_stored":3061186560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1537605632,"data_stored":1537605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":165543936,"data_stored":165543936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1543573504,"data_stored":1543569432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1537212416,"data_stored":1537212416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:33:35.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph status --format=json 2024-02-22T19:33:35.986 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:36.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:35.297818+0000 mgr.y (mgr.14535) 181 : cluster [DBG] pgmap v194: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.20k op/s; 124/214410 objects misplaced (0.058%); 2.3 MiB/s, 37 objects/s recovering 2024-02-22T19:33:36.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:35.362407+0000 osd.5 (osd.5) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:36.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:35.362458+0000 osd.5 (osd.5) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:36.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:35.399496+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e131: 8 total, 8 up, 7 in 2024-02-22T19:33:36.674 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:35.297818+0000 mgr.y (mgr.14535) 181 : cluster [DBG] pgmap v194: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.20k op/s; 124/214410 objects misplaced (0.058%); 2.3 MiB/s, 37 objects/s recovering 2024-02-22T19:33:36.674 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:35.362407+0000 osd.5 (osd.5) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:36.674 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:35.362458+0000 osd.5 (osd.5) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:36.674 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:35.399496+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e131: 8 total, 8 up, 7 in 2024-02-22T19:33:36.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:35.297818+0000 mgr.y (mgr.14535) 181 : cluster [DBG] pgmap v194: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.20k op/s; 124/214410 objects misplaced (0.058%); 2.3 MiB/s, 37 objects/s recovering 2024-02-22T19:33:36.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:35.362407+0000 osd.5 (osd.5) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:36.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:35.362458+0000 osd.5 (osd.5) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:36.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:35.399496+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e131: 8 total, 8 up, 7 in 2024-02-22T19:33:36.799 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:36.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:36.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:36.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:36.969 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:36.970 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:37.071 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:37.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:33:37.849 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:37.850 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:37.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:37.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:37.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:38.030 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:38.032 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:38.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:38.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:38.168 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:38.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:37.300415+0000 mgr.y (mgr.14535) 182 : cluster [DBG] pgmap v196: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 714 B/s rd, 0 B/s wr, 522 op/s; 124/214410 objects misplaced (0.058%); 1.0 MiB/s, 16 objects/s recovering 2024-02-22T19:33:38.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:38.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19162 7.0 7:049100ab:::benchmark_data_smithi053_107660_object19161:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.132188+0000", 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.156382308, 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012938493000000001, 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19162 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132188+0000", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132188+0000", 2024-02-22T19:33:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132186+0000", 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0939999999999999e-05 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132217+0000", 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0973e-05 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132228+0000", 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2610000000000001e-06 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132234+0000", 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1192999999999997e-05 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132275+0000", 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0587999999999994e-05 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132365+0000", 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100864 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132466+0000", 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00148449 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133951+0000", 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032656590000000002 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.137216+0000", 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0077983369999999998 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.145015+0000", 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4248999999999999e-05 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.145099+0000", 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7319999999999999e-05 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.145126+0000", 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012938493000000001 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19170 7.0 7:099f3196:::benchmark_data_smithi053_107660_object19169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.132770+0000", 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.15580013100001, 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013757603, 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19170 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132770+0000", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132770+0000", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132769+0000", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7450999999999998e-05 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132806+0000", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4518999999999999e-05 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132821+0000", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3330000000000002e-06 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132827+0000", 2024-02-22T19:33:38.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00037433600000000002 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133202+0000", 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115468 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133317+0000", 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015316 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133470+0000", 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00062185599999999995 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.134092+0000", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003163646 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.137256+0000", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0092206400000000004 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.146476+0000", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9346e-05 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.146506+0000", 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1888000000000001e-05 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.146528+0000", 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013757603 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19192 7.0 7:0c92f274:::benchmark_data_smithi053_107660_object19191:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.148536+0000", 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.140033835, 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006757702, 2024-02-22T19:33:38.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19192 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148536+0000", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148536+0000", 2024-02-22T19:33:38.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148534+0000", 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1286000000000003e-05 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148586+0000", 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7541e-05 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148603+0000", 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2600000000000005e-06 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148611+0000", 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102261 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148714+0000", 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000128057 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148842+0000", 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000163614 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.149005+0000", 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015786750000000001 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.150584+0000", 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018755429999999999 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.152460+0000", 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027746709999999998 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.155234+0000", 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4789999999999997e-05 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.155269+0000", 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5043000000000001e-05 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.155294+0000", 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006757702 2024-02-22T19:33:38.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19339 7.0 7:03559344:::benchmark_data_smithi053_107660_object19338:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.243068+0000", 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.04550256100001, 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014996083, 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19339 2024-02-22T19:33:38.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243068+0000", 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243068+0000", 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243066+0000", 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7571000000000003e-05 2024-02-22T19:33:38.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243103+0000", 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1447000000000001e-05 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243115+0000", 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4299999999999997e-06 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243120+0000", 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3835000000000002e-05 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243174+0000", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100698 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243275+0000", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000101266 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243376+0000", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00076734500000000001 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.244144+0000", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081373880000000006 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.252281+0000", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056537219999999999 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.257935+0000", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5027000000000001e-05 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.257980+0000", 2024-02-22T19:33:38.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4048000000000006e-05 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.258064+0000", 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014996083 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19495 7.0 7:03bef8f6:::benchmark_data_smithi053_107660_object19494:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.332319+0000", 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.956251616, 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010829564999999999, 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19495 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332319+0000", 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332319+0000", 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332317+0000", 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3709999999999999e-05 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332371+0000", 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0648999999999999e-05 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332382+0000", 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3790000000000003e-06 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332388+0000", 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1232000000000001e-05 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332429+0000", 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014674600000000001 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332576+0000", 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015044 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332727+0000", 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00071417799999999999 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.333441+0000", 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024283830000000001 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.335869+0000", 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0071718609999999999 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.343041+0000", 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3146999999999997e-05 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.343104+0000", 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4029000000000001e-05 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.343148+0000", 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010829564999999999 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19582 7.0 7:04cf3c1f:::benchmark_data_smithi053_107660_object19581:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.382535+0000", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.90603541199999, 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070217949999999999, 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19582 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382535+0000", 2024-02-22T19:33:38.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382535+0000", 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382532+0000", 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8909000000000001e-05 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382591+0000", 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7160000000000003e-05 2024-02-22T19:33:38.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382628+0000", 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2578999999999999e-05 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382641+0000", 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018372399999999999 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382825+0000", 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015238099999999999 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382977+0000", 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017174399999999999 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383149+0000", 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014893879999999999 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.384638+0000", 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00052996900000000003 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.385168+0000", 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043303209999999998 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.389498+0000", 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3080000000000002e-05 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.389532+0000", 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5000000000000001e-05 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.389557+0000", 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070217949999999999 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19587 7.0 7:0712d9c9:::benchmark_data_smithi053_107660_object19586:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.382667+0000", 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.905903464, 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083673589999999996, 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19587 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382667+0000", 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382667+0000", 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:38.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382665+0000", 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1635999999999999e-05 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382707+0000", 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0740000000000001e-05 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382718+0000", 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0060000000000003e-06 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382725+0000", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0010469699999999999 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383772+0000", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124206 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383896+0000", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7546999999999994e-05 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383983+0000", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00075819799999999997 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.384742+0000", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000459751 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.385201+0000", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005775662 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.390977+0000", 2024-02-22T19:33:38.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0908999999999998e-05 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.391008+0000", 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6064e-05 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.391034+0000", 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083673589999999996 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19598 7.0 7:04b15c7c:::benchmark_data_smithi053_107660_object19597:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.393641+0000", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.894928792, 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089943990000000001, 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19598 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393641+0000", 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393641+0000", 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393640+0000", 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1282e-05 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393701+0000", 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3458000000000001e-05 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393714+0000", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.418e-06 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393722+0000", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1810000000000001e-05 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393764+0000", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6761000000000006e-05 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393840+0000", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110919 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393951+0000", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001293137 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.395244+0000", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002559221 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.397804+0000", 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004777051 2024-02-22T19:33:38.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.402581+0000", 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3751000000000003e-05 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.402614+0000", 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1347000000000001e-05 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.402636+0000", 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089943990000000001 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19665 7.0 7:0fd8bd0c:::benchmark_data_smithi053_107660_object19664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.432947+0000", 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.85562344500001, 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093526809999999998, 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19665 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.432947+0000", 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.432947+0000", 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.432945+0000", 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1642000000000002e-05 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433026+0000", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5486e-05 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433052+0000", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5729999999999998e-06 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433057+0000", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8538999999999997e-05 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433106+0000", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8690000000000006e-05 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433195+0000", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104849 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433299+0000", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00090149700000000002 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.434201+0000", 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002709595 2024-02-22T19:33:38.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.436910+0000", 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052803090000000004 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.442191+0000", 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4300000000000004e-05 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.442255+0000", 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4342000000000003e-05 2024-02-22T19:33:38.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.442299+0000", 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093526809999999998 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19707 7.0 7:0b2b3dfb:::benchmark_data_smithi053_107660_object19706:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.454436+0000", 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.834134046, 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0079920819999999993, 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19707 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454436+0000", 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454436+0000", 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454435+0000", 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0632e-05 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454476+0000", 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8410000000000006e-06 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454485+0000", 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1099999999999996e-06 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454489+0000", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4212000000000003e-05 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454533+0000", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123671 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454657+0000", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000175818 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454832+0000", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00075767599999999999 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.455590+0000", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061248409999999998 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.461715+0000", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060546099999999995 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.462320+0000", 2024-02-22T19:33:38.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9639000000000003e-05 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.462380+0000", 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8124000000000002e-05 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.462428+0000", 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0079920819999999993 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.346 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19721 7.0 7:0e922d33:::benchmark_data_smithi053_107660_object19720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.466702+0000", 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.821868427, 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090205489999999992, 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19721 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466702+0000", 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466702+0000", 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466700+0000", 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1017999999999998e-05 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466751+0000", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2935999999999999e-05 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466764+0000", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.725e-06 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466770+0000", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6407999999999999e-05 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466827+0000", 2024-02-22T19:33:38.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016757999999999999 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466994+0000", 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018296600000000001 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.467177+0000", 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001057987 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.468235+0000", 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064652999999999998 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.468882+0000", 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067415519999999996 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.475623+0000", 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7846000000000001e-05 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.475681+0000", 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0992999999999999e-05 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.475722+0000", 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090205489999999992 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20011 7.0 7:05e9e5be:::benchmark_data_smithi053_107660_object20010:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.617516+0000", 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.671054126, 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081888140000000009, 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20011 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617516+0000", 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617516+0000", 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999967 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617513+0000", 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0525999999999997e-05 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617563+0000", 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5023000000000001e-05 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617588+0000", 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2320000000000001e-05 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617601+0000", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5715000000000001e-05 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617697+0000", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3256000000000004e-05 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617780+0000", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000175458 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617955+0000", 2024-02-22T19:33:38.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00093616999999999995 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.618891+0000", 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056099899999999996 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.624501+0000", 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001148114 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.625649+0000", 2024-02-22T19:33:38.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2036000000000003e-05 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.625682+0000", 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3323e-05 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.625705+0000", 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081888140000000009 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20051 7.0 7:02db49fd:::benchmark_data_smithi053_107660_object20050:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.639182+0000", 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.64938807999999, 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0088055890000000008, 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20051 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639182+0000", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639182+0000", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639181+0000", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5690000000000003e-05 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639247+0000", 2024-02-22T19:33:38.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2836e-05 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639260+0000", 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9449999999999999e-06 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639268+0000", 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001110703 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.640379+0000", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000132364 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.640512+0000", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123743 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.640635+0000", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00068257599999999995 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.641318+0000", 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001252967 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.642571+0000", 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053208530000000004 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.647892+0000", 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4314000000000002e-05 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.647946+0000", 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1668999999999997e-05 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.647988+0000", 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0088055890000000008 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20366 7.0 7:0c62a8f1:::benchmark_data_smithi053_107660_object20365:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.780949+0000", 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.50762096299999, 2024-02-22T19:33:38.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073489389999999996, 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20366 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.780949+0000", 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.780949+0000", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.780947+0000", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3261999999999999e-05 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781000+0000", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5065999999999999e-05 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781015+0000", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.233e-06 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781022+0000", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8088999999999999e-05 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781080+0000", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016692500000000001 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781247+0000", 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9491e-05 2024-02-22T19:33:38.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781347+0000", 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001557938 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782905+0000", 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000526221 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.783431+0000", 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004772933 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788204+0000", 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1786000000000002e-05 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788256+0000", 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2366000000000001e-05 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788298+0000", 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073489389999999996 2024-02-22T19:33:38.368 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20367 7.0 7:090e9347:::benchmark_data_smithi053_107660_object20366:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.781154+0000", 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.507415837, 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072720479999999997, 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20367 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.369 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781154+0000", 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781154+0000", 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781153+0000", 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9843999999999998e-05 2024-02-22T19:33:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781213+0000", 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2939999999999994e-06 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781223+0000", 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1700000000000002e-06 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781229+0000", 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00079298900000000002 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782022+0000", 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000112934 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782135+0000", 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016562200000000001 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782300+0000", 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001390929 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.783691+0000", 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026650099999999999 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.783958+0000", 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043807660000000003 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788338+0000", 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8968999999999998e-05 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788387+0000", 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8951e-05 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788426+0000", 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072720479999999997 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20451 7.0 7:00a00ce1:::benchmark_data_smithi053_107660_object20450:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.817018+0000", 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.471552155, 2024-02-22T19:33:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087138210000000001, 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20451 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817018+0000", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817018+0000", 2024-02-22T19:33:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817016+0000", 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3357999999999999e-05 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817079+0000", 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5642e-05 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817095+0000", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7360000000000001e-06 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817103+0000", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2648999999999997e-05 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817155+0000", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013609900000000001 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817292+0000", 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015985500000000001 2024-02-22T19:33:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817451+0000", 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00083612599999999997 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.818287+0000", 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020579639999999998 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.820345+0000", 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053291049999999998 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.825675+0000", 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3754000000000002e-05 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.825708+0000", 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.353e-05 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.825732+0000", 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087138210000000001 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20519 7.0 7:0f3a20e9:::benchmark_data_smithi053_107660_object20518:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.845539+0000", 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.44303092, 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084815829999999991, 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20519 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845539+0000", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845539+0000", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845538+0000", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3948999999999994e-05 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845602+0000", 2024-02-22T19:33:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3067999999999999e-05 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845615+0000", 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0070000000000003e-06 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845621+0000", 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014366870000000001 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847058+0000", 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102033 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847160+0000", 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014512800000000001 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847305+0000", 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064053899999999997 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847945+0000", 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017828989999999999 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.849728+0000", 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042388959999999998 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.853967+0000", 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1578999999999997e-05 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.853999+0000", 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2167999999999998e-05 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.854021+0000", 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084815829999999991 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20643 7.0 7:0874e533:::benchmark_data_smithi053_107660_object20642:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.897311+0000", 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.39125913300001, 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0077622109999999998, 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20643 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897311+0000", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897311+0000", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897310+0000", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9523000000000003e-05 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897399+0000", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1929e-05 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897411+0000", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897417+0000", 2024-02-22T19:33:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3964999999999998e-05 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897451+0000", 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012626900000000001 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897577+0000", 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000129837 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897707+0000", 2024-02-22T19:33:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00096261199999999995 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.898670+0000", 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016538670000000001 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.900324+0000", 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046556480000000001 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.904979+0000", 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5241000000000002e-05 2024-02-22T19:33:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.905034+0000", 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8747e-05 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.905073+0000", 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0077622109999999998 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20650 7.0 7:055267f9:::benchmark_data_smithi053_107660_object20649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:33:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.386210855, 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073057419999999996, 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20650 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5485000000000002e-05 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902414+0000", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.144e-06 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902420+0000", 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.558e-06 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902426+0000", 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00071791599999999998 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.903144+0000", 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013517 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.903279+0000", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017210399999999999 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.903451+0000", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000651338 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.904102+0000", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012351790000000001 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.905337+0000", 2024-02-22T19:33:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042757710000000003 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.909613+0000", 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1090000000000002e-05 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.909644+0000", 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0721999999999999e-05 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.909665+0000", 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073057419999999996 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20673 7.0 7:0c980cb3:::benchmark_data_smithi053_107660_object20672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.915300+0000", 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 109.37327009800001, 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0068715149999999999, 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20673 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915300+0000", 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915300+0000", 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915299+0000", 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8996000000000003e-05 2024-02-22T19:33:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915348+0000", 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2449000000000001e-05 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915360+0000", 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3000000000000001e-06 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915366+0000", 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0315000000000002e-05 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915416+0000", 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6032999999999994e-05 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915502+0000", 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100528 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915602+0000", 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00079250499999999999 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.916395+0000", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033575110000000001 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.919752+0000", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023228369999999999 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.922075+0000", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4342999999999998e-05 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.922130+0000", 2024-02-22T19:33:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1965999999999998e-05 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.922172+0000", 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0068715149999999999 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:38.401 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:33:38.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:38.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:37.300415+0000 mgr.y (mgr.14535) 182 : cluster [DBG] pgmap v196: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 714 B/s rd, 0 B/s wr, 522 op/s; 124/214410 objects misplaced (0.058%); 1.0 MiB/s, 16 objects/s recovering 2024-02-22T19:33:38.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:37.300415+0000 mgr.y (mgr.14535) 182 : cluster [DBG] pgmap v196: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 714 B/s rd, 0 B/s wr, 522 op/s; 124/214410 objects misplaced (0.058%); 1.0 MiB/s, 16 objects/s recovering 2024-02-22T19:33:38.536 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:38.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:38.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:38.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:39.024 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:39.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:39.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:39.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:33:39.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:39.190 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:39.195 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:39.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:39.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:39.342 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:39.390 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:39.546 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:39.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:39.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:39.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:39.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:39.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:39.749 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:33:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:39] "GET /metrics HTTP/1.1" 200 175580 "" "Prometheus/2.33.4" 2024-02-22T19:33:39.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:39.833 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:39.966 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:39.969 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:39.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:40.011 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:40.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:40.096 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:33:40.103 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:40.156 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:40.159 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:40.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:40.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:40.296 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:39.084165+0000 mon.a (mon.0) 867 : cluster [DBG] osdmap e132: 8 total, 8 up, 7 in 2024-02-22T19:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:39.301327+0000 mgr.y (mgr.14535) 183 : cluster [DBG] pgmap v198: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 941 op/s; 12/203046 objects degraded (0.006%); 1.6 MiB/s, 28 objects/s recovering 2024-02-22T19:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:39.730642+0000 mon.a (mon.0) 868 : audit [INF] from='client.? 172.21.15.53:0/2667543452' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:33:40.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:39.084165+0000 mon.a (mon.0) 867 : cluster [DBG] osdmap e132: 8 total, 8 up, 7 in 2024-02-22T19:33:40.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:39.301327+0000 mgr.y (mgr.14535) 183 : cluster [DBG] pgmap v198: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 941 op/s; 12/203046 objects degraded (0.006%); 1.6 MiB/s, 28 objects/s recovering 2024-02-22T19:33:40.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:39.730642+0000 mon.a (mon.0) 868 : audit [INF] from='client.? 172.21.15.53:0/2667543452' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:33:40.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:39.084165+0000 mon.a (mon.0) 867 : cluster [DBG] osdmap e132: 8 total, 8 up, 7 in 2024-02-22T19:33:40.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:39.301327+0000 mgr.y (mgr.14535) 183 : cluster [DBG] pgmap v198: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 941 op/s; 12/203046 objects degraded (0.006%); 1.6 MiB/s, 28 objects/s recovering 2024-02-22T19:33:40.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:39.730642+0000 mon.a (mon.0) 868 : audit [INF] from='client.? 172.21.15.53:0/2667543452' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:33:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:33:40.391 INFO:teuthology.orchestra.run.smithi053.stdout:{"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1708630133,"num_in_osds":7,"osd_in_since":1708630308,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":146},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":148,"num_pools":7,"num_objects":67682,"data_bytes":4422964924,"bytes_used":15523635200,"bytes_avail":656441614336,"bytes_total":671965249536,"inactive_pgs_ratio":0.0067567569203674793,"degraded_objects":12,"degraded_total":203046,"degraded_ratio":5.909990839514199e-05,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":1686320,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":13500416,"num_keys_recovered":0,"read_bytes_sec":1279,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":940},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.53:8443/","prometheus":"http://172.21.15.53:9283/"}},"servicemap":{"epoch":13,"modified":"2024-02-22T19:30:25.194204+0000","services":{"rgw":{"daemons":{"summary":"","24371":{"start_epoch":12,"start_stamp":"2024-02-22T19:29:08.193725+0000","gid":24371,"addr":"172.21.15.53:0/65330745","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi053","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi053","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"8","realm_id":"","realm_name":"","zone_id":"2d4caf58-b951-44e6-ad1d-667e6ee8308c","zone_name":"default","zonegroup_id":"a2ce501a-2ab0-4645-bd76-4ac5a6518222","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{"30da65d0-18bc-46a0-b277-07c6ffcad469":{"message":"Global Recovery Event (95s)\n [===========================.] (remaining: 1s)","progress":0.98648649454116821,"add_to_ceph_s":true},"53dee76a-7cc6-40e4-a276-5eae2690ea89":{"message":"Rebalancing after osd.5 marked out (105s)\n [===========================.] (remaining: 3s)","progress":0.97151237726211548,"add_to_ceph_s":false}}} 2024-02-22T19:33:40.747 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:40.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:40.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:40.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:40.910 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:41.019 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:41.025 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:41.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:41.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:41.196 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:41.285 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:111993 7.d 7:b80f50ca:::benchmark_data_smithi053_107660_object22786:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:41.282274+0000", 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0033209519999999998, 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033443489999999999, 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 111993 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282274+0000", 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282274+0000", 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282272+0000", 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3780000000000003e-06 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282279+0000", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1221e-05 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282291+0000", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.113e-06 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282296+0000", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5963e-05 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:33:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282342+0000", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7415000000000002e-05 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282429+0000", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3791999999999999e-05 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,1", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.282523+0000", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026772419999999998 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:41.285200+0000", 2024-02-22T19:33:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002926615 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 1 2024-02-22T19:33:41.290 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:33:41.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:40.079198+0000 mon.a (mon.0) 869 : cluster [WRN] Health check update: Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:33:41.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:40.079892+0000 mon.a (mon.0) 870 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:41.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:40.087890+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.53:0/2667543452' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:33:41.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:40.087933+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e133: 8 total, 8 up, 7 in 2024-02-22T19:33:41.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:40.389519+0000 mon.a (mon.0) 873 : audit [DBG] from='client.? 172.21.15.53:0/1153386808' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-22T19:33:41.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:40.079198+0000 mon.a (mon.0) 869 : cluster [WRN] Health check update: Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:33:41.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:40.079892+0000 mon.a (mon.0) 870 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:41.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:40.087890+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.53:0/2667543452' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:33:41.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:40.087933+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e133: 8 total, 8 up, 7 in 2024-02-22T19:33:41.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:40.389519+0000 mon.a (mon.0) 873 : audit [DBG] from='client.? 172.21.15.53:0/1153386808' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-22T19:33:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:40.079198+0000 mon.a (mon.0) 869 : cluster [WRN] Health check update: Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:33:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:40.079892+0000 mon.a (mon.0) 870 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:33:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:40.087890+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.53:0/2667543452' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:33:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:40.087933+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e133: 8 total, 8 up, 7 in 2024-02-22T19:33:41.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:40.389519+0000 mon.a (mon.0) 873 : audit [DBG] from='client.? 172.21.15.53:0/1153386808' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-22T19:33:41.561 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:41.564 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:41.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:41.665 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:41.697 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:41.898 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:41.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:41.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:42.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:42.033 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:42.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:42.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:42.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:33:42.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:42.199 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:42.224 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:42.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:42.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:42.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:41.109328+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e134: 8 total, 8 up, 7 in 2024-02-22T19:33:42.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:41.301697+0000 mgr.y (mgr.14535) 184 : cluster [DBG] pgmap v201: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 866 B/s rd, 0 B/s wr, 641 op/s; 12/203046 objects degraded (0.006%); 975 KiB/s, 18 objects/s recovering 2024-02-22T19:33:42.241 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:41.109328+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e134: 8 total, 8 up, 7 in 2024-02-22T19:33:42.241 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:41.301697+0000 mgr.y (mgr.14535) 184 : cluster [DBG] pgmap v201: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 866 B/s rd, 0 B/s wr, 641 op/s; 12/203046 objects degraded (0.006%); 975 KiB/s, 18 objects/s recovering 2024-02-22T19:33:42.266 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-02-22T19:33:42.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:33:42.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:41.109328+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e134: 8 total, 8 up, 7 in 2024-02-22T19:33:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:41.301697+0000 mgr.y (mgr.14535) 184 : cluster [DBG] pgmap v201: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 866 B/s rd, 0 B/s wr, 641 op/s; 12/203046 objects degraded (0.006%); 975 KiB/s, 18 objects/s recovering 2024-02-22T19:33:42.392 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:42.431 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:42.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:42.692 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:42.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:42.826 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:42.907 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:42.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:43.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:43.068 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:43.113 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:43.116 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:43.116 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:43.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:43.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:43.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:43.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:43.466 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.009629+0000 osd.0 (osd.0) 59 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:33:43.466 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.010463+0000 osd.0 (osd.0) 60 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:33:43.466 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.112824+0000 mon.a (mon.0) 875 : cluster [DBG] osdmap e135: 8 total, 8 up, 7 in 2024-02-22T19:33:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:43.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.009629+0000 osd.0 (osd.0) 59 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:33:43.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.010463+0000 osd.0 (osd.0) 60 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:33:43.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.112824+0000 mon.a (mon.0) 875 : cluster [DBG] osdmap e135: 8 total, 8 up, 7 in 2024-02-22T19:33:43.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.009629+0000 osd.0 (osd.0) 59 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:33:43.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.010463+0000 osd.0 (osd.0) 60 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:33:43.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.112824+0000 mon.a (mon.0) 875 : cluster [DBG] osdmap e135: 8 total, 8 up, 7 in 2024-02-22T19:33:43.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:33:44.322 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:44.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.749762+0000 osd.2 (osd.2) 35 : cluster [DBG] 5.d scrub starts 2024-02-22T19:33:44.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.757747+0000 osd.2 (osd.2) 36 : cluster [DBG] 5.d scrub ok 2024-02-22T19:33:44.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.856205+0000 osd.3 (osd.3) 61 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:33:44.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:42.862559+0000 osd.3 (osd.3) 62 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:33:44.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:43.036783+0000 osd.0 (osd.0) 61 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:33:44.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:43.048323+0000 osd.0 (osd.0) 62 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:33:44.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:43.302308+0000 mgr.y (mgr.14535) 185 : cluster [DBG] pgmap v203: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 898 op/s; 12/203046 objects degraded (0.006%); 1.3 MiB/s, 26 objects/s recovering 2024-02-22T19:33:44.328 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:44.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:44.343 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:33:44.344 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:33:44.344 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:33:44.344 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:33:44.344 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:33:44.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:44.490 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.749762+0000 osd.2 (osd.2) 35 : cluster [DBG] 5.d scrub starts 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.757747+0000 osd.2 (osd.2) 36 : cluster [DBG] 5.d scrub ok 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.856205+0000 osd.3 (osd.3) 61 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:42.862559+0000 osd.3 (osd.3) 62 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:43.036783+0000 osd.0 (osd.0) 61 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:43.048323+0000 osd.0 (osd.0) 62 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:43.302308+0000 mgr.y (mgr.14535) 185 : cluster [DBG] pgmap v203: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 898 op/s; 12/203046 objects degraded (0.006%); 1.3 MiB/s, 26 objects/s recovering 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.749762+0000 osd.2 (osd.2) 35 : cluster [DBG] 5.d scrub starts 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.757747+0000 osd.2 (osd.2) 36 : cluster [DBG] 5.d scrub ok 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.856205+0000 osd.3 (osd.3) 61 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:42.862559+0000 osd.3 (osd.3) 62 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:43.036783+0000 osd.0 (osd.0) 61 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:33:44.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:43.048323+0000 osd.0 (osd.0) 62 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:33:44.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:43.302308+0000 mgr.y (mgr.14535) 185 : cluster [DBG] pgmap v203: 148 pgs: 1 peering, 1 active+recovering+undersized+degraded+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 898 op/s; 12/203046 objects degraded (0.006%); 1.3 MiB/s, 26 objects/s recovering 2024-02-22T19:33:44.497 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:44.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:33:44.767 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:44.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:44.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:44.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:44.911 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:33:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:33:45.161 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:43.789185+0000 osd.2 (osd.2) 37 : cluster [DBG] 4.f scrub starts 2024-02-22T19:33:45.161 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:43.827073+0000 osd.2 (osd.2) 38 : cluster [DBG] 4.f scrub ok 2024-02-22T19:33:45.161 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.260318+0000 osd.7 (osd.7) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:45.161 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.260373+0000 osd.7 (osd.7) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:45.161 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:44.465241+0000 mgr.y (mgr.14535) 186 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:45.161 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:44.910550+0000 mon.b (mon.1) 42 : audit [DBG] from='client.? 172.21.15.53:0/3648761057' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:45.162 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:45.165 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:45.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:45.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:33:45.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:45.336 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:45.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:43.789185+0000 osd.2 (osd.2) 37 : cluster [DBG] 4.f scrub starts 2024-02-22T19:33:45.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:43.827073+0000 osd.2 (osd.2) 38 : cluster [DBG] 4.f scrub ok 2024-02-22T19:33:45.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.260318+0000 osd.7 (osd.7) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:45.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.260373+0000 osd.7 (osd.7) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:45.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:44.465241+0000 mgr.y (mgr.14535) 186 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:45.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:44.910550+0000 mon.b (mon.1) 42 : audit [DBG] from='client.? 172.21.15.53:0/3648761057' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:45.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:43.789185+0000 osd.2 (osd.2) 37 : cluster [DBG] 4.f scrub starts 2024-02-22T19:33:45.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:43.827073+0000 osd.2 (osd.2) 38 : cluster [DBG] 4.f scrub ok 2024-02-22T19:33:45.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.260318+0000 osd.7 (osd.7) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:45.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.260373+0000 osd.7 (osd.7) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:45.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:44.465241+0000 mgr.y (mgr.14535) 186 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:45.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:44.910550+0000 mon.b (mon.1) 42 : audit [DBG] from='client.? 172.21.15.53:0/3648761057' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:45.804 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:33:45.806 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:33:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.688043+0000 osd.1 (osd.1) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.688100+0000 osd.1 (osd.1) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.754027+0000 osd.1 (osd.1) 43 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.757105+0000 osd.1 (osd.1) 44 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.819611+0000 osd.2 (osd.2) 39 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:44.828560+0000 osd.2 (osd.2) 40 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.148139+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e136: 8 total, 8 up, 7 in 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.304402+0000 mgr.y (mgr.14535) 187 : cluster [DBG] pgmap v205: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 980 B/s rd, 0 B/s wr, 624 op/s; 0 B/s, 2 objects/s recovering 2024-02-22T19:33:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.522378+0000 osd.7 (osd.7) 55 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.688043+0000 osd.1 (osd.1) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.688100+0000 osd.1 (osd.1) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.754027+0000 osd.1 (osd.1) 43 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.757105+0000 osd.1 (osd.1) 44 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.819611+0000 osd.2 (osd.2) 39 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:44.828560+0000 osd.2 (osd.2) 40 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.148139+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e136: 8 total, 8 up, 7 in 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.304402+0000 mgr.y (mgr.14535) 187 : cluster [DBG] pgmap v205: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 980 B/s rd, 0 B/s wr, 624 op/s; 0 B/s, 2 objects/s recovering 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.522378+0000 osd.7 (osd.7) 55 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.688043+0000 osd.1 (osd.1) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:33:46.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.688100+0000 osd.1 (osd.1) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.754027+0000 osd.1 (osd.1) 43 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.757105+0000 osd.1 (osd.1) 44 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.819611+0000 osd.2 (osd.2) 39 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:44.828560+0000 osd.2 (osd.2) 40 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.148139+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e136: 8 total, 8 up, 7 in 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.304402+0000 mgr.y (mgr.14535) 187 : cluster [DBG] pgmap v205: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 980 B/s rd, 0 B/s wr, 624 op/s; 0 B/s, 2 objects/s recovering 2024-02-22T19:33:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.522378+0000 osd.7 (osd.7) 55 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:33:47.482 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:47.482 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.527950+0000 osd.7 (osd.7) 56 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:33:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.795010+0000 osd.2 (osd.2) 41 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:33:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.802749+0000 osd.2 (osd.2) 42 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:33:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:45.999260+0000 osd.0 (osd.0) 63 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:33:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.001286+0000 osd.0 (osd.0) 64 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.157794+0000 mon.a (mon.0) 877 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized) 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.761883+0000 osd.6 (osd.6) 27 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.763192+0000 osd.6 (osd.6) 28 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.527950+0000 osd.7 (osd.7) 56 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.795010+0000 osd.2 (osd.2) 41 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.802749+0000 osd.2 (osd.2) 42 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:45.999260+0000 osd.0 (osd.0) 63 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.001286+0000 osd.0 (osd.0) 64 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.157794+0000 mon.a (mon.0) 877 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized) 2024-02-22T19:33:47.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.761883+0000 osd.6 (osd.6) 27 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.763192+0000 osd.6 (osd.6) 28 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.527950+0000 osd.7 (osd.7) 56 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.795010+0000 osd.2 (osd.2) 41 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.802749+0000 osd.2 (osd.2) 42 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:45.999260+0000 osd.0 (osd.0) 63 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.001286+0000 osd.0 (osd.0) 64 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.157794+0000 mon.a (mon.0) 877 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/203046 objects degraded (0.006%), 1 pg degraded, 1 pg undersized) 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.761883+0000 osd.6 (osd.6) 27 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:33:47.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.763192+0000 osd.6 (osd.6) 28 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:33:48.110 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:48.119 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:48.126 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:48.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:48.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:33:48.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:48.282 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.717920+0000 osd.1 (osd.1) 45 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.724454+0000 osd.1 (osd.1) 46 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.822637+0000 osd.2 (osd.2) 43 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:46.825671+0000 osd.2 (osd.2) 44 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:47.309397+0000 mgr.y (mgr.14535) 188 : cluster [DBG] pgmap v206: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 825 B/s rd, 0 B/s wr, 525 op/s; 0 B/s, 2 objects/s recovering 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.717920+0000 osd.1 (osd.1) 45 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.724454+0000 osd.1 (osd.1) 46 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:33:48.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.822637+0000 osd.2 (osd.2) 43 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:33:48.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:46.825671+0000 osd.2 (osd.2) 44 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:33:48.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:47.309397+0000 mgr.y (mgr.14535) 188 : cluster [DBG] pgmap v206: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 825 B/s rd, 0 B/s wr, 525 op/s; 0 B/s, 2 objects/s recovering 2024-02-22T19:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.717920+0000 osd.1 (osd.1) 45 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:33:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.724454+0000 osd.1 (osd.1) 46 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:33:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.822637+0000 osd.2 (osd.2) 43 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:33:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:46.825671+0000 osd.2 (osd.2) 44 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:33:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:47.309397+0000 mgr.y (mgr.14535) 188 : cluster [DBG] pgmap v206: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 825 B/s rd, 0 B/s wr, 525 op/s; 0 B/s, 2 objects/s recovering 2024-02-22T19:33:49.468 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:47.747538+0000 osd.1 (osd.1) 47 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:33:49.468 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:47.757471+0000 osd.1 (osd.1) 48 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:33:49.469 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:33:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:49] "GET /metrics HTTP/1.1" 200 175580 "" "Prometheus/2.33.4" 2024-02-22T19:33:49.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:47.747538+0000 osd.1 (osd.1) 47 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:33:49.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:47.757471+0000 osd.1 (osd.1) 48 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:33:49.472 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:49.475 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:49.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:49.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:47.747538+0000 osd.1 (osd.1) 47 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:33:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:47.757471+0000 osd.1 (osd.1) 48 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:33:49.640 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:49.761 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:49.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:49.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:50.158 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:50.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:50.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:50.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:50.363 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:50.431 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:50.434 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:50.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:50.434 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:48.762564+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.b scrub starts 2024-02-22T19:33:50.434 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:48.766699+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.b scrub ok 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:48.827366+0000 osd.3 (osd.3) 63 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:48.835023+0000 osd.3 (osd.3) 64 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:48.895844+0000 osd.4 (osd.4) 57 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:48.898793+0000 osd.4 (osd.4) 58 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:49.022407+0000 osd.0 (osd.0) 65 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:49.035611+0000 osd.0 (osd.0) 66 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:49.312060+0000 mgr.y (mgr.14535) 189 : cluster [DBG] pgmap v207: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.00k op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:49.798019+0000 osd.6 (osd.6) 29 : cluster [DBG] 6.a scrub starts 2024-02-22T19:33:50.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:49.804836+0000 osd.6 (osd.6) 30 : cluster [DBG] 6.a scrub ok 2024-02-22T19:33:50.444 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:50.444 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:48.762564+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.b scrub starts 2024-02-22T19:33:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:48.766699+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.b scrub ok 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:48.827366+0000 osd.3 (osd.3) 63 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:48.835023+0000 osd.3 (osd.3) 64 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:48.895844+0000 osd.4 (osd.4) 57 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:48.898793+0000 osd.4 (osd.4) 58 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:49.022407+0000 osd.0 (osd.0) 65 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:49.035611+0000 osd.0 (osd.0) 66 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:49.312060+0000 mgr.y (mgr.14535) 189 : cluster [DBG] pgmap v207: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.00k op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:49.798019+0000 osd.6 (osd.6) 29 : cluster [DBG] 6.a scrub starts 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:49.804836+0000 osd.6 (osd.6) 30 : cluster [DBG] 6.a scrub ok 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:48.762564+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.b scrub starts 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:48.766699+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.b scrub ok 2024-02-22T19:33:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:48.827366+0000 osd.3 (osd.3) 63 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:48.835023+0000 osd.3 (osd.3) 64 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:48.895844+0000 osd.4 (osd.4) 57 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:48.898793+0000 osd.4 (osd.4) 58 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:49.022407+0000 osd.0 (osd.0) 65 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:49.035611+0000 osd.0 (osd.0) 66 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:49.312060+0000 mgr.y (mgr.14535) 189 : cluster [DBG] pgmap v207: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.00k op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:49.798019+0000 osd.6 (osd.6) 29 : cluster [DBG] 6.a scrub starts 2024-02-22T19:33:50.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:49.804836+0000 osd.6 (osd.6) 30 : cluster [DBG] 6.a scrub ok 2024-02-22T19:33:50.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:50.568 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:50.677 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:50.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:50.839 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:50.916 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:50.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:50.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:51.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:51.083 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:51.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:33:51.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:33:51.596 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:52.048 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:52.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:52.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:33:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:33:52.395 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.057727+0000 osd.0 (osd.0) 67 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:33:52.395 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.065817+0000 osd.0 (osd.0) 68 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:33:52.395 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.313392+0000 mgr.y (mgr.14535) 190 : cluster [DBG] pgmap v208: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 874 op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:52.395 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.800040+0000 osd.6 (osd.6) 31 : cluster [DBG] 3.a scrub starts 2024-02-22T19:33:52.395 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.802959+0000 osd.6 (osd.6) 32 : cluster [DBG] 3.a scrub ok 2024-02-22T19:33:52.396 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.057727+0000 osd.0 (osd.0) 67 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:33:52.398 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.057727+0000 osd.0 (osd.0) 67 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:33:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.065817+0000 osd.0 (osd.0) 68 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:33:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.313392+0000 mgr.y (mgr.14535) 190 : cluster [DBG] pgmap v208: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 874 op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.800040+0000 osd.6 (osd.6) 31 : cluster [DBG] 3.a scrub starts 2024-02-22T19:33:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.802959+0000 osd.6 (osd.6) 32 : cluster [DBG] 3.a scrub ok 2024-02-22T19:33:52.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:52.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:52.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:52.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.065817+0000 osd.0 (osd.0) 68 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:33:52.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.313392+0000 mgr.y (mgr.14535) 190 : cluster [DBG] pgmap v208: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 874 op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:52.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.800040+0000 osd.6 (osd.6) 31 : cluster [DBG] 3.a scrub starts 2024-02-22T19:33:52.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.802959+0000 osd.6 (osd.6) 32 : cluster [DBG] 3.a scrub ok 2024-02-22T19:33:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:53.194 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:53.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:53.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:53.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.940062+0000 osd.4 (osd.4) 59 : cluster [DBG] 4.e scrub starts 2024-02-22T19:33:53.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:51.953381+0000 osd.4 (osd.4) 60 : cluster [DBG] 4.e scrub ok 2024-02-22T19:33:53.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:52.332322+0000 mgr.y (mgr.14535) 191 : audit [DBG] from='client.24824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:53.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:52.777347+0000 osd.3 (osd.3) 65 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:33:53.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:52.788043+0000 osd.3 (osd.3) 66 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:33:53.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:53.344418+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:53.436 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:53.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:53.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:53.514 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.940062+0000 osd.4 (osd.4) 59 : cluster [DBG] 4.e scrub starts 2024-02-22T19:33:53.514 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:51.953381+0000 osd.4 (osd.4) 60 : cluster [DBG] 4.e scrub ok 2024-02-22T19:33:53.514 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:52.332322+0000 mgr.y (mgr.14535) 191 : audit [DBG] from='client.24824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:53.514 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:52.777347+0000 osd.3 (osd.3) 65 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:33:53.514 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:52.788043+0000 osd.3 (osd.3) 66 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:33:53.514 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:53.344418+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:53.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:53.553 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-02-22T19:33:51.313153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3896710844,"num_objects":59652,"num_object_clones":0,"num_object_copies":178956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59652,"num_whiteouts":0,"num_read":996,"num_read_kb":748,"num_write":119378,"num_write_kb":5709098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5178,"num_bytes_recovered":332925147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77951,"ondisk_log_size":77951,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":13629512,"kb_used_data":11511952,"kb_used_omap":0,"kb_used_meta":2117376,"kb_avail":642586552,"statfs":{"total":671965249536,"available":658008629248,"internally_reserved":0,"allocated":11788238848,"data_stored":11785513043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2168193024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-526254080,"num_objects":-8030,"num_object_clones":0,"num_object_copies":-24090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8030,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8030,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.195793"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114757+0000","last_change":"2024-02-22T19:31:50.004483+0000","last_active":"2024-02-22T19:33:42.114757+0000","last_peered":"2024-02-22T19:33:42.114757+0000","last_clean":"2024-02-22T19:33:42.114757+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:33:42.114757+0000","last_undegraded":"2024-02-22T19:33:42.114757+0000","last_fullsized":"2024-02-22T19:33:42.114757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":98,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112408+0000","last_change":"2024-02-22T19:31:50.003253+0000","last_active":"2024-02-22T19:33:42.112408+0000","last_peered":"2024-02-22T19:33:42.112408+0000","last_clean":"2024-02-22T19:33:42.112408+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:33:42.112408+0000","last_undegraded":"2024-02-22T19:33:42.112408+0000","last_fullsized":"2024-02-22T19:33:42.112408+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:24.056067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:24.056067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":105,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157887+0000","last_change":"2024-02-22T19:31:50.013054+0000","last_active":"2024-02-22T19:33:42.157887+0000","last_peered":"2024-02-22T19:33:42.157887+0000","last_clean":"2024-02-22T19:33:42.157887+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:33:42.157887+0000","last_undegraded":"2024-02-22T19:33:42.157887+0000","last_fullsized":"2024-02-22T19:33:42.157887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:12.445206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:12.445206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":107,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114541+0000","last_change":"2024-02-22T19:30:18.087006+0000","last_active":"2024-02-22T19:33:42.114541+0000","last_peered":"2024-02-22T19:33:42.114541+0000","last_clean":"2024-02-22T19:33:42.114541+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:33:42.114541+0000","last_undegraded":"2024-02-22T19:33:42.114541+0000","last_fullsized":"2024-02-22T19:33:42.114541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:18.086939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.086939+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":84,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115080+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:33:42.115080+0000","last_peered":"2024-02-22T19:33:42.115080+0000","last_clean":"2024-02-22T19:33:42.115080+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:33:42.115080+0000","last_undegraded":"2024-02-22T19:33:42.115080+0000","last_fullsized":"2024-02-22T19:33:42.115080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114646+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:33:42.114646+0000","last_peered":"2024-02-22T19:33:42.114646+0000","last_clean":"2024-02-22T19:33:42.114646+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:33:42.114646+0000","last_undegraded":"2024-02-22T19:33:42.114646+0000","last_fullsized":"2024-02-22T19:33:42.114646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":90,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116016+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:33:42.116016+0000","last_peered":"2024-02-22T19:33:42.116016+0000","last_clean":"2024-02-22T19:33:42.116016+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:33:42.116016+0000","last_undegraded":"2024-02-22T19:33:42.116016+0000","last_fullsized":"2024-02-22T19:33:42.116016+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":113,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115047+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:33:42.115047+0000","last_peered":"2024-02-22T19:33:42.115047+0000","last_clean":"2024-02-22T19:33:42.115047+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:33:42.115047+0000","last_undegraded":"2024-02-22T19:33:42.115047+0000","last_fullsized":"2024-02-22T19:33:42.115047+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.156419+0000","last_change":"2024-02-22T19:30:21.682977+0000","last_active":"2024-02-22T19:33:42.156419+0000","last_peered":"2024-02-22T19:33:42.156419+0000","last_clean":"2024-02-22T19:33:42.156419+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:33:42.156419+0000","last_undegraded":"2024-02-22T19:33:42.156419+0000","last_fullsized":"2024-02-22T19:33:42.156419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.682837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.682837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":53,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.156804+0000","last_change":"2024-02-22T19:33:42.010538+0000","last_active":"2024-02-22T19:33:42.156804+0000","last_peered":"2024-02-22T19:33:42.156804+0000","last_clean":"2024-02-22T19:33:42.156804+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:33:42.156804+0000","last_undegraded":"2024-02-22T19:33:42.156804+0000","last_fullsized":"2024-02-22T19:33:42.156804+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.010467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.010467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.802849+0000","last_change":"2024-02-22T19:33:45.802849+0000","last_active":"2024-02-22T19:33:45.802849+0000","last_peered":"2024-02-22T19:33:45.802849+0000","last_clean":"2024-02-22T19:33:45.802849+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:33:45.802849+0000","last_undegraded":"2024-02-22T19:33:45.802849+0000","last_fullsized":"2024-02-22T19:33:45.802849+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.802756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.802756+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":107,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115015+0000","last_change":"2024-02-22T19:30:14.069609+0000","last_active":"2024-02-22T19:33:42.115015+0000","last_peered":"2024-02-22T19:33:42.115015+0000","last_clean":"2024-02-22T19:33:42.115015+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:33:42.115015+0000","last_undegraded":"2024-02-22T19:33:42.115015+0000","last_fullsized":"2024-02-22T19:33:42.115015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:14.069532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.069532+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112282+0000","last_change":"2024-02-22T19:31:50.005065+0000","last_active":"2024-02-22T19:33:42.112282+0000","last_peered":"2024-02-22T19:33:42.112282+0000","last_clean":"2024-02-22T19:33:42.112282+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:33:42.112282+0000","last_undegraded":"2024-02-22T19:33:42.112282+0000","last_fullsized":"2024-02-22T19:33:42.112282+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150114+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:33:42.150114+0000","last_peered":"2024-02-22T19:33:42.150114+0000","last_clean":"2024-02-22T19:33:42.150114+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:33:42.150114+0000","last_undegraded":"2024-02-22T19:33:42.150114+0000","last_fullsized":"2024-02-22T19:33:42.150114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":60,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114799+0000","last_change":"2024-02-22T19:31:50.439339+0000","last_active":"2024-02-22T19:33:42.114799+0000","last_peered":"2024-02-22T19:33:42.114799+0000","last_clean":"2024-02-22T19:33:42.114799+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:33:42.114799+0000","last_undegraded":"2024-02-22T19:33:42.114799+0000","last_fullsized":"2024-02-22T19:33:42.114799+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:31:23.683223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.683223+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":101,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115119+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:33:42.115119+0000","last_peered":"2024-02-22T19:33:42.115119+0000","last_clean":"2024-02-22T19:33:42.115119+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:33:42.115119+0000","last_undegraded":"2024-02-22T19:33:42.115119+0000","last_fullsized":"2024-02-22T19:33:42.115119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115659+0000","last_change":"2024-02-22T19:31:50.009328+0000","last_active":"2024-02-22T19:33:42.115659+0000","last_peered":"2024-02-22T19:33:42.115659+0000","last_clean":"2024-02-22T19:33:42.115659+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:33:42.115659+0000","last_undegraded":"2024-02-22T19:33:42.115659+0000","last_fullsized":"2024-02-22T19:33:42.115659+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113229+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:33:42.113229+0000","last_peered":"2024-02-22T19:33:42.113229+0000","last_clean":"2024-02-22T19:33:42.113229+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:33:42.113229+0000","last_undegraded":"2024-02-22T19:33:42.113229+0000","last_fullsized":"2024-02-22T19:33:42.113229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157087+0000","last_change":"2024-02-22T19:31:26.454221+0000","last_active":"2024-02-22T19:33:42.157087+0000","last_peered":"2024-02-22T19:33:42.157087+0000","last_clean":"2024-02-22T19:33:42.157087+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:33:42.157087+0000","last_undegraded":"2024-02-22T19:33:42.157087+0000","last_fullsized":"2024-02-22T19:33:42.157087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.454155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.454155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":116,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114182+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:33:42.114182+0000","last_peered":"2024-02-22T19:33:42.114182+0000","last_clean":"2024-02-22T19:33:42.114182+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:33:42.114182+0000","last_undegraded":"2024-02-22T19:33:42.114182+0000","last_fullsized":"2024-02-22T19:33:42.114182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":98,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.835132+0000","last_change":"2024-02-22T19:33:48.835132+0000","last_active":"2024-02-22T19:33:48.835132+0000","last_peered":"2024-02-22T19:33:48.835132+0000","last_clean":"2024-02-22T19:33:48.835132+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:33:48.835132+0000","last_undegraded":"2024-02-22T19:33:48.835132+0000","last_fullsized":"2024-02-22T19:33:48.835132+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.835030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.835030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114926+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:33:42.114926+0000","last_peered":"2024-02-22T19:33:42.114926+0000","last_clean":"2024-02-22T19:33:42.114926+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:33:42.114926+0000","last_undegraded":"2024-02-22T19:33:42.114926+0000","last_fullsized":"2024-02-22T19:33:42.114926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":93,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115102+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:33:42.115102+0000","last_peered":"2024-02-22T19:33:42.115102+0000","last_clean":"2024-02-22T19:33:42.115102+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:33:42.115102+0000","last_undegraded":"2024-02-22T19:33:42.115102+0000","last_fullsized":"2024-02-22T19:33:42.115102+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":116,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115594+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:33:42.115594+0000","last_peered":"2024-02-22T19:33:42.115594+0000","last_clean":"2024-02-22T19:33:42.115594+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:33:42.115594+0000","last_undegraded":"2024-02-22T19:33:42.115594+0000","last_fullsized":"2024-02-22T19:33:42.115594+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":99,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.151003+0000","last_change":"2024-02-22T19:31:50.095022+0000","last_active":"2024-02-22T19:33:42.151003+0000","last_peered":"2024-02-22T19:33:42.151003+0000","last_clean":"2024-02-22T19:33:42.151003+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:33:42.151003+0000","last_undegraded":"2024-02-22T19:33:42.151003+0000","last_fullsized":"2024-02-22T19:33:42.151003+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":158,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.148306+0000","last_change":"2024-02-22T19:30:16.686114+0000","last_active":"2024-02-22T19:33:45.148306+0000","last_peered":"2024-02-22T19:33:45.148306+0000","last_clean":"2024-02-22T19:33:45.148306+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:33:45.148306+0000","last_undegraded":"2024-02-22T19:33:45.148306+0000","last_fullsized":"2024-02-22T19:33:45.148306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":93,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112473+0000","last_change":"2024-02-22T19:30:06.976185+0000","last_active":"2024-02-22T19:33:42.112473+0000","last_peered":"2024-02-22T19:33:42.112473+0000","last_clean":"2024-02-22T19:33:42.112473+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:33:42.112473+0000","last_undegraded":"2024-02-22T19:33:42.112473+0000","last_fullsized":"2024-02-22T19:33:42.112473+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:06.976107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.976107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":104,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157858+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:33:42.157858+0000","last_peered":"2024-02-22T19:33:42.157858+0000","last_clean":"2024-02-22T19:33:42.157858+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:33:42.157858+0000","last_undegraded":"2024-02-22T19:33:42.157858+0000","last_fullsized":"2024-02-22T19:33:42.157858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":98,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113418+0000","last_change":"2024-02-22T19:31:50.009891+0000","last_active":"2024-02-22T19:33:42.113418+0000","last_peered":"2024-02-22T19:33:42.113418+0000","last_clean":"2024-02-22T19:33:42.113418+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:33:42.113418+0000","last_undegraded":"2024-02-22T19:33:42.113418+0000","last_fullsized":"2024-02-22T19:33:42.113418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.158159+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:33:42.158159+0000","last_peered":"2024-02-22T19:33:42.158159+0000","last_clean":"2024-02-22T19:33:42.158159+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:33:42.158159+0000","last_undegraded":"2024-02-22T19:33:42.158159+0000","last_fullsized":"2024-02-22T19:33:42.158159+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":90,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114940+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:33:42.114940+0000","last_peered":"2024-02-22T19:33:42.114940+0000","last_clean":"2024-02-22T19:33:42.114940+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:33:42.114940+0000","last_undegraded":"2024-02-22T19:33:42.114940+0000","last_fullsized":"2024-02-22T19:33:42.114940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":165,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.763313+0000","last_change":"2024-02-22T19:33:46.763313+0000","last_active":"2024-02-22T19:33:46.763313+0000","last_peered":"2024-02-22T19:33:46.763313+0000","last_clean":"2024-02-22T19:33:46.763313+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:33:46.763313+0000","last_undegraded":"2024-02-22T19:33:46.763313+0000","last_fullsized":"2024-02-22T19:33:46.763313+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:46.763202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.763202+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150037+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:33:42.150037+0000","last_peered":"2024-02-22T19:33:42.150037+0000","last_clean":"2024-02-22T19:33:42.150037+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:33:42.150037+0000","last_undegraded":"2024-02-22T19:33:42.150037+0000","last_fullsized":"2024-02-22T19:33:42.150037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":86,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.146874+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:33:42.146874+0000","last_peered":"2024-02-22T19:33:42.146874+0000","last_clean":"2024-02-22T19:33:42.146874+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:33:42.146874+0000","last_undegraded":"2024-02-22T19:33:42.146874+0000","last_fullsized":"2024-02-22T19:33:42.146874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":105,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:43.048479+0000","last_change":"2024-02-22T19:33:43.048434+0000","last_active":"2024-02-22T19:33:43.048479+0000","last_peered":"2024-02-22T19:33:43.048479+0000","last_clean":"2024-02-22T19:33:43.048479+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:33:43.048479+0000","last_undegraded":"2024-02-22T19:33:43.048479+0000","last_fullsized":"2024-02-22T19:33:43.048479+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:43.048330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.048330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":94,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114952+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:33:42.114952+0000","last_peered":"2024-02-22T19:33:42.114952+0000","last_clean":"2024-02-22T19:33:42.114952+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:33:42.114952+0000","last_undegraded":"2024-02-22T19:33:42.114952+0000","last_fullsized":"2024-02-22T19:33:42.114952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":84,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115227+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:33:42.115227+0000","last_peered":"2024-02-22T19:33:42.115227+0000","last_clean":"2024-02-22T19:33:42.115227+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:33:42.115227+0000","last_undegraded":"2024-02-22T19:33:42.115227+0000","last_fullsized":"2024-02-22T19:33:42.115227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150134+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:33:42.150134+0000","last_peered":"2024-02-22T19:33:42.150134+0000","last_clean":"2024-02-22T19:33:42.150134+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:33:42.150134+0000","last_undegraded":"2024-02-22T19:33:42.150134+0000","last_fullsized":"2024-02-22T19:33:42.150134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.528051+0000","last_change":"2024-02-22T19:33:45.528051+0000","last_active":"2024-02-22T19:33:45.528051+0000","last_peered":"2024-02-22T19:33:45.528051+0000","last_clean":"2024-02-22T19:33:45.528051+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:33:45.528051+0000","last_undegraded":"2024-02-22T19:33:45.528051+0000","last_fullsized":"2024-02-22T19:33:45.528051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.527956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.527956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":107,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115006+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:33:42.115006+0000","last_peered":"2024-02-22T19:33:42.115006+0000","last_clean":"2024-02-22T19:33:42.115006+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:33:42.115006+0000","last_undegraded":"2024-02-22T19:33:42.115006+0000","last_fullsized":"2024-02-22T19:33:42.115006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":94,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.156171+0000","last_change":"2024-02-22T19:31:50.014237+0000","last_active":"2024-02-22T19:33:42.156171+0000","last_peered":"2024-02-22T19:33:42.156171+0000","last_clean":"2024-02-22T19:33:42.156171+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:33:42.156171+0000","last_undegraded":"2024-02-22T19:33:42.156171+0000","last_fullsized":"2024-02-22T19:33:42.156171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":86,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115052+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:33:42.115052+0000","last_peered":"2024-02-22T19:33:42.115052+0000","last_clean":"2024-02-22T19:33:42.115052+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:33:42.115052+0000","last_undegraded":"2024-02-22T19:33:42.115052+0000","last_fullsized":"2024-02-22T19:33:42.115052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150248+0000","last_change":"2024-02-22T19:31:23.009097+0000","last_active":"2024-02-22T19:33:42.150248+0000","last_peered":"2024-02-22T19:33:42.150248+0000","last_clean":"2024-02-22T19:33:42.150248+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:33:42.150248+0000","last_undegraded":"2024-02-22T19:33:42.150248+0000","last_fullsized":"2024-02-22T19:33:42.150248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.009021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:47.757576+0000","last_change":"2024-02-22T19:33:47.757576+0000","last_active":"2024-02-22T19:33:47.757576+0000","last_peered":"2024-02-22T19:33:47.757576+0000","last_clean":"2024-02-22T19:33:47.757576+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:33:47.757576+0000","last_undegraded":"2024-02-22T19:33:47.757576+0000","last_fullsized":"2024-02-22T19:33:47.757576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:47.757479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:47.757479+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":98,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.862673+0000","last_change":"2024-02-22T19:33:42.862634+0000","last_active":"2024-02-22T19:33:42.862673+0000","last_peered":"2024-02-22T19:33:42.862673+0000","last_clean":"2024-02-22T19:33:42.862673+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:33:42.862673+0000","last_undegraded":"2024-02-22T19:33:42.862673+0000","last_fullsized":"2024-02-22T19:33:42.862673+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.862563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.862563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":98,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112535+0000","last_change":"2024-02-22T19:31:50.012381+0000","last_active":"2024-02-22T19:33:42.112535+0000","last_peered":"2024-02-22T19:33:42.112535+0000","last_clean":"2024-02-22T19:33:42.112535+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:33:42.112535+0000","last_undegraded":"2024-02-22T19:33:42.112535+0000","last_fullsized":"2024-02-22T19:33:42.112535+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":93,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114785+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:33:42.114785+0000","last_peered":"2024-02-22T19:33:42.114785+0000","last_clean":"2024-02-22T19:33:42.114785+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:33:42.114785+0000","last_undegraded":"2024-02-22T19:33:42.114785+0000","last_fullsized":"2024-02-22T19:33:42.114785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":110,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114955+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:33:42.114955+0000","last_peered":"2024-02-22T19:33:42.114955+0000","last_clean":"2024-02-22T19:33:42.114955+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:33:42.114955+0000","last_undegraded":"2024-02-22T19:33:42.114955+0000","last_fullsized":"2024-02-22T19:33:42.114955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157519+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:33:42.157519+0000","last_peered":"2024-02-22T19:33:42.157519+0000","last_clean":"2024-02-22T19:33:42.157519+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:33:42.157519+0000","last_undegraded":"2024-02-22T19:33:42.157519+0000","last_fullsized":"2024-02-22T19:33:42.157519+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":96,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155200+0000","last_change":"2024-02-22T19:31:50.105130+0000","last_active":"2024-02-22T19:33:45.155200+0000","last_peered":"2024-02-22T19:33:45.155200+0000","last_clean":"2024-02-22T19:33:45.155200+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:33:45.155200+0000","last_undegraded":"2024-02-22T19:33:45.155200+0000","last_fullsized":"2024-02-22T19:33:45.155200+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:29.679183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.679183+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":91,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150481+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:33:42.150481+0000","last_peered":"2024-02-22T19:33:42.150481+0000","last_clean":"2024-02-22T19:33:42.150481+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:33:42.150481+0000","last_undegraded":"2024-02-22T19:33:42.150481+0000","last_fullsized":"2024-02-22T19:33:42.150481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":108,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115438+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:33:42.115438+0000","last_peered":"2024-02-22T19:33:42.115438+0000","last_clean":"2024-02-22T19:33:42.115438+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:33:42.115438+0000","last_undegraded":"2024-02-22T19:33:42.115438+0000","last_fullsized":"2024-02-22T19:33:42.115438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116003+0000","last_change":"2024-02-22T19:31:50.016234+0000","last_active":"2024-02-22T19:33:42.116003+0000","last_peered":"2024-02-22T19:33:42.116003+0000","last_clean":"2024-02-22T19:33:42.116003+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:33:42.116003+0000","last_undegraded":"2024-02-22T19:33:42.116003+0000","last_fullsized":"2024-02-22T19:33:42.116003+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.057632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":152,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.178161+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:33:45.178161+0000","last_peered":"2024-02-22T19:33:45.178161+0000","last_clean":"2024-02-22T19:33:45.178161+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:33:45.178161+0000","last_undegraded":"2024-02-22T19:33:45.178161+0000","last_fullsized":"2024-02-22T19:33:45.178161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.156610+0000","last_change":"2024-02-22T19:31:11.415809+0000","last_active":"2024-02-22T19:33:42.156610+0000","last_peered":"2024-02-22T19:33:42.156610+0000","last_clean":"2024-02-22T19:33:42.156610+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:33:42.156610+0000","last_undegraded":"2024-02-22T19:33:42.156610+0000","last_fullsized":"2024-02-22T19:33:42.156610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.415718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":116,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150519+0000","last_change":"2024-02-22T19:31:50.352892+0000","last_active":"2024-02-22T19:33:42.150519+0000","last_peered":"2024-02-22T19:33:42.150519+0000","last_clean":"2024-02-22T19:33:42.150519+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:33:42.150519+0000","last_undegraded":"2024-02-22T19:33:42.150519+0000","last_fullsized":"2024-02-22T19:33:42.150519+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.685295+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114490+0000","last_change":"2024-02-22T19:30:13.178015+0000","last_active":"2024-02-22T19:33:42.114490+0000","last_peered":"2024-02-22T19:33:42.114490+0000","last_clean":"2024-02-22T19:33:42.114490+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:33:42.114490+0000","last_undegraded":"2024-02-22T19:33:42.114490+0000","last_fullsized":"2024-02-22T19:33:42.114490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.177942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.177942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":86,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116242+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:33:42.116242+0000","last_peered":"2024-02-22T19:33:42.116242+0000","last_clean":"2024-02-22T19:33:42.116242+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:33:42.116242+0000","last_undegraded":"2024-02-22T19:33:42.116242+0000","last_fullsized":"2024-02-22T19:33:42.116242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":91,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.149805+0000","last_change":"2024-02-22T19:30:22.592910+0000","last_active":"2024-02-22T19:33:42.149805+0000","last_peered":"2024-02-22T19:33:42.149805+0000","last_clean":"2024-02-22T19:33:42.149805+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:33:42.149805+0000","last_undegraded":"2024-02-22T19:33:42.149805+0000","last_fullsized":"2024-02-22T19:33:42.149805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.592774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":110,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.001358+0000","last_change":"2024-02-22T19:33:46.001358+0000","last_active":"2024-02-22T19:33:46.001358+0000","last_peered":"2024-02-22T19:33:46.001358+0000","last_clean":"2024-02-22T19:33:46.001358+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:33:46.001358+0000","last_undegraded":"2024-02-22T19:33:46.001358+0000","last_fullsized":"2024-02-22T19:33:46.001358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:46.001290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.001290+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150174+0000","last_change":"2024-02-22T19:30:13.625185+0000","last_active":"2024-02-22T19:33:42.150174+0000","last_peered":"2024-02-22T19:33:42.150174+0000","last_clean":"2024-02-22T19:33:42.150174+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:33:42.150174+0000","last_undegraded":"2024-02-22T19:33:42.150174+0000","last_fullsized":"2024-02-22T19:33:42.150174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:13.625119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:13.625119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116122+0000","last_change":"2024-02-22T19:31:50.013893+0000","last_active":"2024-02-22T19:33:42.116122+0000","last_peered":"2024-02-22T19:33:42.116122+0000","last_clean":"2024-02-22T19:33:42.116122+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:33:42.116122+0000","last_undegraded":"2024-02-22T19:33:42.116122+0000","last_fullsized":"2024-02-22T19:33:42.116122+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":101,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.146711+0000","last_change":"2024-02-22T19:31:50.006289+0000","last_active":"2024-02-22T19:33:42.146711+0000","last_peered":"2024-02-22T19:33:42.146711+0000","last_clean":"2024-02-22T19:33:42.146711+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:33:42.146711+0000","last_undegraded":"2024-02-22T19:33:42.146711+0000","last_fullsized":"2024-02-22T19:33:42.146711+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":97,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116365+0000","last_change":"2024-02-22T19:30:05.723159+0000","last_active":"2024-02-22T19:33:42.116365+0000","last_peered":"2024-02-22T19:33:42.116365+0000","last_clean":"2024-02-22T19:33:42.116365+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:33:42.116365+0000","last_undegraded":"2024-02-22T19:33:42.116365+0000","last_fullsized":"2024-02-22T19:33:42.116365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:30:05.723058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:05.723058+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"136'7449","reported_seq":10068,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.246850+0000","last_change":"2024-02-22T19:32:06.969062+0000","last_active":"2024-02-22T19:33:49.246850+0000","last_peered":"2024-02-22T19:33:49.246850+0000","last_clean":"2024-02-22T19:33:49.246850+0000","last_became_active":"2024-02-22T19:32:06.966453+0000","last_became_peered":"2024-02-22T19:32:06.966453+0000","last_unstale":"2024-02-22T19:33:49.246850+0000","last_undegraded":"2024-02-22T19:33:49.246850+0000","last_fullsized":"2024-02-22T19:33:49.246850+0000","mapping_epoch":104,"log_start":"108'2900","ondisk_log_start":"108'2900","created":93,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4549,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":243466240,"num_objects":3715,"num_object_clones":0,"num_object_copies":11145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7449,"num_write_kb":357248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.147171+0000","last_change":"2024-02-22T19:31:50.006727+0000","last_active":"2024-02-22T19:33:42.147171+0000","last_peered":"2024-02-22T19:33:42.147171+0000","last_clean":"2024-02-22T19:33:42.147171+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:33:42.147171+0000","last_undegraded":"2024-02-22T19:33:42.147171+0000","last_fullsized":"2024-02-22T19:33:42.147171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.461857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.461857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":106,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.151370+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:33:42.151370+0000","last_peered":"2024-02-22T19:33:42.151370+0000","last_clean":"2024-02-22T19:33:42.151370+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:33:42.151370+0000","last_undegraded":"2024-02-22T19:33:42.151370+0000","last_fullsized":"2024-02-22T19:33:42.151370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":101,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.149987+0000","last_change":"2024-02-22T19:31:50.611447+0000","last_active":"2024-02-22T19:33:42.149987+0000","last_peered":"2024-02-22T19:33:42.149987+0000","last_clean":"2024-02-22T19:33:42.149987+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:33:42.149987+0000","last_undegraded":"2024-02-22T19:33:42.149987+0000","last_fullsized":"2024-02-22T19:33:42.149987+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":153,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.151504+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:33:45.151504+0000","last_peered":"2024-02-22T19:33:45.151504+0000","last_clean":"2024-02-22T19:33:45.151504+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:33:45.151504+0000","last_undegraded":"2024-02-22T19:33:45.151504+0000","last_fullsized":"2024-02-22T19:33:45.151504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"136'7405","reported_seq":10023,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.229302+0000","last_change":"2024-02-22T19:33:42.253020+0000","last_active":"2024-02-22T19:33:49.229302+0000","last_peered":"2024-02-22T19:33:49.229302+0000","last_clean":"2024-02-22T19:33:49.229302+0000","last_became_active":"2024-02-22T19:33:42.130070+0000","last_became_peered":"2024-02-22T19:33:42.130070+0000","last_unstale":"2024-02-22T19:33:49.229302+0000","last_undegraded":"2024-02-22T19:33:49.229302+0000","last_fullsized":"2024-02-22T19:33:49.229302+0000","mapping_epoch":134,"log_start":"111'3200","ondisk_log_start":"111'3200","created":93,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4205,"ondisk_log_size":4205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":242548736,"num_objects":3701,"num_object_clones":0,"num_object_copies":11103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7405,"num_write_kb":355392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1273,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":84,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115163+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:33:42.115163+0000","last_peered":"2024-02-22T19:33:42.115163+0000","last_clean":"2024-02-22T19:33:42.115163+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:33:42.115163+0000","last_undegraded":"2024-02-22T19:33:42.115163+0000","last_fullsized":"2024-02-22T19:33:42.115163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":91,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.151011+0000","last_change":"2024-02-22T19:30:31.557520+0000","last_active":"2024-02-22T19:33:42.151011+0000","last_peered":"2024-02-22T19:33:42.151011+0000","last_clean":"2024-02-22T19:33:42.151011+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:33:42.151011+0000","last_undegraded":"2024-02-22T19:33:42.151011+0000","last_fullsized":"2024-02-22T19:33:42.151011+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.557421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.557421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":114,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115666+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:33:42.115666+0000","last_peered":"2024-02-22T19:33:42.115666+0000","last_clean":"2024-02-22T19:33:42.115666+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:33:42.115666+0000","last_undegraded":"2024-02-22T19:33:42.115666+0000","last_fullsized":"2024-02-22T19:33:42.115666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113818+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:33:42.113818+0000","last_peered":"2024-02-22T19:33:42.113818+0000","last_clean":"2024-02-22T19:33:42.113818+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:33:42.113818+0000","last_undegraded":"2024-02-22T19:33:42.113818+0000","last_fullsized":"2024-02-22T19:33:42.113818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"136'7521","reported_seq":7572,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.270581+0000","last_change":"2024-02-22T19:31:33.014127+0000","last_active":"2024-02-22T19:33:49.270581+0000","last_peered":"2024-02-22T19:33:49.270581+0000","last_clean":"2024-02-22T19:33:49.270581+0000","last_became_active":"2024-02-22T19:31:33.013760+0000","last_became_peered":"2024-02-22T19:31:33.013760+0000","last_unstale":"2024-02-22T19:33:49.270581+0000","last_undegraded":"2024-02-22T19:33:49.270581+0000","last_fullsized":"2024-02-22T19:33:49.270581+0000","mapping_epoch":93,"log_start":"106'2600","ondisk_log_start":"106'2600","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4921,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":244514816,"num_objects":3731,"num_object_clones":0,"num_object_copies":11193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7521,"num_write_kb":360064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150667+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:33:42.150667+0000","last_peered":"2024-02-22T19:33:42.150667+0000","last_clean":"2024-02-22T19:33:42.150667+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:33:42.150667+0000","last_undegraded":"2024-02-22T19:33:42.150667+0000","last_fullsized":"2024-02-22T19:33:42.150667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.766810+0000","last_change":"2024-02-22T19:33:48.766810+0000","last_active":"2024-02-22T19:33:48.766810+0000","last_peered":"2024-02-22T19:33:48.766810+0000","last_clean":"2024-02-22T19:33:48.766810+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:33:48.766810+0000","last_undegraded":"2024-02-22T19:33:48.766810+0000","last_fullsized":"2024-02-22T19:33:48.766810+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.766706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.766706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":105,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150437+0000","last_change":"2024-02-22T19:31:49.999189+0000","last_active":"2024-02-22T19:33:42.150437+0000","last_peered":"2024-02-22T19:33:42.150437+0000","last_clean":"2024-02-22T19:33:42.150437+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:33:42.150437+0000","last_undegraded":"2024-02-22T19:33:42.150437+0000","last_fullsized":"2024-02-22T19:33:42.150437+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:15.882992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.882992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":119,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.147022+0000","last_change":"2024-02-22T19:30:20.713429+0000","last_active":"2024-02-22T19:33:42.147022+0000","last_peered":"2024-02-22T19:33:42.147022+0000","last_clean":"2024-02-22T19:33:42.147022+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:33:42.147022+0000","last_undegraded":"2024-02-22T19:33:42.147022+0000","last_fullsized":"2024-02-22T19:33:42.147022+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:20.713330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:20.713330+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"136'7390","reported_seq":7441,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.230746+0000","last_change":"2024-02-22T19:31:33.015434+0000","last_active":"2024-02-22T19:33:49.230746+0000","last_peered":"2024-02-22T19:33:49.230746+0000","last_clean":"2024-02-22T19:33:49.230746+0000","last_became_active":"2024-02-22T19:31:33.015132+0000","last_became_peered":"2024-02-22T19:31:33.015132+0000","last_unstale":"2024-02-22T19:33:49.230746+0000","last_undegraded":"2024-02-22T19:33:49.230746+0000","last_fullsized":"2024-02-22T19:33:49.230746+0000","mapping_epoch":93,"log_start":"102'2100","ondisk_log_start":"102'2100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5290,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":239206400,"num_objects":3650,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7390,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.156543+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:33:42.156543+0000","last_peered":"2024-02-22T19:33:42.156543+0000","last_clean":"2024-02-22T19:33:42.156543+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:33:42.156543+0000","last_undegraded":"2024-02-22T19:33:42.156543+0000","last_fullsized":"2024-02-22T19:33:42.156543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113581+0000","last_change":"2024-02-22T19:30:39.210227+0000","last_active":"2024-02-22T19:33:42.113581+0000","last_peered":"2024-02-22T19:33:42.113581+0000","last_clean":"2024-02-22T19:33:42.113581+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:33:42.113581+0000","last_undegraded":"2024-02-22T19:33:42.113581+0000","last_fullsized":"2024-02-22T19:33:42.113581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.210162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.210162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115457+0000","last_change":"2024-02-22T19:31:50.014265+0000","last_active":"2024-02-22T19:33:42.115457+0000","last_peered":"2024-02-22T19:33:42.115457+0000","last_clean":"2024-02-22T19:33:42.115457+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:33:42.115457+0000","last_undegraded":"2024-02-22T19:33:42.115457+0000","last_fullsized":"2024-02-22T19:33:42.115457+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":124,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157224+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:33:42.157224+0000","last_peered":"2024-02-22T19:33:42.157224+0000","last_clean":"2024-02-22T19:33:42.157224+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:33:42.157224+0000","last_undegraded":"2024-02-22T19:33:42.157224+0000","last_fullsized":"2024-02-22T19:33:42.157224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"136'7385","reported_seq":7436,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.197459+0000","last_change":"2024-02-22T19:31:33.014009+0000","last_active":"2024-02-22T19:33:49.197459+0000","last_peered":"2024-02-22T19:33:49.197459+0000","last_clean":"2024-02-22T19:33:49.197459+0000","last_became_active":"2024-02-22T19:31:33.013715+0000","last_became_peered":"2024-02-22T19:31:33.013715+0000","last_unstale":"2024-02-22T19:33:49.197459+0000","last_undegraded":"2024-02-22T19:33:49.197459+0000","last_fullsized":"2024-02-22T19:33:49.197459+0000","mapping_epoch":93,"log_start":"102'2000","ondisk_log_start":"102'2000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5385,"ondisk_log_size":5385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":239534080,"num_objects":3655,"num_object_clones":0,"num_object_copies":10965,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7385,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":84,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116277+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:33:42.116277+0000","last_peered":"2024-02-22T19:33:42.116277+0000","last_clean":"2024-02-22T19:33:42.116277+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:33:42.116277+0000","last_undegraded":"2024-02-22T19:33:42.116277+0000","last_fullsized":"2024-02-22T19:33:42.116277+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112309+0000","last_change":"2024-02-22T19:31:50.010144+0000","last_active":"2024-02-22T19:33:42.112309+0000","last_peered":"2024-02-22T19:33:42.112309+0000","last_clean":"2024-02-22T19:33:42.112309+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:33:42.112309+0000","last_undegraded":"2024-02-22T19:33:42.112309+0000","last_fullsized":"2024-02-22T19:33:42.112309+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.692694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":93,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114801+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:33:42.114801+0000","last_peered":"2024-02-22T19:33:42.114801+0000","last_clean":"2024-02-22T19:33:42.114801+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:33:42.114801+0000","last_undegraded":"2024-02-22T19:33:42.114801+0000","last_fullsized":"2024-02-22T19:33:42.114801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":202,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.724609+0000","last_change":"2024-02-22T19:33:46.724569+0000","last_active":"2024-02-22T19:33:46.724609+0000","last_peered":"2024-02-22T19:33:46.724609+0000","last_clean":"2024-02-22T19:33:46.724609+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:33:46.724609+0000","last_undegraded":"2024-02-22T19:33:46.724609+0000","last_fullsized":"2024-02-22T19:33:46.724609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:33:46.724462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.724462+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157564+0000","last_change":"2024-02-22T19:30:23.636614+0000","last_active":"2024-02-22T19:33:42.157564+0000","last_peered":"2024-02-22T19:33:42.157564+0000","last_clean":"2024-02-22T19:33:42.157564+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:33:42.157564+0000","last_undegraded":"2024-02-22T19:33:42.157564+0000","last_fullsized":"2024-02-22T19:33:42.157564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.636547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.636547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"136'7341","reported_seq":7392,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.254478+0000","last_change":"2024-02-22T19:31:33.008987+0000","last_active":"2024-02-22T19:33:49.254478+0000","last_peered":"2024-02-22T19:33:49.254478+0000","last_clean":"2024-02-22T19:33:49.254478+0000","last_became_active":"2024-02-22T19:31:33.008726+0000","last_became_peered":"2024-02-22T19:31:33.008726+0000","last_unstale":"2024-02-22T19:33:49.254478+0000","last_undegraded":"2024-02-22T19:33:49.254478+0000","last_fullsized":"2024-02-22T19:33:49.254478+0000","mapping_epoch":93,"log_start":"106'2500","ondisk_log_start":"106'2500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4841,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":242155520,"num_objects":3695,"num_object_clones":0,"num_object_copies":11085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":353152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150598+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:33:42.150598+0000","last_peered":"2024-02-22T19:33:42.150598+0000","last_clean":"2024-02-22T19:33:42.150598+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:33:42.150598+0000","last_undegraded":"2024-02-22T19:33:42.150598+0000","last_fullsized":"2024-02-22T19:33:42.150598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":93,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112012+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:33:42.112012+0000","last_peered":"2024-02-22T19:33:42.112012+0000","last_clean":"2024-02-22T19:33:42.112012+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:33:42.112012+0000","last_undegraded":"2024-02-22T19:33:42.112012+0000","last_fullsized":"2024-02-22T19:33:42.112012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:44.828699+0000","last_change":"2024-02-22T19:33:44.828664+0000","last_active":"2024-02-22T19:33:44.828699+0000","last_peered":"2024-02-22T19:33:44.828699+0000","last_clean":"2024-02-22T19:33:44.828699+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:33:44.828699+0000","last_undegraded":"2024-02-22T19:33:44.828699+0000","last_fullsized":"2024-02-22T19:33:44.828699+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:44.828568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.828568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":100,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114530+0000","last_change":"2024-02-22T19:31:50.458765+0000","last_active":"2024-02-22T19:33:42.114530+0000","last_peered":"2024-02-22T19:33:42.114530+0000","last_clean":"2024-02-22T19:33:42.114530+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:33:42.114530+0000","last_undegraded":"2024-02-22T19:33:42.114530+0000","last_fullsized":"2024-02-22T19:33:42.114530+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.825747+0000","last_change":"2024-02-22T19:33:46.825747+0000","last_active":"2024-02-22T19:33:46.825747+0000","last_peered":"2024-02-22T19:33:46.825747+0000","last_clean":"2024-02-22T19:33:46.825747+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:33:46.825747+0000","last_undegraded":"2024-02-22T19:33:46.825747+0000","last_fullsized":"2024-02-22T19:33:46.825747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:46.825679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.825679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"136'7433","reported_seq":7484,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.283625+0000","last_change":"2024-02-22T19:31:33.014815+0000","last_active":"2024-02-22T19:33:49.283625+0000","last_peered":"2024-02-22T19:33:49.283625+0000","last_clean":"2024-02-22T19:33:49.283625+0000","last_became_active":"2024-02-22T19:31:33.014489+0000","last_became_peered":"2024-02-22T19:31:33.014489+0000","last_unstale":"2024-02-22T19:33:49.283625+0000","last_undegraded":"2024-02-22T19:33:49.283625+0000","last_fullsized":"2024-02-22T19:33:49.283625+0000","mapping_epoch":93,"log_start":"106'2500","ondisk_log_start":"106'2500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4933,"ondisk_log_size":4933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":244252672,"num_objects":3727,"num_object_clones":0,"num_object_copies":11181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7433,"num_write_kb":357120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116221+0000","last_change":"2024-02-22T19:31:50.014877+0000","last_active":"2024-02-22T19:33:42.116221+0000","last_peered":"2024-02-22T19:33:42.116221+0000","last_clean":"2024-02-22T19:33:42.116221+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:33:42.116221+0000","last_undegraded":"2024-02-22T19:33:42.116221+0000","last_fullsized":"2024-02-22T19:33:42.116221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.149817+0000","last_change":"2024-02-22T19:31:11.906662+0000","last_active":"2024-02-22T19:33:42.149817+0000","last_peered":"2024-02-22T19:33:42.149817+0000","last_clean":"2024-02-22T19:33:42.149817+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:33:42.149817+0000","last_undegraded":"2024-02-22T19:33:42.149817+0000","last_fullsized":"2024-02-22T19:33:42.149817+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.906561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.906561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":99,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.123325+0000","last_change":"2024-02-22T19:31:50.009641+0000","last_active":"2024-02-22T19:33:42.123325+0000","last_peered":"2024-02-22T19:33:42.123325+0000","last_clean":"2024-02-22T19:33:42.123325+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:33:42.123325+0000","last_undegraded":"2024-02-22T19:33:42.123325+0000","last_fullsized":"2024-02-22T19:33:42.123325+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":117,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.147505+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:33:42.147505+0000","last_peered":"2024-02-22T19:33:42.147505+0000","last_clean":"2024-02-22T19:33:42.147505+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:33:42.147505+0000","last_undegraded":"2024-02-22T19:33:42.147505+0000","last_fullsized":"2024-02-22T19:33:42.147505+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.151027+0000","last_change":"2024-02-22T19:31:49.995726+0000","last_active":"2024-02-22T19:33:42.151027+0000","last_peered":"2024-02-22T19:33:42.151027+0000","last_clean":"2024-02-22T19:33:42.151027+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:33:42.151027+0000","last_undegraded":"2024-02-22T19:33:42.151027+0000","last_fullsized":"2024-02-22T19:33:42.151027+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"136'7443","reported_seq":7492,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.182982+0000","last_change":"2024-02-22T19:31:33.010431+0000","last_active":"2024-02-22T19:33:49.182982+0000","last_peered":"2024-02-22T19:33:49.182982+0000","last_clean":"2024-02-22T19:33:49.182982+0000","last_became_active":"2024-02-22T19:31:33.010187+0000","last_became_peered":"2024-02-22T19:31:33.010187+0000","last_unstale":"2024-02-22T19:33:49.182982+0000","last_undegraded":"2024-02-22T19:33:49.182982+0000","last_fullsized":"2024-02-22T19:33:49.182982+0000","mapping_epoch":93,"log_start":"106'2600","ondisk_log_start":"106'2600","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4843,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":242417664,"num_objects":3699,"num_object_clones":0,"num_object_copies":11097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7443,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":96,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113631+0000","last_change":"2024-02-22T19:31:50.012369+0000","last_active":"2024-02-22T19:33:42.113631+0000","last_peered":"2024-02-22T19:33:42.113631+0000","last_clean":"2024-02-22T19:33:42.113631+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:33:42.113631+0000","last_undegraded":"2024-02-22T19:33:42.113631+0000","last_fullsized":"2024-02-22T19:33:42.113631+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":91,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.156340+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:33:42.156340+0000","last_peered":"2024-02-22T19:33:42.156340+0000","last_clean":"2024-02-22T19:33:42.156340+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:33:42.156340+0000","last_undegraded":"2024-02-22T19:33:42.156340+0000","last_fullsized":"2024-02-22T19:33:42.156340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":381,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:47.574144+0000","last_change":"2024-02-22T19:31:29.001184+0000","last_active":"2024-02-22T19:33:47.574144+0000","last_peered":"2024-02-22T19:33:47.574144+0000","last_clean":"2024-02-22T19:33:47.574144+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:33:47.574144+0000","last_undegraded":"2024-02-22T19:33:47.574144+0000","last_fullsized":"2024-02-22T19:33:47.574144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:31:29.001098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.001098+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":277,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":107,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157622+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:33:42.157622+0000","last_peered":"2024-02-22T19:33:42.157622+0000","last_clean":"2024-02-22T19:33:42.157622+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:33:42.157622+0000","last_undegraded":"2024-02-22T19:33:42.157622+0000","last_fullsized":"2024-02-22T19:33:42.157622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112618+0000","last_change":"2024-02-22T19:31:50.010171+0000","last_active":"2024-02-22T19:33:42.112618+0000","last_peered":"2024-02-22T19:33:42.112618+0000","last_clean":"2024-02-22T19:33:42.112618+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:33:42.112618+0000","last_undegraded":"2024-02-22T19:33:42.112618+0000","last_fullsized":"2024-02-22T19:33:42.112618+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"136'7453","reported_seq":10148,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.198408+0000","last_change":"2024-02-22T19:32:07.977104+0000","last_active":"2024-02-22T19:33:49.198408+0000","last_peered":"2024-02-22T19:33:49.198408+0000","last_clean":"2024-02-22T19:33:49.198408+0000","last_became_active":"2024-02-22T19:32:07.973466+0000","last_became_peered":"2024-02-22T19:32:07.973466+0000","last_unstale":"2024-02-22T19:33:49.198408+0000","last_undegraded":"2024-02-22T19:33:49.198408+0000","last_fullsized":"2024-02-22T19:33:49.198408+0000","mapping_epoch":105,"log_start":"108'2900","ondisk_log_start":"108'2900","created":93,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4553,"ondisk_log_size":4553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":241762304,"num_objects":3689,"num_object_clones":0,"num_object_copies":11067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":86048768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112363+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:33:42.112363+0000","last_peered":"2024-02-22T19:33:42.112363+0000","last_clean":"2024-02-22T19:33:42.112363+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:33:42.112363+0000","last_undegraded":"2024-02-22T19:33:42.112363+0000","last_fullsized":"2024-02-22T19:33:42.112363+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":102,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113828+0000","last_change":"2024-02-22T19:31:50.010143+0000","last_active":"2024-02-22T19:33:42.113828+0000","last_peered":"2024-02-22T19:33:42.113828+0000","last_clean":"2024-02-22T19:33:42.113828+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:33:42.113828+0000","last_undegraded":"2024-02-22T19:33:42.113828+0000","last_fullsized":"2024-02-22T19:33:42.113828+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":152,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157011+0000","last_change":"2024-02-22T19:31:50.036817+0000","last_active":"2024-02-22T19:33:42.157011+0000","last_peered":"2024-02-22T19:33:42.157011+0000","last_clean":"2024-02-22T19:33:42.157011+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:33:42.157011+0000","last_undegraded":"2024-02-22T19:33:42.157011+0000","last_fullsized":"2024-02-22T19:33:42.157011+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":97,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.147073+0000","last_change":"2024-02-22T19:31:50.006906+0000","last_active":"2024-02-22T19:33:42.147073+0000","last_peered":"2024-02-22T19:33:42.147073+0000","last_clean":"2024-02-22T19:33:42.147073+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:33:42.147073+0000","last_undegraded":"2024-02-22T19:33:42.147073+0000","last_fullsized":"2024-02-22T19:33:42.147073+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"136'7286","reported_seq":7337,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.186443+0000","last_change":"2024-02-22T19:31:33.012064+0000","last_active":"2024-02-22T19:33:49.186443+0000","last_peered":"2024-02-22T19:33:49.186443+0000","last_clean":"2024-02-22T19:33:49.186443+0000","last_became_active":"2024-02-22T19:31:33.011750+0000","last_became_peered":"2024-02-22T19:31:33.011750+0000","last_unstale":"2024-02-22T19:33:49.186443+0000","last_undegraded":"2024-02-22T19:33:49.186443+0000","last_fullsized":"2024-02-22T19:33:49.186443+0000","mapping_epoch":93,"log_start":"108'2800","ondisk_log_start":"108'2800","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4486,"ondisk_log_size":4486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":247201792,"num_objects":3772,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":353856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.035679+0000","last_change":"2024-02-22T19:33:49.035679+0000","last_active":"2024-02-22T19:33:49.035679+0000","last_peered":"2024-02-22T19:33:49.035679+0000","last_clean":"2024-02-22T19:33:49.035679+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:33:49.035679+0000","last_undegraded":"2024-02-22T19:33:49.035679+0000","last_fullsized":"2024-02-22T19:33:49.035679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.035615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.035615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115521+0000","last_change":"2024-02-22T19:31:50.006662+0000","last_active":"2024-02-22T19:33:42.115521+0000","last_peered":"2024-02-22T19:33:42.115521+0000","last_clean":"2024-02-22T19:33:42.115521+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:33:42.115521+0000","last_undegraded":"2024-02-22T19:33:42.115521+0000","last_fullsized":"2024-02-22T19:33:42.115521+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":132,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:44.757272+0000","last_change":"2024-02-22T19:33:44.757231+0000","last_active":"2024-02-22T19:33:44.757272+0000","last_peered":"2024-02-22T19:33:44.757272+0000","last_clean":"2024-02-22T19:33:44.757272+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:33:44.757272+0000","last_undegraded":"2024-02-22T19:33:44.757272+0000","last_fullsized":"2024-02-22T19:33:44.757272+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:33:44.757114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.757114+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":166,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172556+0000","last_change":"2024-02-22T19:31:50.440080+0000","last_active":"2024-02-22T19:33:45.172556+0000","last_peered":"2024-02-22T19:33:45.172556+0000","last_clean":"2024-02-22T19:33:45.172556+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:33:45.172556+0000","last_undegraded":"2024-02-22T19:33:45.172556+0000","last_fullsized":"2024-02-22T19:33:45.172556+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"136'7400","reported_seq":7451,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.259235+0000","last_change":"2024-02-22T19:31:33.012817+0000","last_active":"2024-02-22T19:33:49.259235+0000","last_peered":"2024-02-22T19:33:49.259235+0000","last_clean":"2024-02-22T19:33:49.259235+0000","last_became_active":"2024-02-22T19:31:33.012470+0000","last_became_peered":"2024-02-22T19:31:33.012470+0000","last_unstale":"2024-02-22T19:33:49.259235+0000","last_undegraded":"2024-02-22T19:33:49.259235+0000","last_fullsized":"2024-02-22T19:33:49.259235+0000","mapping_epoch":93,"log_start":"111'3200","ondisk_log_start":"111'3200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4200,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":246546432,"num_objects":3762,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7400,"num_write_kb":357184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":85,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.150757+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:33:42.150757+0000","last_peered":"2024-02-22T19:33:42.150757+0000","last_clean":"2024-02-22T19:33:42.150757+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:33:42.150757+0000","last_undegraded":"2024-02-22T19:33:42.150757+0000","last_fullsized":"2024-02-22T19:33:42.150757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":91,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.157705+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:33:42.157705+0000","last_peered":"2024-02-22T19:33:42.157705+0000","last_clean":"2024-02-22T19:33:42.157705+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:33:42.157705+0000","last_undegraded":"2024-02-22T19:33:42.157705+0000","last_fullsized":"2024-02-22T19:33:42.157705+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":144,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115568+0000","last_change":"2024-02-22T19:31:50.644540+0000","last_active":"2024-02-22T19:33:42.115568+0000","last_peered":"2024-02-22T19:33:42.115568+0000","last_clean":"2024-02-22T19:33:42.115568+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:33:42.115568+0000","last_undegraded":"2024-02-22T19:33:42.115568+0000","last_fullsized":"2024-02-22T19:33:42.115568+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":152,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173678+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:33:45.173678+0000","last_peered":"2024-02-22T19:33:45.173678+0000","last_clean":"2024-02-22T19:33:45.173678+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:33:45.173678+0000","last_undegraded":"2024-02-22T19:33:45.173678+0000","last_fullsized":"2024-02-22T19:33:45.173678+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"136'7643","reported_seq":7694,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.271667+0000","last_change":"2024-02-22T19:31:33.013118+0000","last_active":"2024-02-22T19:33:49.271667+0000","last_peered":"2024-02-22T19:33:49.271667+0000","last_clean":"2024-02-22T19:33:49.271667+0000","last_became_active":"2024-02-22T19:31:33.012002+0000","last_became_peered":"2024-02-22T19:31:33.012002+0000","last_unstale":"2024-02-22T19:33:49.271667+0000","last_undegraded":"2024-02-22T19:33:49.271667+0000","last_fullsized":"2024-02-22T19:33:49.271667+0000","mapping_epoch":93,"log_start":"113'3500","ondisk_log_start":"113'3500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4143,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":244908032,"num_objects":3737,"num_object_clones":0,"num_object_copies":11211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7643,"num_write_kb":364160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114847+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:33:42.114847+0000","last_peered":"2024-02-22T19:33:42.114847+0000","last_clean":"2024-02-22T19:33:42.114847+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:33:42.114847+0000","last_undegraded":"2024-02-22T19:33:42.114847+0000","last_fullsized":"2024-02-22T19:33:42.114847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":90,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.116124+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:33:42.116124+0000","last_peered":"2024-02-22T19:33:42.116124+0000","last_clean":"2024-02-22T19:33:42.116124+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:33:42.116124+0000","last_undegraded":"2024-02-22T19:33:42.116124+0000","last_fullsized":"2024-02-22T19:33:42.116124+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":120,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115270+0000","last_change":"2024-02-22T19:30:11.662827+0000","last_active":"2024-02-22T19:33:42.115270+0000","last_peered":"2024-02-22T19:33:42.115270+0000","last_clean":"2024-02-22T19:33:42.115270+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:33:42.115270+0000","last_undegraded":"2024-02-22T19:33:42.115270+0000","last_fullsized":"2024-02-22T19:33:42.115270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:30:11.662740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:11.662740+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":94,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115454+0000","last_change":"2024-02-22T19:31:26.190739+0000","last_active":"2024-02-22T19:33:42.115454+0000","last_peered":"2024-02-22T19:33:42.115454+0000","last_clean":"2024-02-22T19:33:42.115454+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:33:42.115454+0000","last_undegraded":"2024-02-22T19:33:42.115454+0000","last_fullsized":"2024-02-22T19:33:42.115454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.190632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.190632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"136'7486","reported_seq":7537,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.197482+0000","last_change":"2024-02-22T19:31:33.012589+0000","last_active":"2024-02-22T19:33:49.197482+0000","last_peered":"2024-02-22T19:33:49.197482+0000","last_clean":"2024-02-22T19:33:49.197482+0000","last_became_active":"2024-02-22T19:31:33.012188+0000","last_became_peered":"2024-02-22T19:31:33.012188+0000","last_unstale":"2024-02-22T19:33:49.197482+0000","last_undegraded":"2024-02-22T19:33:49.197482+0000","last_fullsized":"2024-02-22T19:33:49.197482+0000","mapping_epoch":93,"log_start":"102'2100","ondisk_log_start":"102'2100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5386,"ondisk_log_size":5386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":249954304,"num_objects":3814,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7486,"num_write_kb":361600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.112941+0000","last_change":"2024-02-22T19:31:50.009247+0000","last_active":"2024-02-22T19:33:42.112941+0000","last_peered":"2024-02-22T19:33:42.112941+0000","last_clean":"2024-02-22T19:33:42.112941+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:33:42.112941+0000","last_undegraded":"2024-02-22T19:33:42.112941+0000","last_fullsized":"2024-02-22T19:33:42.112941+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:25.682652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:25.682652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":168,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175697+0000","last_change":"2024-02-22T19:31:50.016299+0000","last_active":"2024-02-22T19:33:45.175697+0000","last_peered":"2024-02-22T19:33:45.175697+0000","last_clean":"2024-02-22T19:33:45.175697+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:33:45.175697+0000","last_undegraded":"2024-02-22T19:33:45.175697+0000","last_fullsized":"2024-02-22T19:33:45.175697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":95,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115046+0000","last_change":"2024-02-22T19:31:14.073980+0000","last_active":"2024-02-22T19:33:42.115046+0000","last_peered":"2024-02-22T19:33:42.115046+0000","last_clean":"2024-02-22T19:33:42.115046+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:33:42.115046+0000","last_undegraded":"2024-02-22T19:33:42.115046+0000","last_fullsized":"2024-02-22T19:33:42.115046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.073908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.073908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":88,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:43.827229+0000","last_change":"2024-02-22T19:33:43.827188+0000","last_active":"2024-02-22T19:33:43.827229+0000","last_peered":"2024-02-22T19:33:43.827229+0000","last_clean":"2024-02-22T19:33:43.827229+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:33:43.827229+0000","last_undegraded":"2024-02-22T19:33:43.827229+0000","last_fullsized":"2024-02-22T19:33:43.827229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:33:43.827087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.827087+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"136'7651","reported_seq":7703,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.158432+0000","last_change":"2024-02-22T19:31:33.011737+0000","last_active":"2024-02-22T19:33:49.158432+0000","last_peered":"2024-02-22T19:33:49.158432+0000","last_clean":"2024-02-22T19:33:49.158432+0000","last_became_active":"2024-02-22T19:31:33.011438+0000","last_became_peered":"2024-02-22T19:31:33.011438+0000","last_unstale":"2024-02-22T19:33:49.158432+0000","last_undegraded":"2024-02-22T19:33:49.158432+0000","last_fullsized":"2024-02-22T19:33:49.158432+0000","mapping_epoch":93,"log_start":"102'2200","ondisk_log_start":"102'2200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5451,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":247857176,"num_objects":3783,"num_object_clones":0,"num_object_copies":11349,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7651,"num_write_kb":365825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":93,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115358+0000","last_change":"2024-02-22T19:31:50.006646+0000","last_active":"2024-02-22T19:33:42.115358+0000","last_peered":"2024-02-22T19:33:42.115358+0000","last_clean":"2024-02-22T19:33:42.115358+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:33:42.115358+0000","last_undegraded":"2024-02-22T19:33:42.115358+0000","last_fullsized":"2024-02-22T19:33:42.115358+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":48,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115952+0000","last_change":"2024-02-22T19:31:50.016416+0000","last_active":"2024-02-22T19:33:42.115952+0000","last_peered":"2024-02-22T19:33:42.115952+0000","last_clean":"2024-02-22T19:33:42.115952+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:33:42.115952+0000","last_undegraded":"2024-02-22T19:33:42.115952+0000","last_fullsized":"2024-02-22T19:33:42.115952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":110,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.898875+0000","last_change":"2024-02-22T19:33:48.898875+0000","last_active":"2024-02-22T19:33:48.898875+0000","last_peered":"2024-02-22T19:33:48.898875+0000","last_clean":"2024-02-22T19:33:48.898875+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:33:48.898875+0000","last_undegraded":"2024-02-22T19:33:48.898875+0000","last_fullsized":"2024-02-22T19:33:48.898875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.898801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.898801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":110,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115609+0000","last_change":"2024-02-22T19:30:06.957298+0000","last_active":"2024-02-22T19:33:42.115609+0000","last_peered":"2024-02-22T19:33:42.115609+0000","last_clean":"2024-02-22T19:33:42.115609+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:33:42.115609+0000","last_undegraded":"2024-02-22T19:33:42.115609+0000","last_fullsized":"2024-02-22T19:33:42.115609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:06.957233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:06.957233+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"136'7226","reported_seq":9799,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.284209+0000","last_change":"2024-02-22T19:33:40.144966+0000","last_active":"2024-02-22T19:33:49.284209+0000","last_peered":"2024-02-22T19:33:49.284209+0000","last_clean":"2024-02-22T19:33:49.284209+0000","last_became_active":"2024-02-22T19:33:40.096849+0000","last_became_peered":"2024-02-22T19:33:40.096849+0000","last_unstale":"2024-02-22T19:33:49.284209+0000","last_undegraded":"2024-02-22T19:33:49.284209+0000","last_fullsized":"2024-02-22T19:33:49.284209+0000","mapping_epoch":132,"log_start":"102'1900","ondisk_log_start":"102'1900","created":93,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5326,"ondisk_log_size":5326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":235667456,"num_objects":3596,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7226,"num_write_kb":346304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1254,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":87,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114664+0000","last_change":"2024-02-22T19:30:35.181887+0000","last_active":"2024-02-22T19:33:42.114664+0000","last_peered":"2024-02-22T19:33:42.114664+0000","last_clean":"2024-02-22T19:33:42.114664+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:33:42.114664+0000","last_undegraded":"2024-02-22T19:33:42.114664+0000","last_fullsized":"2024-02-22T19:33:42.114664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.181816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.181816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.113963+0000","last_change":"2024-02-22T19:30:21.094504+0000","last_active":"2024-02-22T19:33:42.113963+0000","last_peered":"2024-02-22T19:33:42.113963+0000","last_clean":"2024-02-22T19:33:42.113963+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:33:42.113963+0000","last_undegraded":"2024-02-22T19:33:42.113963+0000","last_fullsized":"2024-02-22T19:33:42.113963+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:21.094370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:21.094370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":90,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.146863+0000","last_change":"2024-02-22T19:30:04.608437+0000","last_active":"2024-02-22T19:33:42.146863+0000","last_peered":"2024-02-22T19:33:42.146863+0000","last_clean":"2024-02-22T19:33:42.146863+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:33:42.146863+0000","last_undegraded":"2024-02-22T19:33:42.146863+0000","last_fullsized":"2024-02-22T19:33:42.146863+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:04.608337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:04.608337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":119,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115237+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:33:42.115237+0000","last_peered":"2024-02-22T19:33:42.115237+0000","last_clean":"2024-02-22T19:33:42.115237+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:33:42.115237+0000","last_undegraded":"2024-02-22T19:33:42.115237+0000","last_fullsized":"2024-02-22T19:33:42.115237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"136'7438","reported_seq":7489,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.281684+0000","last_change":"2024-02-22T19:31:33.014534+0000","last_active":"2024-02-22T19:33:49.281684+0000","last_peered":"2024-02-22T19:33:49.281684+0000","last_clean":"2024-02-22T19:33:49.281684+0000","last_became_active":"2024-02-22T19:31:33.014231+0000","last_became_peered":"2024-02-22T19:31:33.014231+0000","last_unstale":"2024-02-22T19:33:49.281684+0000","last_undegraded":"2024-02-22T19:33:49.281684+0000","last_fullsized":"2024-02-22T19:33:49.281684+0000","mapping_epoch":93,"log_start":"106'2500","ondisk_log_start":"106'2500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4938,"ondisk_log_size":4938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":244711424,"num_objects":3734,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7438,"num_write_kb":357504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.115751+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:33:42.115751+0000","last_peered":"2024-02-22T19:33:42.115751+0000","last_clean":"2024-02-22T19:33:42.115751+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:33:42.115751+0000","last_undegraded":"2024-02-22T19:33:42.115751+0000","last_fullsized":"2024-02-22T19:33:42.115751+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":173,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.165290+0000","last_change":"2024-02-22T19:33:42.757859+0000","last_active":"2024-02-22T19:33:45.165290+0000","last_peered":"2024-02-22T19:33:45.165290+0000","last_clean":"2024-02-22T19:33:45.165290+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:33:45.165290+0000","last_undegraded":"2024-02-22T19:33:45.165290+0000","last_fullsized":"2024-02-22T19:33:45.165290+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:33:42.757755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.757755+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":99,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114974+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:33:42.114974+0000","last_peered":"2024-02-22T19:33:42.114974+0000","last_clean":"2024-02-22T19:33:42.114974+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:33:42.114974+0000","last_undegraded":"2024-02-22T19:33:42.114974+0000","last_fullsized":"2024-02-22T19:33:42.114974+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":106,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-02-22T19:33:42.114915+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:33:42.114915+0000","last_peered":"2024-02-22T19:33:42.114915+0000","last_clean":"2024-02-22T19:33:42.114915+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:33:42.114915+0000","last_undegraded":"2024-02-22T19:33:42.114915+0000","last_fullsized":"2024-02-22T19:33:42.114915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3896705048,"num_objects":59460,"num_object_clones":0,"num_object_copies":178380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59460,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":118950,"num_write_kb":5709057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5115,"num_bytes_recovered":332922880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11821854720,"data_stored":11821842504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77450,"ondisk_log_size":77450,"up":48,"acting":48,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":277,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266749,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1505528,"kb_used_data":1208304,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":92239624,"statfs":{"total":95995035648,"available":94453374976,"internally_reserved":0,"allocated":1237303296,"data_stored":1236913835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":5.0609999999999999,"5min":1.506,"15min":0.88400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":42.597999999999999,"5min":42.597999999999999,"15min":42.597999999999999},"last":6.6580000000000004},{"interface":"front","average":{"1min":3.5640000000000001,"5min":1.1499999999999999,"15min":0.73599999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":42.155999999999999,"5min":42.155999999999999,"15min":42.155999999999999},"last":3.6349999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.6139999999999999,"5min":1.1719999999999999,"15min":0.77600000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":38.188000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":4.2999999999999998,"5min":1.3089999999999999,"15min":0.79900000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":37.938000000000002,"5min":37.938000000000002,"15min":37.938000000000002},"last":1.415}]},{"osd":2,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":1.226,"15min":0.78200000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":41.640999999999998,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.434},{"interface":"front","average":{"1min":4.2530000000000001,"5min":1.3560000000000001,"15min":0.85199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":41.722000000000001,"5min":41.722000000000001,"15min":41.722000000000001},"last":1.7250000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1479999999999997,"5min":1.329,"15min":0.84499999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":41.457000000000001,"5min":41.457000000000001,"15min":41.457000000000001},"last":3.149},{"interface":"front","average":{"1min":3.6549999999999998,"5min":1.204,"15min":0.81899999999999995},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":41.311999999999998,"5min":41.311999999999998,"15min":41.311999999999998},"last":4.8460000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":1.024}]},{"osd":6,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4889999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5539999999999998,"5min":12.067,"15min":12.067},"last":0.502}]}]},{"osd":6,"up_from":45,"seq":193273528386,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1518816,"kb_used_data":1209304,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92226336,"statfs":{"total":95995035648,"available":94439768064,"internally_reserved":0,"allocated":1238327296,"data_stored":1237961503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.7719999999999998,"5min":1.002,"15min":0.68000000000000005},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":17.885000000000002,"5min":17.885000000000002,"15min":17.885000000000002},"last":5.1909999999999998},{"interface":"front","average":{"1min":2.5139999999999998,"5min":1.1120000000000001,"15min":0.71299999999999997},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.871}]},{"osd":1,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":0.80600000000000005,"15min":0.64000000000000001},"min":{"1min":0.29099999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":11.301,"5min":11.301,"15min":11.301},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":0.81399999999999995,"15min":0.62},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.8410000000000002,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":1.093,"15min":0.72099999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":9.2319999999999993,"5min":12.859999999999999,"15min":12.859999999999999},"last":5.9950000000000001},{"interface":"front","average":{"1min":1.54,"5min":0.82899999999999996,"15min":0.64400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":5.0030000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":1.51,"15min":0.86399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.327,"5min":38.762,"15min":38.762},"last":0.88400000000000001},{"interface":"front","average":{"1min":3.7050000000000001,"5min":1.788,"15min":0.96599999999999997},"min":{"1min":0.379,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":24.242999999999999,"5min":35.612000000000002,"15min":35.612000000000002},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44800000000000001,"15min":0.43099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.33600000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48899999999999999,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.39700000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48999999999999999,"15min":0.51000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.43099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1991056,"kb_used_data":1685640,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91754096,"statfs":{"total":95995035648,"available":93956194304,"internally_reserved":0,"allocated":1726095360,"data_stored":1725681791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":0.73799999999999999,"15min":0.46400000000000002},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":3.0569999999999999,"5min":0.93500000000000005,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":11.352,"5min":11.352,"15min":11.352},"last":3.4729999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":4.5469999999999997,"5min":1.3220000000000001,"15min":0.70199999999999996},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":15.683,"5min":15.683,"15min":15.683},"last":0.67100000000000004},{"interface":"front","average":{"1min":4.9130000000000003,"5min":1.3400000000000001,"15min":0.70799999999999996},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":13.461,"5min":13.461,"15min":13.461},"last":3.3849999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:33:34 2024","interfaces":[{"interface":"back","average":{"1min":5.0599999999999996,"5min":1.9970000000000001,"15min":0.97699999999999998},"min":{"1min":0.24399999999999999,"5min":0.223,"15min":0.223},"max":{"1min":11.932,"5min":25.145,"15min":25.145},"last":0.93700000000000006},{"interface":"front","average":{"1min":3.8420000000000001,"5min":1.8959999999999999,"15min":0.97799999999999998},"min":{"1min":0.40300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":11.625,"5min":22.25,"15min":22.25},"last":1.264}]},{"osd":4,"last update":"Thu Feb 22 19:32:50 2024","interfaces":[{"interface":"back","average":{"1min":2.5350000000000001,"5min":0.95899999999999996,"15min":0.73499999999999999},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":15.542999999999999,"5min":15.542999999999999,"15min":15.542999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.226,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.40400000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:33:09 2024","interfaces":[{"interface":"back","average":{"1min":3.1429999999999998,"5min":1.1819999999999999,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":15.587999999999999,"5min":15.587999999999999,"15min":15.587999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":2.694,"5min":1.0149999999999999,"15min":0.72299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.5839999999999996,"5min":7.5839999999999996,"15min":7.5839999999999996},"last":2.1840000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:33:40 2024","interfaces":[{"interface":"back","average":{"1min":2.1549999999999998,"5min":0.99299999999999999,"15min":0.69299999999999995},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.4199999999999999,"5min":1.333,"15min":0.83899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.032,"5min":15.651999999999999,"15min":15.651999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.5579999999999998,"5min":0.97299999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":2.1419999999999999},{"interface":"front","average":{"1min":1.706,"5min":0.82399999999999995,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":0.871}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2206904,"kb_used_data":1905584,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91538248,"statfs":{"total":95995035648,"available":93735165952,"internally_reserved":0,"allocated":1951318016,"data_stored":1950928929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":3.2610000000000001,"5min":1.262,"15min":0.66000000000000003},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":12.305,"5min":12.305,"15min":12.305},"last":2.1269999999999998},{"interface":"front","average":{"1min":2.6440000000000001,"5min":1.327,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":13.013999999999999,"5min":13.013999999999999,"15min":13.013999999999999},"last":1.534}]},{"osd":2,"last update":"Thu Feb 22 19:33:06 2024","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":1.335,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":19.927,"5min":19.927,"15min":19.927},"last":0.40999999999999998},{"interface":"front","average":{"1min":4.2110000000000003,"5min":1.327,"15min":0.69099999999999995},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":17.907,"5min":17.907,"15min":17.907},"last":1.4139999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:33:27 2024","interfaces":[{"interface":"back","average":{"1min":3.5150000000000001,"5min":1.2989999999999999,"15min":0.72299999999999998},"min":{"1min":0.22900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":13.06,"5min":13.06,"15min":13.06},"last":1.496},{"interface":"front","average":{"1min":4.0919999999999996,"5min":1.5349999999999999,"15min":0.81200000000000006},"min":{"1min":0.39000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":13.837999999999999,"5min":22.936,"15min":22.936},"last":1.4610000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":0.89500000000000002,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":2.649,"5min":1.423,"15min":0.86799999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":13.874000000000001,"5min":13.874000000000001,"15min":13.874000000000001},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":1.26,"15min":0.79800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":13.903,"5min":13.903,"15min":13.903},"last":0.499},{"interface":"front","average":{"1min":1.7270000000000001,"5min":0.81100000000000005,"15min":0.64900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.8920000000000003,"5min":6.8920000000000003,"15min":6.8920000000000003},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:33:33 2024","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":0.90700000000000003,"15min":0.69799999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.984,"5min":6.984,"15min":6.984},"last":0.42699999999999999},{"interface":"front","average":{"1min":2.0569999999999999,"5min":1.038,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:33:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":0.84399999999999997,"15min":0.66400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9660000000000002,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":2.4279999999999999,"5min":0.95299999999999996,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":6.0800000000000001,"5min":6.0800000000000001,"15min":6.0800000000000001},"last":0.55500000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2204972,"kb_used_data":1907748,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":91540180,"statfs":{"total":95995035648,"available":93737144320,"internally_reserved":0,"allocated":1953533952,"data_stored":1953156075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":1.1719999999999999,"15min":0.63600000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":11.898999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":1.194},{"interface":"front","average":{"1min":3.6859999999999999,"5min":1.4239999999999999,"15min":0.71599999999999997},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":16.138999999999999,"5min":18.059999999999999,"15min":18.059999999999999},"last":1.381}]},{"osd":1,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":0.96199999999999997,"15min":0.57199999999999995},"min":{"1min":0.21199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.183,"5min":10.183,"15min":10.183},"last":1.357},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.73799999999999999,"15min":0.51600000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":9.7189999999999994,"5min":9.7189999999999994,"15min":9.7189999999999994},"last":0.374}]},{"osd":3,"last update":"Thu Feb 22 19:33:32 2024","interfaces":[{"interface":"back","average":{"1min":4.016,"5min":1.6559999999999999,"15min":0.86099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":30.071000000000002,"5min":30.071000000000002,"15min":30.071000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":5.2720000000000002,"5min":1.9850000000000001,"15min":0.95999999999999996},"min":{"1min":0.39900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":32.119,"5min":32.119,"15min":32.119},"last":4.8019999999999996}]},{"osd":4,"last update":"Thu Feb 22 19:32:53 2024","interfaces":[{"interface":"back","average":{"1min":2.1669999999999998,"5min":0.90400000000000003,"15min":0.68200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":6.0449999999999999,"5min":6.0449999999999999,"15min":6.0449999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.67500000000000004,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.34499999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:33:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":0.747,"15min":0.66500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.42899999999999999},{"interface":"front","average":{"1min":2.1669999999999998,"5min":0.97999999999999998,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0739999999999998,"5min":6.0739999999999998,"15min":6.0739999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:39 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.1579999999999999,"15min":0.78800000000000003},"min":{"1min":0.375,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":9.9689999999999994,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":0.81200000000000006,"15min":0.69099999999999995},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.4100000000000001,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":0.40200000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":0.94799999999999995,"15min":0.70199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.1029999999999998,"5min":6.1029999999999998,"15min":6.1029999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.73699999999999999,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":9.5660000000000007,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.67700000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2706560,"kb_used_data":2396920,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91038592,"statfs":{"total":95995035648,"available":93223518208,"internally_reserved":0,"allocated":2454446080,"data_stored":2454049133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1909999999999998,"5min":1.5309999999999999,"15min":0.80400000000000005},"min":{"1min":0.20899999999999999,"5min":0.193,"15min":0.193},"max":{"1min":12.929,"5min":12.929,"15min":12.929},"last":6.1710000000000003},{"interface":"front","average":{"1min":4.774,"5min":1.7390000000000001,"15min":0.90500000000000003},"min":{"1min":0.36099999999999999,"5min":0.247,"15min":0.247},"max":{"1min":14.831,"5min":14.831,"15min":14.831},"last":3.992}]},{"osd":1,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1600000000000001,"5min":1.728,"15min":0.89000000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":17.009,"5min":17.009,"15min":17.009},"last":3.7610000000000001},{"interface":"front","average":{"1min":2.843,"5min":1.3380000000000001,"15min":0.752},"min":{"1min":0.38600000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":7.7619999999999996,"5min":10.279,"15min":10.279},"last":1.7769999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1660000000000004,"5min":1.7090000000000001,"15min":0.89800000000000002},"min":{"1min":0.34499999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":10.714,"5min":10.714,"15min":10.714},"last":0.309},{"interface":"front","average":{"1min":4.2519999999999998,"5min":1.712,"15min":0.88900000000000001},"min":{"1min":0.46100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":2.1389999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:32:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3359999999999999,"5min":0.99099999999999999,"15min":0.747},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":11.241,"5min":11.241,"15min":11.241},"last":2.9689999999999999},{"interface":"front","average":{"1min":2.1680000000000001,"5min":0.95699999999999996,"15min":0.72299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.3700000000000001,"5min":6.7089999999999996,"15min":6.7089999999999996},"last":0.34000000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":1.125,"15min":0.76800000000000002},"min":{"1min":0.437,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.8550000000000004,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":2.1179999999999999},{"interface":"front","average":{"1min":3.3239999999999998,"5min":1.2689999999999999,"15min":0.81100000000000005},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":12.744999999999999,"5min":12.744999999999999,"15min":12.744999999999999},"last":3}]},{"osd":6,"last update":"Thu Feb 22 19:33:42 2024","interfaces":[{"interface":"back","average":{"1min":3.21,"5min":1.363,"15min":0.84599999999999997},"min":{"1min":0.496,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.898999999999999,"5min":12.898999999999999,"15min":12.898999999999999},"last":2.0619999999999998},{"interface":"front","average":{"1min":3.036,"5min":1.4590000000000001,"15min":0.92000000000000004},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":12.776999999999999,"5min":12.776999999999999,"15min":12.776999999999999},"last":3.0230000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:32:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":1.0780000000000001,"15min":0.74299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":10.055,"5min":10.055,"15min":10.055},"last":1.75},{"interface":"front","average":{"1min":2.3319999999999999,"5min":1.022,"15min":0.746},"min":{"1min":0.45700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.8200000000000003,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":2.089}]}]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1495676,"kb_used_data":1198452,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":92249476,"statfs":{"total":95995035648,"available":94463463424,"internally_reserved":0,"allocated":1227214848,"data_stored":1226821777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":2.3029999999999999,"5min":0.88100000000000001,"15min":0.63800000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":24.009,"5min":24.009,"15min":24.009},"last":2.3719999999999999},{"interface":"front","average":{"1min":2.7599999999999998,"5min":0.98499999999999999,"15min":0.69199999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":24.263000000000002,"5min":24.263000000000002,"15min":24.263000000000002},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":0.68100000000000005,"15min":0.59399999999999997},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":2.6749999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.59999999999999998,"15min":0.53500000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":2.5019999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1709999999999998,"5min":0.86399999999999999,"15min":0.66900000000000004},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":23.082000000000001,"5min":23.082000000000001,"15min":23.082000000000001},"last":2.98},{"interface":"front","average":{"1min":2.6110000000000002,"5min":0.95899999999999996,"15min":0.68100000000000005},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":23.138999999999999,"5min":23.138999999999999,"15min":23.138999999999999},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:32:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":0.749,"15min":0.61899999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":2.492},{"interface":"front","average":{"1min":1.5109999999999999,"5min":0.73499999999999999,"15min":0.59899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.0149999999999997,"5min":7.0149999999999997,"15min":7.0149999999999997},"last":2.9390000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":2.1120000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:33:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64800000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":2.0499999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.498,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55800000000000005,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.32800000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":2.6349999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":2.0880000000000001}]}]},{"osd":5,"up_from":0,"seq":167503724615,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1940328448,"data_stored":1940324376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1709051904,"data_stored":1709047832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1935867904,"data_stored":1935867904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2435186688,"data_stored":2435186688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1219166208,"data_stored":1219166208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":132251648,"data_stored":132251648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1228214272,"data_stored":1228210200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1221787648,"data_stored":1221787648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:33:53.600 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:53.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.940062+0000 osd.4 (osd.4) 59 : cluster [DBG] 4.e scrub starts 2024-02-22T19:33:53.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:51.953381+0000 osd.4 (osd.4) 60 : cluster [DBG] 4.e scrub ok 2024-02-22T19:33:53.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:52.332322+0000 mgr.y (mgr.14535) 191 : audit [DBG] from='client.24824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:33:53.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:52.777347+0000 osd.3 (osd.3) 65 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:33:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:52.788043+0000 osd.3 (osd.3) 66 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:33:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:53.344418+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:53.770 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:53.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:53.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:53.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:53.931 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:53.968 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:53.978 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:53.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:54.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:54.115 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:54.136 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:54.173 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:54.176 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:54.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:54.218 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:33:54.277 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:33:54.336 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:54.468 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:33:54.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:54.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:33:54.476 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:33:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:33:54.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:33:54.609 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:54.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:52.884430+0000 osd.2 (osd.2) 47 : cluster [DBG] 6.f scrub starts 2024-02-22T19:33:54.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:52.891920+0000 osd.2 (osd.2) 48 : cluster [DBG] 6.f scrub ok 2024-02-22T19:33:54.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:52.932033+0000 osd.4 (osd.4) 61 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:52.938435+0000 osd.4 (osd.4) 62 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.313792+0000 mgr.y (mgr.14535) 192 : cluster [DBG] pgmap v209: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 803 op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:53.542724+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.648220+0000 osd.7 (osd.7) 57 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.652461+0000 osd.7 (osd.7) 58 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.778962+0000 osd.6 (osd.6) 33 : cluster [DBG] 4.a scrub starts 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.782460+0000 osd.6 (osd.6) 34 : cluster [DBG] 4.a scrub ok 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:54.474383+0000 mon.a (mon.0) 880 : audit [INF] from='client.? 172.21.15.53:0/379278029' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:52.884430+0000 osd.2 (osd.2) 47 : cluster [DBG] 6.f scrub starts 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:52.891920+0000 osd.2 (osd.2) 48 : cluster [DBG] 6.f scrub ok 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:52.932033+0000 osd.4 (osd.4) 61 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:33:54.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:52.938435+0000 osd.4 (osd.4) 62 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.313792+0000 mgr.y (mgr.14535) 192 : cluster [DBG] pgmap v209: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 803 op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:53.542724+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.648220+0000 osd.7 (osd.7) 57 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.652461+0000 osd.7 (osd.7) 58 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.778962+0000 osd.6 (osd.6) 33 : cluster [DBG] 4.a scrub starts 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.782460+0000 osd.6 (osd.6) 34 : cluster [DBG] 4.a scrub ok 2024-02-22T19:33:54.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:54.474383+0000 mon.a (mon.0) 880 : audit [INF] from='client.? 172.21.15.53:0/379278029' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:54.815 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:54.820 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:52.884430+0000 osd.2 (osd.2) 47 : cluster [DBG] 6.f scrub starts 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:52.891920+0000 osd.2 (osd.2) 48 : cluster [DBG] 6.f scrub ok 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:52.932033+0000 osd.4 (osd.4) 61 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:52.938435+0000 osd.4 (osd.4) 62 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.313792+0000 mgr.y (mgr.14535) 192 : cluster [DBG] pgmap v209: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 803 op/s; 0 B/s, 1 objects/s recovering 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:53.542724+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.648220+0000 osd.7 (osd.7) 57 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.652461+0000 osd.7 (osd.7) 58 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.778962+0000 osd.6 (osd.6) 33 : cluster [DBG] 4.a scrub starts 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.782460+0000 osd.6 (osd.6) 34 : cluster [DBG] 4.a scrub ok 2024-02-22T19:33:54.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:54.474383+0000 mon.a (mon.0) 880 : audit [INF] from='client.? 172.21.15.53:0/379278029' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:33:54.824 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:54.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:54.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:55.471 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:55.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:55.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:55.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:55.614 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.931494+0000 osd.4 (osd.4) 63 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:53.940370+0000 osd.4 (osd.4) 64 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:54.475793+0000 mgr.y (mgr.14535) 193 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.540032+0000 mon.a (mon.0) 881 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.540065+0000 mon.a (mon.0) 882 : cluster [INF] Cluster is now healthy 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:54.557098+0000 mon.a (mon.0) 883 : audit [INF] from='client.? 172.21.15.53:0/379278029' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.557155+0000 mon.a (mon.0) 884 : cluster [DBG] osdmap e137: 8 total, 8 up, 7 in 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.603491+0000 osd.7 (osd.7) 59 : cluster [DBG] 3.e scrub starts 2024-02-22T19:33:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.606409+0000 osd.7 (osd.7) 60 : cluster [DBG] 3.e scrub ok 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.742798+0000 osd.3 (osd.3) 67 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:54.749054+0000 osd.3 (osd.3) 68 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.931494+0000 osd.4 (osd.4) 63 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:53.940370+0000 osd.4 (osd.4) 64 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:54.475793+0000 mgr.y (mgr.14535) 193 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.540032+0000 mon.a (mon.0) 881 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.540065+0000 mon.a (mon.0) 882 : cluster [INF] Cluster is now healthy 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:54.557098+0000 mon.a (mon.0) 883 : audit [INF] from='client.? 172.21.15.53:0/379278029' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.557155+0000 mon.a (mon.0) 884 : cluster [DBG] osdmap e137: 8 total, 8 up, 7 in 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.603491+0000 osd.7 (osd.7) 59 : cluster [DBG] 3.e scrub starts 2024-02-22T19:33:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.606409+0000 osd.7 (osd.7) 60 : cluster [DBG] 3.e scrub ok 2024-02-22T19:33:55.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.742798+0000 osd.3 (osd.3) 67 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:33:55.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:54.749054+0000 osd.3 (osd.3) 68 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.931494+0000 osd.4 (osd.4) 63 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:53.940370+0000 osd.4 (osd.4) 64 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:54.475793+0000 mgr.y (mgr.14535) 193 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.540032+0000 mon.a (mon.0) 881 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.540065+0000 mon.a (mon.0) 882 : cluster [INF] Cluster is now healthy 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:54.557098+0000 mon.a (mon.0) 883 : audit [INF] from='client.? 172.21.15.53:0/379278029' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.557155+0000 mon.a (mon.0) 884 : cluster [DBG] osdmap e137: 8 total, 8 up, 7 in 2024-02-22T19:33:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.603491+0000 osd.7 (osd.7) 59 : cluster [DBG] 3.e scrub starts 2024-02-22T19:33:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.606409+0000 osd.7 (osd.7) 60 : cluster [DBG] 3.e scrub ok 2024-02-22T19:33:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.742798+0000 osd.3 (osd.3) 67 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:33:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:54.749054+0000 osd.3 (osd.3) 68 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:33:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:55.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:55.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:56.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:56.078 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:56.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:56.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:56.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:33:56.445 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:56.469 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:33:56.469 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:33:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:33:56.486 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:56.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:33:56.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:33:56.577 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:55.314162+0000 mgr.y (mgr.14535) 194 : cluster [DBG] pgmap v211: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 940 op/s 2024-02-22T19:33:56.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:56.639 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:56.674 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:56.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:56.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:33:56.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:33:56.807 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:55.314162+0000 mgr.y (mgr.14535) 194 : cluster [DBG] pgmap v211: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 940 op/s 2024-02-22T19:33:56.809 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:56.832 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:33:56.844 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:56.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:33:56.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:33:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:55.314162+0000 mgr.y (mgr.14535) 194 : cluster [DBG] pgmap v211: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 940 op/s 2024-02-22T19:33:57.079 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:33:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:57.129 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:33:57.181 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:33:57.233 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:33:57.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:33:57.267 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:33:57.300 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:33:57.325 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:33:57.327 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:57.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:33:57.343 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:33:57.373 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:33:57.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:57.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:33:57.535 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:57.669 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:57.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:57.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:57.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:33:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:57.828 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:33:57.830 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:57.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:33:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:55.745391+0000 osd.1 (osd.1) 49 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:33:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:55.749964+0000 osd.1 (osd.1) 50 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:33:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:56.629798+0000 osd.7 (osd.7) 61 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:33:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:56.634366+0000 osd.7 (osd.7) 62 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:33:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:33:57.047266+0000 mon.a (mon.0) 885 : audit [DBG] from='client.? 172.21.15.53:0/1185936608' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:57.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:55.745391+0000 osd.1 (osd.1) 49 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:55.749964+0000 osd.1 (osd.1) 50 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:56.629798+0000 osd.7 (osd.7) 61 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:56.634366+0000 osd.7 (osd.7) 62 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:33:57.047266+0000 mon.a (mon.0) 885 : audit [DBG] from='client.? 172.21.15.53:0/1185936608' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:55.745391+0000 osd.1 (osd.1) 49 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:55.749964+0000 osd.1 (osd.1) 50 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:56.629798+0000 osd.7 (osd.7) 61 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:56.634366+0000 osd.7 (osd.7) 62 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:33:58.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:33:57.047266+0000 mon.a (mon.0) 885 : audit [DBG] from='client.? 172.21.15.53:0/1185936608' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:33:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:58.136 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:58.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:58.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:58.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:33:58.313 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:58.608 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:57.316880+0000 mgr.y (mgr.14535) 195 : cluster [DBG] pgmap v212: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 940 op/s 2024-02-22T19:33:58.860 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:57.316880+0000 mgr.y (mgr.14535) 195 : cluster [DBG] pgmap v212: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 940 op/s 2024-02-22T19:33:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:57.316880+0000 mgr.y (mgr.14535) 195 : cluster [DBG] pgmap v212: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 940 op/s 2024-02-22T19:33:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:33:59.200 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:59.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:33:59.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:33:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:59.506 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:33:59] "GET /metrics HTTP/1.1" 200 175570 "" "Prometheus/2.33.4" 2024-02-22T19:33:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:33:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:33:59.581 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:59.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:33:59.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:33:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:33:59.989 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:33:59.995 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:33:59.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:57.768544+0000 osd.1 (osd.1) 51 : cluster [DBG] 5.c scrub starts 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:57.772319+0000 osd.1 (osd.1) 52 : cluster [DBG] 5.c scrub ok 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:57.871438+0000 osd.2 (osd.2) 49 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:57.874672+0000 osd.2 (osd.2) 50 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.175389+0000 osd.0 (osd.0) 69 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.185183+0000 osd.0 (osd.0) 70 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:34:00.000 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.830604+0000 osd.4 (osd.4) 65 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:34:00.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.835366+0000 osd.4 (osd.4) 66 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:34:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:57.768544+0000 osd.1 (osd.1) 51 : cluster [DBG] 5.c scrub starts 2024-02-22T19:34:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:57.772319+0000 osd.1 (osd.1) 52 : cluster [DBG] 5.c scrub ok 2024-02-22T19:34:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:57.871438+0000 osd.2 (osd.2) 49 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:34:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:57.874672+0000 osd.2 (osd.2) 50 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:34:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.175389+0000 osd.0 (osd.0) 69 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:34:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.185183+0000 osd.0 (osd.0) 70 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:34:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.830604+0000 osd.4 (osd.4) 65 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:34:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:33:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.835366+0000 osd.4 (osd.4) 66 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:34:00.096 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:57.768544+0000 osd.1 (osd.1) 51 : cluster [DBG] 5.c scrub starts 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:57.772319+0000 osd.1 (osd.1) 52 : cluster [DBG] 5.c scrub ok 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:57.871438+0000 osd.2 (osd.2) 49 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:57.874672+0000 osd.2 (osd.2) 50 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.175389+0000 osd.0 (osd.0) 69 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.185183+0000 osd.0 (osd.0) 70 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:34:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.830604+0000 osd.4 (osd.4) 65 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:34:00.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:33:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.835366+0000 osd.4 (osd.4) 66 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:34:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:00.131 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:00.153 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:00.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:00.158 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:00.258 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:00.392 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:34:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:34:00.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:34:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.769611+0000 osd.1 (osd.1) 53 : cluster [DBG] 5.19 scrub starts 2024-02-22T19:34:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.781460+0000 osd.1 (osd.1) 54 : cluster [DBG] 5.19 scrub ok 2024-02-22T19:34:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.906867+0000 osd.2 (osd.2) 51 : cluster [DBG] 5.a scrub starts 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:58.910109+0000 osd.2 (osd.2) 52 : cluster [DBG] 5.a scrub ok 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:59.317738+0000 mgr.y (mgr.14535) 196 : cluster [DBG] pgmap v213: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:59.947109+0000 osd.2 (osd.2) 53 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:33:59.950581+0000 osd.2 (osd.2) 54 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.769611+0000 osd.1 (osd.1) 53 : cluster [DBG] 5.19 scrub starts 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.781460+0000 osd.1 (osd.1) 54 : cluster [DBG] 5.19 scrub ok 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.906867+0000 osd.2 (osd.2) 51 : cluster [DBG] 5.a scrub starts 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:58.910109+0000 osd.2 (osd.2) 52 : cluster [DBG] 5.a scrub ok 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:59.317738+0000 mgr.y (mgr.14535) 196 : cluster [DBG] pgmap v213: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:59.947109+0000 osd.2 (osd.2) 53 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:34:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:33:59.950581+0000 osd.2 (osd.2) 54 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:34:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.769611+0000 osd.1 (osd.1) 53 : cluster [DBG] 5.19 scrub starts 2024-02-22T19:34:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.781460+0000 osd.1 (osd.1) 54 : cluster [DBG] 5.19 scrub ok 2024-02-22T19:34:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.906867+0000 osd.2 (osd.2) 51 : cluster [DBG] 5.a scrub starts 2024-02-22T19:34:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:58.910109+0000 osd.2 (osd.2) 52 : cluster [DBG] 5.a scrub ok 2024-02-22T19:34:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:59.317738+0000 mgr.y (mgr.14535) 196 : cluster [DBG] pgmap v213: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:01.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:59.947109+0000 osd.2 (osd.2) 53 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:34:01.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:33:59.950581+0000 osd.2 (osd.2) 54 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:00.367281+0000 mgr.y (mgr.14535) 197 : audit [DBG] from='client.24833 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:00.712255+0000 osd.1 (osd.1) 55 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:00.713481+0000 osd.1 (osd.1) 56 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:00.778803+0000 osd.6 (osd.6) 35 : cluster [DBG] 5.1f deep-scrub starts 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:00.780570+0000 osd.6 (osd.6) 36 : cluster [DBG] 5.1f deep-scrub ok 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.110401+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:34:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.123398+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:00.367281+0000 mgr.y (mgr.14535) 197 : audit [DBG] from='client.24833 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:00.712255+0000 osd.1 (osd.1) 55 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:00.713481+0000 osd.1 (osd.1) 56 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:00.778803+0000 osd.6 (osd.6) 35 : cluster [DBG] 5.1f deep-scrub starts 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:00.780570+0000 osd.6 (osd.6) 36 : cluster [DBG] 5.1f deep-scrub ok 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.110401+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:34:01.844 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.123398+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:34:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:01.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:01.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:01.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:00.367281+0000 mgr.y (mgr.14535) 197 : audit [DBG] from='client.24833 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:00.712255+0000 osd.1 (osd.1) 55 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:00.713481+0000 osd.1 (osd.1) 56 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:00.778803+0000 osd.6 (osd.6) 35 : cluster [DBG] 5.1f deep-scrub starts 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:00.780570+0000 osd.6 (osd.6) 36 : cluster [DBG] 5.1f deep-scrub ok 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.110401+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:34:01.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.123398+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:34:01.998 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:02.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:02.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:02.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:02.158 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:02.183 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:02.186 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:02.186 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:02.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:02.349 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:02.352 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-02-22T19:33:59.317047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3300660924,"num_objects":50557,"num_object_clones":0,"num_object_copies":151671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50557,"num_whiteouts":0,"num_read":1006,"num_read_kb":758,"num_write":128473,"num_write_kb":5709098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5178,"num_bytes_recovered":332925147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78146,"ondisk_log_size":78146,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":11900764,"kb_used_data":9750180,"kb_used_omap":0,"kb_used_meta":2150144,"kb_avail":644315300,"statfs":{"total":671965249536,"available":659778867200,"internally_reserved":0,"allocated":9984184320,"data_stored":9981419602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2201747456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,7,14],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-596049920,"num_objects":-9095,"num_object_clones":0,"num_object_copies":-27285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9095,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9095,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005614"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174694+0000","last_change":"2024-02-22T19:31:50.004483+0000","last_active":"2024-02-22T19:33:45.174694+0000","last_peered":"2024-02-22T19:33:45.174694+0000","last_clean":"2024-02-22T19:33:45.174694+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:33:45.174694+0000","last_undegraded":"2024-02-22T19:33:45.174694+0000","last_fullsized":"2024-02-22T19:33:45.174694+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":104,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.781565+0000","last_change":"2024-02-22T19:33:58.781565+0000","last_active":"2024-02-22T19:33:58.781565+0000","last_peered":"2024-02-22T19:33:58.781565+0000","last_clean":"2024-02-22T19:33:58.781565+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:33:58.781565+0000","last_undegraded":"2024-02-22T19:33:58.781565+0000","last_fullsized":"2024-02-22T19:33:58.781565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.781466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.781466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":106,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.170559+0000","last_change":"2024-02-22T19:31:50.013054+0000","last_active":"2024-02-22T19:33:45.170559+0000","last_peered":"2024-02-22T19:33:45.170559+0000","last_clean":"2024-02-22T19:33:45.170559+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:33:45.170559+0000","last_undegraded":"2024-02-22T19:33:45.170559+0000","last_fullsized":"2024-02-22T19:33:45.170559+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:12.445206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:12.445206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:53.940471+0000","last_change":"2024-02-22T19:33:53.940446+0000","last_active":"2024-02-22T19:33:53.940471+0000","last_peered":"2024-02-22T19:33:53.940471+0000","last_clean":"2024-02-22T19:33:53.940471+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:33:53.940471+0000","last_undegraded":"2024-02-22T19:33:53.940471+0000","last_fullsized":"2024-02-22T19:33:53.940471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:53.940375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.940375+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173436+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:33:45.173436+0000","last_peered":"2024-02-22T19:33:45.173436+0000","last_clean":"2024-02-22T19:33:45.173436+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:33:45.173436+0000","last_undegraded":"2024-02-22T19:33:45.173436+0000","last_fullsized":"2024-02-22T19:33:45.173436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.176419+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:33:45.176419+0000","last_peered":"2024-02-22T19:33:45.176419+0000","last_clean":"2024-02-22T19:33:45.176419+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:33:45.176419+0000","last_undegraded":"2024-02-22T19:33:45.176419+0000","last_fullsized":"2024-02-22T19:33:45.176419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.180307+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:33:45.180307+0000","last_peered":"2024-02-22T19:33:45.180307+0000","last_clean":"2024-02-22T19:33:45.180307+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:33:45.180307+0000","last_undegraded":"2024-02-22T19:33:45.180307+0000","last_fullsized":"2024-02-22T19:33:45.180307+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":114,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173044+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:33:45.173044+0000","last_peered":"2024-02-22T19:33:45.173044+0000","last_clean":"2024-02-22T19:33:45.173044+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:33:45.173044+0000","last_undegraded":"2024-02-22T19:33:45.173044+0000","last_fullsized":"2024-02-22T19:33:45.173044+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:51.065944+0000","last_change":"2024-02-22T19:33:51.065916+0000","last_active":"2024-02-22T19:33:51.065944+0000","last_peered":"2024-02-22T19:33:51.065944+0000","last_clean":"2024-02-22T19:33:51.065944+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:33:51.065944+0000","last_undegraded":"2024-02-22T19:33:51.065944+0000","last_fullsized":"2024-02-22T19:33:51.065944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:51.065825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.065825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":54,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172039+0000","last_change":"2024-02-22T19:33:42.010538+0000","last_active":"2024-02-22T19:33:45.172039+0000","last_peered":"2024-02-22T19:33:45.172039+0000","last_clean":"2024-02-22T19:33:45.172039+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:33:45.172039+0000","last_undegraded":"2024-02-22T19:33:45.172039+0000","last_fullsized":"2024-02-22T19:33:45.172039+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.010467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.010467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.802887+0000","last_change":"2024-02-22T19:33:45.802849+0000","last_active":"2024-02-22T19:33:45.802887+0000","last_peered":"2024-02-22T19:33:45.802887+0000","last_clean":"2024-02-22T19:33:45.802887+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:33:45.802887+0000","last_undegraded":"2024-02-22T19:33:45.802887+0000","last_fullsized":"2024-02-22T19:33:45.802887+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.802756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.802756+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:52.938558+0000","last_change":"2024-02-22T19:33:52.938521+0000","last_active":"2024-02-22T19:33:52.938558+0000","last_peered":"2024-02-22T19:33:52.938558+0000","last_clean":"2024-02-22T19:33:52.938558+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:33:52.938558+0000","last_undegraded":"2024-02-22T19:33:52.938558+0000","last_fullsized":"2024-02-22T19:33:52.938558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:52.938440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.938440+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.153639+0000","last_change":"2024-02-22T19:31:50.005065+0000","last_active":"2024-02-22T19:33:45.153639+0000","last_peered":"2024-02-22T19:33:45.153639+0000","last_clean":"2024-02-22T19:33:45.153639+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:33:45.153639+0000","last_undegraded":"2024-02-22T19:33:45.153639+0000","last_fullsized":"2024-02-22T19:33:45.153639+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:18.496740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.496740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.151831+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:33:45.151831+0000","last_peered":"2024-02-22T19:33:45.151831+0000","last_clean":"2024-02-22T19:33:45.151831+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:33:45.151831+0000","last_undegraded":"2024-02-22T19:33:45.151831+0000","last_fullsized":"2024-02-22T19:33:45.151831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":61,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175133+0000","last_change":"2024-02-22T19:31:50.439339+0000","last_active":"2024-02-22T19:33:45.175133+0000","last_peered":"2024-02-22T19:33:45.175133+0000","last_clean":"2024-02-22T19:33:45.175133+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:33:45.175133+0000","last_undegraded":"2024-02-22T19:33:45.175133+0000","last_fullsized":"2024-02-22T19:33:45.175133+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:31:23.683223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.683223+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":102,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175472+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:33:45.175472+0000","last_peered":"2024-02-22T19:33:45.175472+0000","last_clean":"2024-02-22T19:33:45.175472+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:33:45.175472+0000","last_undegraded":"2024-02-22T19:33:45.175472+0000","last_fullsized":"2024-02-22T19:33:45.175472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175620+0000","last_change":"2024-02-22T19:31:50.009328+0000","last_active":"2024-02-22T19:33:45.175620+0000","last_peered":"2024-02-22T19:33:45.175620+0000","last_clean":"2024-02-22T19:33:45.175620+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:33:45.175620+0000","last_undegraded":"2024-02-22T19:33:45.175620+0000","last_fullsized":"2024-02-22T19:33:45.175620+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155336+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:33:45.155336+0000","last_peered":"2024-02-22T19:33:45.155336+0000","last_clean":"2024-02-22T19:33:45.155336+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:33:45.155336+0000","last_undegraded":"2024-02-22T19:33:45.155336+0000","last_fullsized":"2024-02-22T19:33:45.155336+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172345+0000","last_change":"2024-02-22T19:31:26.454221+0000","last_active":"2024-02-22T19:33:45.172345+0000","last_peered":"2024-02-22T19:33:45.172345+0000","last_clean":"2024-02-22T19:33:45.172345+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:33:45.172345+0000","last_undegraded":"2024-02-22T19:33:45.172345+0000","last_fullsized":"2024-02-22T19:33:45.172345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.454155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.454155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":117,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.159111+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:33:45.159111+0000","last_peered":"2024-02-22T19:33:45.159111+0000","last_clean":"2024-02-22T19:33:45.159111+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:33:45.159111+0000","last_undegraded":"2024-02-22T19:33:45.159111+0000","last_fullsized":"2024-02-22T19:33:45.159111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.835161+0000","last_change":"2024-02-22T19:33:48.835132+0000","last_active":"2024-02-22T19:33:48.835161+0000","last_peered":"2024-02-22T19:33:48.835161+0000","last_clean":"2024-02-22T19:33:48.835161+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:33:48.835161+0000","last_undegraded":"2024-02-22T19:33:48.835161+0000","last_fullsized":"2024-02-22T19:33:48.835161+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.835030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.835030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175271+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:33:45.175271+0000","last_peered":"2024-02-22T19:33:45.175271+0000","last_clean":"2024-02-22T19:33:45.175271+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:33:45.175271+0000","last_undegraded":"2024-02-22T19:33:45.175271+0000","last_fullsized":"2024-02-22T19:33:45.175271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174942+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:33:45.174942+0000","last_peered":"2024-02-22T19:33:45.174942+0000","last_clean":"2024-02-22T19:33:45.174942+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:33:45.174942+0000","last_undegraded":"2024-02-22T19:33:45.174942+0000","last_fullsized":"2024-02-22T19:33:45.174942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":117,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174791+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:33:45.174791+0000","last_peered":"2024-02-22T19:33:45.174791+0000","last_clean":"2024-02-22T19:33:45.174791+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:33:45.174791+0000","last_undegraded":"2024-02-22T19:33:45.174791+0000","last_fullsized":"2024-02-22T19:33:45.174791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152206+0000","last_change":"2024-02-22T19:31:50.095022+0000","last_active":"2024-02-22T19:33:45.152206+0000","last_peered":"2024-02-22T19:33:45.152206+0000","last_clean":"2024-02-22T19:33:45.152206+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:33:45.152206+0000","last_undegraded":"2024-02-22T19:33:45.152206+0000","last_fullsized":"2024-02-22T19:33:45.152206+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-22T19:30:37.617754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:37.617754+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":163,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.781746+0000","last_change":"2024-02-22T19:33:58.781746+0000","last_active":"2024-02-22T19:33:58.781746+0000","last_peered":"2024-02-22T19:33:58.781746+0000","last_clean":"2024-02-22T19:33:58.781746+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:33:58.781746+0000","last_undegraded":"2024-02-22T19:33:58.781746+0000","last_fullsized":"2024-02-22T19:33:58.781746+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:16.686047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.686047+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":99,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:55.750063+0000","last_change":"2024-02-22T19:33:55.750063+0000","last_active":"2024-02-22T19:33:55.750063+0000","last_peered":"2024-02-22T19:33:55.750063+0000","last_clean":"2024-02-22T19:33:55.750063+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:33:55.750063+0000","last_undegraded":"2024-02-22T19:33:55.750063+0000","last_fullsized":"2024-02-22T19:33:55.750063+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:55.749971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:55.749971+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":105,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171592+0000","last_change":"2024-02-22T19:30:16.731533+0000","last_active":"2024-02-22T19:33:45.171592+0000","last_peered":"2024-02-22T19:33:45.171592+0000","last_clean":"2024-02-22T19:33:45.171592+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:33:45.171592+0000","last_undegraded":"2024-02-22T19:33:45.171592+0000","last_fullsized":"2024-02-22T19:33:45.171592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:16.731454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:16.731454+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155525+0000","last_change":"2024-02-22T19:31:50.009891+0000","last_active":"2024-02-22T19:33:45.155525+0000","last_peered":"2024-02-22T19:33:45.155525+0000","last_clean":"2024-02-22T19:33:45.155525+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:33:45.155525+0000","last_undegraded":"2024-02-22T19:33:45.155525+0000","last_fullsized":"2024-02-22T19:33:45.155525+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171882+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:33:45.171882+0000","last_peered":"2024-02-22T19:33:45.171882+0000","last_clean":"2024-02-22T19:33:45.171882+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:33:45.171882+0000","last_undegraded":"2024-02-22T19:33:45.171882+0000","last_fullsized":"2024-02-22T19:33:45.171882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173183+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:33:45.173183+0000","last_peered":"2024-02-22T19:33:45.173183+0000","last_clean":"2024-02-22T19:33:45.173183+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:33:45.173183+0000","last_undegraded":"2024-02-22T19:33:45.173183+0000","last_fullsized":"2024-02-22T19:33:45.173183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":166,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.763354+0000","last_change":"2024-02-22T19:33:46.763313+0000","last_active":"2024-02-22T19:33:46.763354+0000","last_peered":"2024-02-22T19:33:46.763354+0000","last_clean":"2024-02-22T19:33:46.763354+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:33:46.763354+0000","last_undegraded":"2024-02-22T19:33:46.763354+0000","last_fullsized":"2024-02-22T19:33:46.763354+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:46.763202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.763202+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.151747+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:33:45.151747+0000","last_peered":"2024-02-22T19:33:45.151747+0000","last_clean":"2024-02-22T19:33:45.151747+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:33:45.151747+0000","last_undegraded":"2024-02-22T19:33:45.151747+0000","last_fullsized":"2024-02-22T19:33:45.151747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":87,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.147614+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:33:45.147614+0000","last_peered":"2024-02-22T19:33:45.147614+0000","last_clean":"2024-02-22T19:33:45.147614+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:33:45.147614+0000","last_undegraded":"2024-02-22T19:33:45.147614+0000","last_fullsized":"2024-02-22T19:33:45.147614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":106,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.170729+0000","last_change":"2024-02-22T19:33:43.048434+0000","last_active":"2024-02-22T19:33:45.170729+0000","last_peered":"2024-02-22T19:33:45.170729+0000","last_clean":"2024-02-22T19:33:45.170729+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:33:45.170729+0000","last_undegraded":"2024-02-22T19:33:45.170729+0000","last_fullsized":"2024-02-22T19:33:45.170729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:43.048330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.048330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":95,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173267+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:33:45.173267+0000","last_peered":"2024-02-22T19:33:45.173267+0000","last_clean":"2024-02-22T19:33:45.173267+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:33:45.173267+0000","last_undegraded":"2024-02-22T19:33:45.173267+0000","last_fullsized":"2024-02-22T19:33:45.173267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173675+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:33:45.173675+0000","last_peered":"2024-02-22T19:33:45.173675+0000","last_clean":"2024-02-22T19:33:45.173675+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:33:45.173675+0000","last_undegraded":"2024-02-22T19:33:45.173675+0000","last_fullsized":"2024-02-22T19:33:45.173675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.153165+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:33:45.153165+0000","last_peered":"2024-02-22T19:33:45.153165+0000","last_clean":"2024-02-22T19:33:45.153165+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:33:45.153165+0000","last_undegraded":"2024-02-22T19:33:45.153165+0000","last_fullsized":"2024-02-22T19:33:45.153165+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":98,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.528085+0000","last_change":"2024-02-22T19:33:45.528051+0000","last_active":"2024-02-22T19:33:45.528085+0000","last_peered":"2024-02-22T19:33:45.528085+0000","last_clean":"2024-02-22T19:33:45.528085+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:33:45.528085+0000","last_undegraded":"2024-02-22T19:33:45.528085+0000","last_fullsized":"2024-02-22T19:33:45.528085+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.527956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.527956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173009+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:33:45.173009+0000","last_peered":"2024-02-22T19:33:45.173009+0000","last_clean":"2024-02-22T19:33:45.173009+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:33:45.173009+0000","last_undegraded":"2024-02-22T19:33:45.173009+0000","last_fullsized":"2024-02-22T19:33:45.173009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":95,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.168815+0000","last_change":"2024-02-22T19:31:50.014237+0000","last_active":"2024-02-22T19:33:45.168815+0000","last_peered":"2024-02-22T19:33:45.168815+0000","last_clean":"2024-02-22T19:33:45.168815+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:33:45.168815+0000","last_undegraded":"2024-02-22T19:33:45.168815+0000","last_fullsized":"2024-02-22T19:33:45.168815+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":87,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173357+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:33:45.173357+0000","last_peered":"2024-02-22T19:33:45.173357+0000","last_clean":"2024-02-22T19:33:45.173357+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:33:45.173357+0000","last_undegraded":"2024-02-22T19:33:45.173357+0000","last_fullsized":"2024-02-22T19:33:45.173357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152221+0000","last_change":"2024-02-22T19:31:23.009097+0000","last_active":"2024-02-22T19:33:45.152221+0000","last_peered":"2024-02-22T19:33:45.152221+0000","last_clean":"2024-02-22T19:33:45.152221+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:33:45.152221+0000","last_undegraded":"2024-02-22T19:33:45.152221+0000","last_fullsized":"2024-02-22T19:33:45.152221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.009021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:47.757619+0000","last_change":"2024-02-22T19:33:47.757576+0000","last_active":"2024-02-22T19:33:47.757619+0000","last_peered":"2024-02-22T19:33:47.757619+0000","last_clean":"2024-02-22T19:33:47.757619+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:33:47.757619+0000","last_undegraded":"2024-02-22T19:33:47.757619+0000","last_fullsized":"2024-02-22T19:33:47.757619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:47.757479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:47.757479+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172846+0000","last_change":"2024-02-22T19:33:42.862634+0000","last_active":"2024-02-22T19:33:45.172846+0000","last_peered":"2024-02-22T19:33:45.172846+0000","last_clean":"2024-02-22T19:33:45.172846+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:33:45.172846+0000","last_undegraded":"2024-02-22T19:33:45.172846+0000","last_fullsized":"2024-02-22T19:33:45.172846+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.862563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.862563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.156306+0000","last_change":"2024-02-22T19:31:50.012381+0000","last_active":"2024-02-22T19:33:45.156306+0000","last_peered":"2024-02-22T19:33:45.156306+0000","last_clean":"2024-02-22T19:33:45.156306+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:33:45.156306+0000","last_undegraded":"2024-02-22T19:33:45.156306+0000","last_fullsized":"2024-02-22T19:33:45.156306+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175176+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:33:45.175176+0000","last_peered":"2024-02-22T19:33:45.175176+0000","last_clean":"2024-02-22T19:33:45.175176+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:33:45.175176+0000","last_undegraded":"2024-02-22T19:33:45.175176+0000","last_fullsized":"2024-02-22T19:33:45.175176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175337+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:33:45.175337+0000","last_peered":"2024-02-22T19:33:45.175337+0000","last_clean":"2024-02-22T19:33:45.175337+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:33:45.175337+0000","last_undegraded":"2024-02-22T19:33:45.175337+0000","last_fullsized":"2024-02-22T19:33:45.175337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171314+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:33:45.171314+0000","last_peered":"2024-02-22T19:33:45.171314+0000","last_clean":"2024-02-22T19:33:45.171314+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:33:45.171314+0000","last_undegraded":"2024-02-22T19:33:45.171314+0000","last_fullsized":"2024-02-22T19:33:45.171314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":99,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.567693+0000","last_change":"2024-02-22T19:31:50.105130+0000","last_active":"2024-02-22T19:33:54.567693+0000","last_peered":"2024-02-22T19:33:54.567693+0000","last_clean":"2024-02-22T19:33:54.567693+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:33:54.567693+0000","last_undegraded":"2024-02-22T19:33:54.567693+0000","last_fullsized":"2024-02-22T19:33:54.567693+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:29.679183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.679183+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154155+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:33:45.154155+0000","last_peered":"2024-02-22T19:33:45.154155+0000","last_clean":"2024-02-22T19:33:45.154155+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:33:45.154155+0000","last_undegraded":"2024-02-22T19:33:45.154155+0000","last_fullsized":"2024-02-22T19:33:45.154155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177832+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:33:45.177832+0000","last_peered":"2024-02-22T19:33:45.177832+0000","last_clean":"2024-02-22T19:33:45.177832+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:33:45.177832+0000","last_undegraded":"2024-02-22T19:33:45.177832+0000","last_fullsized":"2024-02-22T19:33:45.177832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.835432+0000","last_change":"2024-02-22T19:33:58.835432+0000","last_active":"2024-02-22T19:33:58.835432+0000","last_peered":"2024-02-22T19:33:58.835432+0000","last_clean":"2024-02-22T19:33:58.835432+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:33:58.835432+0000","last_undegraded":"2024-02-22T19:33:58.835432+0000","last_fullsized":"2024-02-22T19:33:58.835432+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.835370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.835370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":155,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.574295+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:33:54.574295+0000","last_peered":"2024-02-22T19:33:54.574295+0000","last_clean":"2024-02-22T19:33:54.574295+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:33:54.574295+0000","last_undegraded":"2024-02-22T19:33:54.574295+0000","last_fullsized":"2024-02-22T19:33:54.574295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.169557+0000","last_change":"2024-02-22T19:31:11.415809+0000","last_active":"2024-02-22T19:33:45.169557+0000","last_peered":"2024-02-22T19:33:45.169557+0000","last_clean":"2024-02-22T19:33:45.169557+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:33:45.169557+0000","last_undegraded":"2024-02-22T19:33:45.169557+0000","last_fullsized":"2024-02-22T19:33:45.169557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.415718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":117,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152529+0000","last_change":"2024-02-22T19:31:50.352892+0000","last_active":"2024-02-22T19:33:45.152529+0000","last_peered":"2024-02-22T19:33:45.152529+0000","last_clean":"2024-02-22T19:33:45.152529+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:33:45.152529+0000","last_undegraded":"2024-02-22T19:33:45.152529+0000","last_fullsized":"2024-02-22T19:33:45.152529+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.685295+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":93,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:57.874765+0000","last_change":"2024-02-22T19:33:57.874765+0000","last_active":"2024-02-22T19:33:57.874765+0000","last_peered":"2024-02-22T19:33:57.874765+0000","last_clean":"2024-02-22T19:33:57.874765+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:33:57.874765+0000","last_undegraded":"2024-02-22T19:33:57.874765+0000","last_fullsized":"2024-02-22T19:33:57.874765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.874679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.874679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":87,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173357+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:33:45.173357+0000","last_peered":"2024-02-22T19:33:45.173357+0000","last_clean":"2024-02-22T19:33:45.173357+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:33:45.173357+0000","last_undegraded":"2024-02-22T19:33:45.173357+0000","last_fullsized":"2024-02-22T19:33:45.173357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:53.652573+0000","last_change":"2024-02-22T19:33:53.652545+0000","last_active":"2024-02-22T19:33:53.652573+0000","last_peered":"2024-02-22T19:33:53.652573+0000","last_clean":"2024-02-22T19:33:53.652573+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:33:53.652573+0000","last_undegraded":"2024-02-22T19:33:53.652573+0000","last_fullsized":"2024-02-22T19:33:53.652573+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:53.652468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.652468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.001397+0000","last_change":"2024-02-22T19:33:46.001358+0000","last_active":"2024-02-22T19:33:46.001397+0000","last_peered":"2024-02-22T19:33:46.001397+0000","last_clean":"2024-02-22T19:33:46.001397+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:33:46.001397+0000","last_undegraded":"2024-02-22T19:33:46.001397+0000","last_fullsized":"2024-02-22T19:33:46.001397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:46.001290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.001290+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":91,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:56.634433+0000","last_change":"2024-02-22T19:33:56.634433+0000","last_active":"2024-02-22T19:33:56.634433+0000","last_peered":"2024-02-22T19:33:56.634433+0000","last_clean":"2024-02-22T19:33:56.634433+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:33:56.634433+0000","last_undegraded":"2024-02-22T19:33:56.634433+0000","last_fullsized":"2024-02-22T19:33:56.634433+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:56.634370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:56.634370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.180485+0000","last_change":"2024-02-22T19:31:50.013893+0000","last_active":"2024-02-22T19:33:45.180485+0000","last_peered":"2024-02-22T19:33:45.180485+0000","last_clean":"2024-02-22T19:33:45.180485+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:33:45.180485+0000","last_undegraded":"2024-02-22T19:33:45.180485+0000","last_fullsized":"2024-02-22T19:33:45.180485+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":102,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.147636+0000","last_change":"2024-02-22T19:31:50.006289+0000","last_active":"2024-02-22T19:33:45.147636+0000","last_peered":"2024-02-22T19:33:45.147636+0000","last_clean":"2024-02-22T19:33:45.147636+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:33:45.147636+0000","last_undegraded":"2024-02-22T19:33:45.147636+0000","last_fullsized":"2024-02-22T19:33:45.147636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:52.788186+0000","last_change":"2024-02-22T19:33:52.788152+0000","last_active":"2024-02-22T19:33:52.788186+0000","last_peered":"2024-02-22T19:33:52.788186+0000","last_clean":"2024-02-22T19:33:52.788186+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:33:52.788186+0000","last_undegraded":"2024-02-22T19:33:52.788186+0000","last_fullsized":"2024-02-22T19:33:52.788186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:33:52.788050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.788050+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"137'7992","reported_seq":10611,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.225398+0000","last_change":"2024-02-22T19:32:06.969062+0000","last_active":"2024-02-22T19:33:59.225398+0000","last_peered":"2024-02-22T19:33:59.225398+0000","last_clean":"2024-02-22T19:33:59.225398+0000","last_became_active":"2024-02-22T19:32:06.966453+0000","last_became_peered":"2024-02-22T19:32:06.966453+0000","last_unstale":"2024-02-22T19:33:59.225398+0000","last_undegraded":"2024-02-22T19:33:59.225398+0000","last_fullsized":"2024-02-22T19:33:59.225398+0000","mapping_epoch":104,"log_start":"113'3400","ondisk_log_start":"113'3400","created":93,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4592,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":207880192,"num_objects":3172,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7992,"num_write_kb":357248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":54,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.804958+0000","last_change":"2024-02-22T19:33:49.804924+0000","last_active":"2024-02-22T19:33:49.804958+0000","last_peered":"2024-02-22T19:33:49.804958+0000","last_clean":"2024-02-22T19:33:49.804958+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:33:49.804958+0000","last_undegraded":"2024-02-22T19:33:49.804958+0000","last_fullsized":"2024-02-22T19:33:49.804958+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.804842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.804842+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":107,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152380+0000","last_change":"2024-02-22T19:30:17.547632+0000","last_active":"2024-02-22T19:33:45.152380+0000","last_peered":"2024-02-22T19:33:45.152380+0000","last_clean":"2024-02-22T19:33:45.152380+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:33:45.152380+0000","last_undegraded":"2024-02-22T19:33:45.152380+0000","last_fullsized":"2024-02-22T19:33:45.152380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.547567+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":102,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.153076+0000","last_change":"2024-02-22T19:31:50.611447+0000","last_active":"2024-02-22T19:33:45.153076+0000","last_peered":"2024-02-22T19:33:45.153076+0000","last_clean":"2024-02-22T19:33:45.153076+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:33:45.153076+0000","last_undegraded":"2024-02-22T19:33:45.153076+0000","last_fullsized":"2024-02-22T19:33:45.153076+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":156,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.576796+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:33:54.576796+0000","last_peered":"2024-02-22T19:33:54.576796+0000","last_clean":"2024-02-22T19:33:54.576796+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:33:54.576796+0000","last_undegraded":"2024-02-22T19:33:54.576796+0000","last_fullsized":"2024-02-22T19:33:54.576796+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"137'7965","reported_seq":10584,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.310778+0000","last_change":"2024-02-22T19:33:42.253020+0000","last_active":"2024-02-22T19:33:59.310778+0000","last_peered":"2024-02-22T19:33:59.310778+0000","last_clean":"2024-02-22T19:33:59.310778+0000","last_became_active":"2024-02-22T19:33:42.130070+0000","last_became_peered":"2024-02-22T19:33:42.130070+0000","last_unstale":"2024-02-22T19:33:59.310778+0000","last_undegraded":"2024-02-22T19:33:59.310778+0000","last_fullsized":"2024-02-22T19:33:59.310778+0000","mapping_epoch":134,"log_start":"117'3800","ondisk_log_start":"117'3800","created":93,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4165,"ondisk_log_size":4165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":205848576,"num_objects":3141,"num_object_clones":0,"num_object_copies":9423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7965,"num_write_kb":355392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1273,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173514+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:33:45.173514+0000","last_peered":"2024-02-22T19:33:45.173514+0000","last_clean":"2024-02-22T19:33:45.173514+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:33:45.173514+0000","last_undegraded":"2024-02-22T19:33:45.173514+0000","last_fullsized":"2024-02-22T19:33:45.173514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":97,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.606520+0000","last_change":"2024-02-22T19:33:54.606520+0000","last_active":"2024-02-22T19:33:54.606520+0000","last_peered":"2024-02-22T19:33:54.606520+0000","last_clean":"2024-02-22T19:33:54.606520+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:33:54.606520+0000","last_undegraded":"2024-02-22T19:33:54.606520+0000","last_fullsized":"2024-02-22T19:33:54.606520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:54.606418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.606418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174883+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:33:45.174883+0000","last_peered":"2024-02-22T19:33:45.174883+0000","last_clean":"2024-02-22T19:33:45.174883+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:33:45.174883+0000","last_undegraded":"2024-02-22T19:33:45.174883+0000","last_fullsized":"2024-02-22T19:33:45.174883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.166524+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:33:45.166524+0000","last_peered":"2024-02-22T19:33:45.166524+0000","last_clean":"2024-02-22T19:33:45.166524+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:33:45.166524+0000","last_undegraded":"2024-02-22T19:33:45.166524+0000","last_fullsized":"2024-02-22T19:33:45.166524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"137'8070","reported_seq":8122,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.269094+0000","last_change":"2024-02-22T19:31:33.014127+0000","last_active":"2024-02-22T19:33:59.269094+0000","last_peered":"2024-02-22T19:33:59.269094+0000","last_clean":"2024-02-22T19:33:59.269094+0000","last_became_active":"2024-02-22T19:31:33.013760+0000","last_became_peered":"2024-02-22T19:31:33.013760+0000","last_unstale":"2024-02-22T19:33:59.269094+0000","last_undegraded":"2024-02-22T19:33:59.269094+0000","last_fullsized":"2024-02-22T19:33:59.269094+0000","mapping_epoch":93,"log_start":"111'3200","ondisk_log_start":"111'3200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4870,"ondisk_log_size":4870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208535552,"num_objects":3182,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8070,"num_write_kb":360064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152948+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:33:45.152948+0000","last_peered":"2024-02-22T19:33:45.152948+0000","last_clean":"2024-02-22T19:33:45.152948+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:33:45.152948+0000","last_undegraded":"2024-02-22T19:33:45.152948+0000","last_fullsized":"2024-02-22T19:33:45.152948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.766849+0000","last_change":"2024-02-22T19:33:48.766810+0000","last_active":"2024-02-22T19:33:48.766849+0000","last_peered":"2024-02-22T19:33:48.766849+0000","last_clean":"2024-02-22T19:33:48.766849+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:33:48.766849+0000","last_undegraded":"2024-02-22T19:33:48.766849+0000","last_fullsized":"2024-02-22T19:33:48.766849+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.766706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.766706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":106,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152429+0000","last_change":"2024-02-22T19:31:49.999189+0000","last_active":"2024-02-22T19:33:45.152429+0000","last_peered":"2024-02-22T19:33:45.152429+0000","last_clean":"2024-02-22T19:33:45.152429+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:33:45.152429+0000","last_undegraded":"2024-02-22T19:33:45.152429+0000","last_fullsized":"2024-02-22T19:33:45.152429+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:15.882992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.882992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":125,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:53.782561+0000","last_change":"2024-02-22T19:33:53.782532+0000","last_active":"2024-02-22T19:33:53.782561+0000","last_peered":"2024-02-22T19:33:53.782561+0000","last_clean":"2024-02-22T19:33:53.782561+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:33:53.782561+0000","last_undegraded":"2024-02-22T19:33:53.782561+0000","last_fullsized":"2024-02-22T19:33:53.782561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:33:53.782465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.782465+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"137'7949","reported_seq":8001,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.268175+0000","last_change":"2024-02-22T19:31:33.015434+0000","last_active":"2024-02-22T19:33:59.268175+0000","last_peered":"2024-02-22T19:33:59.268175+0000","last_clean":"2024-02-22T19:33:59.268175+0000","last_became_active":"2024-02-22T19:31:33.015132+0000","last_became_peered":"2024-02-22T19:31:33.015132+0000","last_unstale":"2024-02-22T19:33:59.268175+0000","last_undegraded":"2024-02-22T19:33:59.268175+0000","last_fullsized":"2024-02-22T19:33:59.268175+0000","mapping_epoch":93,"log_start":"106'2600","ondisk_log_start":"106'2600","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5349,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":202571776,"num_objects":3091,"num_object_clones":0,"num_object_copies":9273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.170135+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:33:45.170135+0000","last_peered":"2024-02-22T19:33:45.170135+0000","last_clean":"2024-02-22T19:33:45.170135+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:33:45.170135+0000","last_undegraded":"2024-02-22T19:33:45.170135+0000","last_fullsized":"2024-02-22T19:33:45.170135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":95,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.910213+0000","last_change":"2024-02-22T19:33:58.910213+0000","last_active":"2024-02-22T19:33:58.910213+0000","last_peered":"2024-02-22T19:33:58.910213+0000","last_clean":"2024-02-22T19:33:58.910213+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:33:58.910213+0000","last_undegraded":"2024-02-22T19:33:58.910213+0000","last_fullsized":"2024-02-22T19:33:58.910213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.910117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.910117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177902+0000","last_change":"2024-02-22T19:31:50.014265+0000","last_active":"2024-02-22T19:33:45.177902+0000","last_peered":"2024-02-22T19:33:45.177902+0000","last_clean":"2024-02-22T19:33:45.177902+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:33:45.177902+0000","last_undegraded":"2024-02-22T19:33:45.177902+0000","last_fullsized":"2024-02-22T19:33:45.177902+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":125,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172477+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:33:45.172477+0000","last_peered":"2024-02-22T19:33:45.172477+0000","last_clean":"2024-02-22T19:33:45.172477+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:33:45.172477+0000","last_undegraded":"2024-02-22T19:33:45.172477+0000","last_fullsized":"2024-02-22T19:33:45.172477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"137'7942","reported_seq":7994,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.142005+0000","last_change":"2024-02-22T19:31:33.014009+0000","last_active":"2024-02-22T19:33:59.142005+0000","last_peered":"2024-02-22T19:33:59.142005+0000","last_clean":"2024-02-22T19:33:59.142005+0000","last_became_active":"2024-02-22T19:31:33.013715+0000","last_became_peered":"2024-02-22T19:31:33.013715+0000","last_unstale":"2024-02-22T19:33:59.142005+0000","last_undegraded":"2024-02-22T19:33:59.142005+0000","last_fullsized":"2024-02-22T19:33:59.142005+0000","mapping_epoch":93,"log_start":"106'2500","ondisk_log_start":"106'2500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5442,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":203030528,"num_objects":3098,"num_object_clones":0,"num_object_copies":9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.180658+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:33:45.180658+0000","last_peered":"2024-02-22T19:33:45.180658+0000","last_clean":"2024-02-22T19:33:45.180658+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:33:45.180658+0000","last_undegraded":"2024-02-22T19:33:45.180658+0000","last_fullsized":"2024-02-22T19:33:45.180658+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.156100+0000","last_change":"2024-02-22T19:31:50.010144+0000","last_active":"2024-02-22T19:33:45.156100+0000","last_peered":"2024-02-22T19:33:45.156100+0000","last_clean":"2024-02-22T19:33:45.156100+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:33:45.156100+0000","last_undegraded":"2024-02-22T19:33:45.156100+0000","last_fullsized":"2024-02-22T19:33:45.156100+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.692694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175198+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:33:45.175198+0000","last_peered":"2024-02-22T19:33:45.175198+0000","last_clean":"2024-02-22T19:33:45.175198+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:33:45.175198+0000","last_undegraded":"2024-02-22T19:33:45.175198+0000","last_fullsized":"2024-02-22T19:33:45.175198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":205,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.566953+0000","last_change":"2024-02-22T19:33:46.724569+0000","last_active":"2024-02-22T19:33:54.566953+0000","last_peered":"2024-02-22T19:33:54.566953+0000","last_clean":"2024-02-22T19:33:54.566953+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:33:54.566953+0000","last_undegraded":"2024-02-22T19:33:54.566953+0000","last_fullsized":"2024-02-22T19:33:54.566953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:33:46.724462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.724462+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":93,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.185263+0000","last_change":"2024-02-22T19:33:58.185263+0000","last_active":"2024-02-22T19:33:58.185263+0000","last_peered":"2024-02-22T19:33:58.185263+0000","last_clean":"2024-02-22T19:33:58.185263+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:33:58.185263+0000","last_undegraded":"2024-02-22T19:33:58.185263+0000","last_fullsized":"2024-02-22T19:33:58.185263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.185187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.185187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"137'7889","reported_seq":7941,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.269696+0000","last_change":"2024-02-22T19:31:33.008987+0000","last_active":"2024-02-22T19:33:59.269696+0000","last_peered":"2024-02-22T19:33:59.269696+0000","last_clean":"2024-02-22T19:33:59.269696+0000","last_became_active":"2024-02-22T19:31:33.008726+0000","last_became_peered":"2024-02-22T19:31:33.008726+0000","last_unstale":"2024-02-22T19:33:59.269696+0000","last_undegraded":"2024-02-22T19:33:59.269696+0000","last_fullsized":"2024-02-22T19:33:59.269696+0000","mapping_epoch":93,"log_start":"109'3000","ondisk_log_start":"109'3000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4889,"ondisk_log_size":4889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":206241792,"num_objects":3147,"num_object_clones":0,"num_object_copies":9441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7889,"num_write_kb":353152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152537+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:33:45.152537+0000","last_peered":"2024-02-22T19:33:45.152537+0000","last_clean":"2024-02-22T19:33:45.152537+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:33:45.152537+0000","last_undegraded":"2024-02-22T19:33:45.152537+0000","last_fullsized":"2024-02-22T19:33:45.152537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155357+0000","last_change":"2024-02-22T19:30:36.106987+0000","last_active":"2024-02-22T19:33:45.155357+0000","last_peered":"2024-02-22T19:33:45.155357+0000","last_clean":"2024-02-22T19:33:45.155357+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:33:45.155357+0000","last_undegraded":"2024-02-22T19:33:45.155357+0000","last_fullsized":"2024-02-22T19:33:45.155357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.106920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.106920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.158966+0000","last_change":"2024-02-22T19:33:44.828664+0000","last_active":"2024-02-22T19:33:45.158966+0000","last_peered":"2024-02-22T19:33:45.158966+0000","last_clean":"2024-02-22T19:33:45.158966+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:33:45.158966+0000","last_undegraded":"2024-02-22T19:33:45.158966+0000","last_fullsized":"2024-02-22T19:33:45.158966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:44.828568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.828568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":101,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.159854+0000","last_change":"2024-02-22T19:31:50.458765+0000","last_active":"2024-02-22T19:33:45.159854+0000","last_peered":"2024-02-22T19:33:45.159854+0000","last_clean":"2024-02-22T19:33:45.159854+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:33:45.159854+0000","last_undegraded":"2024-02-22T19:33:45.159854+0000","last_fullsized":"2024-02-22T19:33:45.159854+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:30:28.128752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.128752+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.825770+0000","last_change":"2024-02-22T19:33:46.825747+0000","last_active":"2024-02-22T19:33:46.825770+0000","last_peered":"2024-02-22T19:33:46.825770+0000","last_clean":"2024-02-22T19:33:46.825770+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:33:46.825770+0000","last_undegraded":"2024-02-22T19:33:46.825770+0000","last_fullsized":"2024-02-22T19:33:46.825770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:46.825679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.825679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"137'8004","reported_seq":8056,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.268783+0000","last_change":"2024-02-22T19:31:33.014815+0000","last_active":"2024-02-22T19:33:59.268783+0000","last_peered":"2024-02-22T19:33:59.268783+0000","last_clean":"2024-02-22T19:33:59.268783+0000","last_became_active":"2024-02-22T19:31:33.014489+0000","last_became_peered":"2024-02-22T19:31:33.014489+0000","last_unstale":"2024-02-22T19:33:59.268783+0000","last_undegraded":"2024-02-22T19:33:59.268783+0000","last_fullsized":"2024-02-22T19:33:59.268783+0000","mapping_epoch":93,"log_start":"109'3100","ondisk_log_start":"109'3100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4904,"ondisk_log_size":4904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":206831616,"num_objects":3156,"num_object_clones":0,"num_object_copies":9468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8004,"num_write_kb":357120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177528+0000","last_change":"2024-02-22T19:31:50.014877+0000","last_active":"2024-02-22T19:33:45.177528+0000","last_peered":"2024-02-22T19:33:45.177528+0000","last_clean":"2024-02-22T19:33:45.177528+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:33:45.177528+0000","last_undegraded":"2024-02-22T19:33:45.177528+0000","last_fullsized":"2024-02-22T19:33:45.177528+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152994+0000","last_change":"2024-02-22T19:31:11.906662+0000","last_active":"2024-02-22T19:33:45.152994+0000","last_peered":"2024-02-22T19:33:45.152994+0000","last_clean":"2024-02-22T19:33:45.152994+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:33:45.152994+0000","last_undegraded":"2024-02-22T19:33:45.152994+0000","last_fullsized":"2024-02-22T19:33:45.152994+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.906561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.906561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173831+0000","last_change":"2024-02-22T19:31:50.009641+0000","last_active":"2024-02-22T19:33:45.173831+0000","last_peered":"2024-02-22T19:33:45.173831+0000","last_clean":"2024-02-22T19:33:45.173831+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:33:45.173831+0000","last_undegraded":"2024-02-22T19:33:45.173831+0000","last_fullsized":"2024-02-22T19:33:45.173831+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":118,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.148719+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:33:45.148719+0000","last_peered":"2024-02-22T19:33:45.148719+0000","last_clean":"2024-02-22T19:33:45.148719+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:33:45.148719+0000","last_undegraded":"2024-02-22T19:33:45.148719+0000","last_fullsized":"2024-02-22T19:33:45.148719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152287+0000","last_change":"2024-02-22T19:31:49.995726+0000","last_active":"2024-02-22T19:33:45.152287+0000","last_peered":"2024-02-22T19:33:45.152287+0000","last_clean":"2024-02-22T19:33:45.152287+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:33:45.152287+0000","last_undegraded":"2024-02-22T19:33:45.152287+0000","last_fullsized":"2024-02-22T19:33:45.152287+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"137'8007","reported_seq":8058,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.211671+0000","last_change":"2024-02-22T19:31:33.010431+0000","last_active":"2024-02-22T19:33:59.211671+0000","last_peered":"2024-02-22T19:33:59.211671+0000","last_clean":"2024-02-22T19:33:59.211671+0000","last_became_active":"2024-02-22T19:31:33.010187+0000","last_became_peered":"2024-02-22T19:31:33.010187+0000","last_unstale":"2024-02-22T19:33:59.211671+0000","last_undegraded":"2024-02-22T19:33:59.211671+0000","last_fullsized":"2024-02-22T19:33:59.211671+0000","mapping_epoch":93,"log_start":"110'3200","ondisk_log_start":"110'3200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4807,"ondisk_log_size":4807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":205455360,"num_objects":3135,"num_object_clones":0,"num_object_copies":9405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8007,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154191+0000","last_change":"2024-02-22T19:31:50.012369+0000","last_active":"2024-02-22T19:33:45.154191+0000","last_peered":"2024-02-22T19:33:45.154191+0000","last_clean":"2024-02-22T19:33:45.154191+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:33:45.154191+0000","last_undegraded":"2024-02-22T19:33:45.154191+0000","last_fullsized":"2024-02-22T19:33:45.154191+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.169943+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:33:45.169943+0000","last_peered":"2024-02-22T19:33:45.169943+0000","last_clean":"2024-02-22T19:33:45.169943+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:33:45.169943+0000","last_undegraded":"2024-02-22T19:33:45.169943+0000","last_fullsized":"2024-02-22T19:33:45.169943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":392,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:57.584190+0000","last_change":"2024-02-22T19:31:29.001184+0000","last_active":"2024-02-22T19:33:57.584190+0000","last_peered":"2024-02-22T19:33:57.584190+0000","last_clean":"2024-02-22T19:33:57.584190+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:33:57.584190+0000","last_undegraded":"2024-02-22T19:33:57.584190+0000","last_fullsized":"2024-02-22T19:33:57.584190+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:31:29.001098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.001098+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":292,"num_read_kb":287,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171394+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:33:45.171394+0000","last_peered":"2024-02-22T19:33:45.171394+0000","last_clean":"2024-02-22T19:33:45.171394+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:33:45.171394+0000","last_undegraded":"2024-02-22T19:33:45.171394+0000","last_fullsized":"2024-02-22T19:33:45.171394+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154164+0000","last_change":"2024-02-22T19:31:50.010171+0000","last_active":"2024-02-22T19:33:45.154164+0000","last_peered":"2024-02-22T19:33:45.154164+0000","last_clean":"2024-02-22T19:33:45.154164+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:33:45.154164+0000","last_undegraded":"2024-02-22T19:33:45.154164+0000","last_fullsized":"2024-02-22T19:33:45.154164+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.584822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.584822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"137'8032","reported_seq":10728,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.190910+0000","last_change":"2024-02-22T19:32:07.977104+0000","last_active":"2024-02-22T19:33:59.190910+0000","last_peered":"2024-02-22T19:33:59.190910+0000","last_clean":"2024-02-22T19:33:59.190910+0000","last_became_active":"2024-02-22T19:32:07.973466+0000","last_became_peered":"2024-02-22T19:32:07.973466+0000","last_unstale":"2024-02-22T19:33:59.190910+0000","last_undegraded":"2024-02-22T19:33:59.190910+0000","last_fullsized":"2024-02-22T19:33:59.190910+0000","mapping_epoch":105,"log_start":"113'3400","ondisk_log_start":"113'3400","created":93,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4632,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":203816960,"num_objects":3110,"num_object_clones":0,"num_object_copies":9330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8032,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":86048768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154675+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:33:45.154675+0000","last_peered":"2024-02-22T19:33:45.154675+0000","last_clean":"2024-02-22T19:33:45.154675+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:33:45.154675+0000","last_undegraded":"2024-02-22T19:33:45.154675+0000","last_fullsized":"2024-02-22T19:33:45.154675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154381+0000","last_change":"2024-02-22T19:31:50.010143+0000","last_active":"2024-02-22T19:33:45.154381+0000","last_peered":"2024-02-22T19:33:45.154381+0000","last_clean":"2024-02-22T19:33:45.154381+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:33:45.154381+0000","last_undegraded":"2024-02-22T19:33:45.154381+0000","last_fullsized":"2024-02-22T19:33:45.154381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":153,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172271+0000","last_change":"2024-02-22T19:31:50.036817+0000","last_active":"2024-02-22T19:33:45.172271+0000","last_peered":"2024-02-22T19:33:45.172271+0000","last_clean":"2024-02-22T19:33:45.172271+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:33:45.172271+0000","last_undegraded":"2024-02-22T19:33:45.172271+0000","last_fullsized":"2024-02-22T19:33:45.172271+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:30:22.683514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.683514+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":98,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.147739+0000","last_change":"2024-02-22T19:31:50.006906+0000","last_active":"2024-02-22T19:33:45.147739+0000","last_peered":"2024-02-22T19:33:45.147739+0000","last_clean":"2024-02-22T19:33:45.147739+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:33:45.147739+0000","last_undegraded":"2024-02-22T19:33:45.147739+0000","last_fullsized":"2024-02-22T19:33:45.147739+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"137'7861","reported_seq":7913,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.196363+0000","last_change":"2024-02-22T19:31:33.012064+0000","last_active":"2024-02-22T19:33:59.196363+0000","last_peered":"2024-02-22T19:33:59.196363+0000","last_clean":"2024-02-22T19:33:59.196363+0000","last_became_active":"2024-02-22T19:31:33.011750+0000","last_became_peered":"2024-02-22T19:31:33.011750+0000","last_unstale":"2024-02-22T19:33:59.196363+0000","last_undegraded":"2024-02-22T19:33:59.196363+0000","last_fullsized":"2024-02-22T19:33:59.196363+0000","mapping_epoch":93,"log_start":"113'3300","ondisk_log_start":"113'3300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4561,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":209518592,"num_objects":3197,"num_object_clones":0,"num_object_copies":9591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7861,"num_write_kb":353856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.035705+0000","last_change":"2024-02-22T19:33:49.035679+0000","last_active":"2024-02-22T19:33:49.035705+0000","last_peered":"2024-02-22T19:33:49.035705+0000","last_clean":"2024-02-22T19:33:49.035705+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:33:49.035705+0000","last_undegraded":"2024-02-22T19:33:49.035705+0000","last_fullsized":"2024-02-22T19:33:49.035705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.035615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.035615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174684+0000","last_change":"2024-02-22T19:31:50.006662+0000","last_active":"2024-02-22T19:33:45.174684+0000","last_peered":"2024-02-22T19:33:45.174684+0000","last_clean":"2024-02-22T19:33:45.174684+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:33:45.174684+0000","last_undegraded":"2024-02-22T19:33:45.174684+0000","last_fullsized":"2024-02-22T19:33:45.174684+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":133,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.156399+0000","last_change":"2024-02-22T19:33:44.757231+0000","last_active":"2024-02-22T19:33:45.156399+0000","last_peered":"2024-02-22T19:33:45.156399+0000","last_clean":"2024-02-22T19:33:45.156399+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:33:45.156399+0000","last_undegraded":"2024-02-22T19:33:45.156399+0000","last_fullsized":"2024-02-22T19:33:45.156399+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:33:44.757114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.757114+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":169,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.576485+0000","last_change":"2024-02-22T19:31:50.440080+0000","last_active":"2024-02-22T19:33:54.576485+0000","last_peered":"2024-02-22T19:33:54.576485+0000","last_clean":"2024-02-22T19:33:54.576485+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:33:54.576485+0000","last_undegraded":"2024-02-22T19:33:54.576485+0000","last_fullsized":"2024-02-22T19:33:54.576485+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"137'8001","reported_seq":8053,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.225275+0000","last_change":"2024-02-22T19:31:33.012817+0000","last_active":"2024-02-22T19:33:59.225275+0000","last_peered":"2024-02-22T19:33:59.225275+0000","last_clean":"2024-02-22T19:33:59.225275+0000","last_became_active":"2024-02-22T19:31:33.012470+0000","last_became_peered":"2024-02-22T19:31:33.012470+0000","last_unstale":"2024-02-22T19:33:59.225275+0000","last_undegraded":"2024-02-22T19:33:59.225275+0000","last_fullsized":"2024-02-22T19:33:59.225275+0000","mapping_epoch":93,"log_start":"117'3800","ondisk_log_start":"117'3800","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4201,"ondisk_log_size":4201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":207159296,"num_objects":3161,"num_object_clones":0,"num_object_copies":9483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8001,"num_write_kb":357184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154357+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:33:45.154357+0000","last_peered":"2024-02-22T19:33:45.154357+0000","last_clean":"2024-02-22T19:33:45.154357+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:33:45.154357+0000","last_undegraded":"2024-02-22T19:33:45.154357+0000","last_fullsized":"2024-02-22T19:33:45.154357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171462+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:33:45.171462+0000","last_peered":"2024-02-22T19:33:45.171462+0000","last_clean":"2024-02-22T19:33:45.171462+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:33:45.171462+0000","last_undegraded":"2024-02-22T19:33:45.171462+0000","last_fullsized":"2024-02-22T19:33:45.171462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":145,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174774+0000","last_change":"2024-02-22T19:31:50.644540+0000","last_active":"2024-02-22T19:33:45.174774+0000","last_peered":"2024-02-22T19:33:45.174774+0000","last_clean":"2024-02-22T19:33:45.174774+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:33:45.174774+0000","last_undegraded":"2024-02-22T19:33:45.174774+0000","last_fullsized":"2024-02-22T19:33:45.174774+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":155,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.570954+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:33:54.570954+0000","last_peered":"2024-02-22T19:33:54.570954+0000","last_clean":"2024-02-22T19:33:54.570954+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:33:54.570954+0000","last_undegraded":"2024-02-22T19:33:54.570954+0000","last_fullsized":"2024-02-22T19:33:54.570954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"137'8190","reported_seq":8242,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.257292+0000","last_change":"2024-02-22T19:31:33.013118+0000","last_active":"2024-02-22T19:33:59.257292+0000","last_peered":"2024-02-22T19:33:59.257292+0000","last_clean":"2024-02-22T19:33:59.257292+0000","last_became_active":"2024-02-22T19:31:33.012002+0000","last_became_peered":"2024-02-22T19:31:33.012002+0000","last_unstale":"2024-02-22T19:33:59.257292+0000","last_undegraded":"2024-02-22T19:33:59.257292+0000","last_fullsized":"2024-02-22T19:33:59.257292+0000","mapping_epoch":93,"log_start":"117'4000","ondisk_log_start":"117'4000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4190,"ondisk_log_size":4190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":209059840,"num_objects":3190,"num_object_clones":0,"num_object_copies":9570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8190,"num_write_kb":364160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174707+0000","last_change":"2024-02-22T19:30:29.099766+0000","last_active":"2024-02-22T19:33:45.174707+0000","last_peered":"2024-02-22T19:33:45.174707+0000","last_clean":"2024-02-22T19:33:45.174707+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:33:45.174707+0000","last_undegraded":"2024-02-22T19:33:45.174707+0000","last_fullsized":"2024-02-22T19:33:45.174707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.099683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.099683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177373+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:33:45.177373+0000","last_peered":"2024-02-22T19:33:45.177373+0000","last_clean":"2024-02-22T19:33:45.177373+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:33:45.177373+0000","last_undegraded":"2024-02-22T19:33:45.177373+0000","last_fullsized":"2024-02-22T19:33:45.177373+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":128,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.749170+0000","last_change":"2024-02-22T19:33:54.749170+0000","last_active":"2024-02-22T19:33:54.749170+0000","last_peered":"2024-02-22T19:33:54.749170+0000","last_clean":"2024-02-22T19:33:54.749170+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:33:54.749170+0000","last_undegraded":"2024-02-22T19:33:54.749170+0000","last_fullsized":"2024-02-22T19:33:54.749170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:33:54.749061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.749061+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":95,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174663+0000","last_change":"2024-02-22T19:31:26.190739+0000","last_active":"2024-02-22T19:33:45.174663+0000","last_peered":"2024-02-22T19:33:45.174663+0000","last_clean":"2024-02-22T19:33:45.174663+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:33:45.174663+0000","last_undegraded":"2024-02-22T19:33:45.174663+0000","last_fullsized":"2024-02-22T19:33:45.174663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.190632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.190632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"137'8063","reported_seq":8115,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.194435+0000","last_change":"2024-02-22T19:31:33.012589+0000","last_active":"2024-02-22T19:33:59.194435+0000","last_peered":"2024-02-22T19:33:59.194435+0000","last_clean":"2024-02-22T19:33:59.194435+0000","last_became_active":"2024-02-22T19:31:33.012188+0000","last_became_peered":"2024-02-22T19:31:33.012188+0000","last_unstale":"2024-02-22T19:33:59.194435+0000","last_undegraded":"2024-02-22T19:33:59.194435+0000","last_fullsized":"2024-02-22T19:33:59.194435+0000","mapping_epoch":93,"log_start":"107'2700","ondisk_log_start":"107'2700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5363,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":212140032,"num_objects":3237,"num_object_clones":0,"num_object_copies":9711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":361600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155031+0000","last_change":"2024-02-22T19:31:50.009247+0000","last_active":"2024-02-22T19:33:45.155031+0000","last_peered":"2024-02-22T19:33:45.155031+0000","last_clean":"2024-02-22T19:33:45.155031+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:33:45.155031+0000","last_undegraded":"2024-02-22T19:33:45.155031+0000","last_fullsized":"2024-02-22T19:33:45.155031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:25.682652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:25.682652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":171,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.573719+0000","last_change":"2024-02-22T19:31:50.016299+0000","last_active":"2024-02-22T19:33:54.573719+0000","last_peered":"2024-02-22T19:33:54.573719+0000","last_clean":"2024-02-22T19:33:54.573719+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:33:54.573719+0000","last_undegraded":"2024-02-22T19:33:54.573719+0000","last_fullsized":"2024-02-22T19:33:54.573719+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":96,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173489+0000","last_change":"2024-02-22T19:31:14.073980+0000","last_active":"2024-02-22T19:33:45.173489+0000","last_peered":"2024-02-22T19:33:45.173489+0000","last_clean":"2024-02-22T19:33:45.173489+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:33:45.173489+0000","last_undegraded":"2024-02-22T19:33:45.173489+0000","last_fullsized":"2024-02-22T19:33:45.173489+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.073908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.073908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":89,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.159714+0000","last_change":"2024-02-22T19:33:43.827188+0000","last_active":"2024-02-22T19:33:45.159714+0000","last_peered":"2024-02-22T19:33:45.159714+0000","last_clean":"2024-02-22T19:33:45.159714+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:33:45.159714+0000","last_undegraded":"2024-02-22T19:33:45.159714+0000","last_fullsized":"2024-02-22T19:33:45.159714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:33:43.827087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.827087+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"137'8229","reported_seq":8282,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.168323+0000","last_change":"2024-02-22T19:31:33.011737+0000","last_active":"2024-02-22T19:33:59.168323+0000","last_peered":"2024-02-22T19:33:59.168323+0000","last_clean":"2024-02-22T19:33:59.168323+0000","last_became_active":"2024-02-22T19:31:33.011438+0000","last_became_peered":"2024-02-22T19:31:33.011438+0000","last_unstale":"2024-02-22T19:33:59.168323+0000","last_undegraded":"2024-02-22T19:33:59.168323+0000","last_fullsized":"2024-02-22T19:33:59.168323+0000","mapping_epoch":93,"log_start":"107'2800","ondisk_log_start":"107'2800","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5429,"ondisk_log_size":5429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":209977368,"num_objects":3205,"num_object_clones":0,"num_object_copies":9615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8229,"num_write_kb":365825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177752+0000","last_change":"2024-02-22T19:31:50.006646+0000","last_active":"2024-02-22T19:33:45.177752+0000","last_peered":"2024-02-22T19:33:45.177752+0000","last_clean":"2024-02-22T19:33:45.177752+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:33:45.177752+0000","last_undegraded":"2024-02-22T19:33:45.177752+0000","last_fullsized":"2024-02-22T19:33:45.177752+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.176040+0000","last_change":"2024-02-22T19:31:50.016416+0000","last_active":"2024-02-22T19:33:45.176040+0000","last_peered":"2024-02-22T19:33:45.176040+0000","last_clean":"2024-02-22T19:33:45.176040+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:33:45.176040+0000","last_undegraded":"2024-02-22T19:33:45.176040+0000","last_fullsized":"2024-02-22T19:33:45.176040+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.898915+0000","last_change":"2024-02-22T19:33:48.898875+0000","last_active":"2024-02-22T19:33:48.898915+0000","last_peered":"2024-02-22T19:33:48.898915+0000","last_clean":"2024-02-22T19:33:48.898915+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:33:48.898915+0000","last_undegraded":"2024-02-22T19:33:48.898915+0000","last_fullsized":"2024-02-22T19:33:48.898915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.898801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.898801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":116,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:51.953487+0000","last_change":"2024-02-22T19:33:51.953460+0000","last_active":"2024-02-22T19:33:51.953487+0000","last_peered":"2024-02-22T19:33:51.953487+0000","last_clean":"2024-02-22T19:33:51.953487+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:33:51.953487+0000","last_undegraded":"2024-02-22T19:33:51.953487+0000","last_fullsized":"2024-02-22T19:33:51.953487+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:51.953387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.953387+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"137'7822","reported_seq":10396,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.238573+0000","last_change":"2024-02-22T19:33:40.144966+0000","last_active":"2024-02-22T19:33:59.238573+0000","last_peered":"2024-02-22T19:33:59.238573+0000","last_clean":"2024-02-22T19:33:59.238573+0000","last_became_active":"2024-02-22T19:33:40.096849+0000","last_became_peered":"2024-02-22T19:33:40.096849+0000","last_unstale":"2024-02-22T19:33:59.238573+0000","last_undegraded":"2024-02-22T19:33:59.238573+0000","last_fullsized":"2024-02-22T19:33:59.238573+0000","mapping_epoch":132,"log_start":"106'2500","ondisk_log_start":"106'2500","created":93,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5322,"ondisk_log_size":5322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":196608000,"num_objects":3000,"num_object_clones":0,"num_object_copies":9000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7822,"num_write_kb":346304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1254,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":93,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:52.892072+0000","last_change":"2024-02-22T19:33:52.892034+0000","last_active":"2024-02-22T19:33:52.892072+0000","last_peered":"2024-02-22T19:33:52.892072+0000","last_clean":"2024-02-22T19:33:52.892072+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:33:52.892072+0000","last_undegraded":"2024-02-22T19:33:52.892072+0000","last_fullsized":"2024-02-22T19:33:52.892072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:52.891927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.891927+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":95,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:57.772428+0000","last_change":"2024-02-22T19:33:57.772428+0000","last_active":"2024-02-22T19:33:57.772428+0000","last_peered":"2024-02-22T19:33:57.772428+0000","last_clean":"2024-02-22T19:33:57.772428+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:33:57.772428+0000","last_undegraded":"2024-02-22T19:33:57.772428+0000","last_fullsized":"2024-02-22T19:33:57.772428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.772326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.772326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":96,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:51.803101+0000","last_change":"2024-02-22T19:33:51.803060+0000","last_active":"2024-02-22T19:33:51.803101+0000","last_peered":"2024-02-22T19:33:51.803101+0000","last_clean":"2024-02-22T19:33:51.803101+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:33:51.803101+0000","last_undegraded":"2024-02-22T19:33:51.803101+0000","last_fullsized":"2024-02-22T19:33:51.803101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:51.802966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.802966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":120,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.176101+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:33:45.176101+0000","last_peered":"2024-02-22T19:33:45.176101+0000","last_clean":"2024-02-22T19:33:45.176101+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:33:45.176101+0000","last_undegraded":"2024-02-22T19:33:45.176101+0000","last_fullsized":"2024-02-22T19:33:45.176101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"137'8029","reported_seq":8081,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.244693+0000","last_change":"2024-02-22T19:31:33.014534+0000","last_active":"2024-02-22T19:33:59.244693+0000","last_peered":"2024-02-22T19:33:59.244693+0000","last_clean":"2024-02-22T19:33:59.244693+0000","last_became_active":"2024-02-22T19:31:33.014231+0000","last_became_peered":"2024-02-22T19:31:33.014231+0000","last_unstale":"2024-02-22T19:33:59.244693+0000","last_undegraded":"2024-02-22T19:33:59.244693+0000","last_fullsized":"2024-02-22T19:33:59.244693+0000","mapping_epoch":93,"log_start":"109'3100","ondisk_log_start":"109'3100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4929,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":205979648,"num_objects":3143,"num_object_clones":0,"num_object_copies":9429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8029,"num_write_kb":357504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175778+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:33:45.175778+0000","last_peered":"2024-02-22T19:33:45.175778+0000","last_clean":"2024-02-22T19:33:45.175778+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:33:45.175778+0000","last_undegraded":"2024-02-22T19:33:45.175778+0000","last_fullsized":"2024-02-22T19:33:45.175778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":176,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.578755+0000","last_change":"2024-02-22T19:33:42.757859+0000","last_active":"2024-02-22T19:33:54.578755+0000","last_peered":"2024-02-22T19:33:54.578755+0000","last_clean":"2024-02-22T19:33:54.578755+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:33:54.578755+0000","last_undegraded":"2024-02-22T19:33:54.578755+0000","last_fullsized":"2024-02-22T19:33:54.578755+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:33:42.757755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.757755+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173268+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:33:45.173268+0000","last_peered":"2024-02-22T19:33:45.173268+0000","last_clean":"2024-02-22T19:33:45.173268+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:33:45.173268+0000","last_undegraded":"2024-02-22T19:33:45.173268+0000","last_fullsized":"2024-02-22T19:33:45.173268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":107,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175273+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:33:45.175273+0000","last_peered":"2024-02-22T19:33:45.175273+0000","last_clean":"2024-02-22T19:33:45.175273+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:33:45.175273+0000","last_undegraded":"2024-02-22T19:33:45.175273+0000","last_fullsized":"2024-02-22T19:33:45.175273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3300655128,"num_objects":50365,"num_object_clones":0,"num_object_copies":151095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128045,"num_write_kb":5709057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5115,"num_bytes_recovered":332922880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9995104256,"data_stored":9995092040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77645,"ondisk_log_size":77645,"up":48,"acting":48,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":292,"num_read_kb":287,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266751,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1331204,"kb_used_data":1021692,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92413948,"statfs":{"total":95995035648,"available":94631882752,"internally_reserved":0,"allocated":1046212608,"data_stored":1045817588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":5.0609999999999999,"5min":1.506,"15min":0.88400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":42.597999999999999,"5min":42.597999999999999,"15min":42.597999999999999},"last":5.8460000000000001},{"interface":"front","average":{"1min":3.5640000000000001,"5min":1.1499999999999999,"15min":0.73599999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":42.155999999999999,"5min":42.155999999999999,"15min":42.155999999999999},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.6139999999999999,"5min":1.1719999999999999,"15min":0.77600000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":38.188000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":3.6749999999999998},{"interface":"front","average":{"1min":4.2999999999999998,"5min":1.3089999999999999,"15min":0.79900000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":37.938000000000002,"5min":37.938000000000002,"15min":37.938000000000002},"last":1.284}]},{"osd":2,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":1.226,"15min":0.78200000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":41.640999999999998,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":4.2530000000000001,"5min":1.3560000000000001,"15min":0.85199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":41.722000000000001,"5min":41.722000000000001,"15min":41.722000000000001},"last":2.984}]},{"osd":3,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1479999999999997,"5min":1.329,"15min":0.84499999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":41.457000000000001,"5min":41.457000000000001,"15min":41.457000000000001},"last":3.798},{"interface":"front","average":{"1min":3.6549999999999998,"5min":1.204,"15min":0.81899999999999995},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":41.311999999999998,"5min":41.311999999999998,"15min":41.311999999999998},"last":6.1840000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.317},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.41699999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4889999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5539999999999998,"5min":12.067,"15min":12.067},"last":0.39000000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1332076,"kb_used_data":1026660,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92413076,"statfs":{"total":95995035648,"available":94630989824,"internally_reserved":0,"allocated":1051299840,"data_stored":1050928488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.7719999999999998,"5min":1.002,"15min":0.68000000000000005},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":17.885000000000002,"5min":17.885000000000002,"15min":17.885000000000002},"last":4.8949999999999996},{"interface":"front","average":{"1min":2.5139999999999998,"5min":1.1120000000000001,"15min":0.71299999999999997},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.21199999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":0.80600000000000005,"15min":0.64000000000000001},"min":{"1min":0.29099999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":11.301,"5min":11.301,"15min":11.301},"last":0.23599999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":0.81399999999999995,"15min":0.62},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.8410000000000002,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":2.3660000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":1.093,"15min":0.72099999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":9.2319999999999993,"5min":12.859999999999999,"15min":12.859999999999999},"last":1.337},{"interface":"front","average":{"1min":1.54,"5min":0.82899999999999996,"15min":0.64400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":1.395}]},{"osd":3,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":1.51,"15min":0.86399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.327,"5min":38.762,"15min":38.762},"last":1.748},{"interface":"front","average":{"1min":3.7050000000000001,"5min":1.788,"15min":0.96599999999999997},"min":{"1min":0.379,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":24.242999999999999,"5min":35.612000000000002,"15min":35.612000000000002},"last":1.155}]},{"osd":4,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44800000000000001,"15min":0.43099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48899999999999999,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.33300000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48999999999999999,"15min":0.51000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.40000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1734876,"kb_used_data":1429460,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92010276,"statfs":{"total":95995035648,"available":94218522624,"internally_reserved":0,"allocated":1463767040,"data_stored":1463347912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":4.0599999999999996,"5min":1.4630000000000001,"15min":0.71299999999999997},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":5.4359999999999999},{"interface":"front","average":{"1min":3.5409999999999999,"5min":1.5509999999999999,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":9.5,"5min":11.352,"15min":11.352},"last":0.371}]},{"osd":2,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":4.5469999999999997,"5min":1.3220000000000001,"15min":0.70199999999999996},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":15.683,"5min":15.683,"15min":15.683},"last":2.3700000000000001},{"interface":"front","average":{"1min":4.9130000000000003,"5min":1.3400000000000001,"15min":0.70799999999999996},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":13.461,"5min":13.461,"15min":13.461},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:33:34 2024","interfaces":[{"interface":"back","average":{"1min":5.0599999999999996,"5min":1.9970000000000001,"15min":0.97699999999999998},"min":{"1min":0.24399999999999999,"5min":0.223,"15min":0.223},"max":{"1min":11.932,"5min":25.145,"15min":25.145},"last":0.214},{"interface":"front","average":{"1min":3.8420000000000001,"5min":1.8959999999999999,"15min":0.97799999999999998},"min":{"1min":0.40300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":11.625,"5min":22.25,"15min":22.25},"last":2.4100000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:53 2024","interfaces":[{"interface":"back","average":{"1min":2.4260000000000002,"5min":1.319,"15min":0.85499999999999998},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.9580000000000002,"5min":15.542999999999999,"15min":15.542999999999999},"last":2.4359999999999999},{"interface":"front","average":{"1min":2.1890000000000001,"5min":0.996,"15min":0.71699999999999997},"min":{"1min":0.314,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.245}]},{"osd":5,"last update":"Thu Feb 22 19:33:09 2024","interfaces":[{"interface":"back","average":{"1min":3.1429999999999998,"5min":1.1819999999999999,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":15.587999999999999,"5min":15.587999999999999,"15min":15.587999999999999},"last":2.4769999999999999},{"interface":"front","average":{"1min":2.694,"5min":1.0149999999999999,"15min":0.72299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.5839999999999996,"5min":7.5839999999999996,"15min":7.5839999999999996},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:40 2024","interfaces":[{"interface":"back","average":{"1min":2.1549999999999998,"5min":0.99299999999999999,"15min":0.69299999999999995},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.314},{"interface":"front","average":{"1min":2.4199999999999999,"5min":1.333,"15min":0.83899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.032,"5min":15.651999999999999,"15min":15.651999999999999},"last":2.5059999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.5579999999999998,"5min":0.97299999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.706,"5min":0.82399999999999995,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":0.33900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1905412,"kb_used_data":1607868,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":91839740,"statfs":{"total":95995035648,"available":94043893760,"internally_reserved":0,"allocated":1646456832,"data_stored":1646062186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":3.2610000000000001,"5min":1.262,"15min":0.66000000000000003},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":12.305,"5min":12.305,"15min":12.305},"last":0.222},{"interface":"front","average":{"1min":2.6440000000000001,"5min":1.327,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":13.013999999999999,"5min":13.013999999999999,"15min":13.013999999999999},"last":1.841}]},{"osd":2,"last update":"Thu Feb 22 19:33:06 2024","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":1.335,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":19.927,"5min":19.927,"15min":19.927},"last":4.3419999999999996},{"interface":"front","average":{"1min":4.2110000000000003,"5min":1.327,"15min":0.69099999999999995},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":17.907,"5min":17.907,"15min":17.907},"last":6.968}]},{"osd":3,"last update":"Thu Feb 22 19:33:27 2024","interfaces":[{"interface":"back","average":{"1min":3.5150000000000001,"5min":1.2989999999999999,"15min":0.72299999999999998},"min":{"1min":0.22900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":13.06,"5min":13.06,"15min":13.06},"last":4.0860000000000003},{"interface":"front","average":{"1min":4.0919999999999996,"5min":1.5349999999999999,"15min":0.81200000000000006},"min":{"1min":0.39000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":13.837999999999999,"5min":22.936,"15min":22.936},"last":8.9629999999999992}]},{"osd":4,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":0.89500000000000002,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":2.649,"5min":1.423,"15min":0.86799999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":13.874000000000001,"5min":13.874000000000001,"15min":13.874000000000001},"last":4.2519999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":1.26,"15min":0.79800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":13.903,"5min":13.903,"15min":13.903},"last":4.2839999999999998},{"interface":"front","average":{"1min":1.7270000000000001,"5min":0.81100000000000005,"15min":0.64900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.8920000000000003,"5min":6.8920000000000003,"15min":6.8920000000000003},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:33 2024","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":0.90700000000000003,"15min":0.69799999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.984,"5min":6.984,"15min":6.984},"last":0.48799999999999999},{"interface":"front","average":{"1min":2.0569999999999999,"5min":1.038,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Feb 22 19:33:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":0.84399999999999997,"15min":0.66400000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9660000000000002,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":2.4279999999999999,"5min":0.95299999999999996,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":6.0800000000000001,"5min":6.0800000000000001,"15min":6.0800000000000001},"last":0.53700000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1925880,"kb_used_data":1612208,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":91819272,"statfs":{"total":95995035648,"available":94022934528,"internally_reserved":0,"allocated":1650900992,"data_stored":1650517556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":1.1719999999999999,"15min":0.63600000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":11.898999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":5.2389999999999999},{"interface":"front","average":{"1min":3.6859999999999999,"5min":1.4239999999999999,"15min":0.71599999999999997},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":16.138999999999999,"5min":18.059999999999999,"15min":18.059999999999999},"last":1.633}]},{"osd":1,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":0.96199999999999997,"15min":0.57199999999999995},"min":{"1min":0.21199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.183,"5min":10.183,"15min":10.183},"last":6.3289999999999997},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.73799999999999999,"15min":0.51600000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":9.7189999999999994,"5min":9.7189999999999994,"15min":9.7189999999999994},"last":5.6059999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:33:32 2024","interfaces":[{"interface":"back","average":{"1min":4.016,"5min":1.6559999999999999,"15min":0.86099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":30.071000000000002,"5min":30.071000000000002,"15min":30.071000000000002},"last":5.282},{"interface":"front","average":{"1min":5.2720000000000002,"5min":1.9850000000000001,"15min":0.95999999999999996},"min":{"1min":0.39900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":32.119,"5min":32.119,"15min":32.119},"last":1.7430000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:53 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.1319999999999999,"15min":0.75800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.226,"5min":6.226,"15min":6.226},"last":6.3890000000000002},{"interface":"front","average":{"1min":1.119,"5min":0.78100000000000003,"15min":0.65300000000000002},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.3860000000000001,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":5.516}]},{"osd":5,"last update":"Thu Feb 22 19:33:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":0.747,"15min":0.66500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":5.5350000000000001},{"interface":"front","average":{"1min":2.1669999999999998,"5min":0.97999999999999998,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0739999999999998,"5min":6.0739999999999998,"15min":6.0739999999999998},"last":6.4320000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:33:39 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.1579999999999999,"15min":0.78800000000000003},"min":{"1min":0.375,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":9.9689999999999994,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":6.4109999999999996},{"interface":"front","average":{"1min":1.1339999999999999,"5min":0.81200000000000006,"15min":0.69099999999999995},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.4100000000000001,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":5.452}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":0.94799999999999995,"15min":0.70199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.1029999999999998,"5min":6.1029999999999998,"15min":6.1029999999999998},"last":6.3650000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.73699999999999999,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":9.5660000000000007,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":5.4850000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2341196,"kb_used_data":2031684,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91403956,"statfs":{"total":95995035648,"available":93597650944,"internally_reserved":0,"allocated":2080444416,"data_stored":2080041910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1909999999999998,"5min":1.5309999999999999,"15min":0.80400000000000005},"min":{"1min":0.20899999999999999,"5min":0.193,"15min":0.193},"max":{"1min":12.929,"5min":12.929,"15min":12.929},"last":2.1960000000000002},{"interface":"front","average":{"1min":4.774,"5min":1.7390000000000001,"15min":0.90500000000000003},"min":{"1min":0.36099999999999999,"5min":0.247,"15min":0.247},"max":{"1min":14.831,"5min":14.831,"15min":14.831},"last":3.4950000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1600000000000001,"5min":1.728,"15min":0.89000000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":17.009,"5min":17.009,"15min":17.009},"last":2.1280000000000001},{"interface":"front","average":{"1min":2.843,"5min":1.3380000000000001,"15min":0.752},"min":{"1min":0.38600000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":7.7619999999999996,"5min":10.279,"15min":10.279},"last":1.2509999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1660000000000004,"5min":1.7090000000000001,"15min":0.89800000000000002},"min":{"1min":0.34499999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":10.714,"5min":10.714,"15min":10.714},"last":1.218},{"interface":"front","average":{"1min":4.2519999999999998,"5min":1.712,"15min":0.88900000000000001},"min":{"1min":0.46100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":2.8239999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:33:50 2024","interfaces":[{"interface":"back","average":{"1min":2.7719999999999998,"5min":1.4199999999999999,"15min":0.89000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.699,"5min":12.699,"15min":12.699},"last":1.546},{"interface":"front","average":{"1min":2.4489999999999998,"5min":1.325,"15min":0.84599999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":11.366,"5min":11.366,"15min":11.366},"last":1.276}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":1.125,"15min":0.76800000000000002},"min":{"1min":0.437,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.8550000000000004,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":2.2639999999999998},{"interface":"front","average":{"1min":3.3239999999999998,"5min":1.2689999999999999,"15min":0.81100000000000005},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":12.744999999999999,"5min":12.744999999999999,"15min":12.744999999999999},"last":1.623}]},{"osd":6,"last update":"Thu Feb 22 19:33:42 2024","interfaces":[{"interface":"back","average":{"1min":3.21,"5min":1.363,"15min":0.84599999999999997},"min":{"1min":0.496,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.898999999999999,"5min":12.898999999999999,"15min":12.898999999999999},"last":2.2909999999999999},{"interface":"front","average":{"1min":3.036,"5min":1.4590000000000001,"15min":0.92000000000000004},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":12.776999999999999,"5min":12.776999999999999,"15min":12.776999999999999},"last":1.6000000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:33:56 2024","interfaces":[{"interface":"back","average":{"1min":1.897,"5min":1.3420000000000001,"15min":0.83099999999999996},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":11.348000000000001,"5min":11.348000000000001,"15min":11.348000000000001},"last":1.302},{"interface":"front","average":{"1min":2.7709999999999999,"5min":1.454,"15min":0.89000000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":12.846,"5min":12.846,"15min":12.846},"last":2.2349999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1330120,"kb_used_data":1020608,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92415032,"statfs":{"total":95995035648,"available":94632992768,"internally_reserved":0,"allocated":1045102592,"data_stored":1044703962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.4830000000000001,"5min":1.2749999999999999,"15min":0.76900000000000002},"min":{"1min":0.313,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.635999999999999,"5min":24.009,"15min":24.009},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.8480000000000001,"5min":1.2450000000000001,"15min":0.77900000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.7229999999999999,"5min":24.263000000000002,"15min":24.263000000000002},"last":4.0129999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9810000000000001,"5min":0.96699999999999997,"15min":0.68899999999999995},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":6.3029999999999999,"5min":6.3029999999999999,"15min":6.3029999999999999},"last":1.389},{"interface":"front","average":{"1min":1.5369999999999999,"5min":0.80700000000000005,"15min":0.60399999999999998},"min":{"1min":0.28000000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":12.445,"5min":12.445,"15min":12.445},"last":1.323}]},{"osd":2,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.107,"15min":0.75},"min":{"1min":0.38,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.8839999999999999,"5min":23.082000000000001,"15min":23.082000000000001},"last":4.4569999999999999},{"interface":"front","average":{"1min":2.3450000000000002,"5min":1.3200000000000001,"15min":0.80100000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.992,"5min":23.138999999999999,"15min":23.138999999999999},"last":4.532}]},{"osd":3,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.9199999999999999,"5min":1.2230000000000001,"15min":0.77700000000000002},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":1.2070000000000001},{"interface":"front","average":{"1min":2.6400000000000001,"5min":1.157,"15min":0.73899999999999999},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":8.9909999999999997,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.4790000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.35799999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:33:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64800000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.311},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.498,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55800000000000005,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.44500000000000001}]}]},{"osd":5,"up_from":0,"seq":167503724617,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1641943040,"data_stored":1641938968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1449857024,"data_stored":1449852952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1637875712,"data_stored":1637875712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2059010048,"data_stored":2059010048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1036910592,"data_stored":1036910592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":92930048,"data_stored":92930048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1042747392,"data_stored":1042743320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1033830400,"data_stored":1033830400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:34:02.358 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-22T19:34:02.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:34:02.359 INFO:tasks.thrashosds.thrasher:fixing pg num pool device_health_metrics 2024-02-22T19:34:02.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:34:02.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:02.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:02.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:02.490 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:02.514 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:02.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:02.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:02.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:02.691 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.320220+0000 mgr.y (mgr.14535) 198 : cluster [DBG] pgmap v214: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:02.691 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.593028+0000 osd.7 (osd.7) 63 : cluster [DBG] 3.f scrub starts 2024-02-22T19:34:02.691 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.596483+0000 osd.7 (osd.7) 64 : cluster [DBG] 3.f scrub ok 2024-02-22T19:34:02.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.731047+0000 osd.1 (osd.1) 57 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:34:02.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.732149+0000 osd.1 (osd.1) 58 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:34:02.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.888168+0000 osd.4 (osd.4) 67 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:34:02.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:01.893903+0000 osd.4 (osd.4) 68 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:34:02.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:02.069318+0000 osd.0 (osd.0) 73 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:34:02.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:02.072814+0000 osd.0 (osd.0) 74 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:34:02.716 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:02.791 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:02.803 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:02.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:02.886 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:02.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:02.934 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:02.957 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:02.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:02.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:02.960 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.320220+0000 mgr.y (mgr.14535) 198 : cluster [DBG] pgmap v214: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:02.960 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.593028+0000 osd.7 (osd.7) 63 : cluster [DBG] 3.f scrub starts 2024-02-22T19:34:02.960 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.596483+0000 osd.7 (osd.7) 64 : cluster [DBG] 3.f scrub ok 2024-02-22T19:34:02.960 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.731047+0000 osd.1 (osd.1) 57 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:34:02.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.732149+0000 osd.1 (osd.1) 58 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:34:02.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.888168+0000 osd.4 (osd.4) 67 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:34:02.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:01.893903+0000 osd.4 (osd.4) 68 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:34:02.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:02.069318+0000 osd.0 (osd.0) 73 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:34:02.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:02.072814+0000 osd.0 (osd.0) 74 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.320220+0000 mgr.y (mgr.14535) 198 : cluster [DBG] pgmap v214: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.593028+0000 osd.7 (osd.7) 63 : cluster [DBG] 3.f scrub starts 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.596483+0000 osd.7 (osd.7) 64 : cluster [DBG] 3.f scrub ok 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.731047+0000 osd.1 (osd.1) 57 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.732149+0000 osd.1 (osd.1) 58 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.888168+0000 osd.4 (osd.4) 67 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:01.893903+0000 osd.4 (osd.4) 68 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:02.069318+0000 osd.0 (osd.0) 73 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:34:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:02.072814+0000 osd.0 (osd.0) 74 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:34:03.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:03.093 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:03.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:03.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:03.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:03.291 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:131820 7.e e137/132 7:7091d32c:::benchmark_data_smithi053_107660_object42613:head v 137'8074, mlcod=137'8070)", 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.537111+0000", 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.016259988, 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016283222999999999, 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.537111+0000", 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.537111+0000", 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.537110+0000", 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8799999999999997e-06 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.537119+0000", 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0919e-05 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.537130+0000", 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2140000000000001e-06 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.537136+0000", 2024-02-22T19:34:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032686529999999998 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.540405+0000", 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1072000000000001e-05 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.540426+0000", 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033147599999999999 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:131828 7.7 e137/93 7:ede12e46:::benchmark_data_smithi053_107660_object42621:head v 137'8179, mlcod=137'8177)", 2024-02-22T19:34:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.542121+0000", 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.011250720000000001, 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011317664999999999, 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542121+0000", 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542121+0000", 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542120+0000", 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5359999999999998e-06 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542126+0000", 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3912e-05 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542140+0000", 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1659999999999999e-06 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542146+0000", 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000371474 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542518+0000", 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9154e-05 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542537+0000", 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000416425 2024-02-22T19:34:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:131829 7.f 7:ff92bed5:::benchmark_data_smithi053_107660_object42622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e137)", 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.542165+0000", 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.011206513, 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011321367000000001, 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 131829 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542165+0000", 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542165+0000", 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542164+0000", 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8760000000000004e-06 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542168+0000", 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2200000000000003e-06 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542176+0000", 2024-02-22T19:34:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6090000000000003e-06 2024-02-22T19:34:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542180+0000", 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030370520000000002 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.545217+0000", 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010710200000000001 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.545324+0000", 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012706800000000001 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.545452+0000", 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054534659999999997 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.550905+0000", 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0087401189999999993 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:131830 7.4 7:20567ea7:::benchmark_data_smithi053_107660_object42623:head [delete] snapc 0=[] ondisk+write+known_if_redirected e137)", 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.542186+0000", 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.011185762, 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011388384, 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 131830 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542186+0000", 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542186+0000", 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542185+0000", 2024-02-22T19:34:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4739999999999999e-06 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542188+0000", 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3680000000000004e-06 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542192+0000", 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5200000000000002e-06 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.542196+0000", 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025226649999999999 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.544718+0000", 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108671 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.544827+0000", 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000120871 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:34:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.544948+0000", 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063521699999999999 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.551300+0000", 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091143390000000008 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:131832 7.6 7:6e874a12:::benchmark_data_smithi053_107660_object42625:head [delete] snapc 0=[] ondisk+write+known_if_redirected e137)", 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.543758+0000", 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0096133220000000005, 2024-02-22T19:34:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098759090000000004, 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 131832 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543758+0000", 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543758+0000", 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543757+0000", 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543764+0000", 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1823e-05 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543776+0000", 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9190000000000002e-06 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543783+0000", 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6833999999999999e-05 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543830+0000", 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103185 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.543933+0000", 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117488 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.544050+0000", 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0072701789999999999 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.551320+0000", 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075624359999999996 2024-02-22T19:34:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:131836 7.7 e137/93 7:e242706f:::benchmark_data_smithi053_107660_object42629:head v 137'8180, mlcod=137'8177)", 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.547832+0000", 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0055394779999999996, 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058494920000000004, 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547832+0000", 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547832+0000", 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547831+0000", 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9259999999999995e-06 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547841+0000", 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2150999999999998e-05 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547863+0000", 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2160000000000003e-06 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547869+0000", 2024-02-22T19:34:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023362900000000001 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.548103+0000", 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8953e-05 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.548122+0000", 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028979899999999998 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:131835 7.d e137/93 7:b299fff2:::benchmark_data_smithi053_107660_object42628:head v 137'8457, mlcod=137'8455)", 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:03.547876+0000", 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0054958000000000003, 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058422730000000003, 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547876+0000", 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547876+0000", 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547875+0000", 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2649999999999998e-06 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547879+0000", 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547884+0000", 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8449999999999996e-06 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547888+0000", 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3339999999999997e-05 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.547982+0000", 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9769e-05 2024-02-22T19:34:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:03.548001+0000", 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012581800000000001 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 7 2024-02-22T19:34:03.688 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:34:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:03.884 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:03.889 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:02.782364+0000 osd.1 (osd.1) 59 : cluster [DBG] 6.19 deep-scrub starts 2024-02-22T19:34:03.889 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:02.783767+0000 osd.1 (osd.1) 60 : cluster [DBG] 6.19 deep-scrub ok 2024-02-22T19:34:03.890 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:03.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:03.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:02.782364+0000 osd.1 (osd.1) 59 : cluster [DBG] 6.19 deep-scrub starts 2024-02-22T19:34:03.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:02.783767+0000 osd.1 (osd.1) 60 : cluster [DBG] 6.19 deep-scrub ok 2024-02-22T19:34:03.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:04.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:02.782364+0000 osd.1 (osd.1) 59 : cluster [DBG] 6.19 deep-scrub starts 2024-02-22T19:34:04.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:02.783767+0000 osd.1 (osd.1) 60 : cluster [DBG] 6.19 deep-scrub ok 2024-02-22T19:34:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:04.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:04.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:04.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:04.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:04.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:04.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:04.638 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:04.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:04.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:04.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:34:04.760 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:04.831 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:04.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:34:04.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:34:04.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:03.320620+0000 mgr.y (mgr.14535) 199 : cluster [DBG] pgmap v215: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:03.577240+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.1c deep-scrub starts 2024-02-22T19:34:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:03.590351+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.1c deep-scrub ok 2024-02-22T19:34:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.046332+0000 osd.4 (osd.4) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.046389+0000 osd.4 (osd.4) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:05.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:03.320620+0000 mgr.y (mgr.14535) 199 : cluster [DBG] pgmap v215: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:03.577240+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.1c deep-scrub starts 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:03.590351+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.1c deep-scrub ok 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.046332+0000 osd.4 (osd.4) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.046389+0000 osd.4 (osd.4) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:03.320620+0000 mgr.y (mgr.14535) 199 : cluster [DBG] pgmap v215: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 909 op/s 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:03.577240+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.1c deep-scrub starts 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:03.590351+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.1c deep-scrub ok 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.046332+0000 osd.4 (osd.4) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.046389+0000 osd.4 (osd.4) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:05.016 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:05.018 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:05.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:05.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:05.150 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:05.171 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:05.176 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:05.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:05.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:05.352 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:04.486243+0000 mgr.y (mgr.14535) 200 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.721894+0000 mon.a (mon.0) 886 : cluster [DBG] osdmap e138: 8 total, 8 up, 7 in 2024-02-22T19:34:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.771001+0000 osd.6 (osd.6) 37 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:34:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.782048+0000 osd.6 (osd.6) 38 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:04.486243+0000 mgr.y (mgr.14535) 200 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.721894+0000 mon.a (mon.0) 886 : cluster [DBG] osdmap e138: 8 total, 8 up, 7 in 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.771001+0000 osd.6 (osd.6) 37 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.782048+0000 osd.6 (osd.6) 38 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:04.486243+0000 mgr.y (mgr.14535) 200 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.721894+0000 mon.a (mon.0) 886 : cluster [DBG] osdmap e138: 8 total, 8 up, 7 in 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.771001+0000 osd.6 (osd.6) 37 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:34:06.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.782048+0000 osd.6 (osd.6) 38 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:34:06.808 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:34:06.993 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:06.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:06.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.872532+0000 osd.3 (osd.3) 69 : cluster [DBG] 3.c scrub starts 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:04.892829+0000 osd.3 (osd.3) 70 : cluster [DBG] 3.c scrub ok 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:05.321440+0000 mgr.y (mgr.14535) 201 : cluster [DBG] pgmap v217: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 892 op/s 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:05.801598+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.53:0/3363509315' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:05.807776+0000 osd.6 (osd.6) 39 : cluster [DBG] 3.10 deep-scrub starts 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:05.811941+0000 osd.6 (osd.6) 40 : cluster [DBG] 3.10 deep-scrub ok 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:05.983525+0000 osd.4 (osd.4) 71 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:05.989940+0000 osd.4 (osd.4) 72 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:34:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:06.004688+0000 osd.0 (osd.0) 75 : cluster [DBG] 3.1 deep-scrub starts 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:06.016188+0000 osd.0 (osd.0) 76 : cluster [DBG] 3.1 deep-scrub ok 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.872532+0000 osd.3 (osd.3) 69 : cluster [DBG] 3.c scrub starts 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:04.892829+0000 osd.3 (osd.3) 70 : cluster [DBG] 3.c scrub ok 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:05.321440+0000 mgr.y (mgr.14535) 201 : cluster [DBG] pgmap v217: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 892 op/s 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:05.801598+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.53:0/3363509315' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:05.807776+0000 osd.6 (osd.6) 39 : cluster [DBG] 3.10 deep-scrub starts 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:05.811941+0000 osd.6 (osd.6) 40 : cluster [DBG] 3.10 deep-scrub ok 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:05.983525+0000 osd.4 (osd.4) 71 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:05.989940+0000 osd.4 (osd.4) 72 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:06.004688+0000 osd.0 (osd.0) 75 : cluster [DBG] 3.1 deep-scrub starts 2024-02-22T19:34:07.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:06.016188+0000 osd.0 (osd.0) 76 : cluster [DBG] 3.1 deep-scrub ok 2024-02-22T19:34:07.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.872532+0000 osd.3 (osd.3) 69 : cluster [DBG] 3.c scrub starts 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:04.892829+0000 osd.3 (osd.3) 70 : cluster [DBG] 3.c scrub ok 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:05.321440+0000 mgr.y (mgr.14535) 201 : cluster [DBG] pgmap v217: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 892 op/s 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:05.801598+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.53:0/3363509315' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:05.807776+0000 osd.6 (osd.6) 39 : cluster [DBG] 3.10 deep-scrub starts 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:05.811941+0000 osd.6 (osd.6) 40 : cluster [DBG] 3.10 deep-scrub ok 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:05.983525+0000 osd.4 (osd.4) 71 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:05.989940+0000 osd.4 (osd.4) 72 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:34:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:06.004688+0000 osd.0 (osd.0) 75 : cluster [DBG] 3.1 deep-scrub starts 2024-02-22T19:34:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:06.016188+0000 osd.0 (osd.0) 76 : cluster [DBG] 3.1 deep-scrub ok 2024-02-22T19:34:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:07.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:07.155 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:07.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:07.314 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:07.618 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:07.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:07.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:07.749 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:07.778 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:07.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:07.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:07.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:34:07.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:34:07.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:34:07.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:34:07.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:34:07.818 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:34:07.868 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:34:07.883 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:07.914 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:34:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:34:07.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:06.603025+0000 osd.7 (osd.7) 67 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:34:07.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:06.606228+0000 osd.7 (osd.7) 68 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:34:07.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:06.763902+0000 mon.a (mon.0) 888 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:07.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:06.787595+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.53:0/3363509315' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:06.787648+0000 mon.a (mon.0) 890 : cluster [DBG] osdmap e139: 8 total, 8 up, 7 in 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:07.540254+0000 mon.c (mon.2) 39 : audit [DBG] from='client.? 172.21.15.53:0/3745701199' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:06.603025+0000 osd.7 (osd.7) 67 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:06.606228+0000 osd.7 (osd.7) 68 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:06.763902+0000 mon.a (mon.0) 888 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:06.787595+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.53:0/3363509315' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:06.787648+0000 mon.a (mon.0) 890 : cluster [DBG] osdmap e139: 8 total, 8 up, 7 in 2024-02-22T19:34:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:07.540254+0000 mon.c (mon.2) 39 : audit [DBG] from='client.? 172.21.15.53:0/3745701199' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:07.968 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:06.603025+0000 osd.7 (osd.7) 67 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:34:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:06.606228+0000 osd.7 (osd.7) 68 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:34:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:06.763902+0000 mon.a (mon.0) 888 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:06.787595+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.53:0/3363509315' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:34:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:06.787648+0000 mon.a (mon.0) 890 : cluster [DBG] osdmap e139: 8 total, 8 up, 7 in 2024-02-22T19:34:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:07.540254+0000 mon.c (mon.2) 39 : audit [DBG] from='client.? 172.21.15.53:0/3745701199' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:08.648 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:08.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:08.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:08.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:08.808 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:08.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:08.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:08.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:08.944 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:08.967 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:08.972 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:34:08.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:08.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:08.981 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:34:09.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:07.321867+0000 mgr.y (mgr.14535) 202 : cluster [DBG] pgmap v219: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 557 op/s 2024-02-22T19:34:09.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:07.321867+0000 mgr.y (mgr.14535) 202 : cluster [DBG] pgmap v219: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 557 op/s 2024-02-22T19:34:09.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:07.321867+0000 mgr.y (mgr.14535) 202 : cluster [DBG] pgmap v219: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 557 op/s 2024-02-22T19:34:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:09.132 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:09.138 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:09.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:09.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:09.300 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:34:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:09] "GET /metrics HTTP/1.1" 200 175579 "" "Prometheus/2.33.4" 2024-02-22T19:34:09.344 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:09.393 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:09.401 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:09.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:09.447 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:09.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:09.597 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:09.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:09.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:09.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:09.758 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:10.325 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-02-22T19:34:07.321605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3008698044,"num_objects":46102,"num_object_clones":0,"num_object_copies":138306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46102,"num_whiteouts":0,"num_read":1011,"num_read_kb":763,"num_write":132928,"num_write_kb":5709098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5178,"num_bytes_recovered":332925147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77903,"ondisk_log_size":77903,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":11014172,"kb_used_data":8896356,"kb_used_omap":0,"kb_used_meta":2117376,"kb_avail":645201892,"statfs":{"total":671965249536,"available":660686737408,"internally_reserved":0,"allocated":9109868544,"data_stored":9107103826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2168193024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-291962880,"num_objects":-4455,"num_object_clones":0,"num_object_copies":-13365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4455,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004558"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174694+0000","last_change":"2024-02-22T19:31:50.004483+0000","last_active":"2024-02-22T19:33:45.174694+0000","last_peered":"2024-02-22T19:33:45.174694+0000","last_clean":"2024-02-22T19:33:45.174694+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:33:45.174694+0000","last_undegraded":"2024-02-22T19:33:45.174694+0000","last_fullsized":"2024-02-22T19:33:45.174694+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:28.096669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.096669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":104,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.781565+0000","last_change":"2024-02-22T19:33:58.781565+0000","last_active":"2024-02-22T19:33:58.781565+0000","last_peered":"2024-02-22T19:33:58.781565+0000","last_clean":"2024-02-22T19:33:58.781565+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:33:58.781565+0000","last_undegraded":"2024-02-22T19:33:58.781565+0000","last_fullsized":"2024-02-22T19:33:58.781565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.781466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.781466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":106,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.170559+0000","last_change":"2024-02-22T19:31:50.013054+0000","last_active":"2024-02-22T19:33:45.170559+0000","last_peered":"2024-02-22T19:33:45.170559+0000","last_clean":"2024-02-22T19:33:45.170559+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:33:45.170559+0000","last_undegraded":"2024-02-22T19:33:45.170559+0000","last_fullsized":"2024-02-22T19:33:45.170559+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:12.445206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:12.445206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:53.940471+0000","last_change":"2024-02-22T19:33:53.940446+0000","last_active":"2024-02-22T19:33:53.940471+0000","last_peered":"2024-02-22T19:33:53.940471+0000","last_clean":"2024-02-22T19:33:53.940471+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:33:53.940471+0000","last_undegraded":"2024-02-22T19:33:53.940471+0000","last_fullsized":"2024-02-22T19:33:53.940471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:53.940375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.940375+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173436+0000","last_change":"2024-02-22T19:30:23.788442+0000","last_active":"2024-02-22T19:33:45.173436+0000","last_peered":"2024-02-22T19:33:45.173436+0000","last_clean":"2024-02-22T19:33:45.173436+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:33:45.173436+0000","last_undegraded":"2024-02-22T19:33:45.173436+0000","last_fullsized":"2024-02-22T19:33:45.173436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:30:23.788367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.176419+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:33:45.176419+0000","last_peered":"2024-02-22T19:33:45.176419+0000","last_clean":"2024-02-22T19:33:45.176419+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:33:45.176419+0000","last_undegraded":"2024-02-22T19:33:45.176419+0000","last_fullsized":"2024-02-22T19:33:45.176419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.180307+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:33:45.180307+0000","last_peered":"2024-02-22T19:33:45.180307+0000","last_clean":"2024-02-22T19:33:45.180307+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:33:45.180307+0000","last_undegraded":"2024-02-22T19:33:45.180307+0000","last_fullsized":"2024-02-22T19:33:45.180307+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":114,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173044+0000","last_change":"2024-02-22T19:30:30.877499+0000","last_active":"2024-02-22T19:33:45.173044+0000","last_peered":"2024-02-22T19:33:45.173044+0000","last_clean":"2024-02-22T19:33:45.173044+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:33:45.173044+0000","last_undegraded":"2024-02-22T19:33:45.173044+0000","last_fullsized":"2024-02-22T19:33:45.173044+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:30.877413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:30.877413+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:51.065944+0000","last_change":"2024-02-22T19:33:51.065916+0000","last_active":"2024-02-22T19:33:51.065944+0000","last_peered":"2024-02-22T19:33:51.065944+0000","last_clean":"2024-02-22T19:33:51.065944+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:33:51.065944+0000","last_undegraded":"2024-02-22T19:33:51.065944+0000","last_fullsized":"2024-02-22T19:33:51.065944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:51.065825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.065825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":54,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172039+0000","last_change":"2024-02-22T19:33:42.010538+0000","last_active":"2024-02-22T19:33:45.172039+0000","last_peered":"2024-02-22T19:33:45.172039+0000","last_clean":"2024-02-22T19:33:45.172039+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:33:45.172039+0000","last_undegraded":"2024-02-22T19:33:45.172039+0000","last_fullsized":"2024-02-22T19:33:45.172039+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.010467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.010467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.802887+0000","last_change":"2024-02-22T19:33:45.802849+0000","last_active":"2024-02-22T19:33:45.802887+0000","last_peered":"2024-02-22T19:33:45.802887+0000","last_clean":"2024-02-22T19:33:45.802887+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:33:45.802887+0000","last_undegraded":"2024-02-22T19:33:45.802887+0000","last_fullsized":"2024-02-22T19:33:45.802887+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.802756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.802756+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:52.938558+0000","last_change":"2024-02-22T19:33:52.938521+0000","last_active":"2024-02-22T19:33:52.938558+0000","last_peered":"2024-02-22T19:33:52.938558+0000","last_clean":"2024-02-22T19:33:52.938558+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:33:52.938558+0000","last_undegraded":"2024-02-22T19:33:52.938558+0000","last_fullsized":"2024-02-22T19:33:52.938558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:52.938440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.938440+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":54,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:02.783853+0000","last_change":"2024-02-22T19:34:02.783853+0000","last_active":"2024-02-22T19:34:02.783853+0000","last_peered":"2024-02-22T19:34:02.783853+0000","last_clean":"2024-02-22T19:34:02.783853+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:34:02.783853+0000","last_undegraded":"2024-02-22T19:34:02.783853+0000","last_fullsized":"2024-02-22T19:34:02.783853+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:34:02.783771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.151831+0000","last_change":"2024-02-22T19:30:41.619608+0000","last_active":"2024-02-22T19:33:45.151831+0000","last_peered":"2024-02-22T19:33:45.151831+0000","last_clean":"2024-02-22T19:33:45.151831+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:33:45.151831+0000","last_undegraded":"2024-02-22T19:33:45.151831+0000","last_fullsized":"2024-02-22T19:33:45.151831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.619448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":61,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175133+0000","last_change":"2024-02-22T19:31:50.439339+0000","last_active":"2024-02-22T19:33:45.175133+0000","last_peered":"2024-02-22T19:33:45.175133+0000","last_clean":"2024-02-22T19:33:45.175133+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:33:45.175133+0000","last_undegraded":"2024-02-22T19:33:45.175133+0000","last_fullsized":"2024-02-22T19:33:45.175133+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:31:23.683223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.683223+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":102,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175472+0000","last_change":"2024-02-22T19:30:40.016834+0000","last_active":"2024-02-22T19:33:45.175472+0000","last_peered":"2024-02-22T19:33:45.175472+0000","last_clean":"2024-02-22T19:33:45.175472+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:33:45.175472+0000","last_undegraded":"2024-02-22T19:33:45.175472+0000","last_fullsized":"2024-02-22T19:33:45.175472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:30:40.016724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.016724+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175620+0000","last_change":"2024-02-22T19:31:50.009328+0000","last_active":"2024-02-22T19:33:45.175620+0000","last_peered":"2024-02-22T19:33:45.175620+0000","last_clean":"2024-02-22T19:33:45.175620+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:33:45.175620+0000","last_undegraded":"2024-02-22T19:33:45.175620+0000","last_fullsized":"2024-02-22T19:33:45.175620+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155336+0000","last_change":"2024-02-22T19:30:41.981429+0000","last_active":"2024-02-22T19:33:45.155336+0000","last_peered":"2024-02-22T19:33:45.155336+0000","last_clean":"2024-02-22T19:33:45.155336+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:33:45.155336+0000","last_undegraded":"2024-02-22T19:33:45.155336+0000","last_fullsized":"2024-02-22T19:33:45.155336+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.981349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172345+0000","last_change":"2024-02-22T19:31:26.454221+0000","last_active":"2024-02-22T19:33:45.172345+0000","last_peered":"2024-02-22T19:33:45.172345+0000","last_clean":"2024-02-22T19:33:45.172345+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:33:45.172345+0000","last_undegraded":"2024-02-22T19:33:45.172345+0000","last_fullsized":"2024-02-22T19:33:45.172345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.454155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.454155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":117,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.159111+0000","last_change":"2024-02-22T19:30:32.144934+0000","last_active":"2024-02-22T19:33:45.159111+0000","last_peered":"2024-02-22T19:33:45.159111+0000","last_clean":"2024-02-22T19:33:45.159111+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:33:45.159111+0000","last_undegraded":"2024-02-22T19:33:45.159111+0000","last_fullsized":"2024-02-22T19:33:45.159111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:32.144787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.144787+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.835161+0000","last_change":"2024-02-22T19:33:48.835132+0000","last_active":"2024-02-22T19:33:48.835161+0000","last_peered":"2024-02-22T19:33:48.835161+0000","last_clean":"2024-02-22T19:33:48.835161+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:33:48.835161+0000","last_undegraded":"2024-02-22T19:33:48.835161+0000","last_fullsized":"2024-02-22T19:33:48.835161+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.835030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.835030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175271+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:33:45.175271+0000","last_peered":"2024-02-22T19:33:45.175271+0000","last_clean":"2024-02-22T19:33:45.175271+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:33:45.175271+0000","last_undegraded":"2024-02-22T19:33:45.175271+0000","last_fullsized":"2024-02-22T19:33:45.175271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174942+0000","last_change":"2024-02-22T19:30:31.137040+0000","last_active":"2024-02-22T19:33:45.174942+0000","last_peered":"2024-02-22T19:33:45.174942+0000","last_clean":"2024-02-22T19:33:45.174942+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:33:45.174942+0000","last_undegraded":"2024-02-22T19:33:45.174942+0000","last_fullsized":"2024-02-22T19:33:45.174942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.136961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.136961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":117,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174791+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:33:45.174791+0000","last_peered":"2024-02-22T19:33:45.174791+0000","last_clean":"2024-02-22T19:33:45.174791+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:33:45.174791+0000","last_undegraded":"2024-02-22T19:33:45.174791+0000","last_fullsized":"2024-02-22T19:33:45.174791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:03.590429+0000","last_change":"2024-02-22T19:34:03.590429+0000","last_active":"2024-02-22T19:34:03.590429+0000","last_peered":"2024-02-22T19:34:03.590429+0000","last_clean":"2024-02-22T19:34:03.590429+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:34:03.590429+0000","last_undegraded":"2024-02-22T19:34:03.590429+0000","last_fullsized":"2024-02-22T19:34:03.590429+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:34:03.590356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"137'9","reported_seq":170,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:00.780662+0000","last_change":"2024-02-22T19:34:00.780638+0000","last_active":"2024-02-22T19:34:00.780662+0000","last_peered":"2024-02-22T19:34:00.780662+0000","last_clean":"2024-02-22T19:34:00.780662+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:34:00.780662+0000","last_undegraded":"2024-02-22T19:34:00.780662+0000","last_fullsized":"2024-02-22T19:34:00.780662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:34:00.780574+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":99,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:55.750063+0000","last_change":"2024-02-22T19:33:55.750063+0000","last_active":"2024-02-22T19:33:55.750063+0000","last_peered":"2024-02-22T19:33:55.750063+0000","last_clean":"2024-02-22T19:33:55.750063+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:33:55.750063+0000","last_undegraded":"2024-02-22T19:33:55.750063+0000","last_fullsized":"2024-02-22T19:33:55.750063+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:55.749971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:55.749971+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":110,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:01.123505+0000","last_change":"2024-02-22T19:34:01.123505+0000","last_active":"2024-02-22T19:34:01.123505+0000","last_peered":"2024-02-22T19:34:01.123505+0000","last_clean":"2024-02-22T19:34:01.123505+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:34:01.123505+0000","last_undegraded":"2024-02-22T19:34:01.123505+0000","last_fullsized":"2024-02-22T19:34:01.123505+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:34:01.123405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.123405+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155525+0000","last_change":"2024-02-22T19:31:50.009891+0000","last_active":"2024-02-22T19:33:45.155525+0000","last_peered":"2024-02-22T19:33:45.155525+0000","last_clean":"2024-02-22T19:33:45.155525+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:33:45.155525+0000","last_undegraded":"2024-02-22T19:33:45.155525+0000","last_fullsized":"2024-02-22T19:33:45.155525+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.070019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.070019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171882+0000","last_change":"2024-02-22T19:30:39.660611+0000","last_active":"2024-02-22T19:33:45.171882+0000","last_peered":"2024-02-22T19:33:45.171882+0000","last_clean":"2024-02-22T19:33:45.171882+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:33:45.171882+0000","last_undegraded":"2024-02-22T19:33:45.171882+0000","last_fullsized":"2024-02-22T19:33:45.171882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.660549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.660549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173183+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:33:45.173183+0000","last_peered":"2024-02-22T19:33:45.173183+0000","last_clean":"2024-02-22T19:33:45.173183+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:33:45.173183+0000","last_undegraded":"2024-02-22T19:33:45.173183+0000","last_fullsized":"2024-02-22T19:33:45.173183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":166,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.763354+0000","last_change":"2024-02-22T19:33:46.763313+0000","last_active":"2024-02-22T19:33:46.763354+0000","last_peered":"2024-02-22T19:33:46.763354+0000","last_clean":"2024-02-22T19:33:46.763354+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:33:46.763354+0000","last_undegraded":"2024-02-22T19:33:46.763354+0000","last_fullsized":"2024-02-22T19:33:46.763354+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:46.763202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.763202+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.151747+0000","last_change":"2024-02-22T19:30:40.624618+0000","last_active":"2024-02-22T19:33:45.151747+0000","last_peered":"2024-02-22T19:33:45.151747+0000","last_clean":"2024-02-22T19:33:45.151747+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:33:45.151747+0000","last_undegraded":"2024-02-22T19:33:45.151747+0000","last_fullsized":"2024-02-22T19:33:45.151747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:40.624542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:40.624542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":87,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.147614+0000","last_change":"2024-02-22T19:30:35.539864+0000","last_active":"2024-02-22T19:33:45.147614+0000","last_peered":"2024-02-22T19:33:45.147614+0000","last_clean":"2024-02-22T19:33:45.147614+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:33:45.147614+0000","last_undegraded":"2024-02-22T19:33:45.147614+0000","last_fullsized":"2024-02-22T19:33:45.147614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.539769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.539769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":106,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.170729+0000","last_change":"2024-02-22T19:33:43.048434+0000","last_active":"2024-02-22T19:33:45.170729+0000","last_peered":"2024-02-22T19:33:45.170729+0000","last_clean":"2024-02-22T19:33:45.170729+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:33:45.170729+0000","last_undegraded":"2024-02-22T19:33:45.170729+0000","last_fullsized":"2024-02-22T19:33:45.170729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:43.048330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.048330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":95,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173267+0000","last_change":"2024-02-22T19:30:47.930353+0000","last_active":"2024-02-22T19:33:45.173267+0000","last_peered":"2024-02-22T19:33:45.173267+0000","last_clean":"2024-02-22T19:33:45.173267+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:33:45.173267+0000","last_undegraded":"2024-02-22T19:33:45.173267+0000","last_fullsized":"2024-02-22T19:33:45.173267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:30:47.930245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:47.930245+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173675+0000","last_change":"2024-02-22T19:30:22.752463+0000","last_active":"2024-02-22T19:33:45.173675+0000","last_peered":"2024-02-22T19:33:45.173675+0000","last_clean":"2024-02-22T19:33:45.173675+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:33:45.173675+0000","last_undegraded":"2024-02-22T19:33:45.173675+0000","last_fullsized":"2024-02-22T19:33:45.173675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.752367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.752367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.153165+0000","last_change":"2024-02-22T19:30:15.602215+0000","last_active":"2024-02-22T19:33:45.153165+0000","last_peered":"2024-02-22T19:33:45.153165+0000","last_clean":"2024-02-22T19:33:45.153165+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:33:45.153165+0000","last_undegraded":"2024-02-22T19:33:45.153165+0000","last_fullsized":"2024-02-22T19:33:45.153165+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:30:15.602101+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":98,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.528085+0000","last_change":"2024-02-22T19:33:45.528051+0000","last_active":"2024-02-22T19:33:45.528085+0000","last_peered":"2024-02-22T19:33:45.528085+0000","last_clean":"2024-02-22T19:33:45.528085+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:33:45.528085+0000","last_undegraded":"2024-02-22T19:33:45.528085+0000","last_fullsized":"2024-02-22T19:33:45.528085+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.527956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.527956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173009+0000","last_change":"2024-02-22T19:30:42.903889+0000","last_active":"2024-02-22T19:33:45.173009+0000","last_peered":"2024-02-22T19:33:45.173009+0000","last_clean":"2024-02-22T19:33:45.173009+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:33:45.173009+0000","last_undegraded":"2024-02-22T19:33:45.173009+0000","last_fullsized":"2024-02-22T19:33:45.173009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:42.903794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.903794+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":95,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.168815+0000","last_change":"2024-02-22T19:31:50.014237+0000","last_active":"2024-02-22T19:33:45.168815+0000","last_peered":"2024-02-22T19:33:45.168815+0000","last_clean":"2024-02-22T19:33:45.168815+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:33:45.168815+0000","last_undegraded":"2024-02-22T19:33:45.168815+0000","last_fullsized":"2024-02-22T19:33:45.168815+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:07.487779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:07.487779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":87,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173357+0000","last_change":"2024-02-22T19:30:48.880367+0000","last_active":"2024-02-22T19:33:45.173357+0000","last_peered":"2024-02-22T19:33:45.173357+0000","last_clean":"2024-02-22T19:33:45.173357+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:33:45.173357+0000","last_undegraded":"2024-02-22T19:33:45.173357+0000","last_fullsized":"2024-02-22T19:33:45.173357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:48.880278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.880278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152221+0000","last_change":"2024-02-22T19:31:23.009097+0000","last_active":"2024-02-22T19:33:45.152221+0000","last_peered":"2024-02-22T19:33:45.152221+0000","last_clean":"2024-02-22T19:33:45.152221+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:33:45.152221+0000","last_undegraded":"2024-02-22T19:33:45.152221+0000","last_fullsized":"2024-02-22T19:33:45.152221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.009021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:47.757619+0000","last_change":"2024-02-22T19:33:47.757576+0000","last_active":"2024-02-22T19:33:47.757619+0000","last_peered":"2024-02-22T19:33:47.757619+0000","last_clean":"2024-02-22T19:33:47.757619+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:33:47.757619+0000","last_undegraded":"2024-02-22T19:33:47.757619+0000","last_fullsized":"2024-02-22T19:33:47.757619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:47.757479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:47.757479+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172846+0000","last_change":"2024-02-22T19:33:42.862634+0000","last_active":"2024-02-22T19:33:45.172846+0000","last_peered":"2024-02-22T19:33:45.172846+0000","last_clean":"2024-02-22T19:33:45.172846+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:33:45.172846+0000","last_undegraded":"2024-02-22T19:33:45.172846+0000","last_fullsized":"2024-02-22T19:33:45.172846+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.862563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.862563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.156306+0000","last_change":"2024-02-22T19:31:50.012381+0000","last_active":"2024-02-22T19:33:45.156306+0000","last_peered":"2024-02-22T19:33:45.156306+0000","last_clean":"2024-02-22T19:33:45.156306+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:33:45.156306+0000","last_undegraded":"2024-02-22T19:33:45.156306+0000","last_fullsized":"2024-02-22T19:33:45.156306+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.021518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.021518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175176+0000","last_change":"2024-02-22T19:30:41.969686+0000","last_active":"2024-02-22T19:33:45.175176+0000","last_peered":"2024-02-22T19:33:45.175176+0000","last_clean":"2024-02-22T19:33:45.175176+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:33:45.175176+0000","last_undegraded":"2024-02-22T19:33:45.175176+0000","last_fullsized":"2024-02-22T19:33:45.175176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:41.969606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.969606+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175337+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:33:45.175337+0000","last_peered":"2024-02-22T19:33:45.175337+0000","last_clean":"2024-02-22T19:33:45.175337+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:33:45.175337+0000","last_undegraded":"2024-02-22T19:33:45.175337+0000","last_fullsized":"2024-02-22T19:33:45.175337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171314+0000","last_change":"2024-02-22T19:30:46.483326+0000","last_active":"2024-02-22T19:33:45.171314+0000","last_peered":"2024-02-22T19:33:45.171314+0000","last_clean":"2024-02-22T19:33:45.171314+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:33:45.171314+0000","last_undegraded":"2024-02-22T19:33:45.171314+0000","last_fullsized":"2024-02-22T19:33:45.171314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:46.483257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.483257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.202016+0000","last_change":"2024-02-22T19:31:50.105130+0000","last_active":"2024-02-22T19:33:59.202016+0000","last_peered":"2024-02-22T19:33:59.202016+0000","last_clean":"2024-02-22T19:33:59.202016+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:33:59.202016+0000","last_undegraded":"2024-02-22T19:33:59.202016+0000","last_fullsized":"2024-02-22T19:33:59.202016+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:29.679183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.679183+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154155+0000","last_change":"2024-02-22T19:30:39.596234+0000","last_active":"2024-02-22T19:33:45.154155+0000","last_peered":"2024-02-22T19:33:45.154155+0000","last_clean":"2024-02-22T19:33:45.154155+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:33:45.154155+0000","last_undegraded":"2024-02-22T19:33:45.154155+0000","last_fullsized":"2024-02-22T19:33:45.154155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:30:39.596165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177832+0000","last_change":"2024-02-22T19:30:32.846419+0000","last_active":"2024-02-22T19:33:45.177832+0000","last_peered":"2024-02-22T19:33:45.177832+0000","last_clean":"2024-02-22T19:33:45.177832+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:33:45.177832+0000","last_undegraded":"2024-02-22T19:33:45.177832+0000","last_fullsized":"2024-02-22T19:33:45.177832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:30:32.846336+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.835432+0000","last_change":"2024-02-22T19:33:58.835432+0000","last_active":"2024-02-22T19:33:58.835432+0000","last_peered":"2024-02-22T19:33:58.835432+0000","last_clean":"2024-02-22T19:33:58.835432+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:33:58.835432+0000","last_undegraded":"2024-02-22T19:33:58.835432+0000","last_fullsized":"2024-02-22T19:33:58.835432+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.835370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.835370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":156,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.201157+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:33:59.201157+0000","last_peered":"2024-02-22T19:33:59.201157+0000","last_clean":"2024-02-22T19:33:59.201157+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:33:59.201157+0000","last_undegraded":"2024-02-22T19:33:59.201157+0000","last_fullsized":"2024-02-22T19:33:59.201157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.169557+0000","last_change":"2024-02-22T19:31:11.415809+0000","last_active":"2024-02-22T19:33:45.169557+0000","last_peered":"2024-02-22T19:33:45.169557+0000","last_clean":"2024-02-22T19:33:45.169557+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:33:45.169557+0000","last_undegraded":"2024-02-22T19:33:45.169557+0000","last_fullsized":"2024-02-22T19:33:45.169557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.415718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":117,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152529+0000","last_change":"2024-02-22T19:31:50.352892+0000","last_active":"2024-02-22T19:33:45.152529+0000","last_peered":"2024-02-22T19:33:45.152529+0000","last_clean":"2024-02-22T19:33:45.152529+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:33:45.152529+0000","last_undegraded":"2024-02-22T19:33:45.152529+0000","last_fullsized":"2024-02-22T19:33:45.152529+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.685295+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":93,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:57.874765+0000","last_change":"2024-02-22T19:33:57.874765+0000","last_active":"2024-02-22T19:33:57.874765+0000","last_peered":"2024-02-22T19:33:57.874765+0000","last_clean":"2024-02-22T19:33:57.874765+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:33:57.874765+0000","last_undegraded":"2024-02-22T19:33:57.874765+0000","last_fullsized":"2024-02-22T19:33:57.874765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.874679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.874679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":87,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173357+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:33:45.173357+0000","last_peered":"2024-02-22T19:33:45.173357+0000","last_clean":"2024-02-22T19:33:45.173357+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:33:45.173357+0000","last_undegraded":"2024-02-22T19:33:45.173357+0000","last_fullsized":"2024-02-22T19:33:45.173357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:53.652573+0000","last_change":"2024-02-22T19:33:53.652545+0000","last_active":"2024-02-22T19:33:53.652573+0000","last_peered":"2024-02-22T19:33:53.652573+0000","last_clean":"2024-02-22T19:33:53.652573+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:33:53.652573+0000","last_undegraded":"2024-02-22T19:33:53.652573+0000","last_fullsized":"2024-02-22T19:33:53.652573+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:53.652468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.652468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.001397+0000","last_change":"2024-02-22T19:33:46.001358+0000","last_active":"2024-02-22T19:33:46.001397+0000","last_peered":"2024-02-22T19:33:46.001397+0000","last_clean":"2024-02-22T19:33:46.001397+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:33:46.001397+0000","last_undegraded":"2024-02-22T19:33:46.001397+0000","last_fullsized":"2024-02-22T19:33:46.001397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:46.001290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.001290+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":91,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:56.634433+0000","last_change":"2024-02-22T19:33:56.634433+0000","last_active":"2024-02-22T19:33:56.634433+0000","last_peered":"2024-02-22T19:33:56.634433+0000","last_clean":"2024-02-22T19:33:56.634433+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:33:56.634433+0000","last_undegraded":"2024-02-22T19:33:56.634433+0000","last_fullsized":"2024-02-22T19:33:56.634433+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:56.634370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:56.634370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.180485+0000","last_change":"2024-02-22T19:31:50.013893+0000","last_active":"2024-02-22T19:33:45.180485+0000","last_peered":"2024-02-22T19:33:45.180485+0000","last_clean":"2024-02-22T19:33:45.180485+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:33:45.180485+0000","last_undegraded":"2024-02-22T19:33:45.180485+0000","last_fullsized":"2024-02-22T19:33:45.180485+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:27.575255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:27.575255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":102,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.147636+0000","last_change":"2024-02-22T19:31:50.006289+0000","last_active":"2024-02-22T19:33:45.147636+0000","last_peered":"2024-02-22T19:33:45.147636+0000","last_clean":"2024-02-22T19:33:45.147636+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:33:45.147636+0000","last_undegraded":"2024-02-22T19:33:45.147636+0000","last_fullsized":"2024-02-22T19:33:45.147636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.656244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.656244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:52.788186+0000","last_change":"2024-02-22T19:33:52.788152+0000","last_active":"2024-02-22T19:33:52.788186+0000","last_peered":"2024-02-22T19:33:52.788186+0000","last_clean":"2024-02-22T19:33:52.788186+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:33:52.788186+0000","last_undegraded":"2024-02-22T19:33:52.788186+0000","last_fullsized":"2024-02-22T19:33:52.788186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:33:52.788050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.788050+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"137'8238","reported_seq":10858,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.250971+0000","last_change":"2024-02-22T19:32:06.969062+0000","last_active":"2024-02-22T19:34:04.250971+0000","last_peered":"2024-02-22T19:34:04.250971+0000","last_clean":"2024-02-22T19:34:04.250971+0000","last_became_active":"2024-02-22T19:32:06.966453+0000","last_became_peered":"2024-02-22T19:32:06.966453+0000","last_unstale":"2024-02-22T19:34:04.250971+0000","last_undegraded":"2024-02-22T19:34:04.250971+0000","last_fullsized":"2024-02-22T19:34:04.250971+0000","mapping_epoch":104,"log_start":"115'3600","ondisk_log_start":"115'3600","created":93,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4638,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":191758336,"num_objects":2926,"num_object_clones":0,"num_object_copies":8778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8238,"num_write_kb":357248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":54,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.804958+0000","last_change":"2024-02-22T19:33:49.804924+0000","last_active":"2024-02-22T19:33:49.804958+0000","last_peered":"2024-02-22T19:33:49.804958+0000","last_clean":"2024-02-22T19:33:49.804958+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:33:49.804958+0000","last_undegraded":"2024-02-22T19:33:49.804958+0000","last_fullsized":"2024-02-22T19:33:49.804958+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.804842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.804842+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":112,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:01.596593+0000","last_change":"2024-02-22T19:34:01.596593+0000","last_active":"2024-02-22T19:34:01.596593+0000","last_peered":"2024-02-22T19:34:01.596593+0000","last_clean":"2024-02-22T19:34:01.596593+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:34:01.596593+0000","last_undegraded":"2024-02-22T19:34:01.596593+0000","last_fullsized":"2024-02-22T19:34:01.596593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:34:01.596491+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.596491+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":102,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.153076+0000","last_change":"2024-02-22T19:31:50.611447+0000","last_active":"2024-02-22T19:33:45.153076+0000","last_peered":"2024-02-22T19:33:45.153076+0000","last_clean":"2024-02-22T19:33:45.153076+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:33:45.153076+0000","last_undegraded":"2024-02-22T19:33:45.153076+0000","last_fullsized":"2024-02-22T19:33:45.153076+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:30:28.601576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:28.601576+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":157,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.198478+0000","last_change":"2024-02-22T19:30:45.659967+0000","last_active":"2024-02-22T19:33:59.198478+0000","last_peered":"2024-02-22T19:33:59.198478+0000","last_clean":"2024-02-22T19:33:59.198478+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:33:59.198478+0000","last_undegraded":"2024-02-22T19:33:59.198478+0000","last_fullsized":"2024-02-22T19:33:59.198478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:45.659896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.659896+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"137'8258","reported_seq":10877,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.272289+0000","last_change":"2024-02-22T19:33:42.253020+0000","last_active":"2024-02-22T19:34:04.272289+0000","last_peered":"2024-02-22T19:34:04.272289+0000","last_clean":"2024-02-22T19:34:04.272289+0000","last_became_active":"2024-02-22T19:33:42.130070+0000","last_became_peered":"2024-02-22T19:33:42.130070+0000","last_unstale":"2024-02-22T19:34:04.272289+0000","last_undegraded":"2024-02-22T19:34:04.272289+0000","last_fullsized":"2024-02-22T19:34:04.272289+0000","mapping_epoch":134,"log_start":"117'4100","ondisk_log_start":"117'4100","created":93,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4158,"ondisk_log_size":4158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":186646528,"num_objects":2848,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8258,"num_write_kb":355392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1273,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173514+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:33:45.173514+0000","last_peered":"2024-02-22T19:33:45.173514+0000","last_clean":"2024-02-22T19:33:45.173514+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:33:45.173514+0000","last_undegraded":"2024-02-22T19:33:45.173514+0000","last_fullsized":"2024-02-22T19:33:45.173514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":97,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.606520+0000","last_change":"2024-02-22T19:33:54.606520+0000","last_active":"2024-02-22T19:33:54.606520+0000","last_peered":"2024-02-22T19:33:54.606520+0000","last_clean":"2024-02-22T19:33:54.606520+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:33:54.606520+0000","last_undegraded":"2024-02-22T19:33:54.606520+0000","last_fullsized":"2024-02-22T19:33:54.606520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:54.606418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.606418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174883+0000","last_change":"2024-02-22T19:30:35.031906+0000","last_active":"2024-02-22T19:33:45.174883+0000","last_peered":"2024-02-22T19:33:45.174883+0000","last_clean":"2024-02-22T19:33:45.174883+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:33:45.174883+0000","last_undegraded":"2024-02-22T19:33:45.174883+0000","last_fullsized":"2024-02-22T19:33:45.174883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:35.031815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.031815+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.166524+0000","last_change":"2024-02-22T19:30:36.176563+0000","last_active":"2024-02-22T19:33:45.166524+0000","last_peered":"2024-02-22T19:33:45.166524+0000","last_clean":"2024-02-22T19:33:45.166524+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:33:45.166524+0000","last_undegraded":"2024-02-22T19:33:45.166524+0000","last_fullsized":"2024-02-22T19:33:45.166524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.176489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.176489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"137'8346","reported_seq":8398,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.256649+0000","last_change":"2024-02-22T19:31:33.014127+0000","last_active":"2024-02-22T19:34:04.256649+0000","last_peered":"2024-02-22T19:34:04.256649+0000","last_clean":"2024-02-22T19:34:04.256649+0000","last_became_active":"2024-02-22T19:31:33.013760+0000","last_became_peered":"2024-02-22T19:31:33.013760+0000","last_unstale":"2024-02-22T19:34:04.256649+0000","last_undegraded":"2024-02-22T19:34:04.256649+0000","last_fullsized":"2024-02-22T19:34:04.256649+0000","mapping_epoch":93,"log_start":"115'3500","ondisk_log_start":"115'3500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4846,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":190447616,"num_objects":2906,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8346,"num_write_kb":360064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152948+0000","last_change":"2024-02-22T19:31:02.751858+0000","last_active":"2024-02-22T19:33:45.152948+0000","last_peered":"2024-02-22T19:33:45.152948+0000","last_clean":"2024-02-22T19:33:45.152948+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:33:45.152948+0000","last_undegraded":"2024-02-22T19:33:45.152948+0000","last_fullsized":"2024-02-22T19:33:45.152948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:02.751782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:02.751782+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.766849+0000","last_change":"2024-02-22T19:33:48.766810+0000","last_active":"2024-02-22T19:33:48.766849+0000","last_peered":"2024-02-22T19:33:48.766849+0000","last_clean":"2024-02-22T19:33:48.766849+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:33:48.766849+0000","last_undegraded":"2024-02-22T19:33:48.766849+0000","last_fullsized":"2024-02-22T19:33:48.766849+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.766706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.766706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":106,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152429+0000","last_change":"2024-02-22T19:31:49.999189+0000","last_active":"2024-02-22T19:33:45.152429+0000","last_peered":"2024-02-22T19:33:45.152429+0000","last_clean":"2024-02-22T19:33:45.152429+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:33:45.152429+0000","last_undegraded":"2024-02-22T19:33:45.152429+0000","last_fullsized":"2024-02-22T19:33:45.152429+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:15.882992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:15.882992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":125,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:53.782561+0000","last_change":"2024-02-22T19:33:53.782532+0000","last_active":"2024-02-22T19:33:53.782561+0000","last_peered":"2024-02-22T19:33:53.782561+0000","last_clean":"2024-02-22T19:33:53.782561+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:33:53.782561+0000","last_undegraded":"2024-02-22T19:33:53.782561+0000","last_fullsized":"2024-02-22T19:33:53.782561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:33:53.782465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.782465+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"137'8257","reported_seq":8309,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.271580+0000","last_change":"2024-02-22T19:31:33.015434+0000","last_active":"2024-02-22T19:34:04.271580+0000","last_peered":"2024-02-22T19:34:04.271580+0000","last_clean":"2024-02-22T19:34:04.271580+0000","last_became_active":"2024-02-22T19:31:33.015132+0000","last_became_peered":"2024-02-22T19:31:33.015132+0000","last_unstale":"2024-02-22T19:34:04.271580+0000","last_undegraded":"2024-02-22T19:34:04.271580+0000","last_fullsized":"2024-02-22T19:34:04.271580+0000","mapping_epoch":93,"log_start":"107'2900","ondisk_log_start":"107'2900","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5357,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":182386688,"num_objects":2783,"num_object_clones":0,"num_object_copies":8349,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.170135+0000","last_change":"2024-02-22T19:31:06.495959+0000","last_active":"2024-02-22T19:33:45.170135+0000","last_peered":"2024-02-22T19:33:45.170135+0000","last_clean":"2024-02-22T19:33:45.170135+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:33:45.170135+0000","last_undegraded":"2024-02-22T19:33:45.170135+0000","last_fullsized":"2024-02-22T19:33:45.170135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.495875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.495875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":95,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.910213+0000","last_change":"2024-02-22T19:33:58.910213+0000","last_active":"2024-02-22T19:33:58.910213+0000","last_peered":"2024-02-22T19:33:58.910213+0000","last_clean":"2024-02-22T19:33:58.910213+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:33:58.910213+0000","last_undegraded":"2024-02-22T19:33:58.910213+0000","last_fullsized":"2024-02-22T19:33:58.910213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.910117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.910117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177902+0000","last_change":"2024-02-22T19:31:50.014265+0000","last_active":"2024-02-22T19:33:45.177902+0000","last_peered":"2024-02-22T19:33:45.177902+0000","last_clean":"2024-02-22T19:33:45.177902+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:33:45.177902+0000","last_undegraded":"2024-02-22T19:33:45.177902+0000","last_fullsized":"2024-02-22T19:33:45.177902+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:17.466666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:17.466666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":125,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.172477+0000","last_change":"2024-02-22T19:30:48.444481+0000","last_active":"2024-02-22T19:33:45.172477+0000","last_peered":"2024-02-22T19:33:45.172477+0000","last_clean":"2024-02-22T19:33:45.172477+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:33:45.172477+0000","last_undegraded":"2024-02-22T19:33:45.172477+0000","last_fullsized":"2024-02-22T19:33:45.172477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:30:48.444357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:48.444357+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"137'8207","reported_seq":8259,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.198916+0000","last_change":"2024-02-22T19:31:33.014009+0000","last_active":"2024-02-22T19:34:04.198916+0000","last_peered":"2024-02-22T19:34:04.198916+0000","last_clean":"2024-02-22T19:34:04.198916+0000","last_became_active":"2024-02-22T19:31:33.013715+0000","last_became_peered":"2024-02-22T19:31:33.013715+0000","last_unstale":"2024-02-22T19:34:04.198916+0000","last_undegraded":"2024-02-22T19:34:04.198916+0000","last_fullsized":"2024-02-22T19:34:04.198916+0000","mapping_epoch":93,"log_start":"107'2800","ondisk_log_start":"107'2800","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5407,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":185663488,"num_objects":2833,"num_object_clones":0,"num_object_copies":8499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8207,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":85,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.180658+0000","last_change":"2024-02-22T19:30:51.890687+0000","last_active":"2024-02-22T19:33:45.180658+0000","last_peered":"2024-02-22T19:33:45.180658+0000","last_clean":"2024-02-22T19:33:45.180658+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:33:45.180658+0000","last_undegraded":"2024-02-22T19:33:45.180658+0000","last_fullsized":"2024-02-22T19:33:45.180658+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:51.890611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:51.890611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.156100+0000","last_change":"2024-02-22T19:31:50.010144+0000","last_active":"2024-02-22T19:33:45.156100+0000","last_peered":"2024-02-22T19:33:45.156100+0000","last_clean":"2024-02-22T19:33:45.156100+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:33:45.156100+0000","last_undegraded":"2024-02-22T19:33:45.156100+0000","last_fullsized":"2024-02-22T19:33:45.156100+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.692694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175198+0000","last_change":"2024-02-22T19:30:22.124106+0000","last_active":"2024-02-22T19:33:45.175198+0000","last_peered":"2024-02-22T19:33:45.175198+0000","last_clean":"2024-02-22T19:33:45.175198+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:33:45.175198+0000","last_undegraded":"2024-02-22T19:33:45.175198+0000","last_fullsized":"2024-02-22T19:33:45.175198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:22.123990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:22.123990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":206,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.199757+0000","last_change":"2024-02-22T19:33:46.724569+0000","last_active":"2024-02-22T19:33:59.199757+0000","last_peered":"2024-02-22T19:33:59.199757+0000","last_clean":"2024-02-22T19:33:59.199757+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:33:59.199757+0000","last_undegraded":"2024-02-22T19:33:59.199757+0000","last_fullsized":"2024-02-22T19:33:59.199757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:33:46.724462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.724462+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":93,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:58.185263+0000","last_change":"2024-02-22T19:33:58.185263+0000","last_active":"2024-02-22T19:33:58.185263+0000","last_peered":"2024-02-22T19:33:58.185263+0000","last_clean":"2024-02-22T19:33:58.185263+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:33:58.185263+0000","last_undegraded":"2024-02-22T19:33:58.185263+0000","last_fullsized":"2024-02-22T19:33:58.185263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.185187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.185187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"137'8150","reported_seq":8202,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.273366+0000","last_change":"2024-02-22T19:31:33.008987+0000","last_active":"2024-02-22T19:34:04.273366+0000","last_peered":"2024-02-22T19:34:04.273366+0000","last_clean":"2024-02-22T19:34:04.273366+0000","last_became_active":"2024-02-22T19:31:33.008726+0000","last_became_peered":"2024-02-22T19:31:33.008726+0000","last_unstale":"2024-02-22T19:34:04.273366+0000","last_undegraded":"2024-02-22T19:34:04.273366+0000","last_fullsized":"2024-02-22T19:34:04.273366+0000","mapping_epoch":93,"log_start":"112'3300","ondisk_log_start":"112'3300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4850,"ondisk_log_size":4850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":189136896,"num_objects":2886,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":353152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152537+0000","last_change":"2024-02-22T19:30:36.620293+0000","last_active":"2024-02-22T19:33:45.152537+0000","last_peered":"2024-02-22T19:33:45.152537+0000","last_clean":"2024-02-22T19:33:45.152537+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:33:45.152537+0000","last_undegraded":"2024-02-22T19:33:45.152537+0000","last_fullsized":"2024-02-22T19:33:45.152537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.620214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":99,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:01.732261+0000","last_change":"2024-02-22T19:34:01.732261+0000","last_active":"2024-02-22T19:34:01.732261+0000","last_peered":"2024-02-22T19:34:01.732261+0000","last_clean":"2024-02-22T19:34:01.732261+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:34:01.732261+0000","last_undegraded":"2024-02-22T19:34:01.732261+0000","last_fullsized":"2024-02-22T19:34:01.732261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:01.732156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.732156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":99,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.158966+0000","last_change":"2024-02-22T19:33:44.828664+0000","last_active":"2024-02-22T19:33:45.158966+0000","last_peered":"2024-02-22T19:33:45.158966+0000","last_clean":"2024-02-22T19:33:45.158966+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:33:45.158966+0000","last_undegraded":"2024-02-22T19:33:45.158966+0000","last_fullsized":"2024-02-22T19:33:45.158966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:44.828568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.828568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.950686+0000","last_change":"2024-02-22T19:33:59.950686+0000","last_active":"2024-02-22T19:33:59.950686+0000","last_peered":"2024-02-22T19:33:59.950686+0000","last_clean":"2024-02-22T19:33:59.950686+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:33:59.950686+0000","last_undegraded":"2024-02-22T19:33:59.950686+0000","last_fullsized":"2024-02-22T19:33:59.950686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:33:59.950586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:59.950586+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:46.825770+0000","last_change":"2024-02-22T19:33:46.825747+0000","last_active":"2024-02-22T19:33:46.825770+0000","last_peered":"2024-02-22T19:33:46.825770+0000","last_clean":"2024-02-22T19:33:46.825770+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:33:46.825770+0000","last_undegraded":"2024-02-22T19:33:46.825770+0000","last_fullsized":"2024-02-22T19:33:46.825770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:46.825679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.825679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"137'8301","reported_seq":8353,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.268444+0000","last_change":"2024-02-22T19:31:33.014815+0000","last_active":"2024-02-22T19:34:04.268444+0000","last_peered":"2024-02-22T19:34:04.268444+0000","last_clean":"2024-02-22T19:34:04.268444+0000","last_became_active":"2024-02-22T19:31:33.014489+0000","last_became_peered":"2024-02-22T19:31:33.014489+0000","last_unstale":"2024-02-22T19:34:04.268444+0000","last_undegraded":"2024-02-22T19:34:04.268444+0000","last_fullsized":"2024-02-22T19:34:04.268444+0000","mapping_epoch":93,"log_start":"113'3400","ondisk_log_start":"113'3400","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4901,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":187367424,"num_objects":2859,"num_object_clones":0,"num_object_copies":8577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8301,"num_write_kb":357120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177528+0000","last_change":"2024-02-22T19:31:50.014877+0000","last_active":"2024-02-22T19:33:45.177528+0000","last_peered":"2024-02-22T19:33:45.177528+0000","last_clean":"2024-02-22T19:33:45.177528+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:33:45.177528+0000","last_undegraded":"2024-02-22T19:33:45.177528+0000","last_fullsized":"2024-02-22T19:33:45.177528+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152994+0000","last_change":"2024-02-22T19:31:11.906662+0000","last_active":"2024-02-22T19:33:45.152994+0000","last_peered":"2024-02-22T19:33:45.152994+0000","last_clean":"2024-02-22T19:33:45.152994+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:33:45.152994+0000","last_undegraded":"2024-02-22T19:33:45.152994+0000","last_fullsized":"2024-02-22T19:33:45.152994+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:11.906561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:31:11.906561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173831+0000","last_change":"2024-02-22T19:31:50.009641+0000","last_active":"2024-02-22T19:33:45.173831+0000","last_peered":"2024-02-22T19:33:45.173831+0000","last_clean":"2024-02-22T19:33:45.173831+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:33:45.173831+0000","last_undegraded":"2024-02-22T19:33:45.173831+0000","last_fullsized":"2024-02-22T19:33:45.173831+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":118,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.148719+0000","last_change":"2024-02-22T19:30:34.566524+0000","last_active":"2024-02-22T19:33:45.148719+0000","last_peered":"2024-02-22T19:33:45.148719+0000","last_clean":"2024-02-22T19:33:45.148719+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:33:45.148719+0000","last_undegraded":"2024-02-22T19:33:45.148719+0000","last_fullsized":"2024-02-22T19:33:45.148719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.566443+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.152287+0000","last_change":"2024-02-22T19:31:49.995726+0000","last_active":"2024-02-22T19:33:45.152287+0000","last_peered":"2024-02-22T19:33:45.152287+0000","last_clean":"2024-02-22T19:33:45.152287+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:33:45.152287+0000","last_undegraded":"2024-02-22T19:33:45.152287+0000","last_fullsized":"2024-02-22T19:33:45.152287+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:49.658226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:49.658226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"137'8311","reported_seq":8362,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.180684+0000","last_change":"2024-02-22T19:31:33.010431+0000","last_active":"2024-02-22T19:34:04.180684+0000","last_peered":"2024-02-22T19:34:04.180684+0000","last_clean":"2024-02-22T19:34:04.180684+0000","last_became_active":"2024-02-22T19:31:33.010187+0000","last_became_peered":"2024-02-22T19:31:33.010187+0000","last_unstale":"2024-02-22T19:34:04.180684+0000","last_undegraded":"2024-02-22T19:34:04.180684+0000","last_fullsized":"2024-02-22T19:34:04.180684+0000","mapping_epoch":93,"log_start":"115'3500","ondisk_log_start":"115'3500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4811,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":185532416,"num_objects":2831,"num_object_clones":0,"num_object_copies":8493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8311,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":97,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154191+0000","last_change":"2024-02-22T19:31:50.012369+0000","last_active":"2024-02-22T19:33:45.154191+0000","last_peered":"2024-02-22T19:33:45.154191+0000","last_clean":"2024-02-22T19:33:45.154191+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:33:45.154191+0000","last_undegraded":"2024-02-22T19:33:45.154191+0000","last_fullsized":"2024-02-22T19:33:45.154191+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:33.113520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:33.113520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.169943+0000","last_change":"2024-02-22T19:30:45.533717+0000","last_active":"2024-02-22T19:33:45.169943+0000","last_peered":"2024-02-22T19:33:45.169943+0000","last_clean":"2024-02-22T19:33:45.169943+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:33:45.169943+0000","last_undegraded":"2024-02-22T19:33:45.169943+0000","last_fullsized":"2024-02-22T19:33:45.169943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.533624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.533624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":397,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:02.589165+0000","last_change":"2024-02-22T19:31:29.001184+0000","last_active":"2024-02-22T19:34:02.589165+0000","last_peered":"2024-02-22T19:34:02.589165+0000","last_clean":"2024-02-22T19:34:02.589165+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:34:02.589165+0000","last_undegraded":"2024-02-22T19:34:02.589165+0000","last_fullsized":"2024-02-22T19:34:02.589165+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:31:29.001098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.001098+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":297,"num_read_kb":292,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171394+0000","last_change":"2024-02-22T19:30:55.536117+0000","last_active":"2024-02-22T19:33:45.171394+0000","last_peered":"2024-02-22T19:33:45.171394+0000","last_clean":"2024-02-22T19:33:45.171394+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:33:45.171394+0000","last_undegraded":"2024-02-22T19:33:45.171394+0000","last_fullsized":"2024-02-22T19:33:45.171394+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:30:55.536040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.536040+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":54,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:00.713551+0000","last_change":"2024-02-22T19:34:00.713551+0000","last_active":"2024-02-22T19:34:00.713551+0000","last_peered":"2024-02-22T19:34:00.713551+0000","last_clean":"2024-02-22T19:34:00.713551+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:34:00.713551+0000","last_undegraded":"2024-02-22T19:34:00.713551+0000","last_fullsized":"2024-02-22T19:34:00.713551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:00.713486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:00.713486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"137'8301","reported_seq":10997,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.221397+0000","last_change":"2024-02-22T19:32:07.977104+0000","last_active":"2024-02-22T19:34:04.221397+0000","last_peered":"2024-02-22T19:34:04.221397+0000","last_clean":"2024-02-22T19:34:04.221397+0000","last_became_active":"2024-02-22T19:32:07.973466+0000","last_became_peered":"2024-02-22T19:32:07.973466+0000","last_unstale":"2024-02-22T19:34:04.221397+0000","last_undegraded":"2024-02-22T19:34:04.221397+0000","last_fullsized":"2024-02-22T19:34:04.221397+0000","mapping_epoch":105,"log_start":"117'3700","ondisk_log_start":"117'3700","created":93,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4601,"ondisk_log_size":4601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":186187776,"num_objects":2841,"num_object_clones":0,"num_object_copies":8523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8301,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":86048768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":88,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154675+0000","last_change":"2024-02-22T19:30:42.994987+0000","last_active":"2024-02-22T19:33:45.154675+0000","last_peered":"2024-02-22T19:33:45.154675+0000","last_clean":"2024-02-22T19:33:45.154675+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:33:45.154675+0000","last_undegraded":"2024-02-22T19:33:45.154675+0000","last_fullsized":"2024-02-22T19:33:45.154675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:30:42.994920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154381+0000","last_change":"2024-02-22T19:31:50.010143+0000","last_active":"2024-02-22T19:33:45.154381+0000","last_peered":"2024-02-22T19:33:45.154381+0000","last_clean":"2024-02-22T19:33:45.154381+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:33:45.154381+0000","last_undegraded":"2024-02-22T19:33:45.154381+0000","last_fullsized":"2024-02-22T19:33:45.154381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.149677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.149677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":158,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:02.072893+0000","last_change":"2024-02-22T19:34:02.072893+0000","last_active":"2024-02-22T19:34:02.072893+0000","last_peered":"2024-02-22T19:34:02.072893+0000","last_clean":"2024-02-22T19:34:02.072893+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:34:02.072893+0000","last_undegraded":"2024-02-22T19:34:02.072893+0000","last_fullsized":"2024-02-22T19:34:02.072893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:34:02.072818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:02.072818+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":98,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.147739+0000","last_change":"2024-02-22T19:31:50.006906+0000","last_active":"2024-02-22T19:33:45.147739+0000","last_peered":"2024-02-22T19:33:45.147739+0000","last_clean":"2024-02-22T19:33:45.147739+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:33:45.147739+0000","last_undegraded":"2024-02-22T19:33:45.147739+0000","last_fullsized":"2024-02-22T19:33:45.147739+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:14.658901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:14.658901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"137'8131","reported_seq":8183,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.189018+0000","last_change":"2024-02-22T19:31:33.012064+0000","last_active":"2024-02-22T19:34:04.189018+0000","last_peered":"2024-02-22T19:34:04.189018+0000","last_clean":"2024-02-22T19:34:04.189018+0000","last_became_active":"2024-02-22T19:31:33.011750+0000","last_became_peered":"2024-02-22T19:31:33.011750+0000","last_unstale":"2024-02-22T19:34:04.189018+0000","last_undegraded":"2024-02-22T19:34:04.189018+0000","last_fullsized":"2024-02-22T19:34:04.189018+0000","mapping_epoch":93,"log_start":"116'3600","ondisk_log_start":"116'3600","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4531,"ondisk_log_size":4531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":191823872,"num_objects":2927,"num_object_clones":0,"num_object_copies":8781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":353856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:49.035705+0000","last_change":"2024-02-22T19:33:49.035679+0000","last_active":"2024-02-22T19:33:49.035705+0000","last_peered":"2024-02-22T19:33:49.035705+0000","last_clean":"2024-02-22T19:33:49.035705+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:33:49.035705+0000","last_undegraded":"2024-02-22T19:33:49.035705+0000","last_fullsized":"2024-02-22T19:33:49.035705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.035615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.035615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174684+0000","last_change":"2024-02-22T19:31:50.006662+0000","last_active":"2024-02-22T19:33:45.174684+0000","last_peered":"2024-02-22T19:33:45.174684+0000","last_clean":"2024-02-22T19:33:45.174684+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:33:45.174684+0000","last_undegraded":"2024-02-22T19:33:45.174684+0000","last_fullsized":"2024-02-22T19:33:45.174684+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:29.616636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:29.616636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":133,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.156399+0000","last_change":"2024-02-22T19:33:44.757231+0000","last_active":"2024-02-22T19:33:45.156399+0000","last_peered":"2024-02-22T19:33:45.156399+0000","last_clean":"2024-02-22T19:33:45.156399+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:33:45.156399+0000","last_undegraded":"2024-02-22T19:33:45.156399+0000","last_fullsized":"2024-02-22T19:33:45.156399+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:33:44.757114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.757114+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":170,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.200947+0000","last_change":"2024-02-22T19:31:50.440080+0000","last_active":"2024-02-22T19:33:59.200947+0000","last_peered":"2024-02-22T19:33:59.200947+0000","last_clean":"2024-02-22T19:33:59.200947+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:33:59.200947+0000","last_undegraded":"2024-02-22T19:33:59.200947+0000","last_fullsized":"2024-02-22T19:33:59.200947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:44.574360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.574360+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"137'8274","reported_seq":8326,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.286892+0000","last_change":"2024-02-22T19:31:33.012817+0000","last_active":"2024-02-22T19:34:04.286892+0000","last_peered":"2024-02-22T19:34:04.286892+0000","last_clean":"2024-02-22T19:34:04.286892+0000","last_became_active":"2024-02-22T19:31:33.012470+0000","last_became_peered":"2024-02-22T19:31:33.012470+0000","last_unstale":"2024-02-22T19:34:04.286892+0000","last_undegraded":"2024-02-22T19:34:04.286892+0000","last_fullsized":"2024-02-22T19:34:04.286892+0000","mapping_epoch":93,"log_start":"117'4100","ondisk_log_start":"117'4100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4174,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":189267968,"num_objects":2888,"num_object_clones":0,"num_object_copies":8664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":357184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":86,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.154357+0000","last_change":"2024-02-22T19:30:38.631764+0000","last_active":"2024-02-22T19:33:45.154357+0000","last_peered":"2024-02-22T19:33:45.154357+0000","last_clean":"2024-02-22T19:33:45.154357+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:33:45.154357+0000","last_undegraded":"2024-02-22T19:33:45.154357+0000","last_fullsized":"2024-02-22T19:33:45.154357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:30:38.631697+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.171462+0000","last_change":"2024-02-22T19:30:50.477813+0000","last_active":"2024-02-22T19:33:45.171462+0000","last_peered":"2024-02-22T19:33:45.171462+0000","last_clean":"2024-02-22T19:33:45.171462+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:33:45.171462+0000","last_undegraded":"2024-02-22T19:33:45.171462+0000","last_fullsized":"2024-02-22T19:33:45.171462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:50.477738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.477738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":145,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174774+0000","last_change":"2024-02-22T19:31:50.644540+0000","last_active":"2024-02-22T19:33:45.174774+0000","last_peered":"2024-02-22T19:33:45.174774+0000","last_clean":"2024-02-22T19:33:45.174774+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:33:45.174774+0000","last_undegraded":"2024-02-22T19:33:45.174774+0000","last_fullsized":"2024-02-22T19:33:45.174774+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":156,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.200275+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:33:59.200275+0000","last_peered":"2024-02-22T19:33:59.200275+0000","last_clean":"2024-02-22T19:33:59.200275+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:33:59.200275+0000","last_undegraded":"2024-02-22T19:33:59.200275+0000","last_fullsized":"2024-02-22T19:33:59.200275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"137'8442","reported_seq":8494,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.281308+0000","last_change":"2024-02-22T19:31:33.013118+0000","last_active":"2024-02-22T19:34:04.281308+0000","last_peered":"2024-02-22T19:34:04.281308+0000","last_clean":"2024-02-22T19:34:04.281308+0000","last_became_active":"2024-02-22T19:31:33.012002+0000","last_became_peered":"2024-02-22T19:31:33.012002+0000","last_unstale":"2024-02-22T19:34:04.281308+0000","last_undegraded":"2024-02-22T19:34:04.281308+0000","last_fullsized":"2024-02-22T19:34:04.281308+0000","mapping_epoch":93,"log_start":"117'4300","ondisk_log_start":"117'4300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4142,"ondisk_log_size":4142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":192544768,"num_objects":2938,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8442,"num_write_kb":364160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":93,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:01.893979+0000","last_change":"2024-02-22T19:34:01.893979+0000","last_active":"2024-02-22T19:34:01.893979+0000","last_peered":"2024-02-22T19:34:01.893979+0000","last_clean":"2024-02-22T19:34:01.893979+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:34:01.893979+0000","last_undegraded":"2024-02-22T19:34:01.893979+0000","last_fullsized":"2024-02-22T19:34:01.893979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:01.893910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.893910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":91,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177373+0000","last_change":"2024-02-22T19:30:36.904777+0000","last_active":"2024-02-22T19:33:45.177373+0000","last_peered":"2024-02-22T19:33:45.177373+0000","last_clean":"2024-02-22T19:33:45.177373+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:33:45.177373+0000","last_undegraded":"2024-02-22T19:33:45.177373+0000","last_fullsized":"2024-02-22T19:33:45.177373+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:36.904715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:36.904715+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":128,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:54.749170+0000","last_change":"2024-02-22T19:33:54.749170+0000","last_active":"2024-02-22T19:33:54.749170+0000","last_peered":"2024-02-22T19:33:54.749170+0000","last_clean":"2024-02-22T19:33:54.749170+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:33:54.749170+0000","last_undegraded":"2024-02-22T19:33:54.749170+0000","last_fullsized":"2024-02-22T19:33:54.749170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:33:54.749061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.749061+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":95,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.174663+0000","last_change":"2024-02-22T19:31:26.190739+0000","last_active":"2024-02-22T19:33:45.174663+0000","last_peered":"2024-02-22T19:33:45.174663+0000","last_clean":"2024-02-22T19:33:45.174663+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:33:45.174663+0000","last_undegraded":"2024-02-22T19:33:45.174663+0000","last_fullsized":"2024-02-22T19:33:45.174663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.190632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.190632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"137'8361","reported_seq":8413,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.188603+0000","last_change":"2024-02-22T19:31:33.012589+0000","last_active":"2024-02-22T19:34:04.188603+0000","last_peered":"2024-02-22T19:34:04.188603+0000","last_clean":"2024-02-22T19:34:04.188603+0000","last_became_active":"2024-02-22T19:31:33.012188+0000","last_became_peered":"2024-02-22T19:31:33.012188+0000","last_unstale":"2024-02-22T19:34:04.188603+0000","last_undegraded":"2024-02-22T19:34:04.188603+0000","last_fullsized":"2024-02-22T19:34:04.188603+0000","mapping_epoch":93,"log_start":"109'3000","ondisk_log_start":"109'3000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5361,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":192610304,"num_objects":2939,"num_object_clones":0,"num_object_copies":8817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8361,"num_write_kb":361600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.155031+0000","last_change":"2024-02-22T19:31:50.009247+0000","last_active":"2024-02-22T19:33:45.155031+0000","last_peered":"2024-02-22T19:33:45.155031+0000","last_clean":"2024-02-22T19:33:45.155031+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:33:45.155031+0000","last_undegraded":"2024-02-22T19:33:45.155031+0000","last_fullsized":"2024-02-22T19:33:45.155031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:25.682652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:25.682652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":172,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.199556+0000","last_change":"2024-02-22T19:31:50.016299+0000","last_active":"2024-02-22T19:33:59.199556+0000","last_peered":"2024-02-22T19:33:59.199556+0000","last_clean":"2024-02-22T19:33:59.199556+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:33:59.199556+0000","last_undegraded":"2024-02-22T19:33:59.199556+0000","last_fullsized":"2024-02-22T19:33:59.199556+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:35.998951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.998951+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":96,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173489+0000","last_change":"2024-02-22T19:31:14.073980+0000","last_active":"2024-02-22T19:33:45.173489+0000","last_peered":"2024-02-22T19:33:45.173489+0000","last_clean":"2024-02-22T19:33:45.173489+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:33:45.173489+0000","last_undegraded":"2024-02-22T19:33:45.173489+0000","last_fullsized":"2024-02-22T19:33:45.173489+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.073908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.073908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":89,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.159714+0000","last_change":"2024-02-22T19:33:43.827188+0000","last_active":"2024-02-22T19:33:45.159714+0000","last_peered":"2024-02-22T19:33:45.159714+0000","last_clean":"2024-02-22T19:33:45.159714+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:33:45.159714+0000","last_undegraded":"2024-02-22T19:33:45.159714+0000","last_fullsized":"2024-02-22T19:33:45.159714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:33:43.827087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.827087+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"137'8505","reported_seq":8558,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.183114+0000","last_change":"2024-02-22T19:31:33.011737+0000","last_active":"2024-02-22T19:34:04.183114+0000","last_peered":"2024-02-22T19:34:04.183114+0000","last_clean":"2024-02-22T19:34:04.183114+0000","last_became_active":"2024-02-22T19:31:33.011438+0000","last_became_peered":"2024-02-22T19:31:33.011438+0000","last_unstale":"2024-02-22T19:34:04.183114+0000","last_undegraded":"2024-02-22T19:34:04.183114+0000","last_fullsized":"2024-02-22T19:34:04.183114+0000","mapping_epoch":93,"log_start":"109'3100","ondisk_log_start":"109'3100","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5405,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":191889432,"num_objects":2929,"num_object_clones":0,"num_object_copies":8787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8505,"num_write_kb":365825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":94,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.177752+0000","last_change":"2024-02-22T19:31:50.006646+0000","last_active":"2024-02-22T19:33:45.177752+0000","last_peered":"2024-02-22T19:33:45.177752+0000","last_clean":"2024-02-22T19:33:45.177752+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:33:45.177752+0000","last_undegraded":"2024-02-22T19:33:45.177752+0000","last_fullsized":"2024-02-22T19:33:45.177752+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":49,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.176040+0000","last_change":"2024-02-22T19:31:50.016416+0000","last_active":"2024-02-22T19:33:45.176040+0000","last_peered":"2024-02-22T19:33:45.176040+0000","last_clean":"2024-02-22T19:33:45.176040+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:33:45.176040+0000","last_undegraded":"2024-02-22T19:33:45.176040+0000","last_fullsized":"2024-02-22T19:33:45.176040+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:31.646595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:31.646595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:48.898915+0000","last_change":"2024-02-22T19:33:48.898875+0000","last_active":"2024-02-22T19:33:48.898915+0000","last_peered":"2024-02-22T19:33:48.898915+0000","last_clean":"2024-02-22T19:33:48.898915+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:33:48.898915+0000","last_undegraded":"2024-02-22T19:33:48.898915+0000","last_fullsized":"2024-02-22T19:33:48.898915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.898801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.898801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":116,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:51.953487+0000","last_change":"2024-02-22T19:33:51.953460+0000","last_active":"2024-02-22T19:33:51.953487+0000","last_peered":"2024-02-22T19:33:51.953487+0000","last_clean":"2024-02-22T19:33:51.953487+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:33:51.953487+0000","last_undegraded":"2024-02-22T19:33:51.953487+0000","last_fullsized":"2024-02-22T19:33:51.953487+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:51.953387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.953387+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"137'8104","reported_seq":10678,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.283167+0000","last_change":"2024-02-22T19:33:40.144966+0000","last_active":"2024-02-22T19:34:04.283167+0000","last_peered":"2024-02-22T19:34:04.283167+0000","last_clean":"2024-02-22T19:34:04.283167+0000","last_became_active":"2024-02-22T19:33:40.096849+0000","last_became_peered":"2024-02-22T19:33:40.096849+0000","last_unstale":"2024-02-22T19:34:04.283167+0000","last_undegraded":"2024-02-22T19:34:04.283167+0000","last_fullsized":"2024-02-22T19:34:04.283167+0000","mapping_epoch":132,"log_start":"107'2800","ondisk_log_start":"107'2800","created":93,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5304,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":178126848,"num_objects":2718,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8104,"num_write_kb":346304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1254,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":93,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:52.892072+0000","last_change":"2024-02-22T19:33:52.892034+0000","last_active":"2024-02-22T19:33:52.892072+0000","last_peered":"2024-02-22T19:33:52.892072+0000","last_clean":"2024-02-22T19:33:52.892072+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:33:52.892072+0000","last_undegraded":"2024-02-22T19:33:52.892072+0000","last_fullsized":"2024-02-22T19:33:52.892072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:52.891927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.891927+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":95,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:57.772428+0000","last_change":"2024-02-22T19:33:57.772428+0000","last_active":"2024-02-22T19:33:57.772428+0000","last_peered":"2024-02-22T19:33:57.772428+0000","last_clean":"2024-02-22T19:33:57.772428+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:33:57.772428+0000","last_undegraded":"2024-02-22T19:33:57.772428+0000","last_fullsized":"2024-02-22T19:33:57.772428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.772326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.772326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":96,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:51.803101+0000","last_change":"2024-02-22T19:33:51.803060+0000","last_active":"2024-02-22T19:33:51.803101+0000","last_peered":"2024-02-22T19:33:51.803101+0000","last_clean":"2024-02-22T19:33:51.803101+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:33:51.803101+0000","last_undegraded":"2024-02-22T19:33:51.803101+0000","last_fullsized":"2024-02-22T19:33:51.803101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:51.802966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.802966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":120,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.176101+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:33:45.176101+0000","last_peered":"2024-02-22T19:33:45.176101+0000","last_clean":"2024-02-22T19:33:45.176101+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:33:45.176101+0000","last_undegraded":"2024-02-22T19:33:45.176101+0000","last_fullsized":"2024-02-22T19:33:45.176101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"137'8314","reported_seq":8366,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:34:04.276826+0000","last_change":"2024-02-22T19:31:33.014534+0000","last_active":"2024-02-22T19:34:04.276826+0000","last_peered":"2024-02-22T19:34:04.276826+0000","last_clean":"2024-02-22T19:34:04.276826+0000","last_became_active":"2024-02-22T19:31:33.014231+0000","last_became_peered":"2024-02-22T19:31:33.014231+0000","last_unstale":"2024-02-22T19:34:04.276826+0000","last_undegraded":"2024-02-22T19:34:04.276826+0000","last_fullsized":"2024-02-22T19:34:04.276826+0000","mapping_epoch":93,"log_start":"113'3400","ondisk_log_start":"113'3400","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4914,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":187301888,"num_objects":2858,"num_object_clones":0,"num_object_copies":8574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8314,"num_write_kb":357504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":90,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175778+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:33:45.175778+0000","last_peered":"2024-02-22T19:33:45.175778+0000","last_clean":"2024-02-22T19:33:45.175778+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:33:45.175778+0000","last_undegraded":"2024-02-22T19:33:45.175778+0000","last_fullsized":"2024-02-22T19:33:45.175778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":177,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-02-22T19:33:59.199255+0000","last_change":"2024-02-22T19:33:42.757859+0000","last_active":"2024-02-22T19:33:59.199255+0000","last_peered":"2024-02-22T19:33:59.199255+0000","last_clean":"2024-02-22T19:33:59.199255+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:33:59.199255+0000","last_undegraded":"2024-02-22T19:33:59.199255+0000","last_fullsized":"2024-02-22T19:33:59.199255+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:33:42.757755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.757755+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.173268+0000","last_change":"2024-02-22T19:30:18.668779+0000","last_active":"2024-02-22T19:33:45.173268+0000","last_peered":"2024-02-22T19:33:45.173268+0000","last_clean":"2024-02-22T19:33:45.173268+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:33:45.173268+0000","last_undegraded":"2024-02-22T19:33:45.173268+0000","last_fullsized":"2024-02-22T19:33:45.173268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:30:18.668713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:18.668713+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":107,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-02-22T19:33:45.175273+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:33:45.175273+0000","last_peered":"2024-02-22T19:33:45.175273+0000","last_clean":"2024-02-22T19:33:45.175273+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:33:45.175273+0000","last_undegraded":"2024-02-22T19:33:45.175273+0000","last_fullsized":"2024-02-22T19:33:45.175273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3008692248,"num_objects":45910,"num_object_clones":0,"num_object_copies":137730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132500,"num_write_kb":5709057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5115,"num_bytes_recovered":332922880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9097785344,"data_stored":9097773128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77400,"ondisk_log_size":77400,"up":48,"acting":48,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":297,"num_read_kb":292,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266752,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1225540,"kb_used_data":928316,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":92519612,"statfs":{"total":95995035648,"available":94740082688,"internally_reserved":0,"allocated":950595584,"data_stored":950200564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":5.0609999999999999,"5min":1.506,"15min":0.88400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":42.597999999999999,"5min":42.597999999999999,"15min":42.597999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":3.5640000000000001,"5min":1.1499999999999999,"15min":0.73599999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":42.155999999999999,"5min":42.155999999999999,"15min":42.155999999999999},"last":0.23599999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.6139999999999999,"5min":1.1719999999999999,"15min":0.77600000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":38.188000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":1.421},{"interface":"front","average":{"1min":4.2999999999999998,"5min":1.3089999999999999,"15min":0.79900000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":37.938000000000002,"5min":37.938000000000002,"15min":37.938000000000002},"last":2.4319999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":3.9580000000000002,"5min":1.226,"15min":0.78200000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":41.640999999999998,"5min":41.640999999999998,"15min":41.640999999999998},"last":3.7799999999999998},{"interface":"front","average":{"1min":4.2530000000000001,"5min":1.3560000000000001,"15min":0.85199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":41.722000000000001,"5min":41.722000000000001,"15min":41.722000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1479999999999997,"5min":1.329,"15min":0.84499999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":41.457000000000001,"5min":41.457000000000001,"15min":41.457000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":3.6549999999999998,"5min":1.204,"15min":0.81899999999999995},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":41.311999999999998,"5min":41.311999999999998,"15min":41.311999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.308},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34699999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.41399999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4889999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5539999999999998,"5min":12.067,"15min":12.067},"last":0.51500000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1250092,"kb_used_data":940580,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92495060,"statfs":{"total":95995035648,"available":94714941440,"internally_reserved":0,"allocated":963153920,"data_stored":962782568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.7719999999999998,"5min":1.002,"15min":0.68000000000000005},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":17.885000000000002,"5min":17.885000000000002,"15min":17.885000000000002},"last":2},{"interface":"front","average":{"1min":2.5139999999999998,"5min":1.1120000000000001,"15min":0.71299999999999997},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":1.518}]},{"osd":1,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":0.80600000000000005,"15min":0.64000000000000001},"min":{"1min":0.29099999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":11.301,"5min":11.301,"15min":11.301},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":0.81399999999999995,"15min":0.62},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.8410000000000002,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":2.2810000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":1.093,"15min":0.72099999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":9.2319999999999993,"5min":12.859999999999999,"15min":12.859999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":1.54,"5min":0.82899999999999996,"15min":0.64400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.7679999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":2.1110000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":1.51,"15min":0.86399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.327,"5min":38.762,"15min":38.762},"last":0.44800000000000001},{"interface":"front","average":{"1min":3.7050000000000001,"5min":1.788,"15min":0.96599999999999997},"min":{"1min":0.379,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":24.242999999999999,"5min":35.612000000000002,"15min":35.612000000000002},"last":3.411}]},{"osd":4,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44800000000000001,"15min":0.43099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.375}]},{"osd":5,"last update":"Thu Feb 22 19:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48899999999999999,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48999999999999999,"15min":0.51000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.48999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1617884,"kb_used_data":1308244,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92127268,"statfs":{"total":95995035648,"available":94338322432,"internally_reserved":0,"allocated":1339641856,"data_stored":1339222728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":4.0599999999999996,"5min":1.4630000000000001,"15min":0.71299999999999997},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":3.6800000000000002},{"interface":"front","average":{"1min":3.5409999999999999,"5min":1.5509999999999999,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":9.5,"5min":11.352,"15min":11.352},"last":1.357}]},{"osd":2,"last update":"Thu Feb 22 19:34:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1959999999999997,"5min":2.073,"15min":0.95599999999999996},"min":{"1min":0.224,"5min":0.222,"15min":0.222},"max":{"1min":16.503,"5min":16.503,"15min":16.503},"last":3.6459999999999999},{"interface":"front","average":{"1min":2.4380000000000002,"5min":1.732,"15min":0.84499999999999997},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":11.715,"5min":13.461,"15min":13.461},"last":1.762}]},{"osd":3,"last update":"Thu Feb 22 19:33:34 2024","interfaces":[{"interface":"back","average":{"1min":5.0599999999999996,"5min":1.9970000000000001,"15min":0.97699999999999998},"min":{"1min":0.24399999999999999,"5min":0.223,"15min":0.223},"max":{"1min":11.932,"5min":25.145,"15min":25.145},"last":1.218},{"interface":"front","average":{"1min":3.8420000000000001,"5min":1.8959999999999999,"15min":0.97799999999999998},"min":{"1min":0.40300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":11.625,"5min":22.25,"15min":22.25},"last":3.589}]},{"osd":4,"last update":"Thu Feb 22 19:33:53 2024","interfaces":[{"interface":"back","average":{"1min":2.4260000000000002,"5min":1.319,"15min":0.85499999999999998},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.9580000000000002,"5min":15.542999999999999,"15min":15.542999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":2.1890000000000001,"5min":0.996,"15min":0.71699999999999997},"min":{"1min":0.314,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:33:09 2024","interfaces":[{"interface":"back","average":{"1min":3.1429999999999998,"5min":1.1819999999999999,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":15.587999999999999,"5min":15.587999999999999,"15min":15.587999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":2.694,"5min":1.0149999999999999,"15min":0.72299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.5839999999999996,"5min":7.5839999999999996,"15min":7.5839999999999996},"last":1.4590000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:40 2024","interfaces":[{"interface":"back","average":{"1min":2.1549999999999998,"5min":0.99299999999999999,"15min":0.69299999999999995},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":1.129},{"interface":"front","average":{"1min":2.4199999999999999,"5min":1.333,"15min":0.83899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.032,"5min":15.651999999999999,"15min":15.651999999999999},"last":0.373}]},{"osd":7,"last update":"Thu Feb 22 19:34:04 2024","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.2490000000000001,"15min":0.76600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.0820000000000001,"15min":0.72399999999999998},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":6.766,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":1.1950000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1771076,"kb_used_data":1469756,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91974076,"statfs":{"total":95995035648,"available":94181453824,"internally_reserved":0,"allocated":1505030144,"data_stored":1504635498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":3.2610000000000001,"5min":1.262,"15min":0.66000000000000003},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":12.305,"5min":12.305,"15min":12.305},"last":0.17699999999999999},{"interface":"front","average":{"1min":2.6440000000000001,"5min":1.327,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":13.013999999999999,"5min":13.013999999999999,"15min":13.013999999999999},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:33:06 2024","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":1.335,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":19.927,"5min":19.927,"15min":19.927},"last":0.52600000000000002},{"interface":"front","average":{"1min":4.2110000000000003,"5min":1.327,"15min":0.69099999999999995},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":17.907,"5min":17.907,"15min":17.907},"last":3.7250000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:33:27 2024","interfaces":[{"interface":"back","average":{"1min":3.5150000000000001,"5min":1.2989999999999999,"15min":0.72299999999999998},"min":{"1min":0.22900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":13.06,"5min":13.06,"15min":13.06},"last":0.34100000000000003},{"interface":"front","average":{"1min":4.0919999999999996,"5min":1.5349999999999999,"15min":0.81200000000000006},"min":{"1min":0.39000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":13.837999999999999,"5min":22.936,"15min":22.936},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":0.89500000000000002,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":3.5979999999999999},{"interface":"front","average":{"1min":2.649,"5min":1.423,"15min":0.86799999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":13.874000000000001,"5min":13.874000000000001,"15min":13.874000000000001},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":1.26,"15min":0.79800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":13.903,"5min":13.903,"15min":13.903},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.7270000000000001,"5min":0.81100000000000005,"15min":0.64900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.8920000000000003,"5min":6.8920000000000003,"15min":6.8920000000000003},"last":3.665}]},{"osd":6,"last update":"Thu Feb 22 19:33:33 2024","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":0.90700000000000003,"15min":0.69799999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.984,"5min":6.984,"15min":6.984},"last":3.637},{"interface":"front","average":{"1min":2.0569999999999999,"5min":1.038,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:34:03 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.024,"15min":0.72399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":6.9199999999999999,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":3.6960000000000002},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.137,"15min":0.77200000000000002},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":7.2720000000000002,"5min":7.2720000000000002,"15min":7.2720000000000002},"last":0.41199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1769400,"kb_used_data":1468080,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91975752,"statfs":{"total":95995035648,"available":94183170048,"internally_reserved":0,"allocated":1503313920,"data_stored":1502930484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":1.1719999999999999,"15min":0.63600000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":11.898999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":3.6859999999999999,"5min":1.4239999999999999,"15min":0.71599999999999997},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":16.138999999999999,"5min":18.059999999999999,"15min":18.059999999999999},"last":3.004}]},{"osd":1,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.6789999999999998,"5min":0.96199999999999997,"15min":0.57199999999999995},"min":{"1min":0.21199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.183,"5min":10.183,"15min":10.183},"last":2.3740000000000001},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.73799999999999999,"15min":0.51600000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":9.7189999999999994,"5min":9.7189999999999994,"15min":9.7189999999999994},"last":1.75}]},{"osd":3,"last update":"Thu Feb 22 19:33:32 2024","interfaces":[{"interface":"back","average":{"1min":4.016,"5min":1.6559999999999999,"15min":0.86099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":30.071000000000002,"5min":30.071000000000002,"15min":30.071000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":5.2720000000000002,"5min":1.9850000000000001,"15min":0.95999999999999996},"min":{"1min":0.39900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":32.119,"5min":32.119,"15min":32.119},"last":1.3560000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:53 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.1319999999999999,"15min":0.75800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.226,"5min":6.226,"15min":6.226},"last":2.2240000000000002},{"interface":"front","average":{"1min":1.119,"5min":0.78100000000000003,"15min":0.65300000000000002},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.3860000000000001,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.27000000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:33:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":0.747,"15min":0.66500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.36599999999999999},{"interface":"front","average":{"1min":2.1669999999999998,"5min":0.97999999999999998,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0739999999999998,"5min":6.0739999999999998,"15min":6.0739999999999998},"last":2.3029999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:33:39 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.1579999999999999,"15min":0.78800000000000003},"min":{"1min":0.375,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":9.9689999999999994,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":2.262},{"interface":"front","average":{"1min":1.1339999999999999,"5min":0.81200000000000006,"15min":0.69099999999999995},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.4100000000000001,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":0.312}]},{"osd":7,"last update":"Thu Feb 22 19:33:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":0.94799999999999995,"15min":0.70199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.1029999999999998,"5min":6.1029999999999998,"15min":6.1029999999999998},"last":2.3370000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.73699999999999999,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":9.5660000000000007,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.34799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2152076,"kb_used_data":1850500,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91593076,"statfs":{"total":95995035648,"available":93791309824,"internally_reserved":0,"allocated":1894912000,"data_stored":1894509494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1909999999999998,"5min":1.5309999999999999,"15min":0.80400000000000005},"min":{"1min":0.20899999999999999,"5min":0.193,"15min":0.193},"max":{"1min":12.929,"5min":12.929,"15min":12.929},"last":0.26200000000000001},{"interface":"front","average":{"1min":4.774,"5min":1.7390000000000001,"15min":0.90500000000000003},"min":{"1min":0.36099999999999999,"5min":0.247,"15min":0.247},"max":{"1min":14.831,"5min":14.831,"15min":14.831},"last":13.733000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1600000000000001,"5min":1.728,"15min":0.89000000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":17.009,"5min":17.009,"15min":17.009},"last":18.259},{"interface":"front","average":{"1min":2.843,"5min":1.3380000000000001,"15min":0.752},"min":{"1min":0.38600000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":7.7619999999999996,"5min":10.279,"15min":10.279},"last":13.154}]},{"osd":2,"last update":"Thu Feb 22 19:33:29 2024","interfaces":[{"interface":"back","average":{"1min":4.1660000000000004,"5min":1.7090000000000001,"15min":0.89800000000000002},"min":{"1min":0.34499999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":10.714,"5min":10.714,"15min":10.714},"last":16.271000000000001},{"interface":"front","average":{"1min":4.2519999999999998,"5min":1.712,"15min":0.88900000000000001},"min":{"1min":0.46100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:33:50 2024","interfaces":[{"interface":"back","average":{"1min":2.7719999999999998,"5min":1.4199999999999999,"15min":0.89000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.699,"5min":12.699,"15min":12.699},"last":13.680999999999999},{"interface":"front","average":{"1min":2.4489999999999998,"5min":1.325,"15min":0.84599999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":11.366,"5min":11.366,"15min":11.366},"last":9.8940000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:33:15 2024","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":1.125,"15min":0.76800000000000002},"min":{"1min":0.437,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.8550000000000004,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.497},{"interface":"front","average":{"1min":3.3239999999999998,"5min":1.2689999999999999,"15min":0.81100000000000005},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":12.744999999999999,"5min":12.744999999999999,"15min":12.744999999999999},"last":13.707000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:42 2024","interfaces":[{"interface":"back","average":{"1min":3.21,"5min":1.363,"15min":0.84599999999999997},"min":{"1min":0.496,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.898999999999999,"5min":12.898999999999999,"15min":12.898999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":3.036,"5min":1.4590000000000001,"15min":0.92000000000000004},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":12.776999999999999,"5min":12.776999999999999,"15min":12.776999999999999},"last":13.637}]},{"osd":7,"last update":"Thu Feb 22 19:33:56 2024","interfaces":[{"interface":"back","average":{"1min":1.897,"5min":1.3420000000000001,"15min":0.83099999999999996},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":11.348000000000001,"5min":11.348000000000001,"15min":11.348000000000001},"last":9.9250000000000007},{"interface":"front","average":{"1min":2.7709999999999999,"5min":1.454,"15min":0.89000000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":12.846,"5min":12.846,"15min":12.846},"last":0.45400000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1228104,"kb_used_data":930880,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":92517048,"statfs":{"total":95995035648,"available":94737457152,"internally_reserved":0,"allocated":953221120,"data_stored":952822490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.4830000000000001,"5min":1.2749999999999999,"15min":0.76900000000000002},"min":{"1min":0.313,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.635999999999999,"5min":24.009,"15min":24.009},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.8480000000000001,"5min":1.2450000000000001,"15min":0.77900000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.7229999999999999,"5min":24.263000000000002,"15min":24.263000000000002},"last":4.3300000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9810000000000001,"5min":0.96699999999999997,"15min":0.68899999999999995},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":6.3029999999999999,"5min":6.3029999999999999,"15min":6.3029999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":0.80700000000000005,"15min":0.60399999999999998},"min":{"1min":0.28000000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":12.445,"5min":12.445,"15min":12.445},"last":1.8380000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.107,"15min":0.75},"min":{"1min":0.38,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.8839999999999999,"5min":23.082000000000001,"15min":23.082000000000001},"last":1.772},{"interface":"front","average":{"1min":2.3450000000000002,"5min":1.3200000000000001,"15min":0.80100000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.992,"5min":23.138999999999999,"15min":23.138999999999999},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.9199999999999999,"5min":1.2230000000000001,"15min":0.77700000000000002},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":1.0509999999999999},{"interface":"front","average":{"1min":2.6400000000000001,"5min":1.157,"15min":0.73899999999999999},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":8.9909999999999997,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.5489999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.38800000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:33:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64800000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.498,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55800000000000005,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.35499999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.47699999999999998}]}]},{"osd":5,"up_from":0,"seq":167503724618,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1494159360,"data_stored":1494155288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1325600768,"data_stored":1325596696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1491533824,"data_stored":1491533824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1875312640,"data_stored":1875312640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":947912704,"data_stored":947912704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73269248,"data_stored":73269248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":954011648,"data_stored":954007576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":935985152,"data_stored":935985152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:34:10.332 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set device_health_metrics pgp_num 1 2024-02-22T19:34:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:10.355 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:10.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:10.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:34:10.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:10.487 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:10.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:10.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:10.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:10.668 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:10.671 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:10.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:10.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:10.912 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:10.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:10.921 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:08.916829+0000 osd.3 (osd.3) 71 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:10.921 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:08.916884+0000 osd.3 (osd.3) 72 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:10.921 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:08.943271+0000 mgr.y (mgr.14535) 203 : audit [DBG] from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:10.921 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:09.322945+0000 mgr.y (mgr.14535) 204 : cluster [DBG] pgmap v220: 148 pgs: 148 active+clean; 2.6 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:34:10.921 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:09.806206+0000 mon.a (mon.0) 891 : cluster [DBG] osdmap e140: 8 total, 8 up, 7 in 2024-02-22T19:34:10.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:10.922 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:08.916829+0000 osd.3 (osd.3) 71 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:10.922 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:08.916884+0000 osd.3 (osd.3) 72 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:10.922 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:08.943271+0000 mgr.y (mgr.14535) 203 : audit [DBG] from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:10.922 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:09.322945+0000 mgr.y (mgr.14535) 204 : cluster [DBG] pgmap v220: 148 pgs: 148 active+clean; 2.6 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:34:10.922 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:09.806206+0000 mon.a (mon.0) 891 : cluster [DBG] osdmap e140: 8 total, 8 up, 7 in 2024-02-22T19:34:11.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:11.075 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:11.077 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:08.916829+0000 osd.3 (osd.3) 71 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:11.077 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:08.916884+0000 osd.3 (osd.3) 72 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:11.077 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:08.943271+0000 mgr.y (mgr.14535) 203 : audit [DBG] from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:11.077 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:09.322945+0000 mgr.y (mgr.14535) 204 : cluster [DBG] pgmap v220: 148 pgs: 148 active+clean; 2.6 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:34:11.077 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:09.806206+0000 mon.a (mon.0) 891 : cluster [DBG] osdmap e140: 8 total, 8 up, 7 in 2024-02-22T19:34:11.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:11.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:11.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:11.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:34:11.234 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:11.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:11.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:11.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:11.369 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:11.392 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:11.394 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:11.395 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:11.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:11.551 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:11.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:11.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:11.683 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:11.713 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:11.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:11.784 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:11.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:11.863 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:12.623 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:34:12.623 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:34:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:11.325388+0000 mgr.y (mgr.14535) 205 : cluster [DBG] pgmap v222: 148 pgs: 148 active+clean; 2.5 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 775 B/s rd, 0 B/s wr, 684 op/s 2024-02-22T19:34:13.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:11.325388+0000 mgr.y (mgr.14535) 205 : cluster [DBG] pgmap v222: 148 pgs: 148 active+clean; 2.5 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 775 B/s rd, 0 B/s wr, 684 op/s 2024-02-22T19:34:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:11.325388+0000 mgr.y (mgr.14535) 205 : cluster [DBG] pgmap v222: 148 pgs: 148 active+clean; 2.5 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 775 B/s rd, 0 B/s wr, 684 op/s 2024-02-22T19:34:13.544 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:34:14.006 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:14.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:14.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:14.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:14.140 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:13.168892+0000 mon.a (mon.0) 892 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:34:14.140 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:13.629574+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:14.142 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:14.166 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:14.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:14.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:14.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:13.168892+0000 mon.a (mon.0) 892 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:34:14.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:13.629574+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:14.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:13.168892+0000 mon.a (mon.0) 892 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:34:14.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:13.629574+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:14.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:14.300 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:14.328 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:14.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:14.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:14.463 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:14.487 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:14.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:14.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:14.591 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:14.667 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:14.713 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:14.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:14.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:14.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:14.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:34:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:34:14.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:34:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:34:14.852 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:14.876 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:14.878 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:14.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:14.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:15.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:13.325711+0000 mgr.y (mgr.14535) 206 : cluster [DBG] pgmap v223: 148 pgs: 148 active+clean; 2.5 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 564 op/s 2024-02-22T19:34:15.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:14.227678+0000 mon.a (mon.0) 894 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:34:15.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:14.237872+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:34:15.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:14.246266+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:15.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:13.325711+0000 mgr.y (mgr.14535) 206 : cluster [DBG] pgmap v223: 148 pgs: 148 active+clean; 2.5 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 564 op/s 2024-02-22T19:34:15.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:14.227678+0000 mon.a (mon.0) 894 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:34:15.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:14.237872+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:34:15.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:14.246266+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:15.038 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:13.325711+0000 mgr.y (mgr.14535) 206 : cluster [DBG] pgmap v223: 148 pgs: 148 active+clean; 2.5 GiB data, 9.7 GiB used, 616 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 564 op/s 2024-02-22T19:34:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:14.227678+0000 mon.a (mon.0) 894 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:34:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:14.237872+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:34:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:14.246266+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:16.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:14.496262+0000 mgr.y (mgr.14535) 207 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:16.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:14.496262+0000 mgr.y (mgr.14535) 207 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:14.496262+0000 mgr.y (mgr.14535) 207 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:15.326379+0000 mgr.y (mgr.14535) 208 : cluster [DBG] pgmap v224: 148 pgs: 148 active+clean; 2.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:34:17.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:15.326379+0000 mgr.y (mgr.14535) 208 : cluster [DBG] pgmap v224: 148 pgs: 148 active+clean; 2.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:34:17.347 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:15.326379+0000 mgr.y (mgr.14535) 208 : cluster [DBG] pgmap v224: 148 pgs: 148 active+clean; 2.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:34:17.352 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:17.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:17.453 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:17.483 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:17.507 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:17.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:17.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:17.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:17.678 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:18.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:17.327955+0000 mgr.y (mgr.14535) 209 : cluster [DBG] pgmap v225: 148 pgs: 148 active+clean; 2.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 890 op/s 2024-02-22T19:34:18.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:17.327955+0000 mgr.y (mgr.14535) 209 : cluster [DBG] pgmap v225: 148 pgs: 148 active+clean; 2.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 890 op/s 2024-02-22T19:34:18.315 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:17.327955+0000 mgr.y (mgr.14535) 209 : cluster [DBG] pgmap v225: 148 pgs: 148 active+clean; 2.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 890 op/s 2024-02-22T19:34:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:18.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:18.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:18.687 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:18.787 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:18.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:18.797 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:18.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:18.956 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:18.959 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:18.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:19.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:19.112 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:19.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:19.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:19.130 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:34:19.131 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 1 pgp_num to 1 2024-02-22T19:34:19.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:18.515068+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.53:0/1299336077' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]: dispatch 2024-02-22T19:34:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:18.515856+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]: dispatch 2024-02-22T19:34:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:18.692381+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.53:0/2417927739' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:34:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:18.515068+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.53:0/1299336077' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]: dispatch 2024-02-22T19:34:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:18.515856+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]: dispatch 2024-02-22T19:34:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:18.692381+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.53:0/2417927739' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:34:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:18.515068+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.53:0/1299336077' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]: dispatch 2024-02-22T19:34:19.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:18.515856+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]: dispatch 2024-02-22T19:34:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:18.692381+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.53:0/2417927739' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:34:19.507 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:34:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:19] "GET /metrics HTTP/1.1" 200 175579 "" "Prometheus/2.33.4" 2024-02-22T19:34:20.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:19.085977+0000 mon.a (mon.0) 899 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:20.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:19.095169+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]': finished 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:19.095241+0000 mon.a (mon.0) 901 : audit [INF] from='client.? 172.21.15.53:0/2417927739' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:19.095275+0000 mon.a (mon.0) 902 : cluster [DBG] osdmap e141: 8 total, 8 up, 7 in 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:19.328737+0000 mgr.y (mgr.14535) 210 : cluster [DBG] pgmap v227: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 948 op/s 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:19.085977+0000 mon.a (mon.0) 899 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:19.095169+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]': finished 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:19.095241+0000 mon.a (mon.0) 901 : audit [INF] from='client.? 172.21.15.53:0/2417927739' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:19.095275+0000 mon.a (mon.0) 902 : cluster [DBG] osdmap e141: 8 total, 8 up, 7 in 2024-02-22T19:34:20.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:19.328737+0000 mgr.y (mgr.14535) 210 : cluster [DBG] pgmap v227: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 948 op/s 2024-02-22T19:34:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:20.386 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:20.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:20.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:20.555 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:19.085977+0000 mon.a (mon.0) 899 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:19.095169+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "1"}]': finished 2024-02-22T19:34:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:19.095241+0000 mon.a (mon.0) 901 : audit [INF] from='client.? 172.21.15.53:0/2417927739' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:34:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:19.095275+0000 mon.a (mon.0) 902 : cluster [DBG] osdmap e141: 8 total, 8 up, 7 in 2024-02-22T19:34:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:19.328737+0000 mgr.y (mgr.14535) 210 : cluster [DBG] pgmap v227: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 948 op/s 2024-02-22T19:34:20.931 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:20.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:20.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:21.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:21.069 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:21.100 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:21.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:21.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:21.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:21.261 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:21.360 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:21.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:21.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:21.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:21.547 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:21.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:21.667 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:21.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:21.949 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:21.955 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:21.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:22.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:22.187 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:22.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:22.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:22.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:22.379 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:22.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:21.330311+0000 mgr.y (mgr.14535) 211 : cluster [DBG] pgmap v228: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 902 op/s 2024-02-22T19:34:22.512 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:22.521 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:22.522 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:22.542 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:34:22.585 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:34:22.610 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:34:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:34:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:689 5.3 5:c609908c:::notify.5:head [watch ping cookie 94129597101696] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44300397000001, 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86272942500000005, 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 689 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154333+0000", 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4004e-05 2024-02-22T19:34:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154347+0000", 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3070000000000001e-05 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154370+0000", 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2280000000000004e-06 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154379+0000", 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108694 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154488+0000", 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3376e-05 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154501+0000", 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85773138400000004 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012232+0000", 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055974900000000001 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012792+0000", 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040998229999999998 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.016892+0000", 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6339000000000002e-05 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.016968+0000", 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6751999999999998e-05 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017065+0000", 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86272942500000005 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:698 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94129597101696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.442608283, 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87369409200000003, 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:22.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 698 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4479999999999996e-06 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154735+0000", 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0008e-05 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154745+0000", 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6600000000000003e-06 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154750+0000", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7904000000000001e-05 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154798+0000", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1430000000000001e-05 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154809+0000", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85813308200000005 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012942+0000", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0219000000000001e-05 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013002+0000", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004174891 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017177+0000", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7285000000000004e-05 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017245+0000", 2024-02-22T19:34:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011180936000000001 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.028426+0000", 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87369409200000003 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:72083 7.6 7:65dd918c:::benchmark_data_smithi053_107660_object72082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:46.985753+0000", 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.611586404999997, 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.058852559999999998, 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 72083 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.985753+0000", 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.985753+0000", 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.985753+0000", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6325999999999999e-05 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.985769+0000", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9439999999999998e-06 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.985773+0000", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9390000000000002e-06 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.985776+0000", 2024-02-22T19:34:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051700200000000002 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.986293+0000", 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151722 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.986445+0000", 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151154 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.986596+0000", 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004531953 2024-02-22T19:34:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.991128+0000", 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016048340000000001 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:46.992733+0000", 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051764983000000001 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.044498+0000", 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9271000000000002e-05 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.044557+0000", 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8977000000000002e-05 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:47.044606+0000", 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.058852559999999998 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:91264 7.4 7:283d5bf0:::benchmark_data_smithi053_107660_object2057:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:13.331142+0000", 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.266197598000005, 2024-02-22T19:34:22.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057630265, 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 91264 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.331142+0000", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.331142+0000", 2024-02-22T19:34:22.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.331141+0000", 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3519999999999997e-06 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.331146+0000", 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8689999999999993e-06 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.331155+0000", 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3650000000000003e-06 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.331161+0000", 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0084589030000000003 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.339620+0000", 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011618800000000001 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.339736+0000", 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000130331 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.339866+0000", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002197849 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.342064+0000", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025265460000000002 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.344590+0000", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044115433000000003 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.388706+0000", 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4975999999999999e-05 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.388741+0000", 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1491000000000001e-05 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.388772+0000", 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057630265 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:91718 7.f 7:f927a21f:::benchmark_data_smithi053_107660_object2511:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:13.888172+0000", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 68.709167588, 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056837049000000001, 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 91718 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888172+0000", 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888172+0000", 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888171+0000", 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6910000000000002e-06 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888177+0000", 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4239999999999999e-06 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888186+0000", 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888192+0000", 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002946368 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.891138+0000", 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102289 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.891240+0000", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115955 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.891356+0000", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047133201 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.938490+0000", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028258290000000002 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.941315+0000", 2024-02-22T19:34:22.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003613439 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.944929+0000", 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4098999999999999e-05 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.944973+0000", 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6211999999999998e-05 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945009+0000", 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056837049000000001 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:91721 7.f 7:ff90f984:::benchmark_data_smithi053_107660_object2514:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:13.888218+0000", 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 68.709121898999996, 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056906680000000001, 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 91721 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888218+0000", 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888218+0000", 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888217+0000", 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3769999999999999e-06 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888220+0000", 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5279999999999997e-06 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888224+0000", 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8e-06 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888228+0000", 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055394270000000004 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.893768+0000", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7385000000000004e-05 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.893865+0000", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011702 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.893982+0000", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044537234000000002 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.938519+0000", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028014450000000001 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.941321+0000", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037199210000000002 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945041+0000", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9789000000000001e-05 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945080+0000", 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4172000000000003e-05 2024-02-22T19:34:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945124+0000", 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056906680000000001 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:91722 7.f 7:f9004fe7:::benchmark_data_smithi053_107660_object2515:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:13.888232+0000", 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 68.709107932999999, 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056997736, 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 91722 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888232+0000", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888232+0000", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888231+0000", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1229999999999998e-06 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888234+0000", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.189e-06 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888238+0000", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4079999999999998e-06 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.888241+0000", 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044761787999999997 2024-02-22T19:34:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.933003+0000", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011395699999999999 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.933117+0000", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127575 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.933244+0000", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053050160000000001 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.938549+0000", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027761259999999999 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.941326+0000", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038296939999999998 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945155+0000", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4245000000000002e-05 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945190+0000", 2024-02-22T19:34:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9941999999999998e-05 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:13.945229+0000", 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056997736 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92964 7.f 7:f76c743f:::benchmark_data_smithi053_107660_object3757:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:15.367040+0000", 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.230299250000002, 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056979265000000001, 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92964 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.367040+0000", 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.367040+0000", 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.367040+0000", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.367043+0000", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2020000000000003e-06 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.367048+0000", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0890000000000002e-06 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.367052+0000", 2024-02-22T19:34:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010886264 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377939+0000", 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6806999999999999e-05 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.378005+0000", 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010012499999999999 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.378106+0000", 2024-02-22T19:34:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041005781999999998 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.419111+0000", 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000206357 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.419318+0000", 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046287189999999999 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.423946+0000", 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8322999999999999e-05 2024-02-22T19:34:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.423985+0000", 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4949e-05 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.424020+0000", 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056979265000000001 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92974 7.6 7:63101ee7:::benchmark_data_smithi053_107660_object3767:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:15.377841+0000", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.219499073999998, 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057954138000000002, 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92974 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377841+0000", 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377841+0000", 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377839+0000", 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8820000000000003e-06 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377846+0000", 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8689999999999999e-06 2024-02-22T19:34:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377856+0000", 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000001e-06 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.377862+0000", 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013521539999999999 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.379214+0000", 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9178000000000005e-05 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.379313+0000", 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.039615943000000001 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.418929+0000", 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013876078999999999 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.432805+0000", 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6028999999999997e-05 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.432841+0000", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028078040000000001 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.435649+0000", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.435751+0000", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4122000000000002e-05 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:15.435795+0000", 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057954138000000002 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:93481 7.6 7:684ba2e0:::benchmark_data_smithi053_107660_object4274:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:16.014655+0000", 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.582684861000004, 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.058183815, 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 93481 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014655+0000", 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014655+0000", 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014654+0000", 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8589999999999998e-06 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014658+0000", 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7429999999999996e-06 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014665+0000", 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.434e-06 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014670+0000", 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9242000000000001e-05 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014719+0000", 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012811699999999999 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014848+0000", 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134611 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.014982+0000", 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048760879999999998 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.019858+0000", 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.043359230999999998 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.063218+0000", 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095389380000000003 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.072756+0000", 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5901e-05 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.072802+0000", 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.629e-05 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:16.072839+0000", 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.058183815 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:107717 7.f 7:f48dbd5e:::benchmark_data_smithi053_107660_object18510:head [delete] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:35.078342+0000", 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.518997814999999, 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056000239, 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 107717 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.078342+0000", 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.078342+0000", 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.078342+0000", 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1490000000000001e-06 2024-02-22T19:34:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.078344+0000", 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0180000000000003e-06 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.078348+0000", 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8179999999999997e-06 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.078352+0000", 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061174280000000003 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.084469+0000", 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013455699999999999 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.084603+0000", 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015179399999999999 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.084755+0000", 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0056764850000000002 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.090432+0000", 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039876700000000001 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.090831+0000", 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.043428432000000003 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.134259+0000", 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8535000000000002e-05 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.134308+0000", 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4610999999999998e-05 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:35.134342+0000", 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056000239 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:114121 7.6 7:6539d17f:::benchmark_data_smithi053_107660_object24914:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:34:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150187+0000", 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.447152424000002, 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064960559000000001, 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114121 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150187+0000", 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150187+0000", 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150185+0000", 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3499999999999999e-06 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150193+0000", 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150201+0000", 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1109999999999996e-06 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150208+0000", 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0375000000000003e-05 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150278+0000", 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109256 2024-02-22T19:34:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150387+0000", 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012612100000000001 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150513+0000", 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064495902999999993 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215009+0000", 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9539e-05 2024-02-22T19:34:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215039+0000", 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4793999999999999e-05 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215053+0000", 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7370000000000001e-05 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215111+0000", 2024-02-22T19:34:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6962000000000003e-05 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215148+0000", 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064960559000000001 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:114127 7.4 7:28c1c1b6:::benchmark_data_smithi053_107660_object24920:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150220+0000", 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.447119868000001, 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065748500000000001, 2024-02-22T19:34:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114127 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150220+0000", 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150220+0000", 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150219+0000", 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9629999999999998e-06 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150222+0000", 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9739999999999999e-06 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150228+0000", 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0260000000000001e-06 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150232+0000", 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8846e-05 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150281+0000", 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112027 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150393+0000", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124556 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150518+0000", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051992050000000001 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155717+0000", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059667896999999998 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215385+0000", 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050588300000000005 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215891+0000", 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2429999999999999e-05 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215933+0000", 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5162e-05 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215968+0000", 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065748500000000001 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114120 7.e e135/132 7:74debd88:::benchmark_data_smithi053_107660_object24913:head v 135'6933, mlcod=135'6931)", 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150481+0000", 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.446858546000001, 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064697999000000006, 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150481+0000", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150481+0000", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150480+0000", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9550000000000003e-06 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150487+0000", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5440000000000004e-06 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150497+0000", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150503+0000", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002534798 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.153038+0000", 2024-02-22T19:34:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8559999999999998e-05 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.153056+0000", 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062094587999999999 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215151+0000", 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8353000000000001e-05 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215179+0000", 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064697999000000006 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114130 7.d e135/93 7:be62d22d:::benchmark_data_smithi053_107660_object24923:head v 135'7356, mlcod=135'7354)", 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.155746+0000", 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.441593867999998, 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064770929000000005, 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155746+0000", 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155746+0000", 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155745+0000", 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9790000000000005e-06 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155755+0000", 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3139999999999993e-06 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155764+0000", 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155770+0000", 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1980999999999998e-05 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155802+0000", 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9924000000000001e-05 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155822+0000", 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064662440000000002 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220484+0000", 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2431000000000002e-05 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220517+0000", 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064770929000000005 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114124 7.7 e135/93 7:e882f8ed:::benchmark_data_smithi053_107660_object24917:head v 135'7086, mlcod=135'7084)", 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.155781+0000", 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.441558794999999, 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064791314000000003, 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155781+0000", 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155781+0000", 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155780+0000", 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3529999999999997e-06 2024-02-22T19:34:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155785+0000", 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1499999999999998e-06 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155790+0000", 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3279999999999999e-06 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155794+0000", 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847999999999998e-05 2024-02-22T19:34:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155833+0000", 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9712999999999999e-05 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155853+0000", 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064696482999999999 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220549+0000", 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2838999999999999e-05 2024-02-22T19:34:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220572+0000", 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064791314000000003 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114129 7.5 e135/93 7:a0f0a8d6:::benchmark_data_smithi053_107660_object24922:head v 135'7146, mlcod=135'7144)", 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.155812+0000", 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.441528073000001, 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064726803999999999, 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155812+0000", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155812+0000", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155811+0000", 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1060000000000002e-06 2024-02-22T19:34:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155815+0000", 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9440000000000005e-06 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155823+0000", 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.794e-06 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155827+0000", 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059616852999999997 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215444+0000", 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9570999999999999e-05 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215464+0000", 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050555859999999999 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220519+0000", 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9337000000000001e-05 2024-02-22T19:34:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220538+0000", 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064726803999999999 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.24670.0:114127 7.4 e135/93 ondisk, result = 0)", 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.158253+0000", 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.439086205000002, 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057718345999999997, 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158253+0000", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158253+0000", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158252+0000", 2024-02-22T19:34:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1167e-05 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158263+0000", 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.055639855000000002 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213903+0000", 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0226e-05 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213913+0000", 2024-02-22T19:34:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001963935 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215877+0000", 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4510000000000003e-06 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215884+0000", 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7468999999999999e-05 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215972+0000", 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057718345999999997 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.24670.0:114121 7.6 e135/93 ondisk, result = 0)", 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.158348+0000", 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.438991891000001, 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056692156, 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158348+0000", 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158348+0000", 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158347+0000", 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2049999999999996e-06 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158354+0000", 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.055567874000000003 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213922+0000", 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5079999999999999e-06 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213927+0000", 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011017240000000001 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215029+0000", 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2789999999999998e-06 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215034+0000", 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1670000000000004e-06 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215040+0000", 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056692156 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.24670.0:114121 7.6 e135/93 ondisk, result = 0)", 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.158647+0000", 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.438693035999997, 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056364886000000003, 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158647+0000", 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158647+0000", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158646+0000", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3319999999999997e-06 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158652+0000", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054808524999999997 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213461+0000", 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2703e-05 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213473+0000", 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001519123 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.214993+0000", 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5920000000000001e-06 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215001+0000", 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.039e-05 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215012+0000", 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056364886000000003 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:22.708 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:34:22.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:22.710 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:34:22.710 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:34:22.710 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:34:22.726 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:34:22.735 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:21.330311+0000 mgr.y (mgr.14535) 211 : cluster [DBG] pgmap v228: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 902 op/s 2024-02-22T19:34:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:22.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:21.330311+0000 mgr.y (mgr.14535) 211 : cluster [DBG] pgmap v228: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 902 op/s 2024-02-22T19:34:22.760 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:22.763 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:22.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:22.773 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:34:22.802 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:34:22.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.809 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.809 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:34:22.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:22.942 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:23.162 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:23.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:23.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:23.414 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:22.498594+0000 mon.b (mon.1) 43 : audit [DBG] from='client.? 172.21.15.53:0/1423238791' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:23.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:22.498594+0000 mon.b (mon.1) 43 : audit [DBG] from='client.? 172.21.15.53:0/1423238791' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:23.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:34:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:22.498594+0000 mon.b (mon.1) 43 : audit [DBG] from='client.? 172.21.15.53:0/1423238791' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:24.096 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:24.106 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:24.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:24.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:34:24.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:24.221 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:24.309 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:24.436 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:24.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:24.440 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:24.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:24.569 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:24.590 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:24.594 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:24.594 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:24.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:24.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:23.332194+0000 mgr.y (mgr.14535) 212 : cluster [DBG] pgmap v229: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 901 op/s 2024-02-22T19:34:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:23.332194+0000 mgr.y (mgr.14535) 212 : cluster [DBG] pgmap v229: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 901 op/s 2024-02-22T19:34:24.770 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:24.827 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:24.834 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:24.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:24.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:34:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:34:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:23.332194+0000 mgr.y (mgr.14535) 212 : cluster [DBG] pgmap v229: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 901 op/s 2024-02-22T19:34:24.853 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:34:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:34:24.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:24.966 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:24.988 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:24.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:24.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:25.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:25.142 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:25.203 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:25.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:25.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:25.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:25.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:34:25.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:24.506475+0000 mgr.y (mgr.14535) 213 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:25.696 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:24.506475+0000 mgr.y (mgr.14535) 213 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:24.506475+0000 mgr.y (mgr.14535) 213 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:25.867 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:25.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:25.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:25.964 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:34:25.965 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:34:25.965 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-02-22T19:34:25.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-22T19:34:25.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.030 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:26.034 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.034 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:26.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:26.191 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:26.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:26.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:26.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:26.481 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.504 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:26.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:25.334080+0000 mgr.y (mgr.14535) 214 : cluster [DBG] pgmap v230: 148 pgs: 148 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 921 op/s 2024-02-22T19:34:26.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:26.636 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.658 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:26.661 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:26.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:25.334080+0000 mgr.y (mgr.14535) 214 : cluster [DBG] pgmap v230: 148 pgs: 148 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 921 op/s 2024-02-22T19:34:26.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:25.334080+0000 mgr.y (mgr.14535) 214 : cluster [DBG] pgmap v230: 148 pgs: 148 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 921 op/s 2024-02-22T19:34:26.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:26.792 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:26.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:26.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:26.949 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:26.973 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:26.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:26.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:27.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:27.107 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:27.134 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:27.138 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:27.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:27.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:27.311 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:27.874 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:27.886 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:27.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:27.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:28.018 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:28.039 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:28.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:28.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:28.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:27.124398+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.53:0/2497165020' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:34:28.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:27.124981+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:34:28.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:27.334499+0000 mgr.y (mgr.14535) 215 : cluster [DBG] pgmap v231: 148 pgs: 148 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 921 op/s 2024-02-22T19:34:28.098 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:27.124398+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.53:0/2497165020' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:34:28.098 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:27.124981+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:34:28.098 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:27.334499+0000 mgr.y (mgr.14535) 215 : cluster [DBG] pgmap v231: 148 pgs: 148 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 921 op/s 2024-02-22T19:34:28.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:28.148 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:34:28.214 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:27.124398+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.53:0/2497165020' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:34:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:27.124981+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:34:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:27.334499+0000 mgr.y (mgr.14535) 215 : cluster [DBG] pgmap v231: 148 pgs: 148 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 921 op/s 2024-02-22T19:34:28.391 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:28.438 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:28.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:28.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:28.600 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:28.975 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:34:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:34:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:154724 7.0 7:003b210e:::benchmark_data_smithi053_107660_object65517:head [delete] snapc 0=[] ondisk+write+known_if_redirected e142)", 2024-02-22T19:34:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.959608+0000", 2024-02-22T19:34:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.012113212, 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012136456, 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 154724 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.959608+0000", 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.959608+0000", 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.959606+0000", 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0000000000000002e-06 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.959612+0000", 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0154e-05 2024-02-22T19:34:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.959623+0000", 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0390000000000001e-06 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.959629+0000", 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037334199999999998 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.960002+0000", 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011418599999999999 2024-02-22T19:34:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.960116+0000", 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012719700000000001 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.960243+0000", 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002443948 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.962687+0000", 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030791999999999998 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.24670.0:154724 7.0 e142/105)", 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.968219+0000", 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0035026929999999999, 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035805020000000002, 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968219+0000", 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968219+0000", 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968218+0000", 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6280000000000002e-06 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968224+0000", 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3597999999999999e-05 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968237+0000", 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968243+0000", 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4878000000000001e-05 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:154731 7.4 e142/93 7:2bb914f2:::benchmark_data_smithi053_107660_object65524:head v 142'9541, mlcod=142'9539)", 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.968606+0000", 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0031154059999999998, 2024-02-22T19:34:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032255600000000001, 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968606+0000", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968606+0000", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968605+0000", 2024-02-22T19:34:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8739999999999997e-06 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968612+0000", 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1194e-05 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968623+0000", 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968629+0000", 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4976999999999999e-05 2024-02-22T19:34:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968684+0000", 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.98e-05 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.968704+0000", 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7942000000000005e-05 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:154729 7.d e142/93)", 2024-02-22T19:34:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.969276+0000", 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0024455129999999999, 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025899759999999999, 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.969276+0000", 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.969276+0000", 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.969275+0000", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6260000000000003e-06 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.969283+0000", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1168e-05 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.969294+0000", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.421e-06 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.969300+0000", 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4343999999999999e-05 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:154734 7.d e142/93)", 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.971065+0000", 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00065610199999999999, 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082888299999999999, 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971065+0000", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971065+0000", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971065+0000", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2080000000000003e-06 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971072+0000", 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5749000000000001e-05 2024-02-22T19:34:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971088+0000", 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7100000000000004e-06 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971093+0000", 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8010000000000001e-05 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:154735 7.4 e142/93 7:22671cf5:::benchmark_data_smithi053_107660_object65528:head v 142'9542, mlcod=142'9539)", 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.971374+0000", 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00034713499999999999, 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054880900000000004, 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971374+0000", 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971374+0000", 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971373+0000", 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1629999999999999e-06 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971381+0000", 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1025e-05 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971392+0000", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971397+0000", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6903000000000001e-05 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971454+0000", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8125999999999999e-05 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.971472+0000", 2024-02-22T19:34:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8195000000000004e-05 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:154715 7.b 7:da8b4c95:::benchmark_data_smithi053_107660_object65508:head [delete] snapc 0=[] ondisk+write+known_if_redirected e142)", 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.953284+0000", 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.018436894999999998, 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018675639000000001, 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 154715 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953284+0000", 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953284+0000", 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953283+0000", 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1899999999999998e-06 2024-02-22T19:34:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953290+0000", 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0915e-05 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953301+0000", 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1020000000000002e-06 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953307+0000", 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028211569999999999 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.956128+0000", 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127062 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.956255+0000", 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015395200000000001 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.956409+0000", 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002455193 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.958865+0000", 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013553840000000001 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.960220+0000", 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069355450000000004 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:154721 7.0 7:0aa80ba9:::benchmark_data_smithi053_107660_object65514:head [delete] snapc 0=[] ondisk+write+known_if_redirected e142)", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:34:28.953321+0000", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.018400439000000001, 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018690121000000001, 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 154721 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953321+0000", 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953321+0000", 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953320+0000", 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8990000000000001e-06 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953323+0000", 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2399999999999998e-06 2024-02-22T19:34:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953329+0000", 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6500000000000002e-06 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953332+0000", 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015248200000000001 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953485+0000", 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012018700000000001 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953605+0000", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000152449 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.953757+0000", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0088212289999999999 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.962579+0000", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052516519999999999 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:34:28.967830+0000", 2024-02-22T19:34:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014509318 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 8 2024-02-22T19:34:29.003 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.090240+0000 mon.a (mon.0) 904 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:28.105518+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.105586+0000 mon.a (mon.0) 906 : cluster [DBG] osdmap e142: 8 total, 8 up, 7 in 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.515349+0000 osd.7 (osd.7) 69 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.519719+0000 osd.7 (osd.7) 70 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.589819+0000 osd.6 (osd.6) 41 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.591722+0000 osd.6 (osd.6) 42 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.090240+0000 mon.a (mon.0) 904 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:28.105518+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:34:29.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.105586+0000 mon.a (mon.0) 906 : cluster [DBG] osdmap e142: 8 total, 8 up, 7 in 2024-02-22T19:34:29.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.515349+0000 osd.7 (osd.7) 69 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:34:29.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.519719+0000 osd.7 (osd.7) 70 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:34:29.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.589819+0000 osd.6 (osd.6) 41 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:34:29.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.591722+0000 osd.6 (osd.6) 42 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:34:29.454 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.090240+0000 mon.a (mon.0) 904 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:28.105518+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.105586+0000 mon.a (mon.0) 906 : cluster [DBG] osdmap e142: 8 total, 8 up, 7 in 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.515349+0000 osd.7 (osd.7) 69 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.519719+0000 osd.7 (osd.7) 70 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.589819+0000 osd.6 (osd.6) 41 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:34:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.591722+0000 osd.6 (osd.6) 42 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:34:29.756 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:34:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:29] "GET /metrics HTTP/1.1" 200 175609 "" "Prometheus/2.33.4" 2024-02-22T19:34:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:29.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:29.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:29.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:29.969 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:30.144 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:30.152 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:30.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:30.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:30.310 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.803695+0000 osd.2 (osd.2) 55 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.806719+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.932623+0000 osd.3 (osd.3) 73 : cluster [DBG] 3.b scrub starts 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:28.943827+0000 osd.3 (osd.3) 74 : cluster [DBG] 3.b scrub ok 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:29.070395+0000 osd.0 (osd.0) 77 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:29.079885+0000 osd.0 (osd.0) 78 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:29.335631+0000 mgr.y (mgr.14535) 216 : cluster [DBG] pgmap v233: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.803695+0000 osd.2 (osd.2) 55 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.806719+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:34:30.485 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.932623+0000 osd.3 (osd.3) 73 : cluster [DBG] 3.b scrub starts 2024-02-22T19:34:30.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:28.943827+0000 osd.3 (osd.3) 74 : cluster [DBG] 3.b scrub ok 2024-02-22T19:34:30.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:29.070395+0000 osd.0 (osd.0) 77 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:34:30.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:29.079885+0000 osd.0 (osd.0) 78 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:34:30.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:29.335631+0000 mgr.y (mgr.14535) 216 : cluster [DBG] pgmap v233: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:34:30.492 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:30.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:30.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:30.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:34:30.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.803695+0000 osd.2 (osd.2) 55 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:34:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.806719+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:34:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.932623+0000 osd.3 (osd.3) 73 : cluster [DBG] 3.b scrub starts 2024-02-22T19:34:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:28.943827+0000 osd.3 (osd.3) 74 : cluster [DBG] 3.b scrub ok 2024-02-22T19:34:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:29.070395+0000 osd.0 (osd.0) 77 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:34:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:29.079885+0000 osd.0 (osd.0) 78 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:34:30.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:29.335631+0000 mgr.y (mgr.14535) 216 : cluster [DBG] pgmap v233: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:34:30.685 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:30.968 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:30.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:30.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:31.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:31.160 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:29.948990+0000 osd.2 (osd.2) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:29.949046+0000 osd.2 (osd.2) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.002787+0000 osd.4 (osd.4) 73 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.007810+0000 osd.4 (osd.4) 74 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.091510+0000 osd.0 (osd.0) 79 : cluster [DBG] 4.b scrub starts 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.097663+0000 osd.0 (osd.0) 80 : cluster [DBG] 4.b scrub ok 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.117506+0000 mon.a (mon.0) 907 : cluster [DBG] osdmap e143: 8 total, 8 up, 7 in 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.496937+0000 osd.7 (osd.7) 71 : cluster [DBG] 5.1a scrub starts 2024-02-22T19:34:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.499861+0000 osd.7 (osd.7) 72 : cluster [DBG] 5.1a scrub ok 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:29.948990+0000 osd.2 (osd.2) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:29.949046+0000 osd.2 (osd.2) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.002787+0000 osd.4 (osd.4) 73 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.007810+0000 osd.4 (osd.4) 74 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.091510+0000 osd.0 (osd.0) 79 : cluster [DBG] 4.b scrub starts 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.097663+0000 osd.0 (osd.0) 80 : cluster [DBG] 4.b scrub ok 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.117506+0000 mon.a (mon.0) 907 : cluster [DBG] osdmap e143: 8 total, 8 up, 7 in 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.496937+0000 osd.7 (osd.7) 71 : cluster [DBG] 5.1a scrub starts 2024-02-22T19:34:31.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.499861+0000 osd.7 (osd.7) 72 : cluster [DBG] 5.1a scrub ok 2024-02-22T19:34:31.450 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:31.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:31.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:31.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:29.948990+0000 osd.2 (osd.2) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:29.949046+0000 osd.2 (osd.2) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.002787+0000 osd.4 (osd.4) 73 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.007810+0000 osd.4 (osd.4) 74 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.091510+0000 osd.0 (osd.0) 79 : cluster [DBG] 4.b scrub starts 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.097663+0000 osd.0 (osd.0) 80 : cluster [DBG] 4.b scrub ok 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.117506+0000 mon.a (mon.0) 907 : cluster [DBG] osdmap e143: 8 total, 8 up, 7 in 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.496937+0000 osd.7 (osd.7) 71 : cluster [DBG] 5.1a scrub starts 2024-02-22T19:34:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.499861+0000 osd.7 (osd.7) 72 : cluster [DBG] 5.1a scrub ok 2024-02-22T19:34:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:31.687 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:31.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:31.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:31.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:31.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:34:31.870 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:32.022 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:32.025 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:32.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:32.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:32.210 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:32.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:32.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:32.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.916205+0000 osd.3 (osd.3) 75 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:34:32.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:30.923703+0000 osd.3 (osd.3) 76 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:34:32.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.020601+0000 osd.1 (osd.1) 61 : cluster [DBG] 5.1d scrub starts 2024-02-22T19:34:32.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.025106+0000 osd.1 (osd.1) 62 : cluster [DBG] 5.1d scrub ok 2024-02-22T19:34:32.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.044686+0000 osd.4 (osd.4) 75 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.047575+0000 osd.4 (osd.4) 76 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.074527+0000 osd.0 (osd.0) 81 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.086514+0000 osd.0 (osd.0) 82 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.337165+0000 mgr.y (mgr.14535) 217 : cluster [DBG] pgmap v235: 148 pgs: 148 active+clean; 1.4 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.12k op/s 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.529487+0000 osd.7 (osd.7) 73 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.916205+0000 osd.3 (osd.3) 75 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:30.923703+0000 osd.3 (osd.3) 76 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.020601+0000 osd.1 (osd.1) 61 : cluster [DBG] 5.1d scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.025106+0000 osd.1 (osd.1) 62 : cluster [DBG] 5.1d scrub ok 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.044686+0000 osd.4 (osd.4) 75 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.047575+0000 osd.4 (osd.4) 76 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.074527+0000 osd.0 (osd.0) 81 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:34:32.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.086514+0000 osd.0 (osd.0) 82 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:34:32.471 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.337165+0000 mgr.y (mgr.14535) 217 : cluster [DBG] pgmap v235: 148 pgs: 148 active+clean; 1.4 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.12k op/s 2024-02-22T19:34:32.471 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.529487+0000 osd.7 (osd.7) 73 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:34:32.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.916205+0000 osd.3 (osd.3) 75 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:30.923703+0000 osd.3 (osd.3) 76 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.020601+0000 osd.1 (osd.1) 61 : cluster [DBG] 5.1d scrub starts 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.025106+0000 osd.1 (osd.1) 62 : cluster [DBG] 5.1d scrub ok 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.044686+0000 osd.4 (osd.4) 75 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.047575+0000 osd.4 (osd.4) 76 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.074527+0000 osd.0 (osd.0) 81 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.086514+0000 osd.0 (osd.0) 82 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.337165+0000 mgr.y (mgr.14535) 217 : cluster [DBG] pgmap v235: 148 pgs: 148 active+clean; 1.4 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.12k op/s 2024-02-22T19:34:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.529487+0000 osd.7 (osd.7) 73 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:34:32.634 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:32.701 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:32.708 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:32.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:32.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 143, 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-02-22T19:22:59.746063+0000", 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-02-22T19:34:30.105160+0000", 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-02-22T19:28:53.274045+0000", 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-02-22T19:31:48.970507+0000", 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5804032, 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-02-22T19:34:32.827 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 18, 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 7, 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-02-22T19:34:32.828 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "pacific", 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "device_health_metrics", 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:26:54.342689+0000", 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.829 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "141", 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-02-22T19:34:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_devicehealth": {} 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "iscsi", 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.555543+0000", 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 3, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 3, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 3, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 3, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 3, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "59", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".rgw.root", 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.986190+0000", 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.839 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "58", 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.841 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.842 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:34:32.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 4, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.log", 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:01.494051+0000", 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.844 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "60", 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.848 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 5, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.control", 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:03.518927+0000", 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.850 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "62", 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.851 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.852 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.853 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 6, 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.meta", 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:05.596341+0000", 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:34:32.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "64", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.856 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.857 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.858 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_bias": 4 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.859 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 7, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "unique_pool_0", 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:31:31.948121+0000", 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:34:32.860 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 16, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 16, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 16, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 16, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 16, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.861 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "96", 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:34:32.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.863 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: "rados": {} 2024-02-22T19:34:32.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab", 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 8, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 99, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.865 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.867 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/162271081", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6802/162271081", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6804/162271081", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6803/162271081", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2", 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 105, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.871 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6805/2973403044", 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6806/2973403044", 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6808/2973403044", 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6807/2973403044", 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83", 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 18, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 132, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.873 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:34:32.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/936980935", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6810/936980935", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6812/936980935", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6811/936980935", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.876 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993", 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 26, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 134, 2024-02-22T19:34:32.877 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-02-22T19:34:32.878 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6813/3878043918", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6814/3878043918", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6816/3878043918", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6815/3878043918", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.880 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211", 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 32, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 99, 2024-02-22T19:34:32.881 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6800", 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6801", 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6803", 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6802", 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6800/2373040976", 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6801/2373040976", 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6803/2373040976", 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6802/2373040976", 2024-02-22T19:34:32.884 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07", 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.885 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 39, 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 93, 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6804", 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6805", 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6807", 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.887 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6806", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6804/2413553331", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6805/2413553331", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6807/2413553331", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6806/2413553331", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e", 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 45, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 99, 2024-02-22T19:34:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6808", 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6809", 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6811", 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6810", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6808/3043274732", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6809/3043274732", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6811/3043274732", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6810/3043274732", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a", 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 53, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 99, 2024-02-22T19:34:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6812", 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6813", 2024-02-22T19:34:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6815", 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:34:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6814", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6812/2605082260", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6813/2605082260", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6815/2605082260", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6814/2605082260", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:33:29.993216+0000", 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:34:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:33:44.688103+0000", 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:34:29.949049+0000", 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:34:08.916886+0000", 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:34:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:34:04.046392+0000", 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:33:35.362460+0000", 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:33:28.201652+0000", 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:33:44.260375+0000", 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.1d", 2024-02-22T19:34:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 4, 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.7", 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:34:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.b", 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.f", 2024-02-22T19:34:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": { 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4161439976": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2578736337": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/762516580": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:34:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3676085495": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3516945073": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/828421682": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4127851809": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/1906929179": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2640420245": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/2854906057": "2024-02-23T19:23:32.485117+0000" 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "ruleset-failure-domain": "osd", 2024-02-22T19:34:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:32.907 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:34:32.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:32.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:33.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:33.062 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:33.329 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:33.332 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:33.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:33.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.534644+0000 osd.7 (osd.7) 74 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.905843+0000 osd.3 (osd.3) 77 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:31.909029+0000 osd.3 (osd.3) 78 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.084006+0000 osd.4 (osd.4) 77 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.094979+0000 osd.4 (osd.4) 78 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.510872+0000 osd.7 (osd.7) 75 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.513906+0000 osd.7 (osd.7) 76 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.603475+0000 osd.6 (osd.6) 43 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.606391+0000 osd.6 (osd.6) 44 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:34:33.461 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:32.823164+0000 mon.a (mon.0) 908 : audit [DBG] from='client.? 172.21.15.53:0/1406240014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:34:33.463 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:33.485 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:33.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:33.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:33.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.534644+0000 osd.7 (osd.7) 74 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:34:33.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.905843+0000 osd.3 (osd.3) 77 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:34:33.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:31.909029+0000 osd.3 (osd.3) 78 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:34:33.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.084006+0000 osd.4 (osd.4) 77 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:34:33.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.094979+0000 osd.4 (osd.4) 78 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:34:33.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.510872+0000 osd.7 (osd.7) 75 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.513906+0000 osd.7 (osd.7) 76 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.603475+0000 osd.6 (osd.6) 43 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.606391+0000 osd.6 (osd.6) 44 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:32.823164+0000 mon.a (mon.0) 908 : audit [DBG] from='client.? 172.21.15.53:0/1406240014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.534644+0000 osd.7 (osd.7) 74 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.905843+0000 osd.3 (osd.3) 77 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:31.909029+0000 osd.3 (osd.3) 78 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.084006+0000 osd.4 (osd.4) 77 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.094979+0000 osd.4 (osd.4) 78 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.510872+0000 osd.7 (osd.7) 75 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.513906+0000 osd.7 (osd.7) 76 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.603475+0000 osd.6 (osd.6) 43 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:34:33.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.606391+0000 osd.6 (osd.6) 44 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:34:33.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:32.823164+0000 mon.a (mon.0) 908 : audit [DBG] from='client.? 172.21.15.53:0/1406240014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:34:33.571 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:33.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:33.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:33.643 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:33.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:33.802 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:34.238 INFO:tasks.thrashosds.thrasher:j is {'epoch': 143, 'fsid': 'b6783888-d1b7-11ee-95bf-87774f69a715', 'created': '2024-02-22T19:22:59.746063+0000', 'modified': '2024-02-22T19:34:30.105160+0000', 'last_up_change': '2024-02-22T19:28:53.274045+0000', 'last_in_change': '2024-02-22T19:31:48.970507+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-02-22T19:31:31.948121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '4326a76a-6144-4859-9b40-5ff69edbbcab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6801', 'nonce': 162271081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 162271081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 162271081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6803', 'nonce': 162271081}]}, 'public_addr': '172.21.15.53:6801/162271081', 'cluster_addr': '172.21.15.53:6802/162271081', 'heartbeat_back_addr': '172.21.15.53:6804/162271081', 'heartbeat_front_addr': '172.21.15.53:6803/162271081', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6805', 'nonce': 2973403044}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2973403044}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2973403044}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6807', 'nonce': 2973403044}]}, 'public_addr': '172.21.15.53:6805/2973403044', 'cluster_addr': '172.21.15.53:6806/2973403044', 'heartbeat_back_addr': '172.21.15.53:6808/2973403044', 'heartbeat_front_addr': '172.21.15.53:6807/2973403044', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 132, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6809', 'nonce': 936980935}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 936980935}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 936980935}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6811', 'nonce': 936980935}]}, 'public_addr': '172.21.15.53:6809/936980935', 'cluster_addr': '172.21.15.53:6810/936980935', 'heartbeat_back_addr': '172.21.15.53:6812/936980935', 'heartbeat_front_addr': '172.21.15.53:6811/936980935', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3f285320-aa28-4fc1-b1c7-456c9aecd993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6813', 'nonce': 3878043918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 3878043918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 3878043918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6815', 'nonce': 3878043918}]}, 'public_addr': '172.21.15.53:6813/3878043918', 'cluster_addr': '172.21.15.53:6814/3878043918', 'heartbeat_back_addr': '172.21.15.53:6816/3878043918', 'heartbeat_front_addr': '172.21.15.53:6815/3878043918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '773e8080-1028-48fe-8f6e-ed6beca60211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2373040976}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6801', 'nonce': 2373040976}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 2373040976}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2373040976}]}, 'public_addr': '172.21.15.64:6800/2373040976', 'cluster_addr': '172.21.15.64:6801/2373040976', 'heartbeat_back_addr': '172.21.15.64:6803/2373040976', 'heartbeat_front_addr': '172.21.15.64:6802/2373040976', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6804', 'nonce': 2413553331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6805', 'nonce': 2413553331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 2413553331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2413553331}]}, 'public_addr': '172.21.15.64:6804/2413553331', 'cluster_addr': '172.21.15.64:6805/2413553331', 'heartbeat_back_addr': '172.21.15.64:6807/2413553331', 'heartbeat_front_addr': '172.21.15.64:6806/2413553331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6808', 'nonce': 3043274732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6809', 'nonce': 3043274732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 3043274732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 3043274732}]}, 'public_addr': '172.21.15.64:6808/3043274732', 'cluster_addr': '172.21.15.64:6809/3043274732', 'heartbeat_back_addr': '172.21.15.64:6811/3043274732', 'heartbeat_front_addr': '172.21.15.64:6810/3043274732', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'cde36201-cb77-4967-9402-6fdbe633b40a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6812', 'nonce': 2605082260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6813', 'nonce': 2605082260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6815', 'nonce': 2605082260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 2605082260}]}, 'public_addr': '172.21.15.64:6812/2605082260', 'cluster_addr': '172.21.15.64:6813/2605082260', 'heartbeat_back_addr': '172.21.15.64:6815/2605082260', 'heartbeat_front_addr': '172.21.15.64:6814/2605082260', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:33:29.993216+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:33:44.688103+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:34:29.949049+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:34:08.916886+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:34:04.046392+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:33:35.362460+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:33:28.201652+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:33:44.260375+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1d', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.53:0/4161439976': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/2578736337': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/762516580': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:6800/3676085495': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:6800/3516945073': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/828421682': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/4127851809': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/1906929179': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:0/2640420245': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:6800/2854906057': '2024-02-23T19:23:32.485117+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-22T19:34:34.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:34:34.239 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:34.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:34.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:34.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:34.401 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:34.404 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:34.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.900779+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:34:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:32.908712+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:34:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.116016+0000 osd.0 (osd.0) 83 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:34:34.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.117543+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:34:34.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.121973+0000 osd.0 (osd.0) 84 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:34:34.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.123445+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:34:34.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.337562+0000 mgr.y (mgr.14535) 218 : cluster [DBG] pgmap v236: 148 pgs: 148 active+clean; 1.4 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s 2024-02-22T19:34:34.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.900779+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:32.908712+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.116016+0000 osd.0 (osd.0) 83 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.117543+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.121973+0000 osd.0 (osd.0) 84 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.123445+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:34:34.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.337562+0000 mgr.y (mgr.14535) 218 : cluster [DBG] pgmap v236: 148 pgs: 148 active+clean; 1.4 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.900779+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:32.908712+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.116016+0000 osd.0 (osd.0) 83 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.117543+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.121973+0000 osd.0 (osd.0) 84 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.123445+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:34:34.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.337562+0000 mgr.y (mgr.14535) 218 : cluster [DBG] pgmap v236: 148 pgs: 148 active+clean; 1.4 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s 2024-02-22T19:34:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:34.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:34.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:34.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:34.691 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:34.714 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:34.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:34.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:34.818 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:34.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:34:34.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:34:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:34:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:34.972 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:34.977 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:34.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:35.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:35.133 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:35.135 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:35.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:35.209 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:34:35.209 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:34:35.223 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:34:35.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:35.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:34:35.311 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:35.311 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:34:35.357 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:34:35.392 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:34:35.443 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:34:35.455 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.455 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.455 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.455 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.456 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.456 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.456 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.456 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.937784+0000 osd.3 (osd.3) 81 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:33.941901+0000 osd.3 (osd.3) 82 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:34.136267+0000 osd.0 (osd.0) 85 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:34.158235+0000 osd.0 (osd.0) 86 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:34.511366+0000 osd.7 (osd.7) 77 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:34.514259+0000 osd.7 (osd.7) 78 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:34.518548+0000 mgr.y (mgr.14535) 219 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:34.562666+0000 osd.6 (osd.6) 45 : cluster [DBG] 7.7 scrub starts 2024-02-22T19:34:35.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:35.036830+0000 osd.6 (osd.6) 46 : cluster [DBG] 7.7 scrub ok 2024-02-22T19:34:35.488 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:35.123221+0000 mon.a (mon.0) 909 : audit [DBG] from='client.? 172.21.15.53:0/1153967772' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:35.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.937784+0000 osd.3 (osd.3) 81 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:34:35.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:33.941901+0000 osd.3 (osd.3) 82 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:34:35.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:34.136267+0000 osd.0 (osd.0) 85 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:34:35.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:34.158235+0000 osd.0 (osd.0) 86 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:34:35.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:34.511366+0000 osd.7 (osd.7) 77 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:34:35.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:34.514259+0000 osd.7 (osd.7) 78 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:34:35.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:34.518548+0000 mgr.y (mgr.14535) 219 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:35.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:34.562666+0000 osd.6 (osd.6) 45 : cluster [DBG] 7.7 scrub starts 2024-02-22T19:34:35.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:35.036830+0000 osd.6 (osd.6) 46 : cluster [DBG] 7.7 scrub ok 2024-02-22T19:34:35.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:35.123221+0000 mon.a (mon.0) 909 : audit [DBG] from='client.? 172.21.15.53:0/1153967772' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.937784+0000 osd.3 (osd.3) 81 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:33.941901+0000 osd.3 (osd.3) 82 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:34.136267+0000 osd.0 (osd.0) 85 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:34.158235+0000 osd.0 (osd.0) 86 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:34.511366+0000 osd.7 (osd.7) 77 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:34.514259+0000 osd.7 (osd.7) 78 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:34:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:34.518548+0000 mgr.y (mgr.14535) 219 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:35.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:34.562666+0000 osd.6 (osd.6) 45 : cluster [DBG] 7.7 scrub starts 2024-02-22T19:34:35.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:35.036830+0000 osd.6 (osd.6) 46 : cluster [DBG] 7.7 scrub ok 2024-02-22T19:34:35.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:35.123221+0000 mon.a (mon.0) 909 : audit [DBG] from='client.? 172.21.15.53:0/1153967772' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:36.304 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:36.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:36.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:36.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:34:36.440 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:34:36.462 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:35.339833+0000 mgr.y (mgr.14535) 220 : cluster [DBG] pgmap v237: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s 2024-02-22T19:34:36.464 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:35.339833+0000 mgr.y (mgr.14535) 220 : cluster [DBG] pgmap v237: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s 2024-02-22T19:34:36.464 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:35.339833+0000 mgr.y (mgr.14535) 220 : cluster [DBG] pgmap v237: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s 2024-02-22T19:34:36.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:36.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:34:36.479 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:36.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:36.597 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:36.628 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:36.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:36.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:36.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:37.287 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:37.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:37.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:37.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:37.410 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:36.002192+0000 osd.1 (osd.1) 63 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:34:37.410 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:36.003294+0000 osd.1 (osd.1) 64 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:34:37.411 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:36.037220+0000 osd.3 (osd.3) 83 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:34:37.411 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:36.044509+0000 osd.3 (osd.3) 84 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:34:37.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:36.002192+0000 osd.1 (osd.1) 63 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:34:37.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:36.003294+0000 osd.1 (osd.1) 64 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:34:37.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:36.037220+0000 osd.3 (osd.3) 83 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:34:37.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:36.044509+0000 osd.3 (osd.3) 84 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:34:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:37.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:37.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:37.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:37.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:36.002192+0000 osd.1 (osd.1) 63 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:34:37.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:36.003294+0000 osd.1 (osd.1) 64 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:34:37.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:36.037220+0000 osd.3 (osd.3) 83 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:34:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:36.044509+0000 osd.3 (osd.3) 84 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:34:37.622 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:37.744 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:34:37.744 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:34:38.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:37.342430+0000 mgr.y (mgr.14535) 221 : cluster [DBG] pgmap v238: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 952 op/s 2024-02-22T19:34:38.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:37.342430+0000 mgr.y (mgr.14535) 221 : cluster [DBG] pgmap v238: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 952 op/s 2024-02-22T19:34:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:37.342430+0000 mgr.y (mgr.14535) 221 : cluster [DBG] pgmap v238: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 952 op/s 2024-02-22T19:34:38.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:34:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:38.054971+0000 osd.1 (osd.1) 65 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:34:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:38.060509+0000 osd.1 (osd.1) 66 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:34:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:38.181647+0000 osd.0 (osd.0) 87 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:34:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:38.184517+0000 osd.0 (osd.0) 88 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:34:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:38.525930+0000 osd.7 (osd.7) 79 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:34:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:38.528930+0000 osd.7 (osd.7) 80 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:34:39.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:38.054971+0000 osd.1 (osd.1) 65 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:34:39.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:38.060509+0000 osd.1 (osd.1) 66 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:34:39.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:38.181647+0000 osd.0 (osd.0) 87 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:34:39.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:38.184517+0000 osd.0 (osd.0) 88 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:34:39.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:38.525930+0000 osd.7 (osd.7) 79 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:34:39.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:38.528930+0000 osd.7 (osd.7) 80 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:34:39.439 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:38.054971+0000 osd.1 (osd.1) 65 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:34:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:38.060509+0000 osd.1 (osd.1) 66 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:34:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:38.181647+0000 osd.0 (osd.0) 87 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:34:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:38.184517+0000 osd.0 (osd.0) 88 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:34:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:38.525930+0000 osd.7 (osd.7) 79 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:34:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:38.528930+0000 osd.7 (osd.7) 80 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:34:39.756 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:34:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:39] "GET /metrics HTTP/1.1" 200 175625 "" "Prometheus/2.33.4" 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.033111+0000 osd.3 (osd.3) 85 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.036995+0000 osd.3 (osd.3) 86 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.080955+0000 osd.4 (osd.4) 81 : cluster [DBG] 5.f scrub starts 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.082183+0000 osd.4 (osd.4) 82 : cluster [DBG] 5.f scrub ok 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.344698+0000 mgr.y (mgr.14535) 222 : cluster [DBG] pgmap v239: 148 pgs: 148 active+clean; 954 MiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.480265+0000 osd.7 (osd.7) 81 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.483736+0000 osd.7 (osd.7) 82 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.033111+0000 osd.3 (osd.3) 85 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.036995+0000 osd.3 (osd.3) 86 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:34:40.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.080955+0000 osd.4 (osd.4) 81 : cluster [DBG] 5.f scrub starts 2024-02-22T19:34:40.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.082183+0000 osd.4 (osd.4) 82 : cluster [DBG] 5.f scrub ok 2024-02-22T19:34:40.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.344698+0000 mgr.y (mgr.14535) 222 : cluster [DBG] pgmap v239: 148 pgs: 148 active+clean; 954 MiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s 2024-02-22T19:34:40.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.480265+0000 osd.7 (osd.7) 81 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:34:40.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.483736+0000 osd.7 (osd.7) 82 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.033111+0000 osd.3 (osd.3) 85 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.036995+0000 osd.3 (osd.3) 86 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.080955+0000 osd.4 (osd.4) 81 : cluster [DBG] 5.f scrub starts 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.082183+0000 osd.4 (osd.4) 82 : cluster [DBG] 5.f scrub ok 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.344698+0000 mgr.y (mgr.14535) 222 : cluster [DBG] pgmap v239: 148 pgs: 148 active+clean; 954 MiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.480265+0000 osd.7 (osd.7) 81 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:34:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.483736+0000 osd.7 (osd.7) 82 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:34:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:39.997607+0000 osd.3 (osd.3) 87 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:34:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:40.000575+0000 osd.3 (osd.3) 88 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:34:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:40.139482+0000 osd.6 (osd.6) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:40.139523+0000 osd.6 (osd.6) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:40.253711+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e144: 8 total, 8 up, 7 in 2024-02-22T19:34:41.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:39.997607+0000 osd.3 (osd.3) 87 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:34:41.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:40.000575+0000 osd.3 (osd.3) 88 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:34:41.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:40.139482+0000 osd.6 (osd.6) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:41.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:40.139523+0000 osd.6 (osd.6) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:41.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:40.253711+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e144: 8 total, 8 up, 7 in 2024-02-22T19:34:41.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:39.997607+0000 osd.3 (osd.3) 87 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:34:41.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:40.000575+0000 osd.3 (osd.3) 88 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:34:41.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:40.139482+0000 osd.6 (osd.6) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:41.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:40.139523+0000 osd.6 (osd.6) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:41.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:40.253711+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e144: 8 total, 8 up, 7 in 2024-02-22T19:34:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:40.855926+0000 osd.2 (osd.2) 59 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:34:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:40.861849+0000 osd.2 (osd.2) 60 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.028668+0000 osd.1 (osd.1) 67 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.030852+0000 osd.1 (osd.1) 68 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.246289+0000 osd.0 (osd.0) 89 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.254441+0000 osd.0 (osd.0) 90 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.347293+0000 mgr.y (mgr.14535) 223 : cluster [DBG] pgmap v241: 148 pgs: 148 active+clean; 903 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 894 op/s 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.452728+0000 osd.6 (osd.6) 49 : cluster [DBG] 7.d scrub starts 2024-02-22T19:34:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.736079+0000 osd.6 (osd.6) 50 : cluster [DBG] 7.d scrub ok 2024-02-22T19:34:42.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:40.855926+0000 osd.2 (osd.2) 59 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:34:42.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:40.861849+0000 osd.2 (osd.2) 60 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:34:42.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.028668+0000 osd.1 (osd.1) 67 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:34:42.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.030852+0000 osd.1 (osd.1) 68 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:34:42.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.246289+0000 osd.0 (osd.0) 89 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:34:42.822 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.254441+0000 osd.0 (osd.0) 90 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:34:42.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.347293+0000 mgr.y (mgr.14535) 223 : cluster [DBG] pgmap v241: 148 pgs: 148 active+clean; 903 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 894 op/s 2024-02-22T19:34:42.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.452728+0000 osd.6 (osd.6) 49 : cluster [DBG] 7.d scrub starts 2024-02-22T19:34:42.823 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.736079+0000 osd.6 (osd.6) 50 : cluster [DBG] 7.d scrub ok 2024-02-22T19:34:42.823 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:40.855926+0000 osd.2 (osd.2) 59 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:34:42.823 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:40.861849+0000 osd.2 (osd.2) 60 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:34:42.823 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.028668+0000 osd.1 (osd.1) 67 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:34:42.824 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.030852+0000 osd.1 (osd.1) 68 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:34:42.824 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.246289+0000 osd.0 (osd.0) 89 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:34:42.824 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.254441+0000 osd.0 (osd.0) 90 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:34:42.824 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.347293+0000 mgr.y (mgr.14535) 223 : cluster [DBG] pgmap v241: 148 pgs: 148 active+clean; 903 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 894 op/s 2024-02-22T19:34:42.824 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.452728+0000 osd.6 (osd.6) 49 : cluster [DBG] 7.d scrub starts 2024-02-22T19:34:42.824 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.736079+0000 osd.6 (osd.6) 50 : cluster [DBG] 7.d scrub ok 2024-02-22T19:34:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.946933+0000 osd.3 (osd.3) 89 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:34:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:41.961781+0000 osd.3 (osd.3) 90 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:34:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:42.433788+0000 osd.7 (osd.7) 83 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:34:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:42.438549+0000 osd.7 (osd.7) 84 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:34:43.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.946933+0000 osd.3 (osd.3) 89 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:34:43.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:41.961781+0000 osd.3 (osd.3) 90 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:34:43.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:42.433788+0000 osd.7 (osd.7) 83 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:34:43.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:42.438549+0000 osd.7 (osd.7) 84 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:34:43.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.946933+0000 osd.3 (osd.3) 89 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:34:43.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:41.961781+0000 osd.3 (osd.3) 90 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:34:43.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:42.433788+0000 osd.7 (osd.7) 83 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:34:43.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:42.438549+0000 osd.7 (osd.7) 84 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:34:44.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:34:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:42.961893+0000 osd.1 (osd.1) 69 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:34:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:42.969181+0000 osd.1 (osd.1) 70 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:34:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.122968+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.e scrub starts 2024-02-22T19:34:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.126149+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.e scrub ok 2024-02-22T19:34:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.224156+0000 osd.0 (osd.0) 91 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:34:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.230655+0000 osd.0 (osd.0) 92 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:34:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.347848+0000 mgr.y (mgr.14535) 224 : cluster [DBG] pgmap v242: 148 pgs: 148 active+clean; 903 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 894 op/s 2024-02-22T19:34:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.429001+0000 osd.7 (osd.7) 85 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:34:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.434025+0000 osd.7 (osd.7) 86 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:34:44.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:42.961893+0000 osd.1 (osd.1) 69 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:34:44.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:42.969181+0000 osd.1 (osd.1) 70 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:34:44.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.122968+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.e scrub starts 2024-02-22T19:34:44.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.126149+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.e scrub ok 2024-02-22T19:34:44.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.224156+0000 osd.0 (osd.0) 91 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:34:44.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.230655+0000 osd.0 (osd.0) 92 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.347848+0000 mgr.y (mgr.14535) 224 : cluster [DBG] pgmap v242: 148 pgs: 148 active+clean; 903 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 894 op/s 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.429001+0000 osd.7 (osd.7) 85 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.434025+0000 osd.7 (osd.7) 86 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:42.961893+0000 osd.1 (osd.1) 69 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:42.969181+0000 osd.1 (osd.1) 70 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.122968+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.e scrub starts 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.126149+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.e scrub ok 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.224156+0000 osd.0 (osd.0) 91 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.230655+0000 osd.0 (osd.0) 92 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.347848+0000 mgr.y (mgr.14535) 224 : cluster [DBG] pgmap v242: 148 pgs: 148 active+clean; 903 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 894 op/s 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.429001+0000 osd.7 (osd.7) 85 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:34:44.737 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.434025+0000 osd.7 (osd.7) 86 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:34:44.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:34:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:34:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:34:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.910952+0000 osd.3 (osd.3) 91 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:34:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:43.915456+0000 osd.3 (osd.3) 92 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:34:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:44.274297+0000 osd.0 (osd.0) 93 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:34:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:44.286052+0000 osd.0 (osd.0) 94 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:34:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:44.429468+0000 osd.7 (osd.7) 87 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:34:45.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:44.432437+0000 osd.7 (osd.7) 88 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:34:45.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:44.523266+0000 mgr.y (mgr.14535) 225 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.910952+0000 osd.3 (osd.3) 91 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:43.915456+0000 osd.3 (osd.3) 92 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:44.274297+0000 osd.0 (osd.0) 93 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:44.286052+0000 osd.0 (osd.0) 94 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:44.429468+0000 osd.7 (osd.7) 87 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:44.432437+0000 osd.7 (osd.7) 88 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:44.523266+0000 mgr.y (mgr.14535) 225 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.910952+0000 osd.3 (osd.3) 91 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:34:45.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:43.915456+0000 osd.3 (osd.3) 92 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:34:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:44.274297+0000 osd.0 (osd.0) 93 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:34:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:44.286052+0000 osd.0 (osd.0) 94 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:34:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:44.429468+0000 osd.7 (osd.7) 87 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:34:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:44.432437+0000 osd.7 (osd.7) 88 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:34:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:44.523266+0000 mgr.y (mgr.14535) 225 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:44.902826+0000 osd.1 (osd.1) 71 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:34:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:44.906454+0000 osd.1 (osd.1) 72 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:34:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:45.348526+0000 mgr.y (mgr.14535) 226 : cluster [DBG] pgmap v243: 148 pgs: 148 active+clean; 610 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 925 op/s 2024-02-22T19:34:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:44.902826+0000 osd.1 (osd.1) 71 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:34:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:44.906454+0000 osd.1 (osd.1) 72 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:34:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:45.348526+0000 mgr.y (mgr.14535) 226 : cluster [DBG] pgmap v243: 148 pgs: 148 active+clean; 610 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 925 op/s 2024-02-22T19:34:46.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:44.902826+0000 osd.1 (osd.1) 71 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:34:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:44.906454+0000 osd.1 (osd.1) 72 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:34:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:45.348526+0000 mgr.y (mgr.14535) 226 : cluster [DBG] pgmap v243: 148 pgs: 148 active+clean; 610 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 925 op/s 2024-02-22T19:34:47.222 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:47.224 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:47.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:47.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:47.389 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:45.940668+0000 osd.3 (osd.3) 93 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:45.948351+0000 osd.3 (osd.3) 94 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:46.113658+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:46.118273+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:46.473125+0000 osd.7 (osd.7) 89 : cluster [DBG] 3.d scrub starts 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:46.476026+0000 osd.7 (osd.7) 90 : cluster [DBG] 3.d scrub ok 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:45.940668+0000 osd.3 (osd.3) 93 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:34:47.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:45.948351+0000 osd.3 (osd.3) 94 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:34:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:46.113658+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:34:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:46.118273+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:34:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:46.473125+0000 osd.7 (osd.7) 89 : cluster [DBG] 3.d scrub starts 2024-02-22T19:34:47.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:46.476026+0000 osd.7 (osd.7) 90 : cluster [DBG] 3.d scrub ok 2024-02-22T19:34:47.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:45.940668+0000 osd.3 (osd.3) 93 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:34:47.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:45.948351+0000 osd.3 (osd.3) 94 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:34:47.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:46.113658+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:34:47.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:46.118273+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:34:47.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:46.473125+0000 osd.7 (osd.7) 89 : cluster [DBG] 3.d scrub starts 2024-02-22T19:34:47.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:46.476026+0000 osd.7 (osd.7) 90 : cluster [DBG] 3.d scrub ok 2024-02-22T19:34:47.621 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:47.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:47.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:47.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:48.096 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:48.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:48.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:48.286 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:48.361 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:34:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:46.950732+0000 osd.3 (osd.3) 95 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:34:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:46.953947+0000 osd.3 (osd.3) 96 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:34:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:47.350823+0000 mgr.y (mgr.14535) 227 : cluster [DBG] pgmap v244: 148 pgs: 148 active+clean; 610 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 925 op/s 2024-02-22T19:34:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:47.522294+0000 osd.7 (osd.7) 91 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:34:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:47.527058+0000 osd.7 (osd.7) 92 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:34:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:48.263648+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.53:0/3051557168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:34:48.681 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:48.686 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:48.687 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:48.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:46.950732+0000 osd.3 (osd.3) 95 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:34:48.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:46.953947+0000 osd.3 (osd.3) 96 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:34:48.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:47.350823+0000 mgr.y (mgr.14535) 227 : cluster [DBG] pgmap v244: 148 pgs: 148 active+clean; 610 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 925 op/s 2024-02-22T19:34:48.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:47.522294+0000 osd.7 (osd.7) 91 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:34:48.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:47.527058+0000 osd.7 (osd.7) 92 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:34:48.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:48.263648+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.53:0/3051557168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:34:48.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:46.950732+0000 osd.3 (osd.3) 95 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:34:48.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:46.953947+0000 osd.3 (osd.3) 96 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:34:48.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:47.350823+0000 mgr.y (mgr.14535) 227 : cluster [DBG] pgmap v244: 148 pgs: 148 active+clean; 610 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 925 op/s 2024-02-22T19:34:48.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:47.522294+0000 osd.7 (osd.7) 91 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:34:48.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:47.527058+0000 osd.7 (osd.7) 92 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:34:48.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:48.263648+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.53:0/3051557168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:34:48.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:48.860 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:49.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:47.832893+0000 osd.2 (osd.2) 61 : cluster [DBG] 7.8 scrub starts 2024-02-22T19:34:49.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:47.956567+0000 osd.1 (osd.1) 73 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:34:49.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:47.963260+0000 osd.1 (osd.1) 74 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.023169+0000 osd.2 (osd.2) 62 : cluster [DBG] 7.8 scrub ok 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.272713+0000 osd.0 (osd.0) 95 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.292120+0000 osd.0 (osd.0) 96 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.319731+0000 mon.a (mon.0) 912 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.319761+0000 mon.a (mon.0) 913 : cluster [INF] Cluster is now healthy 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:48.332748+0000 mon.a (mon.0) 914 : audit [INF] from='client.? 172.21.15.53:0/3051557168' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.332788+0000 mon.a (mon.0) 915 : cluster [DBG] osdmap e145: 8 total, 8 up, 7 in 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:49] "GET /metrics HTTP/1.1" 200 175625 "" "Prometheus/2.33.4" 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:47.832893+0000 osd.2 (osd.2) 61 : cluster [DBG] 7.8 scrub starts 2024-02-22T19:34:49.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:47.956567+0000 osd.1 (osd.1) 73 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:47.963260+0000 osd.1 (osd.1) 74 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.023169+0000 osd.2 (osd.2) 62 : cluster [DBG] 7.8 scrub ok 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.272713+0000 osd.0 (osd.0) 95 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.292120+0000 osd.0 (osd.0) 96 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.319731+0000 mon.a (mon.0) 912 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.319761+0000 mon.a (mon.0) 913 : cluster [INF] Cluster is now healthy 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:48.332748+0000 mon.a (mon.0) 914 : audit [INF] from='client.? 172.21.15.53:0/3051557168' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:34:49.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.332788+0000 mon.a (mon.0) 915 : cluster [DBG] osdmap e145: 8 total, 8 up, 7 in 2024-02-22T19:34:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:47.832893+0000 osd.2 (osd.2) 61 : cluster [DBG] 7.8 scrub starts 2024-02-22T19:34:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:47.956567+0000 osd.1 (osd.1) 73 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:34:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:47.963260+0000 osd.1 (osd.1) 74 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:34:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.023169+0000 osd.2 (osd.2) 62 : cluster [DBG] 7.8 scrub ok 2024-02-22T19:34:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.272713+0000 osd.0 (osd.0) 95 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:34:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.292120+0000 osd.0 (osd.0) 96 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:34:49.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.319731+0000 mon.a (mon.0) 912 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:34:49.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.319761+0000 mon.a (mon.0) 913 : cluster [INF] Cluster is now healthy 2024-02-22T19:34:49.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:48.332748+0000 mon.a (mon.0) 914 : audit [INF] from='client.? 172.21.15.53:0/3051557168' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:34:49.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.332788+0000 mon.a (mon.0) 915 : cluster [DBG] osdmap e145: 8 total, 8 up, 7 in 2024-02-22T19:34:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:49.850 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:49.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:49.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:50.006 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:50.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:50.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:50.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:50.147 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:50.169 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:50.172 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:50.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:50.273 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.562558+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.568328+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.836163+0000 osd.1 (osd.1) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:48.836222+0000 osd.1 (osd.1) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.351709+0000 mgr.y (mgr.14535) 228 : cluster [DBG] pgmap v246: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.366210+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e146: 8 total, 8 up, 7 in 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.408638+0000 osd.6 (osd.6) 51 : cluster [DBG] 7.c scrub starts 2024-02-22T19:34:50.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.521511+0000 osd.6 (osd.6) 52 : cluster [DBG] 7.c scrub ok 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.562558+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.568328+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.836163+0000 osd.1 (osd.1) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:48.836222+0000 osd.1 (osd.1) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.351709+0000 mgr.y (mgr.14535) 228 : cluster [DBG] pgmap v246: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.366210+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e146: 8 total, 8 up, 7 in 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.408638+0000 osd.6 (osd.6) 51 : cluster [DBG] 7.c scrub starts 2024-02-22T19:34:50.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.521511+0000 osd.6 (osd.6) 52 : cluster [DBG] 7.c scrub ok 2024-02-22T19:34:50.511 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:50.522 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:50.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:50.623 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:50.708 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:50.736 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.562558+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:34:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.568328+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:34:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.836163+0000 osd.1 (osd.1) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:48.836222+0000 osd.1 (osd.1) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.351709+0000 mgr.y (mgr.14535) 228 : cluster [DBG] pgmap v246: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:34:50.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.366210+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e146: 8 total, 8 up, 7 in 2024-02-22T19:34:50.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.408638+0000 osd.6 (osd.6) 51 : cluster [DBG] 7.c scrub starts 2024-02-22T19:34:50.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.521511+0000 osd.6 (osd.6) 52 : cluster [DBG] 7.c scrub ok 2024-02-22T19:34:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:34:50.992 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:34:51.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:34:51.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.591949+0000 osd.7 (osd.7) 95 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:34:51.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.595248+0000 osd.7 (osd.7) 96 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:34:51.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:49.822308+0000 osd.2 (osd.2) 63 : cluster [DBG] 7.e scrub starts 2024-02-22T19:34:51.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:50.050779+0000 osd.2 (osd.2) 64 : cluster [DBG] 7.e scrub ok 2024-02-22T19:34:51.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.591949+0000 osd.7 (osd.7) 95 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:34:51.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.595248+0000 osd.7 (osd.7) 96 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:34:51.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:49.822308+0000 osd.2 (osd.2) 63 : cluster [DBG] 7.e scrub starts 2024-02-22T19:34:51.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:50.050779+0000 osd.2 (osd.2) 64 : cluster [DBG] 7.e scrub ok 2024-02-22T19:34:51.807 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:51.811 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:51.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.591949+0000 osd.7 (osd.7) 95 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:34:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.595248+0000 osd.7 (osd.7) 96 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:34:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:49.822308+0000 osd.2 (osd.2) 63 : cluster [DBG] 7.e scrub starts 2024-02-22T19:34:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:50.050779+0000 osd.2 (osd.2) 64 : cluster [DBG] 7.e scrub ok 2024-02-22T19:34:51.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:52.066 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:52.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:52.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:52.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:52.202 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:52.224 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:52.226 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:52.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:52.327 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:52.398 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:50.948547+0000 mgr.y (mgr.14535) 229 : audit [DBG] from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:50.958650+0000 osd.3 (osd.3) 97 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:34:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:50.967429+0000 osd.3 (osd.3) 98 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:34:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:51.352751+0000 mgr.y (mgr.14535) 230 : cluster [DBG] pgmap v248: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:34:52.404 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:50.948547+0000 mgr.y (mgr.14535) 229 : audit [DBG] from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:50.958650+0000 osd.3 (osd.3) 97 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:34:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:50.967429+0000 osd.3 (osd.3) 98 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:34:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:51.352751+0000 mgr.y (mgr.14535) 230 : cluster [DBG] pgmap v248: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:34:52.640 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-02-22T19:34:49.368168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343348924,"num_objects":5432,"num_object_clones":0,"num_object_copies":16296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5432,"num_whiteouts":0,"num_read":1056,"num_read_kb":808,"num_write":173598,"num_write_kb":5709098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5178,"num_bytes_recovered":332925147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77973,"ondisk_log_size":77973,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":3208948,"kb_used_data":1083132,"kb_used_omap":0,"kb_used_meta":2125568,"kb_avail":653007116,"statfs":{"total":671965249536,"available":668679286784,"internally_reserved":0,"allocated":1109127168,"data_stored":1106469984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2176581632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-603324416,"num_objects":-9206,"num_object_clones":0,"num_object_copies":-27618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9206,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9206,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.021119"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":110,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251729+0000","last_change":"2024-02-22T19:34:05.990011+0000","last_active":"2024-02-22T19:34:40.251729+0000","last_peered":"2024-02-22T19:34:40.251729+0000","last_clean":"2024-02-22T19:34:40.251729+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:34:40.251729+0000","last_undegraded":"2024-02-22T19:34:40.251729+0000","last_fullsized":"2024-02-22T19:34:40.251729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:05.989944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:05.989944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":112,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.255365+0000","last_change":"2024-02-22T19:33:58.781565+0000","last_active":"2024-02-22T19:34:40.255365+0000","last_peered":"2024-02-22T19:34:40.255365+0000","last_clean":"2024-02-22T19:34:40.255365+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:34:40.255365+0000","last_undegraded":"2024-02-22T19:34:40.255365+0000","last_fullsized":"2024-02-22T19:34:40.255365+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.781466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.781466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":119,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:44.286204+0000","last_change":"2024-02-22T19:34:44.286165+0000","last_active":"2024-02-22T19:34:44.286204+0000","last_peered":"2024-02-22T19:34:44.286204+0000","last_clean":"2024-02-22T19:34:44.286204+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:34:44.286204+0000","last_undegraded":"2024-02-22T19:34:44.286204+0000","last_fullsized":"2024-02-22T19:34:44.286204+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.286059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.286059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251268+0000","last_change":"2024-02-22T19:33:53.940446+0000","last_active":"2024-02-22T19:34:40.251268+0000","last_peered":"2024-02-22T19:34:40.251268+0000","last_clean":"2024-02-22T19:34:40.251268+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:34:40.251268+0000","last_undegraded":"2024-02-22T19:34:40.251268+0000","last_fullsized":"2024-02-22T19:34:40.251268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:53.940375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.940375+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":98,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273879+0000","last_change":"2024-02-22T19:34:30.923797+0000","last_active":"2024-02-22T19:34:40.273879+0000","last_peered":"2024-02-22T19:34:40.273879+0000","last_clean":"2024-02-22T19:34:40.273879+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:34:40.273879+0000","last_undegraded":"2024-02-22T19:34:40.273879+0000","last_fullsized":"2024-02-22T19:34:40.273879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.923707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.923707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":98,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251386+0000","last_change":"2024-02-22T19:30:52.985496+0000","last_active":"2024-02-22T19:34:40.251386+0000","last_peered":"2024-02-22T19:34:40.251386+0000","last_clean":"2024-02-22T19:34:40.251386+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:34:40.251386+0000","last_undegraded":"2024-02-22T19:34:40.251386+0000","last_fullsized":"2024-02-22T19:34:40.251386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:52.985410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:52.985410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.276415+0000","last_change":"2024-02-22T19:30:56.953302+0000","last_active":"2024-02-22T19:34:40.276415+0000","last_peered":"2024-02-22T19:34:40.276415+0000","last_clean":"2024-02-22T19:34:40.276415+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:34:40.276415+0000","last_undegraded":"2024-02-22T19:34:40.276415+0000","last_fullsized":"2024-02-22T19:34:40.276415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:30:56.953188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":127,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:45.948480+0000","last_change":"2024-02-22T19:34:45.948451+0000","last_active":"2024-02-22T19:34:45.948480+0000","last_peered":"2024-02-22T19:34:45.948480+0000","last_clean":"2024-02-22T19:34:45.948480+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:34:45.948480+0000","last_undegraded":"2024-02-22T19:34:45.948480+0000","last_fullsized":"2024-02-22T19:34:45.948480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:45.948358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:45.948358+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.270990+0000","last_change":"2024-02-22T19:33:51.065916+0000","last_active":"2024-02-22T19:34:40.270990+0000","last_peered":"2024-02-22T19:34:40.270990+0000","last_clean":"2024-02-22T19:34:40.270990+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:34:40.270990+0000","last_undegraded":"2024-02-22T19:34:40.270990+0000","last_fullsized":"2024-02-22T19:34:40.270990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:51.065825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.065825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271290+0000","last_change":"2024-02-22T19:33:42.010538+0000","last_active":"2024-02-22T19:34:40.271290+0000","last_peered":"2024-02-22T19:34:40.271290+0000","last_clean":"2024-02-22T19:34:40.271290+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:34:40.271290+0000","last_undegraded":"2024-02-22T19:34:40.271290+0000","last_fullsized":"2024-02-22T19:34:40.271290+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.010467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.010467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.268871+0000","last_change":"2024-02-22T19:33:45.802849+0000","last_active":"2024-02-22T19:34:40.268871+0000","last_peered":"2024-02-22T19:34:40.268871+0000","last_clean":"2024-02-22T19:34:40.268871+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:34:40.268871+0000","last_undegraded":"2024-02-22T19:34:40.268871+0000","last_fullsized":"2024-02-22T19:34:40.268871+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.802756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.802756+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251945+0000","last_change":"2024-02-22T19:33:52.938521+0000","last_active":"2024-02-22T19:34:40.251945+0000","last_peered":"2024-02-22T19:34:40.251945+0000","last_clean":"2024-02-22T19:34:40.251945+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:34:40.251945+0000","last_undegraded":"2024-02-22T19:34:40.251945+0000","last_fullsized":"2024-02-22T19:34:40.251945+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:52.938440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.938440+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254169+0000","last_change":"2024-02-22T19:34:02.783853+0000","last_active":"2024-02-22T19:34:40.254169+0000","last_peered":"2024-02-22T19:34:40.254169+0000","last_clean":"2024-02-22T19:34:40.254169+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:34:40.254169+0000","last_undegraded":"2024-02-22T19:34:40.254169+0000","last_fullsized":"2024-02-22T19:34:40.254169+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:34:02.783771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271664+0000","last_change":"2024-02-22T19:34:30.499933+0000","last_active":"2024-02-22T19:34:40.271664+0000","last_peered":"2024-02-22T19:34:40.271664+0000","last_clean":"2024-02-22T19:34:40.271664+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:34:40.271664+0000","last_undegraded":"2024-02-22T19:34:40.271664+0000","last_fullsized":"2024-02-22T19:34:40.271664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.499864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.499864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":69,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.253049+0000","last_change":"2024-02-22T19:31:50.439339+0000","last_active":"2024-02-22T19:34:40.253049+0000","last_peered":"2024-02-22T19:34:40.253049+0000","last_clean":"2024-02-22T19:34:40.253049+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:34:40.253049+0000","last_undegraded":"2024-02-22T19:34:40.253049+0000","last_fullsized":"2024-02-22T19:34:40.253049+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:31:23.683223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.683223+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":115,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:46.118418+0000","last_change":"2024-02-22T19:34:46.118383+0000","last_active":"2024-02-22T19:34:46.118418+0000","last_peered":"2024-02-22T19:34:46.118418+0000","last_clean":"2024-02-22T19:34:46.118418+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:34:46.118418+0000","last_undegraded":"2024-02-22T19:34:46.118418+0000","last_fullsized":"2024-02-22T19:34:46.118418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:46.118282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.118282+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.252736+0000","last_change":"2024-02-22T19:31:50.009328+0000","last_active":"2024-02-22T19:34:40.252736+0000","last_peered":"2024-02-22T19:34:40.252736+0000","last_clean":"2024-02-22T19:34:40.252736+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:34:40.252736+0000","last_undegraded":"2024-02-22T19:34:40.252736+0000","last_fullsized":"2024-02-22T19:34:40.252736+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:34.059559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:34.059559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":103,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.256023+0000","last_change":"2024-02-22T19:34:31.025213+0000","last_active":"2024-02-22T19:34:40.256023+0000","last_peered":"2024-02-22T19:34:40.256023+0000","last_clean":"2024-02-22T19:34:40.256023+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:34:40.256023+0000","last_undegraded":"2024-02-22T19:34:40.256023+0000","last_fullsized":"2024-02-22T19:34:40.256023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.025113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.025113+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271462+0000","last_change":"2024-02-22T19:31:26.454221+0000","last_active":"2024-02-22T19:34:40.271462+0000","last_peered":"2024-02-22T19:34:40.271462+0000","last_clean":"2024-02-22T19:34:40.271462+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:34:40.271462+0000","last_undegraded":"2024-02-22T19:34:40.271462+0000","last_fullsized":"2024-02-22T19:34:40.271462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.454155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.454155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":132,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.862002+0000","last_change":"2024-02-22T19:34:40.861961+0000","last_active":"2024-02-22T19:34:40.862002+0000","last_peered":"2024-02-22T19:34:40.862002+0000","last_clean":"2024-02-22T19:34:40.862002+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:34:40.862002+0000","last_undegraded":"2024-02-22T19:34:40.862002+0000","last_fullsized":"2024-02-22T19:34:40.862002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:40.861854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:40.861854+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275663+0000","last_change":"2024-02-22T19:33:48.835132+0000","last_active":"2024-02-22T19:34:40.275663+0000","last_peered":"2024-02-22T19:34:40.275663+0000","last_clean":"2024-02-22T19:34:40.275663+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:34:40.275663+0000","last_undegraded":"2024-02-22T19:34:40.275663+0000","last_fullsized":"2024-02-22T19:34:40.275663+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.835030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.835030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":98,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.253357+0000","last_change":"2024-02-22T19:30:44.976718+0000","last_active":"2024-02-22T19:34:40.253357+0000","last_peered":"2024-02-22T19:34:40.253357+0000","last_clean":"2024-02-22T19:34:40.253357+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:34:40.253357+0000","last_undegraded":"2024-02-22T19:34:40.253357+0000","last_fullsized":"2024-02-22T19:34:40.253357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:44.976653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:44.976653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.252022+0000","last_change":"2024-02-22T19:34:30.007888+0000","last_active":"2024-02-22T19:34:40.252022+0000","last_peered":"2024-02-22T19:34:40.252022+0000","last_clean":"2024-02-22T19:34:40.252022+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:34:40.252022+0000","last_undegraded":"2024-02-22T19:34:40.252022+0000","last_fullsized":"2024-02-22T19:34:40.252022+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.007814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.007814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":125,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.274799+0000","last_change":"2024-02-22T19:31:04.012758+0000","last_active":"2024-02-22T19:34:40.274799+0000","last_peered":"2024-02-22T19:34:40.274799+0000","last_clean":"2024-02-22T19:34:40.274799+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:34:40.274799+0000","last_undegraded":"2024-02-22T19:34:40.274799+0000","last_fullsized":"2024-02-22T19:34:40.274799+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:31:04.012663+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":114,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272239+0000","last_change":"2024-02-22T19:34:03.590429+0000","last_active":"2024-02-22T19:34:40.272239+0000","last_peered":"2024-02-22T19:34:40.272239+0000","last_clean":"2024-02-22T19:34:40.272239+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:34:40.272239+0000","last_undegraded":"2024-02-22T19:34:40.272239+0000","last_fullsized":"2024-02-22T19:34:40.272239+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:34:03.590356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"137'9","reported_seq":187,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.345935+0000","last_change":"2024-02-22T19:34:00.780638+0000","last_active":"2024-02-22T19:34:48.345935+0000","last_peered":"2024-02-22T19:34:48.345935+0000","last_clean":"2024-02-22T19:34:48.345935+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:34:48.345935+0000","last_undegraded":"2024-02-22T19:34:48.345935+0000","last_fullsized":"2024-02-22T19:34:48.345935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:34:00.780574+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.255441+0000","last_change":"2024-02-22T19:33:55.750063+0000","last_active":"2024-02-22T19:34:40.255441+0000","last_peered":"2024-02-22T19:34:40.255441+0000","last_clean":"2024-02-22T19:34:40.255441+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:34:40.255441+0000","last_undegraded":"2024-02-22T19:34:40.255441+0000","last_fullsized":"2024-02-22T19:34:40.255441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:55.749971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:55.749971+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":118,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273832+0000","last_change":"2024-02-22T19:34:01.123505+0000","last_active":"2024-02-22T19:34:40.273832+0000","last_peered":"2024-02-22T19:34:40.273832+0000","last_clean":"2024-02-22T19:34:40.273832+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:34:40.273832+0000","last_undegraded":"2024-02-22T19:34:40.273832+0000","last_fullsized":"2024-02-22T19:34:40.273832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:34:01.123405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.123405+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":112,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:42.969334+0000","last_change":"2024-02-22T19:34:42.969287+0000","last_active":"2024-02-22T19:34:42.969334+0000","last_peered":"2024-02-22T19:34:42.969334+0000","last_clean":"2024-02-22T19:34:42.969334+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:34:42.969334+0000","last_undegraded":"2024-02-22T19:34:42.969334+0000","last_fullsized":"2024-02-22T19:34:42.969334+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.969189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.969189+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.274106+0000","last_change":"2024-02-22T19:34:33.122075+0000","last_active":"2024-02-22T19:34:40.274106+0000","last_peered":"2024-02-22T19:34:40.274106+0000","last_clean":"2024-02-22T19:34:40.274106+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:34:40.274106+0000","last_undegraded":"2024-02-22T19:34:40.274106+0000","last_fullsized":"2024-02-22T19:34:40.274106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:33.121981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.121981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272894+0000","last_change":"2024-02-22T19:30:57.988070+0000","last_active":"2024-02-22T19:34:40.272894+0000","last_peered":"2024-02-22T19:34:40.272894+0000","last_clean":"2024-02-22T19:34:40.272894+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:34:40.272894+0000","last_undegraded":"2024-02-22T19:34:40.272894+0000","last_fullsized":"2024-02-22T19:34:40.272894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:57.987956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:57.987956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":174,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.257724+0000","last_change":"2024-02-22T19:33:46.763313+0000","last_active":"2024-02-22T19:34:40.257724+0000","last_peered":"2024-02-22T19:34:40.257724+0000","last_clean":"2024-02-22T19:34:40.257724+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:34:40.257724+0000","last_undegraded":"2024-02-22T19:34:40.257724+0000","last_fullsized":"2024-02-22T19:34:40.257724+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:46.763202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.763202+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271589+0000","last_change":"2024-02-22T19:34:34.514336+0000","last_active":"2024-02-22T19:34:40.271589+0000","last_peered":"2024-02-22T19:34:40.271589+0000","last_clean":"2024-02-22T19:34:40.271589+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:34:40.271589+0000","last_undegraded":"2024-02-22T19:34:40.271589+0000","last_fullsized":"2024-02-22T19:34:40.271589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:34.514266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:34.514266+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":100,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.258044+0000","last_change":"2024-02-22T19:34:32.606492+0000","last_active":"2024-02-22T19:34:40.258044+0000","last_peered":"2024-02-22T19:34:40.258044+0000","last_clean":"2024-02-22T19:34:40.258044+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:34:40.258044+0000","last_undegraded":"2024-02-22T19:34:40.258044+0000","last_fullsized":"2024-02-22T19:34:40.258044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.606397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.606397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":114,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271951+0000","last_change":"2024-02-22T19:33:43.048434+0000","last_active":"2024-02-22T19:34:40.271951+0000","last_peered":"2024-02-22T19:34:40.271951+0000","last_clean":"2024-02-22T19:34:40.271951+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:34:40.271951+0000","last_undegraded":"2024-02-22T19:34:40.271951+0000","last_fullsized":"2024-02-22T19:34:40.271951+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:43.048330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.048330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":108,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272945+0000","last_change":"2024-02-22T19:34:39.037075+0000","last_active":"2024-02-22T19:34:40.272945+0000","last_peered":"2024-02-22T19:34:40.272945+0000","last_clean":"2024-02-22T19:34:40.272945+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:34:40.272945+0000","last_undegraded":"2024-02-22T19:34:40.272945+0000","last_fullsized":"2024-02-22T19:34:40.272945+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:34:39.036999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.036999+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":98,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275197+0000","last_change":"2024-02-22T19:34:36.044624+0000","last_active":"2024-02-22T19:34:40.275197+0000","last_peered":"2024-02-22T19:34:40.275197+0000","last_clean":"2024-02-22T19:34:40.275197+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:34:40.275197+0000","last_undegraded":"2024-02-22T19:34:40.275197+0000","last_fullsized":"2024-02-22T19:34:40.275197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.044519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.044519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271458+0000","last_change":"2024-02-22T19:34:06.606340+0000","last_active":"2024-02-22T19:34:40.271458+0000","last_peered":"2024-02-22T19:34:40.271458+0000","last_clean":"2024-02-22T19:34:40.271458+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:34:40.271458+0000","last_undegraded":"2024-02-22T19:34:40.271458+0000","last_fullsized":"2024-02-22T19:34:40.271458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:06.606235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:34:06.606235+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272124+0000","last_change":"2024-02-22T19:33:45.528051+0000","last_active":"2024-02-22T19:34:40.272124+0000","last_peered":"2024-02-22T19:34:40.272124+0000","last_clean":"2024-02-22T19:34:40.272124+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:34:40.272124+0000","last_undegraded":"2024-02-22T19:34:40.272124+0000","last_fullsized":"2024-02-22T19:34:40.272124+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:45.527956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:45.527956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:43.915602+0000","last_change":"2024-02-22T19:34:43.915568+0000","last_active":"2024-02-22T19:34:43.915602+0000","last_peered":"2024-02-22T19:34:43.915602+0000","last_clean":"2024-02-22T19:34:43.915602+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:34:43.915602+0000","last_undegraded":"2024-02-22T19:34:43.915602+0000","last_fullsized":"2024-02-22T19:34:43.915602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:43.915463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.915463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":108,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.292273+0000","last_change":"2024-02-22T19:34:48.292233+0000","last_active":"2024-02-22T19:34:48.292273+0000","last_peered":"2024-02-22T19:34:48.292273+0000","last_clean":"2024-02-22T19:34:48.292273+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:34:48.292273+0000","last_undegraded":"2024-02-22T19:34:48.292273+0000","last_fullsized":"2024-02-22T19:34:48.292273+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:48.292131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.292131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":100,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:41.961914+0000","last_change":"2024-02-22T19:34:41.961882+0000","last_active":"2024-02-22T19:34:41.961914+0000","last_peered":"2024-02-22T19:34:41.961914+0000","last_clean":"2024-02-22T19:34:41.961914+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:34:41.961914+0000","last_undegraded":"2024-02-22T19:34:41.961914+0000","last_fullsized":"2024-02-22T19:34:41.961914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.961788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.961788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271614+0000","last_change":"2024-02-22T19:31:23.009097+0000","last_active":"2024-02-22T19:34:40.271614+0000","last_peered":"2024-02-22T19:34:40.271614+0000","last_clean":"2024-02-22T19:34:40.271614+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:34:40.271614+0000","last_undegraded":"2024-02-22T19:34:40.271614+0000","last_fullsized":"2024-02-22T19:34:40.271614+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:31:23.009021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254640+0000","last_change":"2024-02-22T19:33:47.757576+0000","last_active":"2024-02-22T19:34:40.254640+0000","last_peered":"2024-02-22T19:34:40.254640+0000","last_clean":"2024-02-22T19:34:40.254640+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:34:40.254640+0000","last_undegraded":"2024-02-22T19:34:40.254640+0000","last_fullsized":"2024-02-22T19:34:40.254640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:47.757479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:47.757479+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272872+0000","last_change":"2024-02-22T19:33:42.862634+0000","last_active":"2024-02-22T19:34:40.272872+0000","last_peered":"2024-02-22T19:34:40.272872+0000","last_clean":"2024-02-22T19:34:40.272872+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:34:40.272872+0000","last_undegraded":"2024-02-22T19:34:40.272872+0000","last_fullsized":"2024-02-22T19:34:40.272872+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.862563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.862563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":112,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:44.906619+0000","last_change":"2024-02-22T19:34:44.906578+0000","last_active":"2024-02-22T19:34:44.906619+0000","last_peered":"2024-02-22T19:34:44.906619+0000","last_clean":"2024-02-22T19:34:44.906619+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:34:44.906619+0000","last_undegraded":"2024-02-22T19:34:44.906619+0000","last_fullsized":"2024-02-22T19:34:44.906619+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.906461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.906461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.253137+0000","last_change":"2024-02-22T19:34:31.047792+0000","last_active":"2024-02-22T19:34:40.253137+0000","last_peered":"2024-02-22T19:34:40.253137+0000","last_clean":"2024-02-22T19:34:40.253137+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:34:40.253137+0000","last_undegraded":"2024-02-22T19:34:40.253137+0000","last_fullsized":"2024-02-22T19:34:40.253137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.047579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.047579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":119,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251898+0000","last_change":"2024-02-22T19:30:41.008709+0000","last_active":"2024-02-22T19:34:40.251898+0000","last_peered":"2024-02-22T19:34:40.251898+0000","last_clean":"2024-02-22T19:34:40.251898+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:34:40.251898+0000","last_undegraded":"2024-02-22T19:34:40.251898+0000","last_fullsized":"2024-02-22T19:34:40.251898+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:30:41.008629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:41.008629+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271593+0000","last_change":"2024-02-22T19:34:29.079997+0000","last_active":"2024-02-22T19:34:40.271593+0000","last_peered":"2024-02-22T19:34:40.271593+0000","last_clean":"2024-02-22T19:34:40.271593+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:34:40.271593+0000","last_undegraded":"2024-02-22T19:34:40.271593+0000","last_fullsized":"2024-02-22T19:34:40.271593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:29.079891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:29.079891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":117,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.352872+0000","last_change":"2024-02-22T19:31:50.105130+0000","last_active":"2024-02-22T19:34:48.352872+0000","last_peered":"2024-02-22T19:34:48.352872+0000","last_clean":"2024-02-22T19:34:48.352872+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:34:48.352872+0000","last_undegraded":"2024-02-22T19:34:48.352872+0000","last_fullsized":"2024-02-22T19:34:48.352872+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:29.679183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:29.679183+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:42.438701+0000","last_change":"2024-02-22T19:34:42.438663+0000","last_active":"2024-02-22T19:34:42.438701+0000","last_peered":"2024-02-22T19:34:42.438701+0000","last_clean":"2024-02-22T19:34:42.438701+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:34:42.438701+0000","last_undegraded":"2024-02-22T19:34:42.438701+0000","last_fullsized":"2024-02-22T19:34:42.438701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.438557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.438557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":122,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275702+0000","last_change":"2024-02-22T19:34:32.908841+0000","last_active":"2024-02-22T19:34:40.275702+0000","last_peered":"2024-02-22T19:34:40.275702+0000","last_clean":"2024-02-22T19:34:40.275702+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:34:40.275702+0000","last_undegraded":"2024-02-22T19:34:40.275702+0000","last_fullsized":"2024-02-22T19:34:40.275702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:34:32.908721+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.908721+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":110,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.253316+0000","last_change":"2024-02-22T19:33:58.835432+0000","last_active":"2024-02-22T19:34:40.253316+0000","last_peered":"2024-02-22T19:34:40.253316+0000","last_clean":"2024-02-22T19:34:40.253316+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:34:40.253316+0000","last_undegraded":"2024-02-22T19:34:40.253316+0000","last_fullsized":"2024-02-22T19:34:40.253316+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.835370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.835370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":173,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.345135+0000","last_change":"2024-02-22T19:31:08.045693+0000","last_active":"2024-02-22T19:34:48.345135+0000","last_peered":"2024-02-22T19:34:48.345135+0000","last_clean":"2024-02-22T19:34:48.345135+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:34:48.345135+0000","last_undegraded":"2024-02-22T19:34:48.345135+0000","last_fullsized":"2024-02-22T19:34:48.345135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:31:08.045618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:08.045618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":110,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:41.254592+0000","last_change":"2024-02-22T19:34:41.254550+0000","last_active":"2024-02-22T19:34:41.254592+0000","last_peered":"2024-02-22T19:34:41.254592+0000","last_clean":"2024-02-22T19:34:41.254592+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:34:41.254592+0000","last_undegraded":"2024-02-22T19:34:41.254592+0000","last_fullsized":"2024-02-22T19:34:41.254592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.254449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.254449+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":130,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:47.527212+0000","last_change":"2024-02-22T19:34:47.527168+0000","last_active":"2024-02-22T19:34:47.527212+0000","last_peered":"2024-02-22T19:34:47.527212+0000","last_clean":"2024-02-22T19:34:47.527212+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:34:47.527212+0000","last_undegraded":"2024-02-22T19:34:47.527212+0000","last_fullsized":"2024-02-22T19:34:47.527212+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:34:47.527066+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.527066+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.268759+0000","last_change":"2024-02-22T19:33:57.874765+0000","last_active":"2024-02-22T19:34:40.268759+0000","last_peered":"2024-02-22T19:34:40.268759+0000","last_clean":"2024-02-22T19:34:40.268759+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:34:40.268759+0000","last_undegraded":"2024-02-22T19:34:40.268759+0000","last_fullsized":"2024-02-22T19:34:40.268759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.874679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.874679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":95,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275508+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:34:40.275508+0000","last_peered":"2024-02-22T19:34:40.275508+0000","last_clean":"2024-02-22T19:34:40.275508+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:34:40.275508+0000","last_undegraded":"2024-02-22T19:34:40.275508+0000","last_fullsized":"2024-02-22T19:34:40.275508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271328+0000","last_change":"2024-02-22T19:33:53.652545+0000","last_active":"2024-02-22T19:34:40.271328+0000","last_peered":"2024-02-22T19:34:40.271328+0000","last_clean":"2024-02-22T19:34:40.271328+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:34:40.271328+0000","last_undegraded":"2024-02-22T19:34:40.271328+0000","last_fullsized":"2024-02-22T19:34:40.271328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:53.652468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:22.592774+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.652468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":119,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273981+0000","last_change":"2024-02-22T19:33:46.001358+0000","last_active":"2024-02-22T19:34:40.273981+0000","last_peered":"2024-02-22T19:34:40.273981+0000","last_clean":"2024-02-22T19:34:40.273981+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:34:40.273981+0000","last_undegraded":"2024-02-22T19:34:40.273981+0000","last_fullsized":"2024-02-22T19:34:40.273981+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:46.001290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.001290+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271535+0000","last_change":"2024-02-22T19:33:56.634433+0000","last_active":"2024-02-22T19:34:40.271535+0000","last_peered":"2024-02-22T19:34:40.271535+0000","last_clean":"2024-02-22T19:34:40.271535+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:34:40.271535+0000","last_undegraded":"2024-02-22T19:34:40.271535+0000","last_fullsized":"2024-02-22T19:34:40.271535+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:56.634370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:56.634370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.276594+0000","last_change":"2024-02-22T19:34:31.909139+0000","last_active":"2024-02-22T19:34:40.276594+0000","last_peered":"2024-02-22T19:34:40.276594+0000","last_clean":"2024-02-22T19:34:40.276594+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:34:40.276594+0000","last_undegraded":"2024-02-22T19:34:40.276594+0000","last_fullsized":"2024-02-22T19:34:40.276594+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.909036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.909036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":115,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.256659+0000","last_change":"2024-02-22T19:34:05.812010+0000","last_active":"2024-02-22T19:34:40.256659+0000","last_peered":"2024-02-22T19:34:40.256659+0000","last_clean":"2024-02-22T19:34:40.256659+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:34:40.256659+0000","last_undegraded":"2024-02-22T19:34:40.256659+0000","last_fullsized":"2024-02-22T19:34:40.256659+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:34:05.811946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":111,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275744+0000","last_change":"2024-02-22T19:33:52.788152+0000","last_active":"2024-02-22T19:34:40.275744+0000","last_peered":"2024-02-22T19:34:40.275744+0000","last_clean":"2024-02-22T19:34:40.275744+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:34:40.275744+0000","last_undegraded":"2024-02-22T19:34:40.275744+0000","last_fullsized":"2024-02-22T19:34:40.275744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:33:52.788050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.788050+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"145'10858","reported_seq":13484,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.294072+0000","last_change":"2024-02-22T19:32:06.969062+0000","last_active":"2024-02-22T19:34:49.294072+0000","last_peered":"2024-02-22T19:34:49.294072+0000","last_clean":"2024-02-22T19:34:49.294072+0000","last_became_active":"2024-02-22T19:32:06.966453+0000","last_became_peered":"2024-02-22T19:32:06.966453+0000","last_unstale":"2024-02-22T19:34:49.294072+0000","last_undegraded":"2024-02-22T19:34:49.294072+0000","last_fullsized":"2024-02-22T19:34:49.294072+0000","mapping_epoch":104,"log_start":"127'6300","ondisk_log_start":"127'6300","created":93,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4558,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20054016,"num_objects":306,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10858,"num_write_kb":357248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.256871+0000","last_change":"2024-02-22T19:33:49.804924+0000","last_active":"2024-02-22T19:34:40.256871+0000","last_peered":"2024-02-22T19:34:40.256871+0000","last_clean":"2024-02-22T19:34:40.256871+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:34:40.256871+0000","last_undegraded":"2024-02-22T19:34:40.256871+0000","last_fullsized":"2024-02-22T19:34:40.256871+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.804842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.804842+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":120,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271882+0000","last_change":"2024-02-22T19:34:01.596593+0000","last_active":"2024-02-22T19:34:40.271882+0000","last_peered":"2024-02-22T19:34:40.271882+0000","last_clean":"2024-02-22T19:34:40.271882+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:34:40.271882+0000","last_undegraded":"2024-02-22T19:34:40.271882+0000","last_fullsized":"2024-02-22T19:34:40.271882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:34:01.596491+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.596491+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":115,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271369+0000","last_change":"2024-02-22T19:34:28.519793+0000","last_active":"2024-02-22T19:34:40.271369+0000","last_peered":"2024-02-22T19:34:40.271369+0000","last_clean":"2024-02-22T19:34:40.271369+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:34:40.271369+0000","last_undegraded":"2024-02-22T19:34:40.271369+0000","last_fullsized":"2024-02-22T19:34:40.271369+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:28.519724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.519724+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":179,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.347898+0000","last_change":"2024-02-22T19:34:39.483818+0000","last_active":"2024-02-22T19:34:48.347898+0000","last_peered":"2024-02-22T19:34:48.347898+0000","last_clean":"2024-02-22T19:34:48.347898+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:34:48.347898+0000","last_undegraded":"2024-02-22T19:34:48.347898+0000","last_fullsized":"2024-02-22T19:34:48.347898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:34:39.483740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.483740+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"145'10785","reported_seq":13412,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.285186+0000","last_change":"2024-02-22T19:33:42.253020+0000","last_active":"2024-02-22T19:34:49.285186+0000","last_peered":"2024-02-22T19:34:49.285186+0000","last_clean":"2024-02-22T19:34:49.285186+0000","last_became_active":"2024-02-22T19:33:42.130070+0000","last_became_peered":"2024-02-22T19:33:42.130070+0000","last_unstale":"2024-02-22T19:34:49.285186+0000","last_undegraded":"2024-02-22T19:34:49.285186+0000","last_fullsized":"2024-02-22T19:34:49.285186+0000","mapping_epoch":134,"log_start":"130'6600","ondisk_log_start":"130'6600","created":93,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4185,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21037056,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":355392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1273,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":93,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.274611+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:34:40.274611+0000","last_peered":"2024-02-22T19:34:40.274611+0000","last_clean":"2024-02-22T19:34:40.274611+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:34:40.274611+0000","last_undegraded":"2024-02-22T19:34:40.274611+0000","last_fullsized":"2024-02-22T19:34:40.274611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272212+0000","last_change":"2024-02-22T19:33:54.606520+0000","last_active":"2024-02-22T19:34:40.272212+0000","last_peered":"2024-02-22T19:34:40.272212+0000","last_clean":"2024-02-22T19:34:40.272212+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:34:40.272212+0000","last_undegraded":"2024-02-22T19:34:40.272212+0000","last_fullsized":"2024-02-22T19:34:40.272212+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:54.606418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.606418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":128,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251704+0000","last_change":"2024-02-22T19:34:33.123549+0000","last_active":"2024-02-22T19:34:40.251704+0000","last_peered":"2024-02-22T19:34:40.251704+0000","last_clean":"2024-02-22T19:34:40.251704+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:34:40.251704+0000","last_undegraded":"2024-02-22T19:34:40.251704+0000","last_fullsized":"2024-02-22T19:34:40.251704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:33.123452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.123452+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":103,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275276+0000","last_change":"2024-02-22T19:34:28.806832+0000","last_active":"2024-02-22T19:34:40.275276+0000","last_peered":"2024-02-22T19:34:40.275276+0000","last_clean":"2024-02-22T19:34:40.275276+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:34:40.275276+0000","last_undegraded":"2024-02-22T19:34:40.275276+0000","last_fullsized":"2024-02-22T19:34:40.275276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:28.806726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.806726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"145'10929","reported_seq":10989,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.321988+0000","last_change":"2024-02-22T19:31:33.014127+0000","last_active":"2024-02-22T19:34:49.321988+0000","last_peered":"2024-02-22T19:34:49.321988+0000","last_clean":"2024-02-22T19:34:49.321988+0000","last_became_active":"2024-02-22T19:31:33.013760+0000","last_became_peered":"2024-02-22T19:31:33.013760+0000","last_unstale":"2024-02-22T19:34:49.321988+0000","last_undegraded":"2024-02-22T19:34:49.321988+0000","last_fullsized":"2024-02-22T19:34:49.321988+0000","mapping_epoch":93,"log_start":"125'6000","ondisk_log_start":"125'6000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4929,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10929,"num_write_kb":360064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:43.434131+0000","last_change":"2024-02-22T19:34:43.434105+0000","last_active":"2024-02-22T19:34:43.434131+0000","last_peered":"2024-02-22T19:34:43.434131+0000","last_clean":"2024-02-22T19:34:43.434131+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:34:43.434131+0000","last_undegraded":"2024-02-22T19:34:43.434131+0000","last_fullsized":"2024-02-22T19:34:43.434131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:43.434030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.434030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":112,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.269411+0000","last_change":"2024-02-22T19:33:48.766810+0000","last_active":"2024-02-22T19:34:40.269411+0000","last_peered":"2024-02-22T19:34:40.269411+0000","last_clean":"2024-02-22T19:34:40.269411+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:34:40.269411+0000","last_undegraded":"2024-02-22T19:34:40.269411+0000","last_fullsized":"2024-02-22T19:34:40.269411+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.766706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.766706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":119,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:46.476175+0000","last_change":"2024-02-22T19:34:46.476131+0000","last_active":"2024-02-22T19:34:46.476175+0000","last_peered":"2024-02-22T19:34:46.476175+0000","last_clean":"2024-02-22T19:34:46.476175+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:34:46.476175+0000","last_undegraded":"2024-02-22T19:34:46.476175+0000","last_fullsized":"2024-02-22T19:34:46.476175+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.476034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.476034+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":133,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.257078+0000","last_change":"2024-02-22T19:33:53.782532+0000","last_active":"2024-02-22T19:34:40.257078+0000","last_peered":"2024-02-22T19:34:40.257078+0000","last_clean":"2024-02-22T19:34:40.257078+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:34:40.257078+0000","last_undegraded":"2024-02-22T19:34:40.257078+0000","last_fullsized":"2024-02-22T19:34:40.257078+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:33:53.782465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.782465+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"145'10732","reported_seq":10797,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.253170+0000","last_change":"2024-02-22T19:34:48.023284+0000","last_active":"2024-02-22T19:34:49.253170+0000","last_peered":"2024-02-22T19:34:49.253170+0000","last_clean":"2024-02-22T19:34:49.253170+0000","last_became_active":"2024-02-22T19:31:33.015132+0000","last_became_peered":"2024-02-22T19:31:33.015132+0000","last_unstale":"2024-02-22T19:34:49.253170+0000","last_undegraded":"2024-02-22T19:34:49.253170+0000","last_fullsized":"2024-02-22T19:34:49.253170+0000","mapping_epoch":93,"log_start":"124'5400","ondisk_log_start":"124'5400","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'10656","last_scrub_stamp":"2024-02-22T19:34:48.023178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.023178+0000","log_size":5332,"ondisk_log_size":5332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10732,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:43.230809+0000","last_change":"2024-02-22T19:34:43.230772+0000","last_active":"2024-02-22T19:34:43.230809+0000","last_peered":"2024-02-22T19:34:43.230809+0000","last_clean":"2024-02-22T19:34:43.230809+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:34:43.230809+0000","last_undegraded":"2024-02-22T19:34:43.230809+0000","last_fullsized":"2024-02-22T19:34:43.230809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:43.230673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.230673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":103,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271990+0000","last_change":"2024-02-22T19:33:58.910213+0000","last_active":"2024-02-22T19:34:40.271990+0000","last_peered":"2024-02-22T19:34:40.271990+0000","last_clean":"2024-02-22T19:34:40.271990+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:34:40.271990+0000","last_undegraded":"2024-02-22T19:34:40.271990+0000","last_fullsized":"2024-02-22T19:34:40.271990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.910117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.910117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275877+0000","last_change":"2024-02-22T19:34:04.892928+0000","last_active":"2024-02-22T19:34:40.275877+0000","last_peered":"2024-02-22T19:34:40.275877+0000","last_clean":"2024-02-22T19:34:40.275877+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:34:40.275877+0000","last_undegraded":"2024-02-22T19:34:40.275877+0000","last_fullsized":"2024-02-22T19:34:40.275877+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:04.892837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:04.892837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":138,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271539+0000","last_change":"2024-02-22T19:34:30.097726+0000","last_active":"2024-02-22T19:34:40.271539+0000","last_peered":"2024-02-22T19:34:40.271539+0000","last_clean":"2024-02-22T19:34:40.271539+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:34:40.271539+0000","last_undegraded":"2024-02-22T19:34:40.271539+0000","last_fullsized":"2024-02-22T19:34:40.271539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:34:30.097667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.097667+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"145'10730","reported_seq":10795,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.205817+0000","last_change":"2024-02-22T19:34:35.036913+0000","last_active":"2024-02-22T19:34:49.205817+0000","last_peered":"2024-02-22T19:34:49.205817+0000","last_clean":"2024-02-22T19:34:49.205817+0000","last_became_active":"2024-02-22T19:31:33.013715+0000","last_became_peered":"2024-02-22T19:31:33.013715+0000","last_unstale":"2024-02-22T19:34:49.205817+0000","last_undegraded":"2024-02-22T19:34:49.205817+0000","last_fullsized":"2024-02-22T19:34:49.205817+0000","mapping_epoch":93,"log_start":"124'5300","ondisk_log_start":"124'5300","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'9940","last_scrub_stamp":"2024-02-22T19:34:35.036835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:34:35.036835+0000","log_size":5430,"ondisk_log_size":5430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20316160,"num_objects":310,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10730,"num_write_kb":353280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":98,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:46.954050+0000","last_change":"2024-02-22T19:34:46.954016+0000","last_active":"2024-02-22T19:34:46.954050+0000","last_peered":"2024-02-22T19:34:46.954050+0000","last_clean":"2024-02-22T19:34:46.954050+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:34:46.954050+0000","last_undegraded":"2024-02-22T19:34:46.954050+0000","last_fullsized":"2024-02-22T19:34:46.954050+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.953951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.953951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:47.963400+0000","last_change":"2024-02-22T19:34:47.963364+0000","last_active":"2024-02-22T19:34:47.963400+0000","last_peered":"2024-02-22T19:34:47.963400+0000","last_clean":"2024-02-22T19:34:47.963400+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:34:47.963400+0000","last_undegraded":"2024-02-22T19:34:47.963400+0000","last_fullsized":"2024-02-22T19:34:47.963400+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:47.963267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.963267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251754+0000","last_change":"2024-02-22T19:34:32.095049+0000","last_active":"2024-02-22T19:34:40.251754+0000","last_peered":"2024-02-22T19:34:40.251754+0000","last_clean":"2024-02-22T19:34:40.251754+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:34:40.251754+0000","last_undegraded":"2024-02-22T19:34:40.251754+0000","last_fullsized":"2024-02-22T19:34:40.251754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.094985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.094985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":223,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.351281+0000","last_change":"2024-02-22T19:33:46.724569+0000","last_active":"2024-02-22T19:34:48.351281+0000","last_peered":"2024-02-22T19:34:48.351281+0000","last_clean":"2024-02-22T19:34:48.351281+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:34:48.351281+0000","last_undegraded":"2024-02-22T19:34:48.351281+0000","last_fullsized":"2024-02-22T19:34:48.351281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:33:46.724462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.724462+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273350+0000","last_change":"2024-02-22T19:33:58.185263+0000","last_active":"2024-02-22T19:34:40.273350+0000","last_peered":"2024-02-22T19:34:40.273350+0000","last_clean":"2024-02-22T19:34:40.273350+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:34:40.273350+0000","last_undegraded":"2024-02-22T19:34:40.273350+0000","last_fullsized":"2024-02-22T19:34:40.273350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.185187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.185187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"145'10705","reported_seq":10765,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.309286+0000","last_change":"2024-02-22T19:31:33.008987+0000","last_active":"2024-02-22T19:34:49.309286+0000","last_peered":"2024-02-22T19:34:49.309286+0000","last_clean":"2024-02-22T19:34:49.309286+0000","last_became_active":"2024-02-22T19:31:33.008726+0000","last_became_peered":"2024-02-22T19:31:33.008726+0000","last_unstale":"2024-02-22T19:34:49.309286+0000","last_undegraded":"2024-02-22T19:34:49.309286+0000","last_fullsized":"2024-02-22T19:34:49.309286+0000","mapping_epoch":93,"log_start":"125'5800","ondisk_log_start":"125'5800","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4905,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21692416,"num_objects":331,"num_object_clones":0,"num_object_copies":993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10705,"num_write_kb":353152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272049+0000","last_change":"2024-02-22T19:34:38.529035+0000","last_active":"2024-02-22T19:34:40.272049+0000","last_peered":"2024-02-22T19:34:40.272049+0000","last_clean":"2024-02-22T19:34:40.272049+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:34:40.272049+0000","last_undegraded":"2024-02-22T19:34:40.272049+0000","last_fullsized":"2024-02-22T19:34:40.272049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:38.528933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.528933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.255245+0000","last_change":"2024-02-22T19:34:01.732261+0000","last_active":"2024-02-22T19:34:40.255245+0000","last_peered":"2024-02-22T19:34:40.255245+0000","last_clean":"2024-02-22T19:34:40.255245+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:34:40.255245+0000","last_undegraded":"2024-02-22T19:34:40.255245+0000","last_fullsized":"2024-02-22T19:34:40.255245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:01.732156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.732156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.268364+0000","last_change":"2024-02-22T19:33:44.828664+0000","last_active":"2024-02-22T19:34:40.268364+0000","last_peered":"2024-02-22T19:34:40.268364+0000","last_clean":"2024-02-22T19:34:40.268364+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:34:40.268364+0000","last_undegraded":"2024-02-22T19:34:40.268364+0000","last_fullsized":"2024-02-22T19:34:40.268364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:44.828568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.828568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":114,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.269874+0000","last_change":"2024-02-22T19:33:59.950686+0000","last_active":"2024-02-22T19:34:40.269874+0000","last_peered":"2024-02-22T19:34:40.269874+0000","last_clean":"2024-02-22T19:34:40.269874+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:34:40.269874+0000","last_undegraded":"2024-02-22T19:34:40.269874+0000","last_fullsized":"2024-02-22T19:34:40.269874+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:33:59.950586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:59.950586+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":112,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.269850+0000","last_change":"2024-02-22T19:33:46.825747+0000","last_active":"2024-02-22T19:34:40.269850+0000","last_peered":"2024-02-22T19:34:40.269850+0000","last_clean":"2024-02-22T19:34:40.269850+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:34:40.269850+0000","last_undegraded":"2024-02-22T19:34:40.269850+0000","last_fullsized":"2024-02-22T19:34:40.269850+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:46.825679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:46.825679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"145'10828","reported_seq":10888,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.309943+0000","last_change":"2024-02-22T19:31:33.014815+0000","last_active":"2024-02-22T19:34:49.309943+0000","last_peered":"2024-02-22T19:34:49.309943+0000","last_clean":"2024-02-22T19:34:49.309943+0000","last_became_active":"2024-02-22T19:31:33.014489+0000","last_became_peered":"2024-02-22T19:31:33.014489+0000","last_unstale":"2024-02-22T19:34:49.309943+0000","last_undegraded":"2024-02-22T19:34:49.309943+0000","last_fullsized":"2024-02-22T19:34:49.309943+0000","mapping_epoch":93,"log_start":"125'5900","ondisk_log_start":"125'5900","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4928,"ondisk_log_size":4928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21757952,"num_objects":332,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10828,"num_write_kb":357120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":57,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275428+0000","last_change":"2024-02-22T19:31:50.014877+0000","last_active":"2024-02-22T19:34:40.275428+0000","last_peered":"2024-02-22T19:34:40.275428+0000","last_clean":"2024-02-22T19:34:40.275428+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:34:40.275428+0000","last_undegraded":"2024-02-22T19:34:40.275428+0000","last_fullsized":"2024-02-22T19:34:40.275428+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:35.556020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:30:35.556020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:44.432580+0000","last_change":"2024-02-22T19:34:44.432542+0000","last_active":"2024-02-22T19:34:44.432580+0000","last_peered":"2024-02-22T19:34:44.432580+0000","last_clean":"2024-02-22T19:34:44.432580+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:34:44.432580+0000","last_undegraded":"2024-02-22T19:34:44.432580+0000","last_fullsized":"2024-02-22T19:34:44.432580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":108,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273751+0000","last_change":"2024-02-22T19:31:50.009641+0000","last_active":"2024-02-22T19:34:40.273751+0000","last_peered":"2024-02-22T19:34:40.273751+0000","last_clean":"2024-02-22T19:34:40.273751+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:34:40.273751+0000","last_undegraded":"2024-02-22T19:34:40.273751+0000","last_fullsized":"2024-02-22T19:34:40.273751+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:30:45.909289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:30:45.909289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":131,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.257912+0000","last_change":"2024-02-22T19:34:28.591838+0000","last_active":"2024-02-22T19:34:40.257912+0000","last_peered":"2024-02-22T19:34:40.257912+0000","last_clean":"2024-02-22T19:34:40.257912+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:34:40.257912+0000","last_undegraded":"2024-02-22T19:34:40.257912+0000","last_fullsized":"2024-02-22T19:34:40.257912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:34:28.591731+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.591731+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":110,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272320+0000","last_change":"2024-02-22T19:34:31.534714+0000","last_active":"2024-02-22T19:34:40.272320+0000","last_peered":"2024-02-22T19:34:40.272320+0000","last_clean":"2024-02-22T19:34:40.272320+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:34:40.272320+0000","last_undegraded":"2024-02-22T19:34:40.272320+0000","last_fullsized":"2024-02-22T19:34:40.272320+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.534649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.534649+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"145'10829","reported_seq":10888,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.187401+0000","last_change":"2024-02-22T19:31:33.010431+0000","last_active":"2024-02-22T19:34:49.187401+0000","last_peered":"2024-02-22T19:34:49.187401+0000","last_clean":"2024-02-22T19:34:49.187401+0000","last_became_active":"2024-02-22T19:31:33.010187+0000","last_became_peered":"2024-02-22T19:31:33.010187+0000","last_unstale":"2024-02-22T19:34:49.187401+0000","last_undegraded":"2024-02-22T19:34:49.187401+0000","last_fullsized":"2024-02-22T19:34:49.187401+0000","mapping_epoch":93,"log_start":"125'6000","ondisk_log_start":"125'6000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4829,"ondisk_log_size":4829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20512768,"num_objects":313,"num_object_clones":0,"num_object_copies":939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10829,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":110,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254325+0000","last_change":"2024-02-22T19:34:38.060600+0000","last_active":"2024-02-22T19:34:40.254325+0000","last_peered":"2024-02-22T19:34:40.254325+0000","last_clean":"2024-02-22T19:34:40.254325+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:34:40.254325+0000","last_undegraded":"2024-02-22T19:34:40.254325+0000","last_fullsized":"2024-02-22T19:34:40.254325+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:38.060515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.060515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271213+0000","last_change":"2024-02-22T19:34:06.016291+0000","last_active":"2024-02-22T19:34:40.271213+0000","last_peered":"2024-02-22T19:34:40.271213+0000","last_clean":"2024-02-22T19:34:40.271213+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:34:40.271213+0000","last_undegraded":"2024-02-22T19:34:40.271213+0000","last_fullsized":"2024-02-22T19:34:40.271213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:34:06.016195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":455,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.568484+0000","last_change":"2024-02-22T19:34:48.568447+0000","last_active":"2024-02-22T19:34:48.568484+0000","last_peered":"2024-02-22T19:34:48.568484+0000","last_clean":"2024-02-22T19:34:48.568484+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:34:48.568484+0000","last_undegraded":"2024-02-22T19:34:48.568484+0000","last_fullsized":"2024-02-22T19:34:48.568484+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:48.568336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.568336+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":342,"num_read_kb":337,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273625+0000","last_change":"2024-02-22T19:34:38.184599+0000","last_active":"2024-02-22T19:34:40.273625+0000","last_peered":"2024-02-22T19:34:40.273625+0000","last_clean":"2024-02-22T19:34:40.273625+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:34:40.273625+0000","last_undegraded":"2024-02-22T19:34:40.273625+0000","last_fullsized":"2024-02-22T19:34:40.273625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:38.184521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.184521+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254994+0000","last_change":"2024-02-22T19:34:00.713551+0000","last_active":"2024-02-22T19:34:40.254994+0000","last_peered":"2024-02-22T19:34:40.254994+0000","last_clean":"2024-02-22T19:34:40.254994+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:34:40.254994+0000","last_undegraded":"2024-02-22T19:34:40.254994+0000","last_fullsized":"2024-02-22T19:34:40.254994+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:00.713486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:00.713486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"145'10814","reported_seq":13518,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.224469+0000","last_change":"2024-02-22T19:32:07.977104+0000","last_active":"2024-02-22T19:34:49.224469+0000","last_peered":"2024-02-22T19:34:49.224469+0000","last_clean":"2024-02-22T19:34:49.224469+0000","last_became_active":"2024-02-22T19:32:07.973466+0000","last_became_peered":"2024-02-22T19:32:07.973466+0000","last_unstale":"2024-02-22T19:34:49.224469+0000","last_undegraded":"2024-02-22T19:34:49.224469+0000","last_fullsized":"2024-02-22T19:34:49.224469+0000","mapping_epoch":105,"log_start":"127'6200","ondisk_log_start":"127'6200","created":93,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4614,"ondisk_log_size":4614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21495808,"num_objects":328,"num_object_clones":0,"num_object_copies":984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10814,"num_write_kb":356544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":86048768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254319+0000","last_change":"2024-02-22T19:34:36.003355+0000","last_active":"2024-02-22T19:34:40.254319+0000","last_peered":"2024-02-22T19:34:40.254319+0000","last_clean":"2024-02-22T19:34:40.254319+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:34:40.254319+0000","last_undegraded":"2024-02-22T19:34:40.254319+0000","last_fullsized":"2024-02-22T19:34:40.254319+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.003297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.003297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":116,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:41.031000+0000","last_change":"2024-02-22T19:34:41.030962+0000","last_active":"2024-02-22T19:34:41.031000+0000","last_peered":"2024-02-22T19:34:41.031000+0000","last_clean":"2024-02-22T19:34:41.031000+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:34:41.031000+0000","last_undegraded":"2024-02-22T19:34:41.031000+0000","last_fullsized":"2024-02-22T19:34:41.031000+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.030860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.030860+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":166,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.271421+0000","last_change":"2024-02-22T19:34:02.072893+0000","last_active":"2024-02-22T19:34:40.271421+0000","last_peered":"2024-02-22T19:34:40.271421+0000","last_clean":"2024-02-22T19:34:40.271421+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:34:40.271421+0000","last_undegraded":"2024-02-22T19:34:40.271421+0000","last_fullsized":"2024-02-22T19:34:40.271421+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:34:02.072818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:02.072818+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":111,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.257145+0000","last_change":"2024-02-22T19:34:04.782120+0000","last_active":"2024-02-22T19:34:40.257145+0000","last_peered":"2024-02-22T19:34:40.257145+0000","last_clean":"2024-02-22T19:34:40.257145+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:34:40.257145+0000","last_undegraded":"2024-02-22T19:34:40.257145+0000","last_fullsized":"2024-02-22T19:34:40.257145+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:04.782052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:04.782052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"145'10688","reported_seq":10748,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.194545+0000","last_change":"2024-02-22T19:31:33.012064+0000","last_active":"2024-02-22T19:34:49.194545+0000","last_peered":"2024-02-22T19:34:49.194545+0000","last_clean":"2024-02-22T19:34:49.194545+0000","last_became_active":"2024-02-22T19:31:33.011750+0000","last_became_peered":"2024-02-22T19:31:33.011750+0000","last_unstale":"2024-02-22T19:34:49.194545+0000","last_undegraded":"2024-02-22T19:34:49.194545+0000","last_fullsized":"2024-02-22T19:34:49.194545+0000","mapping_epoch":93,"log_start":"127'6200","ondisk_log_start":"127'6200","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4488,"ondisk_log_size":4488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24248320,"num_objects":370,"num_object_clones":0,"num_object_copies":1110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10688,"num_write_kb":353856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.270745+0000","last_change":"2024-02-22T19:33:49.035679+0000","last_active":"2024-02-22T19:34:40.270745+0000","last_peered":"2024-02-22T19:34:40.270745+0000","last_clean":"2024-02-22T19:34:40.270745+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:34:40.270745+0000","last_undegraded":"2024-02-22T19:34:40.270745+0000","last_fullsized":"2024-02-22T19:34:40.270745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:49.035615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:33:49.035615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.274701+0000","last_change":"2024-02-22T19:34:33.941965+0000","last_active":"2024-02-22T19:34:40.274701+0000","last_peered":"2024-02-22T19:34:40.274701+0000","last_clean":"2024-02-22T19:34:40.274701+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:34:40.274701+0000","last_undegraded":"2024-02-22T19:34:40.274701+0000","last_fullsized":"2024-02-22T19:34:40.274701+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:33.941905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.941905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":141,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254503+0000","last_change":"2024-02-22T19:33:44.757231+0000","last_active":"2024-02-22T19:34:40.254503+0000","last_peered":"2024-02-22T19:34:40.254503+0000","last_clean":"2024-02-22T19:34:40.254503+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:34:40.254503+0000","last_undegraded":"2024-02-22T19:34:40.254503+0000","last_fullsized":"2024-02-22T19:34:40.254503+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:33:44.757114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:44.757114+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":192,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.345278+0000","last_change":"2024-02-22T19:34:31.086582+0000","last_active":"2024-02-22T19:34:48.345278+0000","last_peered":"2024-02-22T19:34:48.345278+0000","last_clean":"2024-02-22T19:34:48.345278+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:34:48.345278+0000","last_undegraded":"2024-02-22T19:34:48.345278+0000","last_fullsized":"2024-02-22T19:34:48.345278+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:31.086518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.086518+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"145'10843","reported_seq":10903,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.308565+0000","last_change":"2024-02-22T19:31:33.012817+0000","last_active":"2024-02-22T19:34:49.308565+0000","last_peered":"2024-02-22T19:34:49.308565+0000","last_clean":"2024-02-22T19:34:49.308565+0000","last_became_active":"2024-02-22T19:31:33.012470+0000","last_became_peered":"2024-02-22T19:31:33.012470+0000","last_unstale":"2024-02-22T19:34:49.308565+0000","last_undegraded":"2024-02-22T19:34:49.308565+0000","last_fullsized":"2024-02-22T19:34:49.308565+0000","mapping_epoch":93,"log_start":"130'6700","ondisk_log_start":"130'6700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4143,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20905984,"num_objects":319,"num_object_clones":0,"num_object_copies":957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10843,"num_write_kb":357184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":99,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272399+0000","last_change":"2024-02-22T19:34:32.514011+0000","last_active":"2024-02-22T19:34:40.272399+0000","last_peered":"2024-02-22T19:34:40.272399+0000","last_clean":"2024-02-22T19:34:40.272399+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:34:40.272399+0000","last_undegraded":"2024-02-22T19:34:40.272399+0000","last_fullsized":"2024-02-22T19:34:40.272399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.513915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.513915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273704+0000","last_change":"2024-02-22T19:34:34.158347+0000","last_active":"2024-02-22T19:34:40.273704+0000","last_peered":"2024-02-22T19:34:40.273704+0000","last_clean":"2024-02-22T19:34:40.273704+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:34:40.273704+0000","last_undegraded":"2024-02-22T19:34:40.273704+0000","last_fullsized":"2024-02-22T19:34:40.273704+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:34.158242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:34.158242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":153,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251587+0000","last_change":"2024-02-22T19:31:50.644540+0000","last_active":"2024-02-22T19:34:40.251587+0000","last_peered":"2024-02-22T19:34:40.251587+0000","last_clean":"2024-02-22T19:34:40.251587+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:34:40.251587+0000","last_undegraded":"2024-02-22T19:34:40.251587+0000","last_fullsized":"2024-02-22T19:34:40.251587+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:30:54.022453+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":173,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.345811+0000","last_change":"2024-02-22T19:30:53.878090+0000","last_active":"2024-02-22T19:34:48.345811+0000","last_peered":"2024-02-22T19:34:48.345811+0000","last_clean":"2024-02-22T19:34:48.345811+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:34:48.345811+0000","last_undegraded":"2024-02-22T19:34:48.345811+0000","last_fullsized":"2024-02-22T19:34:48.345811+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:30:53.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:30:53.878001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"145'11054","reported_seq":11114,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.331243+0000","last_change":"2024-02-22T19:31:33.013118+0000","last_active":"2024-02-22T19:34:49.331243+0000","last_peered":"2024-02-22T19:34:49.331243+0000","last_clean":"2024-02-22T19:34:49.331243+0000","last_became_active":"2024-02-22T19:31:33.012002+0000","last_became_peered":"2024-02-22T19:31:33.012002+0000","last_unstale":"2024-02-22T19:34:49.331243+0000","last_undegraded":"2024-02-22T19:34:49.331243+0000","last_fullsized":"2024-02-22T19:34:49.331243+0000","mapping_epoch":93,"log_start":"130'6900","ondisk_log_start":"130'6900","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4154,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11054,"num_write_kb":364160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251807+0000","last_change":"2024-02-22T19:34:01.893979+0000","last_active":"2024-02-22T19:34:40.251807+0000","last_peered":"2024-02-22T19:34:40.251807+0000","last_clean":"2024-02-22T19:34:40.251807+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:34:40.251807+0000","last_undegraded":"2024-02-22T19:34:40.251807+0000","last_fullsized":"2024-02-22T19:34:40.251807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:01.893910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.893910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":104,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275281+0000","last_change":"2024-02-22T19:34:40.000683+0000","last_active":"2024-02-22T19:34:40.275281+0000","last_peered":"2024-02-22T19:34:40.275281+0000","last_clean":"2024-02-22T19:34:40.275281+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:34:40.275281+0000","last_undegraded":"2024-02-22T19:34:40.275281+0000","last_fullsized":"2024-02-22T19:34:40.275281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:40.000583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:40.000583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":136,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273037+0000","last_change":"2024-02-22T19:33:54.749170+0000","last_active":"2024-02-22T19:34:40.273037+0000","last_peered":"2024-02-22T19:34:40.273037+0000","last_clean":"2024-02-22T19:34:40.273037+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:34:40.273037+0000","last_undegraded":"2024-02-22T19:34:40.273037+0000","last_fullsized":"2024-02-22T19:34:40.273037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:33:54.749061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.749061+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":103,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251391+0000","last_change":"2024-02-22T19:31:26.190739+0000","last_active":"2024-02-22T19:34:40.251391+0000","last_peered":"2024-02-22T19:34:40.251391+0000","last_clean":"2024-02-22T19:34:40.251391+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:34:40.251391+0000","last_undegraded":"2024-02-22T19:34:40.251391+0000","last_fullsized":"2024-02-22T19:34:40.251391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:26.190632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:26.190632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"145'10946","reported_seq":11006,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.197783+0000","last_change":"2024-02-22T19:31:33.012589+0000","last_active":"2024-02-22T19:34:49.197783+0000","last_peered":"2024-02-22T19:34:49.197783+0000","last_clean":"2024-02-22T19:34:49.197783+0000","last_became_active":"2024-02-22T19:31:33.012188+0000","last_became_peered":"2024-02-22T19:31:33.012188+0000","last_unstale":"2024-02-22T19:34:49.197783+0000","last_undegraded":"2024-02-22T19:34:49.197783+0000","last_fullsized":"2024-02-22T19:34:49.197783+0000","mapping_epoch":93,"log_start":"124'5500","ondisk_log_start":"124'5500","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5446,"ondisk_log_size":5446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23199744,"num_objects":354,"num_object_clones":0,"num_object_copies":1062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10946,"num_write_kb":361600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":57,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.255738+0000","last_change":"2024-02-22T19:31:50.009247+0000","last_active":"2024-02-22T19:34:40.255738+0000","last_peered":"2024-02-22T19:34:40.255738+0000","last_clean":"2024-02-22T19:34:40.255738+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:34:40.255738+0000","last_undegraded":"2024-02-22T19:34:40.255738+0000","last_fullsized":"2024-02-22T19:34:40.255738+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:25.682652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:25.682652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":194,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.332148+0000","last_change":"2024-02-22T19:34:43.126256+0000","last_active":"2024-02-22T19:34:48.332148+0000","last_peered":"2024-02-22T19:34:48.332148+0000","last_clean":"2024-02-22T19:34:48.332148+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:34:48.332148+0000","last_undegraded":"2024-02-22T19:34:48.332148+0000","last_fullsized":"2024-02-22T19:34:48.332148+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":104,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.272989+0000","last_change":"2024-02-22T19:31:14.073980+0000","last_active":"2024-02-22T19:34:40.272989+0000","last_peered":"2024-02-22T19:34:40.272989+0000","last_clean":"2024-02-22T19:34:40.272989+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:34:40.272989+0000","last_undegraded":"2024-02-22T19:34:40.272989+0000","last_fullsized":"2024-02-22T19:34:40.272989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:14.073908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:31:14.073908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":97,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.269732+0000","last_change":"2024-02-22T19:33:43.827188+0000","last_active":"2024-02-22T19:34:40.269732+0000","last_peered":"2024-02-22T19:34:40.269732+0000","last_clean":"2024-02-22T19:34:40.269732+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:34:40.269732+0000","last_undegraded":"2024-02-22T19:34:40.269732+0000","last_fullsized":"2024-02-22T19:34:40.269732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:33:43.827087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:43.827087+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"145'11075","reported_seq":11141,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.196157+0000","last_change":"2024-02-22T19:34:41.736201+0000","last_active":"2024-02-22T19:34:49.196157+0000","last_peered":"2024-02-22T19:34:49.196157+0000","last_clean":"2024-02-22T19:34:49.196157+0000","last_became_active":"2024-02-22T19:31:33.011438+0000","last_became_peered":"2024-02-22T19:31:33.011438+0000","last_unstale":"2024-02-22T19:34:49.196157+0000","last_undegraded":"2024-02-22T19:34:49.196157+0000","last_fullsized":"2024-02-22T19:34:49.196157+0000","mapping_epoch":93,"log_start":"125'5700","ondisk_log_start":"125'5700","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'10613","last_scrub_stamp":"2024-02-22T19:34:41.736089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.736089+0000","log_size":5375,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23461912,"num_objects":359,"num_object_clones":0,"num_object_copies":1077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11075,"num_write_kb":365825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":102,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.275622+0000","last_change":"2024-02-22T19:31:50.006646+0000","last_active":"2024-02-22T19:34:40.275622+0000","last_peered":"2024-02-22T19:34:40.275622+0000","last_clean":"2024-02-22T19:34:40.275622+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:34:40.275622+0000","last_undegraded":"2024-02-22T19:34:40.275622+0000","last_fullsized":"2024-02-22T19:34:40.275622+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:09.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:09.064536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":62,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.253219+0000","last_change":"2024-02-22T19:34:39.082294+0000","last_active":"2024-02-22T19:34:40.253219+0000","last_peered":"2024-02-22T19:34:40.253219+0000","last_clean":"2024-02-22T19:34:40.253219+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:34:40.253219+0000","last_undegraded":"2024-02-22T19:34:40.253219+0000","last_fullsized":"2024-02-22T19:34:40.253219+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:39.082190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.082190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":119,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.251468+0000","last_change":"2024-02-22T19:33:48.898875+0000","last_active":"2024-02-22T19:34:40.251468+0000","last_peered":"2024-02-22T19:34:40.251468+0000","last_clean":"2024-02-22T19:34:40.251468+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:34:40.251468+0000","last_undegraded":"2024-02-22T19:34:40.251468+0000","last_fullsized":"2024-02-22T19:34:40.251468+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.898801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.898801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":124,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.252654+0000","last_change":"2024-02-22T19:33:51.953460+0000","last_active":"2024-02-22T19:34:40.252654+0000","last_peered":"2024-02-22T19:34:40.252654+0000","last_clean":"2024-02-22T19:34:40.252654+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:34:40.252654+0000","last_undegraded":"2024-02-22T19:34:40.252654+0000","last_fullsized":"2024-02-22T19:34:40.252654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:51.953387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.953387+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"145'10504","reported_seq":13085,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.324602+0000","last_change":"2024-02-22T19:33:40.144966+0000","last_active":"2024-02-22T19:34:49.324602+0000","last_peered":"2024-02-22T19:34:49.324602+0000","last_clean":"2024-02-22T19:34:49.324602+0000","last_became_active":"2024-02-22T19:33:40.096849+0000","last_became_peered":"2024-02-22T19:33:40.096849+0000","last_unstale":"2024-02-22T19:34:49.324602+0000","last_undegraded":"2024-02-22T19:34:49.324602+0000","last_fullsized":"2024-02-22T19:34:49.324602+0000","mapping_epoch":132,"log_start":"124'5200","ondisk_log_start":"124'5200","created":93,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":5304,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10504,"num_write_kb":346304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1254,"num_bytes_recovered":81657856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":101,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.270055+0000","last_change":"2024-02-22T19:33:52.892034+0000","last_active":"2024-02-22T19:34:40.270055+0000","last_peered":"2024-02-22T19:34:40.270055+0000","last_clean":"2024-02-22T19:34:40.270055+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:34:40.270055+0000","last_undegraded":"2024-02-22T19:34:40.270055+0000","last_fullsized":"2024-02-22T19:34:40.270055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:52.891927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.891927+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":103,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.254786+0000","last_change":"2024-02-22T19:33:57.772428+0000","last_active":"2024-02-22T19:34:40.254786+0000","last_peered":"2024-02-22T19:34:40.254786+0000","last_clean":"2024-02-22T19:34:40.254786+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:34:40.254786+0000","last_undegraded":"2024-02-22T19:34:40.254786+0000","last_fullsized":"2024-02-22T19:34:40.254786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.772326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.772326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":104,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.256952+0000","last_change":"2024-02-22T19:33:51.803060+0000","last_active":"2024-02-22T19:34:40.256952+0000","last_peered":"2024-02-22T19:34:40.256952+0000","last_clean":"2024-02-22T19:34:40.256952+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:34:40.256952+0000","last_undegraded":"2024-02-22T19:34:40.256952+0000","last_fullsized":"2024-02-22T19:34:40.256952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:51.802966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.802966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":128,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.252361+0000","last_change":"2024-02-22T19:30:46.953153+0000","last_active":"2024-02-22T19:34:40.252361+0000","last_peered":"2024-02-22T19:34:40.252361+0000","last_clean":"2024-02-22T19:34:40.252361+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:34:40.252361+0000","last_undegraded":"2024-02-22T19:34:40.252361+0000","last_fullsized":"2024-02-22T19:34:40.252361+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:30:46.953061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:46.953061+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"145'10850","reported_seq":10910,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:49.248641+0000","last_change":"2024-02-22T19:31:33.014534+0000","last_active":"2024-02-22T19:34:49.248641+0000","last_peered":"2024-02-22T19:34:49.248641+0000","last_clean":"2024-02-22T19:34:49.248641+0000","last_became_active":"2024-02-22T19:31:33.014231+0000","last_became_peered":"2024-02-22T19:31:33.014231+0000","last_unstale":"2024-02-22T19:34:49.248641+0000","last_undegraded":"2024-02-22T19:34:49.248641+0000","last_fullsized":"2024-02-22T19:34:49.248641+0000","mapping_epoch":93,"log_start":"125'6000","ondisk_log_start":"125'6000","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:31.998251+0000","last_clean_scrub_stamp":"2024-02-22T19:31:31.998251+0000","log_size":4850,"ondisk_log_size":4850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":357504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":98,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.252869+0000","last_change":"2024-02-22T19:30:55.020084+0000","last_active":"2024-02-22T19:34:40.252869+0000","last_peered":"2024-02-22T19:34:40.252869+0000","last_clean":"2024-02-22T19:34:40.252869+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:34:40.252869+0000","last_undegraded":"2024-02-22T19:34:40.252869+0000","last_fullsized":"2024-02-22T19:34:40.252869+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:30:55.019989+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":194,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-02-22T19:34:48.351909+0000","last_change":"2024-02-22T19:33:42.757859+0000","last_active":"2024-02-22T19:34:48.351909+0000","last_peered":"2024-02-22T19:34:48.351909+0000","last_clean":"2024-02-22T19:34:48.351909+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:34:48.351909+0000","last_undegraded":"2024-02-22T19:34:48.351909+0000","last_fullsized":"2024-02-22T19:34:48.351909+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:33:42.757755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.757755+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":113,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.273783+0000","last_change":"2024-02-22T19:34:28.943926+0000","last_active":"2024-02-22T19:34:40.273783+0000","last_peered":"2024-02-22T19:34:40.273783+0000","last_clean":"2024-02-22T19:34:40.273783+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:34:40.273783+0000","last_undegraded":"2024-02-22T19:34:40.273783+0000","last_fullsized":"2024-02-22T19:34:40.273783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:34:28.943835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.943835+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":115,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-02-22T19:34:40.253281+0000","last_change":"2024-02-22T19:30:50.936786+0000","last_active":"2024-02-22T19:34:40.253281+0000","last_peered":"2024-02-22T19:34:40.253281+0000","last_clean":"2024-02-22T19:34:40.253281+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:34:40.253281+0000","last_undegraded":"2024-02-22T19:34:40.253281+0000","last_fullsized":"2024-02-22T19:34:40.253281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:30:50.936709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:30:50.936709+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":343343128,"num_objects":5240,"num_object_clones":0,"num_object_copies":15720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173170,"num_write_kb":5709057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5115,"num_bytes_recovered":332922880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1029910528,"data_stored":1029898312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77470,"ondisk_log_size":77470,"up":48,"acting":48,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":342,"num_read_kb":337,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266761,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":415724,"kb_used_data":118500,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93329428,"statfs":{"total":95995035648,"available":95569334272,"internally_reserved":0,"allocated":121344000,"data_stored":120964342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.702,"15min":0.94999999999999996},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":6.6580000000000004,"5min":42.597999999999999,"15min":42.597999999999999},"last":0.24299999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.377,"15min":0.81100000000000005},"min":{"1min":0.23599999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.3579999999999997,"5min":42.155999999999999,"15min":42.155999999999999},"last":1.996}]},{"osd":1,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.385,"15min":0.84699999999999998},"min":{"1min":0.27000000000000002,"5min":0.248,"15min":0.248},"max":{"1min":8.0679999999999996,"5min":38.188000000000002,"15min":38.188000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.962,"5min":1.5920000000000001,"15min":0.89300000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":7.7220000000000004,"5min":37.938000000000002,"15min":37.938000000000002},"last":2.048}]},{"osd":2,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.329,"15min":0.81599999999999995},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.2220000000000004,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.548,"15min":0.91600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.827,"5min":41.722000000000001,"15min":41.722000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":2.0310000000000001,"5min":1.615,"15min":0.93999999999999995},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":8.0570000000000004,"5min":41.457000000000001,"15min":41.457000000000001},"last":5.5309999999999997},{"interface":"front","average":{"1min":1.7589999999999999,"5min":1.431,"15min":0.89500000000000002},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1840000000000002,"5min":41.311999999999998,"15min":41.311999999999998},"last":0.375}]},{"osd":4,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.27200000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":3.4780000000000002,"5min":3.4780000000000002,"15min":3.4780000000000002},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":2.0139999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.65400000000000003,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.226,"5min":12.199999999999999,"15min":12.199999999999999},"last":2.0310000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.4430000000000001,"5min":12.067,"15min":12.067},"last":0.53800000000000003}]}]},{"osd":6,"up_from":45,"seq":193273528398,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":435348,"kb_used_data":125836,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93309804,"statfs":{"total":95995035648,"available":95549239296,"internally_reserved":0,"allocated":128856064,"data_stored":128500074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.901,"5min":1.2789999999999999,"15min":0.77200000000000002},"min":{"1min":0.223,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":6.9939999999999998,"5min":17.885000000000002,"15min":17.885000000000002},"last":5.8010000000000002},{"interface":"front","average":{"1min":1.034,"5min":1.216,"15min":0.748},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.9849999999999999,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":1.226}]},{"osd":1,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":0.96299999999999997,"15min":0.69199999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.8090000000000002,"5min":11.301,"15min":11.301},"last":1.091},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.94499999999999995,"15min":0.66300000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.9580000000000002,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":2.6930000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.288,"15min":0.78600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":5.9950000000000001,"5min":12.859999999999999,"15min":12.859999999999999},"last":6.0049999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.03,"15min":0.71099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":3.6840000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.6339999999999999,"15min":0.90500000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.4950000000000001,"5min":38.762,"15min":38.762},"last":0.35799999999999998},{"interface":"front","average":{"1min":2.0880000000000001,"5min":2.0939999999999999,"15min":1.0680000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7070000000000007,"5min":35.612000000000002,"15min":35.612000000000002},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.26000000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.27,"5min":1.583,"15min":1.583},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.436,"15min":0.42699999999999999},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.67500000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.28399999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.442},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.056,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.315},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.9570000000000001,"5min":3.593,"15min":3.593},"last":1.0620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":457924,"kb_used_data":156540,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93287228,"statfs":{"total":95995035648,"available":95526121472,"internally_reserved":0,"allocated":160296960,"data_stored":159893194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":4.0599999999999996,"5min":1.4630000000000001,"15min":0.71299999999999997},"min":{"1min":0.20699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":1.3049999999999999},{"interface":"front","average":{"1min":3.5409999999999999,"5min":1.5509999999999999,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":9.5,"5min":11.352,"15min":11.352},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:34:04 2024","interfaces":[{"interface":"back","average":{"1min":4.1959999999999997,"5min":2.073,"15min":0.95599999999999996},"min":{"1min":0.224,"5min":0.222,"15min":0.222},"max":{"1min":16.503,"5min":16.503,"15min":16.503},"last":0.41499999999999998},{"interface":"front","average":{"1min":2.4380000000000002,"5min":1.732,"15min":0.84499999999999997},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":11.715,"5min":13.461,"15min":13.461},"last":11.284000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:34:35 2024","interfaces":[{"interface":"back","average":{"1min":2.1070000000000002,"5min":2.3330000000000002,"15min":1.087},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":5.6120000000000001,"5min":25.145,"15min":25.145},"last":5.7969999999999997},{"interface":"front","average":{"1min":2.8140000000000001,"5min":2.359,"15min":1.131},"min":{"1min":0.32700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":12.154999999999999,"5min":22.25,"15min":22.25},"last":2.2530000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:33:53 2024","interfaces":[{"interface":"back","average":{"1min":2.4260000000000002,"5min":1.319,"15min":0.85499999999999998},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.9580000000000002,"5min":15.542999999999999,"15min":15.542999999999999},"last":0.498},{"interface":"front","average":{"1min":2.1890000000000001,"5min":0.996,"15min":0.71699999999999997},"min":{"1min":0.314,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":3.7210000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:34:11 2024","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.405,"15min":0.873},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.4859999999999998,"5min":15.587999999999999,"15min":15.587999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.772,"5min":1.254,"15min":0.80300000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.878}]},{"osd":6,"last update":"Thu Feb 22 19:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.1830000000000001,"15min":0.75700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.8730000000000002,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":3.7839999999999998},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5209999999999999,"15min":0.90100000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":8.1880000000000006,"5min":15.651999999999999,"15min":15.651999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:34:04 2024","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.2490000000000001,"15min":0.76600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.0820000000000001,"15min":0.72399999999999998},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":6.766,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":3.762}]}]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":480812,"kb_used_data":167204,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93264340,"statfs":{"total":95995035648,"available":95502684160,"internally_reserved":0,"allocated":171216896,"data_stored":170837612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":3.2610000000000001,"5min":1.262,"15min":0.66000000000000003},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":12.305,"5min":12.305,"15min":12.305},"last":4.7990000000000004},{"interface":"front","average":{"1min":2.6440000000000001,"5min":1.327,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":13.013999999999999,"5min":13.013999999999999,"15min":13.013999999999999},"last":2.2370000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:34:10 2024","interfaces":[{"interface":"back","average":{"1min":3.0939999999999999,"5min":1.8740000000000001,"15min":0.85899999999999999},"min":{"1min":0.27600000000000002,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":8.0709999999999997,"5min":19.927,"15min":19.927},"last":6.702},{"interface":"front","average":{"1min":3.23,"5min":1.8919999999999999,"15min":0.88100000000000001},"min":{"1min":0.38600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.710000000000001,"5min":17.907,"15min":17.907},"last":2.8740000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:34:28 2024","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.6479999999999999,"15min":0.83699999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":7.4640000000000004,"5min":13.06,"15min":13.06},"last":1.4039999999999999},{"interface":"front","average":{"1min":3.407,"5min":2.1349999999999998,"15min":1.0089999999999999},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":8.9629999999999992,"5min":22.936,"15min":22.936},"last":5.944}]},{"osd":4,"last update":"Thu Feb 22 19:33:49 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":0.89500000000000002,"15min":0.68100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":2.7850000000000001},{"interface":"front","average":{"1min":2.649,"5min":1.423,"15min":0.86799999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":13.874000000000001,"5min":13.874000000000001,"15min":13.874000000000001},"last":3.319}]},{"osd":5,"last update":"Thu Feb 22 19:34:16 2024","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":1.6060000000000001,"15min":0.91300000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33100000000000002},"max":{"1min":7.6420000000000003,"5min":13.903,"15min":13.903},"last":3.3580000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.016,"15min":0.71699999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.9509999999999996,"5min":6.9509999999999996,"15min":6.9509999999999996},"last":2.8439999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:34:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.125,"15min":0.77100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.7210000000000001,"5min":6.984,"15min":6.984},"last":2.7450000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.2450000000000001,"15min":0.81100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":7.5410000000000004,"5min":7.5410000000000004,"15min":7.5410000000000004},"last":0.35799999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:34:03 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.024,"15min":0.72399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":6.9199999999999999,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":2.8130000000000002},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.137,"15min":0.77200000000000002},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":7.2720000000000002,"5min":7.2720000000000002,"15min":7.2720000000000002},"last":0.39300000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":476000,"kb_used_data":178712,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93269152,"statfs":{"total":95995035648,"available":95507611648,"internally_reserved":0,"allocated":183001088,"data_stored":182633014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":1.996,"5min":1.4910000000000001,"15min":0.745},"min":{"1min":0.26300000000000001,"5min":0.186,"15min":0.186},"max":{"1min":5.3280000000000003,"5min":16.356999999999999,"15min":16.356999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":2.2200000000000002,"5min":1.7809999999999999,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":5.1399999999999997,"5min":18.059999999999999,"15min":18.059999999999999},"last":16.545000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":2.907,"5min":1.454,"15min":0.74099999999999999},"min":{"1min":0.24399999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.9829999999999997,"5min":10.183,"15min":10.183},"last":5.1630000000000003},{"interface":"front","average":{"1min":2.3490000000000002,"5min":1.1160000000000001,"15min":0.64500000000000002},"min":{"1min":0.24299999999999999,"5min":0.193,"15min":0.193},"max":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"last":4}]},{"osd":3,"last update":"Thu Feb 22 19:34:36 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":2.0750000000000002,"15min":1.0009999999999999},"min":{"1min":0.36299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":12.656000000000001,"5min":30.071000000000002,"15min":30.071000000000002},"last":3.9550000000000001},{"interface":"front","average":{"1min":2.3490000000000002,"5min":2.3700000000000001,"15min":1.087},"min":{"1min":0.29299999999999998,"5min":0.192,"15min":0.192},"max":{"1min":10.223000000000001,"5min":32.119,"15min":32.119},"last":5.3570000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:33:53 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.1319999999999999,"15min":0.75800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.226,"5min":6.226,"15min":6.226},"last":5.0540000000000003},{"interface":"front","average":{"1min":1.119,"5min":0.78100000000000003,"15min":0.65300000000000002},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.3860000000000001,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.379}]},{"osd":5,"last update":"Thu Feb 22 19:34:12 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.88700000000000001,"15min":0.71199999999999997},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.5350000000000001,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.46300000000000002},{"interface":"front","average":{"1min":2.3610000000000002,"5min":1.333,"15min":0.83999999999999997},"min":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.4320000000000004,"5min":6.4320000000000004,"15min":6.4320000000000004},"last":5.0910000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:34:40 2024","interfaces":[{"interface":"back","average":{"1min":2.4060000000000001,"5min":1.5189999999999999,"15min":0.90900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4109999999999996,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":5.1150000000000002},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.0249999999999999,"15min":0.76200000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5309999999999997,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":0.40699999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:34:06 2024","interfaces":[{"interface":"back","average":{"1min":2.1110000000000002,"5min":1.254,"15min":0.80400000000000005},"min":{"1min":0.372,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.3650000000000002,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":5.1390000000000002},{"interface":"front","average":{"1min":1.367,"5min":0.89200000000000002,"15min":0.69299999999999995},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.4850000000000003,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.43099999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":529204,"kb_used_data":223724,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93215948,"statfs":{"total":95995035648,"available":95453130752,"internally_reserved":0,"allocated":229093376,"data_stored":228706232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:29 2024","interfaces":[{"interface":"back","average":{"1min":2.6280000000000001,"5min":1.978,"15min":0.94999999999999996},"min":{"1min":0.25900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":8.4990000000000006,"5min":12.929,"15min":12.929},"last":0.77400000000000002},{"interface":"front","average":{"1min":3.1480000000000001,"5min":2.2810000000000001,"15min":1.0820000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":13.839,"5min":14.831,"15min":14.831},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:34:29 2024","interfaces":[{"interface":"back","average":{"1min":3.7919999999999998,"5min":2.3940000000000001,"15min":1.111},"min":{"1min":0.23499999999999999,"5min":0.22,"15min":0.22},"max":{"1min":18.259,"5min":18.259,"15min":18.259},"last":0.61799999999999999},{"interface":"front","average":{"1min":2.7949999999999999,"5min":1.8129999999999999,"15min":0.90800000000000003},"min":{"1min":0.28499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":13.154,"5min":13.154,"15min":13.154},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Feb 22 19:34:29 2024","interfaces":[{"interface":"back","average":{"1min":3.5760000000000001,"5min":2.3290000000000002,"15min":1.1040000000000001},"min":{"1min":0.309,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":16.271000000000001,"5min":16.271000000000001,"15min":16.271000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":2.5089999999999999,"5min":2.1280000000000001,"15min":1.0249999999999999},"min":{"1min":0.28999999999999998,"5min":0.217,"15min":0.217},"max":{"1min":9.6780000000000008,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.127}]},{"osd":4,"last update":"Thu Feb 22 19:33:50 2024","interfaces":[{"interface":"back","average":{"1min":2.7719999999999998,"5min":1.4199999999999999,"15min":0.89000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.699,"5min":12.699,"15min":12.699},"last":0.40999999999999998},{"interface":"front","average":{"1min":2.4489999999999998,"5min":1.325,"15min":0.84599999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":11.366,"5min":11.366,"15min":11.366},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:34:18 2024","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":1.484,"15min":0.88800000000000001},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":12.879,"5min":12.879,"15min":12.879},"last":1.0980000000000001},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.6000000000000001,"15min":0.92100000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":13.707000000000001,"5min":13.707000000000001,"15min":13.707000000000001},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Feb 22 19:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7030000000000001,"5min":1.5860000000000001,"15min":0.92000000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.4199999999999999,"5min":12.898999999999999,"15min":12.898999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":2.6589999999999998,"5min":1.861,"15min":1.054},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":13.803000000000001,"5min":13.803000000000001,"15min":13.803000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Feb 22 19:33:56 2024","interfaces":[{"interface":"back","average":{"1min":1.897,"5min":1.3420000000000001,"15min":0.83099999999999996},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":11.348000000000001,"5min":11.348000000000001,"15min":11.348000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":2.7709999999999999,"5min":1.454,"15min":0.89000000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":12.846,"5min":12.846,"15min":12.846},"last":1.0649999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":413936,"kb_used_data":112616,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93331216,"statfs":{"total":95995035648,"available":95571165184,"internally_reserved":0,"allocated":115318784,"data_stored":114935516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.4830000000000001,"5min":1.2749999999999999,"15min":0.76900000000000002},"min":{"1min":0.313,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.635999999999999,"5min":24.009,"15min":24.009},"last":4.3029999999999999},{"interface":"front","average":{"1min":1.8480000000000001,"5min":1.2450000000000001,"15min":0.77900000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.7229999999999999,"5min":24.263000000000002,"15min":24.263000000000002},"last":2.7919999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9810000000000001,"5min":0.96699999999999997,"15min":0.68899999999999995},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":6.3029999999999999,"5min":6.3029999999999999,"15min":6.3029999999999999},"last":4.0599999999999996},{"interface":"front","average":{"1min":1.5369999999999999,"5min":0.80700000000000005,"15min":0.60399999999999998},"min":{"1min":0.28000000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":12.445,"5min":12.445,"15min":12.445},"last":3.9980000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.107,"15min":0.75},"min":{"1min":0.38,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.8839999999999999,"5min":23.082000000000001,"15min":23.082000000000001},"last":2.2149999999999999},{"interface":"front","average":{"1min":2.3450000000000002,"5min":1.3200000000000001,"15min":0.80100000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.992,"5min":23.138999999999999,"15min":23.138999999999999},"last":3.0089999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.9199999999999999,"5min":1.2230000000000001,"15min":0.77700000000000002},"min":{"1min":0.34300000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":2.702},{"interface":"front","average":{"1min":2.6400000000000001,"5min":1.157,"15min":0.73899999999999999},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":8.9909999999999997,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":2.6400000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.42299999999999999},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":2.1120000000000001,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.495,"15min":0.435},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.212,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.35599999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.46899999999999997},"min":{"1min":0.25900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.0499999999999998,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.48099999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.32500000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.0880000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.373}]}]},{"osd":5,"up_from":0,"seq":167503724627,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":169611264,"data_stored":169607192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149622784,"data_stored":149618712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":165675008,"data_stored":165675008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":214499328,"data_stored":214499328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":110100480,"data_stored":110100480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851968,"data_stored":851968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":111939584,"data_stored":111935512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":107610112,"data_stored":107610112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:34:52.646 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [5, 3, 4] 2024-02-22T19:34:52.646 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.1', '5', '3', '4'] 2024-02-22T19:34:52.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pg-upmap 2.1 5 3 4 2024-02-22T19:34:52.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:50.948547+0000 mgr.y (mgr.14535) 229 : audit [DBG] from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:34:52.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:50.958650+0000 osd.3 (osd.3) 97 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:34:52.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:50.967429+0000 osd.3 (osd.3) 98 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:34:52.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:51.352751+0000 mgr.y (mgr.14535) 230 : cluster [DBG] pgmap v248: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:34:53.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:51.974502+0000 osd.1 (osd.1) 77 : cluster [DBG] 6.d scrub starts 2024-02-22T19:34:53.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:51.981436+0000 osd.1 (osd.1) 78 : cluster [DBG] 6.d scrub ok 2024-02-22T19:34:53.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:52.271785+0000 osd.0 (osd.0) 97 : cluster [DBG] 7.4 scrub starts 2024-02-22T19:34:53.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:52.316904+0000 osd.0 (osd.0) 98 : cluster [DBG] 7.4 scrub ok 2024-02-22T19:34:53.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:53.351073+0000 mon.a (mon.0) 917 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:53.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:51.974502+0000 osd.1 (osd.1) 77 : cluster [DBG] 6.d scrub starts 2024-02-22T19:34:53.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:51.981436+0000 osd.1 (osd.1) 78 : cluster [DBG] 6.d scrub ok 2024-02-22T19:34:53.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:52.271785+0000 osd.0 (osd.0) 97 : cluster [DBG] 7.4 scrub starts 2024-02-22T19:34:53.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:52.316904+0000 osd.0 (osd.0) 98 : cluster [DBG] 7.4 scrub ok 2024-02-22T19:34:53.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:53.351073+0000 mon.a (mon.0) 917 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:51.974502+0000 osd.1 (osd.1) 77 : cluster [DBG] 6.d scrub starts 2024-02-22T19:34:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:51.981436+0000 osd.1 (osd.1) 78 : cluster [DBG] 6.d scrub ok 2024-02-22T19:34:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:52.271785+0000 osd.0 (osd.0) 97 : cluster [DBG] 7.4 scrub starts 2024-02-22T19:34:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:52.316904+0000 osd.0 (osd.0) 98 : cluster [DBG] 7.4 scrub ok 2024-02-22T19:34:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:53.351073+0000 mon.a (mon.0) 917 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:34:54.144 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:34:54.323 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:34:54.352 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:34:54.407 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:34:54.456 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:34:54.503 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:34:54.535 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:34:54.570 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:34:54.628 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:54.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:54.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:52.577304+0000 osd.7 (osd.7) 97 : cluster [DBG] 7.5 deep-scrub starts 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:53.128816+0000 osd.7 (osd.7) 98 : cluster [DBG] 7.5 deep-scrub ok 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:53.274829+0000 osd.0 (osd.0) 99 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:53.279656+0000 osd.0 (osd.0) 100 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:53.355275+0000 mgr.y (mgr.14535) 231 : cluster [DBG] pgmap v249: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 564 op/s 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:53.488757+0000 osd.6 (osd.6) 53 : cluster [DBG] 6.a scrub starts 2024-02-22T19:34:54.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:53.494515+0000 osd.6 (osd.6) 54 : cluster [DBG] 6.a scrub ok 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:54.231993+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.15.53:0/2941209466' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:52.577304+0000 osd.7 (osd.7) 97 : cluster [DBG] 7.5 deep-scrub starts 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:53.128816+0000 osd.7 (osd.7) 98 : cluster [DBG] 7.5 deep-scrub ok 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:53.274829+0000 osd.0 (osd.0) 99 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:53.279656+0000 osd.0 (osd.0) 100 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:53.355275+0000 mgr.y (mgr.14535) 231 : cluster [DBG] pgmap v249: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 564 op/s 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:53.488757+0000 osd.6 (osd.6) 53 : cluster [DBG] 6.a scrub starts 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:53.494515+0000 osd.6 (osd.6) 54 : cluster [DBG] 6.a scrub ok 2024-02-22T19:34:54.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:54.231993+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.15.53:0/2941209466' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:54.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:54.766 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:52.577304+0000 osd.7 (osd.7) 97 : cluster [DBG] 7.5 deep-scrub starts 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:53.128816+0000 osd.7 (osd.7) 98 : cluster [DBG] 7.5 deep-scrub ok 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:53.274829+0000 osd.0 (osd.0) 99 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:53.279656+0000 osd.0 (osd.0) 100 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:53.355275+0000 mgr.y (mgr.14535) 231 : cluster [DBG] pgmap v249: 148 pgs: 148 active+clean; 327 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 564 op/s 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:53.488757+0000 osd.6 (osd.6) 53 : cluster [DBG] 6.a scrub starts 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:53.494515+0000 osd.6 (osd.6) 54 : cluster [DBG] 6.a scrub ok 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:54.231993+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.15.53:0/2941209466' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:34:54.767 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:34:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:34:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:54.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:54.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:54.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:34:54.976 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:55.095 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:34:55.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:55.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:34:55.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:55.273 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:55.402 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Removed 89204 objects 2024-02-22T19:34:55.402 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Clean up completed and total clean up time :104.63 2024-02-22T19:34:55.450 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:34:55.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:55.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:34:55.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:55.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:54.017613+0000 osd.4 (osd.4) 87 : cluster [DBG] 6.e scrub starts 2024-02-22T19:34:55.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:54.019266+0000 osd.4 (osd.4) 88 : cluster [DBG] 6.e scrub ok 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:54.279778+0000 osd.0 (osd.0) 101 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:54.279838+0000 osd.0 (osd.0) 102 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:54.407921+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e147: 8 total, 8 up, 7 in 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:54.533276+0000 mgr.y (mgr.14535) 232 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:54.017613+0000 osd.4 (osd.4) 87 : cluster [DBG] 6.e scrub starts 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:54.019266+0000 osd.4 (osd.4) 88 : cluster [DBG] 6.e scrub ok 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:54.279778+0000 osd.0 (osd.0) 101 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:54.279838+0000 osd.0 (osd.0) 102 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:54.407921+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e147: 8 total, 8 up, 7 in 2024-02-22T19:34:55.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:54.533276+0000 mgr.y (mgr.14535) 232 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:55.599 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-02-22T19:34:55.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-02-22T19:34:55.620 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:55.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:55.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:55.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:54.017613+0000 osd.4 (osd.4) 87 : cluster [DBG] 6.e scrub starts 2024-02-22T19:34:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:54.019266+0000 osd.4 (osd.4) 88 : cluster [DBG] 6.e scrub ok 2024-02-22T19:34:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:54.279778+0000 osd.0 (osd.0) 101 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:54.279838+0000 osd.0 (osd.0) 102 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:54.407921+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e147: 8 total, 8 up, 7 in 2024-02-22T19:34:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:54.533276+0000 mgr.y (mgr.14535) 232 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:34:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:56.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:56.006 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:56.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:56.136 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:56.208 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:34:56.208 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20843 7.0 7:0b664233:::benchmark_data_smithi053_107660_object20842:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.941998869, 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472188600000001, 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20843 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005334+0000", 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4664999999999998e-05 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005359+0000", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1853e-05 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005371+0000", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005378+0000", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016187800000000001 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005539+0000", 2024-02-22T19:34:56.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1446e-05 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005551+0000", 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98239731299999999 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987948+0000", 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4382000000000001e-05 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987963+0000", 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.054996391 2024-02-22T19:34:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.042959+0000", 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041492100000000001 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.043374+0000", 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016716699999999999 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.043541+0000", 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003835649 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047377+0000", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011540490000000001 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048531+0000", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033136839999999999 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051844+0000", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1493999999999999e-05 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051896+0000", 2024-02-22T19:34:56.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065935400000000002 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052555+0000", 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472188600000001 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20848 7.0 7:00506f5c:::benchmark_data_smithi053_107660_object20847:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.929681862, 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0355339840000002, 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20848 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017651+0000", 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6801999999999999e-05 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017678+0000", 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1832e-05 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017690+0000", 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9899999999999997e-06 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017698+0000", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0209e-05 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017748+0000", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017755+0000", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97023272699999996 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987988+0000", 2024-02-22T19:34:56.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0849999999999997e-06 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987996+0000", 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0570709490000001 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045067+0000", 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016401100000000001 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045231+0000", 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158806 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045390+0000", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031485200000000001 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048538+0000", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040701499999999998 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052608+0000", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2032e-05 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052650+0000", 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2859000000000003e-05 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052703+0000", 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000484162 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053187+0000", 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0355339840000002 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.221 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20867 7.b 7:d5ccad63:::benchmark_data_smithi053_107660_object20866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.92258402600001, 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0358723360000002, 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20867 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024749+0000", 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8314e-05 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024777+0000", 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953000000000001e-05 2024-02-22T19:34:56.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024792+0000", 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8079999999999994e-06 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024801+0000", 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9221000000000001e-05 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024860+0000", 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0496000000000001e-05 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024871+0000", 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.964611467 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989482+0000", 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0557000000000001e-05 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989493+0000", 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06140933 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050902+0000", 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015795699999999999 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051060+0000", 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000234997 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051295+0000", 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029240099999999999 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054219+0000", 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001188099 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055407+0000", 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042592480000000002 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059666+0000", 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0272000000000002e-05 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059727+0000", 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089686099999999999 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060624+0000", 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0358723360000002 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20879 7.b 7:d83f7aa7:::benchmark_data_smithi053_107660_object20878:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:34:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.91641850900001, 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0357336730000002, 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20879 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030915+0000", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4876e-05 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030940+0000", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7564e-05 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030957+0000", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5020000000000003e-06 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030965+0000", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2354999999999997e-05 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.031007+0000", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5809999999999999e-06 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.031015+0000", 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.958500719 2024-02-22T19:34:56.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989516+0000", 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5569999999999998e-06 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989523+0000", 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635326220000001 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053056+0000", 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014994699999999999 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053206+0000", 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169927 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053376+0000", 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049563699999999999 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058332+0000", 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079005799999999995 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059122+0000", 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066417830000000001 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.065764+0000", 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0451999999999997e-05 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.065814+0000", 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083617100000000003 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066650+0000", 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0357336730000002 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20885 7.0 7:074624f9:::benchmark_data_smithi053_107660_object20884:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.911418548, 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0216042769999998, 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20885 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035915+0000", 2024-02-22T19:34:56.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7532999999999999e-05 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035942+0000", 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6866000000000001e-05 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035959+0000", 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3299999999999999e-06 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035968+0000", 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0296000000000006e-05 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036038+0000", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2227e-05 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036050+0000", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95196719699999999 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988017+0000", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5079999999999999e-06 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988025+0000", 2024-02-22T19:34:56.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0581795979999999 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046204+0000", 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137909 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046342+0000", 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017299200000000001 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046515+0000", 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046217719999999997 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051137+0000", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003247869 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054385+0000", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022844919999999999 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.056669+0000", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0172e-05 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.056720+0000", 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080141399999999997 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057521+0000", 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0216042769999998 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20888 7.0 7:0390592f:::benchmark_data_smithi053_107660_object20887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.90669482800001, 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.017606019, 2024-02-22T19:34:56.241 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20888 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:56.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040639+0000", 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7110999999999998e-05 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040666+0000", 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4090999999999999e-05 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040680+0000", 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2790000000000004e-06 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040686+0000", 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4023999999999996e-05 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040750+0000", 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2262000000000001e-05 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040763+0000", 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.9472855 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988048+0000", 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8350000000000004e-06 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988056+0000", 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.059385043 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047441+0000", 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012125600000000001 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047562+0000", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017584799999999999 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047738+0000", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054622200000000003 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053200+0000", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001243353 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054444+0000", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031954219999999998 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057639+0000", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4193000000000001e-05 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057693+0000", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055305899999999995 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058246+0000", 2024-02-22T19:34:56.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.017606019 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20889 7.b 7:deffbca2:::benchmark_data_smithi053_107660_object20888:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.90663673500001, 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0267241290000002, 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20889 2024-02-22T19:34:56.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040698+0000", 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7917000000000001e-05 2024-02-22T19:34:56.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040716+0000", 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6410000000000001e-06 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040721+0000", 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0340000000000001e-06 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040724+0000", 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3059000000000001e-05 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040777+0000", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3380000000000001e-05 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040790+0000", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94875468299999999 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989545+0000", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.374e-06 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989552+0000", 2024-02-22T19:34:56.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0647073950000001 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054259+0000", 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142843 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054402+0000", 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018314000000000001 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054585+0000", 2024-02-22T19:34:56.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037587060000000001 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058344+0000", 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002519851 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060864+0000", 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058517530000000003 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066716+0000", 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0093999999999998e-05 2024-02-22T19:34:56.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066776+0000", 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064683000000000004 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.067423+0000", 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0267241290000002 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:701 5.15 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:34:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 185.792525332, 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86739943600000002, 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 701 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154809+0000", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6779999999999999e-05 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154825+0000", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3336000000000002e-05 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154849+0000", 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6969999999999998e-06 2024-02-22T19:34:56.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154854+0000", 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3915999999999998e-05 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154898+0000", 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.413e-06 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154906+0000", 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85827278699999998 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013179+0000", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030207999999999998 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013481+0000", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003549445 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017030+0000", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3808999999999997e-05 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017114+0000", 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005095502 2024-02-22T19:34:56.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.022209+0000", 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86739943600000002 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:22458 7.0 7:0ef0dd72:::benchmark_data_smithi053_107660_object22457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e101)", 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:52.600209+0000", 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 182.34712651000001, 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045517037000000003, 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22458 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600209+0000", 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600209+0000", 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600207+0000", 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4549000000000002e-05 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600231+0000", 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2116999999999999e-05 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600244+0000", 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.011e-06 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.600252+0000", 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036691229999999998 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.603921+0000", 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017633300000000001 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.604097+0000", 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000192058 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.604289+0000", 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021584540000000002 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.606448+0000", 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078427310000000004 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.614290+0000", 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030662571999999999 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.644953+0000", 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9848e-05 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.645013+0000", 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071306099999999997 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:52.645726+0000", 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045517037000000003 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:75672 7.b 7:dfce425d:::benchmark_data_smithi053_107660_object75671:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:51.974833+0000", 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.972502459, 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046512193, 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 75672 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974833+0000", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974833+0000", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974831+0000", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9658999999999999e-05 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974890+0000", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3321e-05 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974904+0000", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7440000000000007e-06 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974911+0000", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031372499999999998 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.978049+0000", 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015122900000000001 2024-02-22T19:34:56.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.978200+0000", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017699400000000001 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.978377+0000", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014419505000000001 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.992796+0000", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0441e-05 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.992817+0000", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028441225000000001 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.021258+0000", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7938000000000001e-05 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.021306+0000", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8973000000000001e-05 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.021345+0000", 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046512193 2024-02-22T19:34:56.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:75674 7.b 7:da5239eb:::benchmark_data_smithi053_107660_object75673:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:51.974925+0000", 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.97241002, 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046526128999999999, 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 75674 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974925+0000", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974925+0000", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974925+0000", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999999e-05 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974967+0000", 2024-02-22T19:34:56.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974977+0000", 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4100000000000001e-06 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.974981+0000", 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058516979999999998 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.980833+0000", 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000165088 2024-02-22T19:34:56.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.980998+0000", 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0093169080000000005 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.990315+0000", 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032628890000000002 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.993578+0000", 2024-02-22T19:34:56.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001250426 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:51.994829+0000", 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026552215000000001 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.021381+0000", 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8050000000000003e-05 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.021419+0000", 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.26e-05 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:52.021451+0000", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046526128999999999 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:78287 7.b 7:de68060c:::benchmark_data_smithi053_107660_object78286:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e121)", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:55.458523+0000", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 119.488812704, 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051227560999999998, 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 78287 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458523+0000", 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458523+0000", 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458522+0000", 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650999999999998e-05 2024-02-22T19:34:56.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458569+0000", 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0439999999999998e-06 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458576+0000", 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5739999999999997e-06 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458580+0000", 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002837642 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.461418+0000", 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136694 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.461555+0000", 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018096400000000001 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.461736+0000", 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078453040000000009 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.469581+0000", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000231277 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.469812+0000", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.039854672000000001 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.509667+0000", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6045999999999999e-05 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.509713+0000", 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7239e-05 2024-02-22T19:34:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.509750+0000", 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051227560999999998 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:78288 7.b 7:dff66ff4:::benchmark_data_smithi053_107660_object78287:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e121)", 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:32:55.458585+0000", 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 119.4887504, 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051268188999999999, 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 78288 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458585+0000", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458585+0000", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458584+0000", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7580999999999998e-05 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458622+0000", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.631e-06 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458628+0000", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9489999999999998e-06 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.458632+0000", 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0082083810000000007 2024-02-22T19:34:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.466840+0000", 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012967 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.466970+0000", 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017253 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.467142+0000", 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004021602 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.471164+0000", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023018729999999999 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.473466+0000", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036319140999999999 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.509785+0000", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8346000000000002e-05 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.509823+0000", 2024-02-22T19:34:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9972999999999999e-05 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:32:55.509853+0000", 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051268188999999999 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92126 7.b 7:d62d0696:::benchmark_data_smithi053_107660_object2919:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.362120+0000", 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.58521516099999, 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059459304999999997, 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92126 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362120+0000", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362120+0000", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362118+0000", 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5909999999999998e-06 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362126+0000", 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2028000000000001e-05 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362138+0000", 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8569999999999997e-06 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362145+0000", 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023871999999999999 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.364532+0000", 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118275 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.364650+0000", 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012999600000000001 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.364780+0000", 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065808530000000002 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371361+0000", 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078305499999999995 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.372144+0000", 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049336468000000001 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421481+0000", 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5105000000000002e-05 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421536+0000", 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3606000000000002e-05 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421579+0000", 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059459304999999997 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92127 7.b 7:daef70d8:::benchmark_data_smithi053_107660_object2920:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.362158+0000", 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.585177202, 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059550767999999997, 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92127 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362158+0000", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362158+0000", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362158+0000", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362160+0000", 2024-02-22T19:34:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1499999999999998e-06 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362166+0000", 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8070000000000001e-06 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.362169+0000", 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048768099999999997 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367046+0000", 2024-02-22T19:34:56.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124838 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367171+0000", 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012241300000000001 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.367293+0000", 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041482159999999997 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371442+0000", 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070585099999999996 2024-02-22T19:34:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.372148+0000", 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049470879000000002 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421618+0000", 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9913999999999998e-05 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421658+0000", 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0494999999999997e-05 2024-02-22T19:34:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.421709+0000", 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059550767999999997 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92133 7.0 7:02ac9df4:::benchmark_data_smithi053_107660_object2926:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.371598+0000", 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.57573752099999, 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054091436, 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92133 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371598+0000", 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371598+0000", 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371596+0000", 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8290000000000008e-06 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371604+0000", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6569999999999997e-06 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371612+0000", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1479999999999999e-06 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371616+0000", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5250999999999998e-05 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371661+0000", 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000113173 2024-02-22T19:34:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371774+0000", 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125294 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371900+0000", 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042051989999999997 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376105+0000", 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033113119999999998 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.379416+0000", 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046177476000000002 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425593+0000", 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3491e-05 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425647+0000", 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2191000000000003e-05 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425689+0000", 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054091436 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92134 7.0 7:0abb5213:::benchmark_data_smithi053_107660_object2927:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.371627+0000", 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.575708117, 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054169565000000003, 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92134 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371627+0000", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371627+0000", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371627+0000", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371629+0000", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4020000000000002e-06 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371632+0000", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4159999999999998e-06 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.371635+0000", 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002643641 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374278+0000", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000122836 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374401+0000", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015393699999999999 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374555+0000", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037701649999999998 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.378325+0000", 2024-02-22T19:34:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001129676 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.379455+0000", 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046270128000000001 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425725+0000", 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9104999999999999e-05 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425764+0000", 2024-02-22T19:34:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2360000000000002e-05 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425797+0000", 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054169565000000003 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:92137 7.0 7:0027a054:::benchmark_data_smithi053_107660_object2930:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:14.374348+0000", 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.57298689, 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051545022000000003, 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 92137 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374348+0000", 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374348+0000", 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374347+0000", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4130000000000003e-06 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374353+0000", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1134999999999999e-05 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374365+0000", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5119999999999997e-06 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.374371+0000", 2024-02-22T19:34:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001757623 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376129+0000", 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010336999999999999 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376232+0000", 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011993999999999999 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.376352+0000", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020245369999999999 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.378377+0000", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011513529999999999 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.379528+0000", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046300133 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425828+0000", 2024-02-22T19:34:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2586999999999998e-05 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425861+0000", 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2746000000000001e-05 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:14.425893+0000", 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.051545022000000003 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:98149 7.0 7:06db0591:::benchmark_data_smithi053_107660_object8942:head [delete] snapc 0=[] ondisk+write+known_if_redirected e126)", 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:22.203944+0000", 2024-02-22T19:34:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 92.743390785000003, 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045276072000000001, 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98149 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.203944+0000", 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.203944+0000", 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.203943+0000", 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3930000000000001e-06 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.203948+0000", 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4559999999999997e-06 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.203954+0000", 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.203959+0000", 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071863999999999999 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.211145+0000", 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011865400000000001 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.211264+0000", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136107 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.211400+0000", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050253429999999998 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.216425+0000", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024842402 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.241268+0000", 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078653679999999993 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.249133+0000", 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6745000000000001e-05 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.249180+0000", 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0871999999999998e-05 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.249221+0000", 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045276072000000001 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:98155 7.0 7:023cc653:::benchmark_data_smithi053_107660_object8948:head [delete] snapc 0=[] ondisk+write+known_if_redirected e126)", 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:22.206674+0000", 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 92.740661204000006, 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045618203000000003, 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98155 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.206674+0000", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.206674+0000", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.206673+0000", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9410000000000001e-06 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:34:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.206680+0000", 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1834000000000001e-05 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.206692+0000", 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2560000000000002e-06 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.206700+0000", 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069507170000000004 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.213651+0000", 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011497000000000001 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.213766+0000", 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031080300000000001 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.214076+0000", 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002623766 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.216700+0000", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024597192 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.241297+0000", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010907536000000001 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.252205+0000", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8121999999999998e-05 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:34:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.252253+0000", 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9276999999999997e-05 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:22.252292+0000", 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045618203000000003 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:34:56.325 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:34:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:34:56.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:55.355732+0000 mgr.y (mgr.14535) 233 : cluster [DBG] pgmap v251: 148 pgs: 148 active+clean; 58 MiB data, 2.3 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-02-22T19:34:56.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:55.457013+0000 osd.6 (osd.6) 55 : cluster [DBG] 3.a scrub starts 2024-02-22T19:34:56.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:55.459909+0000 osd.6 (osd.6) 56 : cluster [DBG] 3.a scrub ok 2024-02-22T19:34:56.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:55.355732+0000 mgr.y (mgr.14535) 233 : cluster [DBG] pgmap v251: 148 pgs: 148 active+clean; 58 MiB data, 2.3 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-02-22T19:34:56.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:55.457013+0000 osd.6 (osd.6) 55 : cluster [DBG] 3.a scrub starts 2024-02-22T19:34:56.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:55.459909+0000 osd.6 (osd.6) 56 : cluster [DBG] 3.a scrub ok 2024-02-22T19:34:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:55.355732+0000 mgr.y (mgr.14535) 233 : cluster [DBG] pgmap v251: 148 pgs: 148 active+clean; 58 MiB data, 2.3 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-02-22T19:34:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:55.457013+0000 osd.6 (osd.6) 55 : cluster [DBG] 3.a scrub starts 2024-02-22T19:34:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:55.459909+0000 osd.6 (osd.6) 56 : cluster [DBG] 3.a scrub ok 2024-02-22T19:34:57.386 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:57.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:57.389 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:57.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:57.524 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:57.545 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:57.548 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:57.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:57.649 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:57.677 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:55.920662+0000 osd.1 (osd.1) 79 : cluster [DBG] 7.b scrub starts 2024-02-22T19:34:57.677 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:55.927886+0000 osd.1 (osd.1) 80 : cluster [DBG] 7.b scrub ok 2024-02-22T19:34:57.677 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:56.104107+0000 osd.4 (osd.4) 89 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:34:57.678 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:56.107554+0000 osd.4 (osd.4) 90 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:34:57.680 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:57.700 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:57.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:57.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:57.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:55.920662+0000 osd.1 (osd.1) 79 : cluster [DBG] 7.b scrub starts 2024-02-22T19:34:57.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:55.927886+0000 osd.1 (osd.1) 80 : cluster [DBG] 7.b scrub ok 2024-02-22T19:34:57.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:56.104107+0000 osd.4 (osd.4) 89 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:34:57.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:56.107554+0000 osd.4 (osd.4) 90 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:34:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:55.920662+0000 osd.1 (osd.1) 79 : cluster [DBG] 7.b scrub starts 2024-02-22T19:34:57.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:55.927886+0000 osd.1 (osd.1) 80 : cluster [DBG] 7.b scrub ok 2024-02-22T19:34:57.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:56.104107+0000 osd.4 (osd.4) 89 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:34:57.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:56.107554+0000 osd.4 (osd.4) 90 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:34:57.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:57.854 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:57.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:57.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:57.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:34:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:58.007 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:34:58.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:58.010 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:34:58.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:34:58.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:34:58.140 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:58.264 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:34:58.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:58.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:34:58.368 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:34:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:34:58.425 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:58.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:34:58.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:34:58.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:57.142184+0000 osd.4 (osd.4) 91 : cluster [DBG] 4.d scrub starts 2024-02-22T19:34:58.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:57.145709+0000 osd.4 (osd.4) 92 : cluster [DBG] 4.d scrub ok 2024-02-22T19:34:58.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:57.356199+0000 mgr.y (mgr.14535) 234 : cluster [DBG] pgmap v252: 148 pgs: 148 active+clean; 58 MiB data, 2.3 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 538 op/s 2024-02-22T19:34:58.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.011672+0000 osd.5 (osd.5) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:58.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.011726+0000 osd.5 (osd.5) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:57.142184+0000 osd.4 (osd.4) 91 : cluster [DBG] 4.d scrub starts 2024-02-22T19:34:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:57.145709+0000 osd.4 (osd.4) 92 : cluster [DBG] 4.d scrub ok 2024-02-22T19:34:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:57.356199+0000 mgr.y (mgr.14535) 234 : cluster [DBG] pgmap v252: 148 pgs: 148 active+clean; 58 MiB data, 2.3 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 538 op/s 2024-02-22T19:34:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.011672+0000 osd.5 (osd.5) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.011726+0000 osd.5 (osd.5) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:58.558 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:58.580 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:34:58.582 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:58.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:34:58.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:34:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:34:58.756 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:34:58.757 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:34:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:57.142184+0000 osd.4 (osd.4) 91 : cluster [DBG] 4.d scrub starts 2024-02-22T19:34:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:57.145709+0000 osd.4 (osd.4) 92 : cluster [DBG] 4.d scrub ok 2024-02-22T19:34:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:57.356199+0000 mgr.y (mgr.14535) 234 : cluster [DBG] pgmap v252: 148 pgs: 148 active+clean; 58 MiB data, 2.3 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 538 op/s 2024-02-22T19:34:58.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.011672+0000 osd.5 (osd.5) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:34:58.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.011726+0000 osd.5 (osd.5) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:34:58.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:34:58.887 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:34:59.440 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:34:59.440 INFO:teuthology.orchestra.run.smithi053.stderr:set 2.1 pg_upmap mapping to [5,3,4] 2024-02-22T19:34:59.559 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:34:59] "GET /metrics HTTP/1.1" 200 175629 "" "Prometheus/2.33.4" 2024-02-22T19:34:59.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:57.923707+0000 osd.1 (osd.1) 81 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:34:59.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:57.925248+0000 osd.1 (osd.1) 82 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:34:59.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.162029+0000 osd.3 (osd.3) 99 : cluster [DBG] 3.18 deep-scrub starts 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.164950+0000 osd.3 (osd.3) 100 : cluster [DBG] 3.18 deep-scrub ok 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.430825+0000 mon.a (mon.0) 920 : cluster [DBG] osdmap e148: 8 total, 8 up, 7 in 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:59.156996+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.53:0/3355005443' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]: dispatch 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:59.157386+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]: dispatch 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:59.372470+0000 mon.a (mon.0) 922 : audit [INF] from='client.? 172.21.15.53:0/57391201' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:57.923707+0000 osd.1 (osd.1) 81 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:57.925248+0000 osd.1 (osd.1) 82 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.162029+0000 osd.3 (osd.3) 99 : cluster [DBG] 3.18 deep-scrub starts 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.164950+0000 osd.3 (osd.3) 100 : cluster [DBG] 3.18 deep-scrub ok 2024-02-22T19:34:59.560 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.430825+0000 mon.a (mon.0) 920 : cluster [DBG] osdmap e148: 8 total, 8 up, 7 in 2024-02-22T19:34:59.561 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:59.156996+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.53:0/3355005443' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]: dispatch 2024-02-22T19:34:59.561 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:59.157386+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]: dispatch 2024-02-22T19:34:59.561 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:34:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:59.372470+0000 mon.a (mon.0) 922 : audit [INF] from='client.? 172.21.15.53:0/57391201' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:34:59.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:34:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:57.923707+0000 osd.1 (osd.1) 81 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:34:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:57.925248+0000 osd.1 (osd.1) 82 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:34:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.162029+0000 osd.3 (osd.3) 99 : cluster [DBG] 3.18 deep-scrub starts 2024-02-22T19:34:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.164950+0000 osd.3 (osd.3) 100 : cluster [DBG] 3.18 deep-scrub ok 2024-02-22T19:34:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.430825+0000 mon.a (mon.0) 920 : cluster [DBG] osdmap e148: 8 total, 8 up, 7 in 2024-02-22T19:34:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:59.156996+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.53:0/3355005443' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]: dispatch 2024-02-22T19:34:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:59.157386+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]: dispatch 2024-02-22T19:34:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:34:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:59.372470+0000 mon.a (mon.0) 922 : audit [INF] from='client.? 172.21.15.53:0/57391201' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:00.036 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:00.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:00.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:00.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:00.168 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:00.725 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.944195+0000 osd.1 (osd.1) 83 : cluster [DBG] 7.0 scrub starts 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:58.962368+0000 osd.1 (osd.1) 84 : cluster [DBG] 7.0 scrub ok 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.138515+0000 osd.4 (osd.4) 93 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.141994+0000 osd.4 (osd.4) 94 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.268755+0000 osd.0 (osd.0) 103 : cluster [DBG] 7.f scrub starts 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.274973+0000 osd.0 (osd.0) 104 : cluster [DBG] 7.f scrub ok 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.356718+0000 mgr.y (mgr.14535) 235 : cluster [DBG] pgmap v254: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 655 op/s 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.429105+0000 mon.a (mon.0) 923 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:59.436243+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]': finished 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:34:59.436337+0000 mon.a (mon.0) 925 : audit [INF] from='client.? 172.21.15.53:0/57391201' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.436408+0000 mon.a (mon.0) 926 : cluster [DBG] osdmap e149: 8 total, 8 up, 7 in 2024-02-22T19:35:00.726 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.944195+0000 osd.1 (osd.1) 83 : cluster [DBG] 7.0 scrub starts 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:58.962368+0000 osd.1 (osd.1) 84 : cluster [DBG] 7.0 scrub ok 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.138515+0000 osd.4 (osd.4) 93 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.141994+0000 osd.4 (osd.4) 94 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.268755+0000 osd.0 (osd.0) 103 : cluster [DBG] 7.f scrub starts 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.274973+0000 osd.0 (osd.0) 104 : cluster [DBG] 7.f scrub ok 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.356718+0000 mgr.y (mgr.14535) 235 : cluster [DBG] pgmap v254: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 655 op/s 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.429105+0000 mon.a (mon.0) 923 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:59.436243+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]': finished 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:34:59.436337+0000 mon.a (mon.0) 925 : audit [INF] from='client.? 172.21.15.53:0/57391201' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:35:00.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.436408+0000 mon.a (mon.0) 926 : cluster [DBG] osdmap e149: 8 total, 8 up, 7 in 2024-02-22T19:35:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.944195+0000 osd.1 (osd.1) 83 : cluster [DBG] 7.0 scrub starts 2024-02-22T19:35:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:58.962368+0000 osd.1 (osd.1) 84 : cluster [DBG] 7.0 scrub ok 2024-02-22T19:35:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.138515+0000 osd.4 (osd.4) 93 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:35:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.141994+0000 osd.4 (osd.4) 94 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:35:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.268755+0000 osd.0 (osd.0) 103 : cluster [DBG] 7.f scrub starts 2024-02-22T19:35:00.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.274973+0000 osd.0 (osd.0) 104 : cluster [DBG] 7.f scrub ok 2024-02-22T19:35:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.356718+0000 mgr.y (mgr.14535) 235 : cluster [DBG] pgmap v254: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 655 op/s 2024-02-22T19:35:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.429105+0000 mon.a (mon.0) 923 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:59.436243+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.1", "id": [5, 3, 4]}]': finished 2024-02-22T19:35:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:34:59.436337+0000 mon.a (mon.0) 925 : audit [INF] from='client.? 172.21.15.53:0/57391201' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:35:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.436408+0000 mon.a (mon.0) 926 : cluster [DBG] osdmap e149: 8 total, 8 up, 7 in 2024-02-22T19:35:01.619 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:01.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:01.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:01.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:01.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:34:59.786903+0000 osd.2 (osd.2) 65 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:35:01.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:34:59.786903+0000 osd.2 (osd.2) 65 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:35:01.760 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:34:59.786903+0000 osd.2 (osd.2) 65 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:35:02.312 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:02.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:02.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:02.325 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:02.325 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:02.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:02.450 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:02.473 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:02.475 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:02.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:01.285926+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:01.285987+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:01.357298+0000 mgr.y (mgr.14535) 236 : cluster [DBG] pgmap v256: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 736 B/s rd, 0 B/s wr, 134 op/s 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:01.439979+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e150: 8 total, 8 up, 7 in 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:01.285926+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:01.285987+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:01.357298+0000 mgr.y (mgr.14535) 236 : cluster [DBG] pgmap v256: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 736 B/s rd, 0 B/s wr, 134 op/s 2024-02-22T19:35:02.575 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:01.439979+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e150: 8 total, 8 up, 7 in 2024-02-22T19:35:02.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:02.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:02.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:02.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:02.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:35:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:01.285926+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:01.285987+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:01.357298+0000 mgr.y (mgr.14535) 236 : cluster [DBG] pgmap v256: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 736 B/s rd, 0 B/s wr, 134 op/s 2024-02-22T19:35:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:01.439979+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e150: 8 total, 8 up, 7 in 2024-02-22T19:35:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:02.783 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:02.783 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:02.884 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:02.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:02.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:03.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:03.065 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:03.265 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:03.265 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:03.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:03.396 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:03.417 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:03.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:03.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:03.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:03.569 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:03.572 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:03.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:03.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:03.701 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:03.721 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:03.724 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:03.724 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:03.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:03.853 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:03.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:03.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:03.976 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:04.004 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:04.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:04.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:04.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:04.263 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:04.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:04.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:04.489 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_0' removed 2024-02-22T19:35:04.516 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:03.357660+0000 mgr.y (mgr.14535) 237 : cluster [DBG] pgmap v258: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 155 op/s 2024-02-22T19:35:04.516 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:04.096539+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.53:0/3966131374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:35:04.516 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:04.096931+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:35:04.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:03.357660+0000 mgr.y (mgr.14535) 237 : cluster [DBG] pgmap v258: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 155 op/s 2024-02-22T19:35:04.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:04.096539+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.53:0/3966131374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:35:04.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:04.096931+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:35:04.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:04.604 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:04.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:35:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:35:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:03.357660+0000 mgr.y (mgr.14535) 237 : cluster [DBG] pgmap v258: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 155 op/s 2024-02-22T19:35:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:04.096539+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.53:0/3966131374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:35:04.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:04.096931+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:35:04.853 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:35:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:35:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:05.453 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:35:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:35:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-02-22T19:35:05.453 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:35:05.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:04.442528+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:35:05.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:04.442608+0000 mon.a (mon.0) 930 : cluster [DBG] osdmap e151: 8 total, 8 up, 7 in 2024-02-22T19:35:05.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:04.543269+0000 mgr.y (mgr.14535) 238 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:04.442528+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:35:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:04.442608+0000 mon.a (mon.0) 930 : cluster [DBG] osdmap e151: 8 total, 8 up, 7 in 2024-02-22T19:35:05.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:04.543269+0000 mgr.y (mgr.14535) 238 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:04.442528+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:35:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:04.442608+0000 mon.a (mon.0) 930 : cluster [DBG] osdmap e151: 8 total, 8 up, 7 in 2024-02-22T19:35:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:04.543269+0000 mgr.y (mgr.14535) 238 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:06.093 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:35:06.093 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:35:06.257 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:06.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:06.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:06.273 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:35:06.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:06.391 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:06.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:05.357992+0000 mgr.y (mgr.14535) 239 : cluster [DBG] pgmap v260: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:35:06.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:05.357992+0000 mgr.y (mgr.14535) 239 : cluster [DBG] pgmap v260: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:35:06.543 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:06.547 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:06.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:06.648 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:06.677 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:06.801 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:06.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:06.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:05.357992+0000 mgr.y (mgr.14535) 239 : cluster [DBG] pgmap v260: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:35:06.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:06.963 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:06.965 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:06.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:07.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:35:07.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:35:07.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:07.098 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:07.460 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-02-22T19:35:07.461 INFO:tasks.radosbench:Beginning radosbench... 2024-02-22T19:35:07.461 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-02-22T19:35:07.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool create unique_pool_1 16 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:06.102576+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:06.102648+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:06.472875+0000 mon.a (mon.0) 931 : cluster [DBG] osdmap e152: 8 total, 8 up, 7 in 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:06.830033+0000 mon.c (mon.2) 42 : audit [DBG] from='client.? 172.21.15.53:0/732462784' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:06.102576+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:06.102648+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:07.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:06.472875+0000 mon.a (mon.0) 931 : cluster [DBG] osdmap e152: 8 total, 8 up, 7 in 2024-02-22T19:35:07.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:06.830033+0000 mon.c (mon.2) 42 : audit [DBG] from='client.? 172.21.15.53:0/732462784' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:06.102576+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:06.102648+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:06.472875+0000 mon.a (mon.0) 931 : cluster [DBG] osdmap e152: 8 total, 8 up, 7 in 2024-02-22T19:35:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:06.830033+0000 mon.c (mon.2) 42 : audit [DBG] from='client.? 172.21.15.53:0/732462784' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:07.358309+0000 mgr.y (mgr.14535) 240 : cluster [DBG] pgmap v262: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:35:08.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:08.345914+0000 mon.a (mon.0) 932 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:08.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:07.358309+0000 mgr.y (mgr.14535) 240 : cluster [DBG] pgmap v262: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:35:08.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:08.345914+0000 mon.a (mon.0) 932 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:07.358309+0000 mgr.y (mgr.14535) 240 : cluster [DBG] pgmap v262: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:35:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:08.345914+0000 mon.a (mon.0) 932 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:08.980 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:08.982 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:08.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:09.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:09.117 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:09.139 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:09.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:09.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:09.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:09.274 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:09.417 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:09.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:09.421 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:09.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:09.550 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:35:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:09] "GET /metrics HTTP/1.1" 200 173294 "" "Prometheus/2.33.4" 2024-02-22T19:35:09.552 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:09.651 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:09.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:09.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:09.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:09.811 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:09.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:09.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:09.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:10.100 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:10.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:10.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:10.205 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:10.235 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:10.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:09.359055+0000 mgr.y (mgr.14535) 241 : cluster [DBG] pgmap v263: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:10.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:09.359055+0000 mgr.y (mgr.14535) 241 : cluster [DBG] pgmap v263: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:09.359055+0000 mgr.y (mgr.14535) 241 : cluster [DBG] pgmap v263: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:11.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:35:11.491 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:35:11.689 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:11.692 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:11.020380+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.53:0/867963664' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:11.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:11.020380+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.53:0/867963664' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:11.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:11.693 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:11.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:11.020380+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.53:0/867963664' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:11.894 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:11.897 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:11.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:11.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:12.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:12.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:12.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:12.209 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:12.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:12.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:12.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:12.342 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:12.441 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:12.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:12.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:12.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:12.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:11.359384+0000 mgr.y (mgr.14535) 242 : cluster [DBG] pgmap v264: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:11.479882+0000 mon.a (mon.0) 934 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:11.487017+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.53:0/867963664' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:11.487078+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e153: 8 total, 8 up, 7 in 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:11.359384+0000 mgr.y (mgr.14535) 242 : cluster [DBG] pgmap v264: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:11.479882+0000 mon.a (mon.0) 934 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:11.487017+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.53:0/867963664' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:12.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:11.487078+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e153: 8 total, 8 up, 7 in 2024-02-22T19:35:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:11.359384+0000 mgr.y (mgr.14535) 242 : cluster [DBG] pgmap v264: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:11.479882+0000 mon.a (mon.0) 934 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:11.487017+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.53:0/867963664' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:11.487078+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e153: 8 total, 8 up, 7 in 2024-02-22T19:35:13.269 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:35:13.270 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:35:14.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:13.359899+0000 mgr.y (mgr.14535) 243 : cluster [DBG] pgmap v266: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:35:14.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:14.252436+0000 mon.a (mon.0) 937 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:35:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:13.359899+0000 mgr.y (mgr.14535) 243 : cluster [DBG] pgmap v266: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:35:14.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:14.252436+0000 mon.a (mon.0) 937 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:35:14.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:35:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:35:14.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:13.359899+0000 mgr.y (mgr.14535) 243 : cluster [DBG] pgmap v266: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:35:14.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:14.252436+0000 mon.a (mon.0) 937 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:35:14.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:35:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:35:15.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:35:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:14.535452+0000 mon.a (mon.0) 938 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:14.552182+0000 mgr.y (mgr.14535) 244 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:16.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:14.535452+0000 mon.a (mon.0) 938 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:16.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:14.552182+0000 mgr.y (mgr.14535) 244 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:16.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:14.535452+0000 mon.a (mon.0) 938 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:16.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:14.552182+0000 mgr.y (mgr.14535) 244 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:16.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:35:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:16.722 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:16.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:16.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:16.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:15.360495+0000 mgr.y (mgr.14535) 245 : cluster [DBG] pgmap v267: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:16.422277+0000 mon.a (mon.0) 939 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:35:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:16.423416+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:35:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:16.427881+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:16.854 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:15.360495+0000 mgr.y (mgr.14535) 245 : cluster [DBG] pgmap v267: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:16.854 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:16.422277+0000 mon.a (mon.0) 939 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:35:16.854 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:16.423416+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:35:16.854 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:16.427881+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:16.854 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:15.360495+0000 mgr.y (mgr.14535) 245 : cluster [DBG] pgmap v267: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:35:16.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:16.422277+0000 mon.a (mon.0) 939 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:35:16.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:16.423416+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:35:16.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:16.427881+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:16.932 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:16.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:16.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:17.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:17.218 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:17.221 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:17.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:17.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:17.665 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:18.002 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:18.008 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:18.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:18.142 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:18.167 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:18.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:17.360843+0000 mgr.y (mgr.14535) 246 : cluster [DBG] pgmap v268: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:35:18.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:17.724623+0000 mon.a (mon.0) 942 : audit [INF] from='client.? 172.21.15.53:0/3166112276' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-02-22T19:35:18.226 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_1' created 2024-02-22T19:35:18.268 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:18.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:18.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:18.449 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:17.360843+0000 mgr.y (mgr.14535) 246 : cluster [DBG] pgmap v268: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:35:18.449 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:17.724623+0000 mon.a (mon.0) 942 : audit [INF] from='client.? 172.21.15.53:0/3166112276' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-02-22T19:35:18.451 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:18.471 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:18.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:17.360843+0000 mgr.y (mgr.14535) 246 : cluster [DBG] pgmap v268: 132 pgs: 132 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-22T19:35:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:17.724623+0000 mon.a (mon.0) 942 : audit [INF] from='client.? 172.21.15.53:0/3166112276' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-02-22T19:35:18.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:18.604 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:18.623 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:18.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:18.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:18.777 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:18.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:18.908 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:18.961 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:18.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:18.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:19.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:19.118 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:19.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:19.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:19.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:19.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:18.216186+0000 mon.a (mon.0) 943 : audit [INF] from='client.? 172.21.15.53:0/3166112276' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-02-22T19:35:19.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:18.216252+0000 mon.a (mon.0) 944 : cluster [DBG] osdmap e154: 8 total, 8 up, 7 in 2024-02-22T19:35:19.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:19.217801+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e155: 8 total, 8 up, 7 in 2024-02-22T19:35:19.229 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:18.216186+0000 mon.a (mon.0) 943 : audit [INF] from='client.? 172.21.15.53:0/3166112276' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-02-22T19:35:19.229 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:18.216252+0000 mon.a (mon.0) 944 : cluster [DBG] osdmap e154: 8 total, 8 up, 7 in 2024-02-22T19:35:19.229 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:19.217801+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e155: 8 total, 8 up, 7 in 2024-02-22T19:35:19.255 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:18.216186+0000 mon.a (mon.0) 943 : audit [INF] from='client.? 172.21.15.53:0/3166112276' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-02-22T19:35:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:18.216252+0000 mon.a (mon.0) 944 : cluster [DBG] osdmap e154: 8 total, 8 up, 7 in 2024-02-22T19:35:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:19.217801+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e155: 8 total, 8 up, 7 in 2024-02-22T19:35:19.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:35:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:19] "GET /metrics HTTP/1.1" 200 173294 "" "Prometheus/2.33.4" 2024-02-22T19:35:20.105 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:20.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:20.109 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:20.210 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:20.235 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:35:20.244 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:20.266 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:20.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:20.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:20.338 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:20.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:20.420 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:19.361473+0000 mgr.y (mgr.14535) 247 : cluster [DBG] pgmap v271: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:35:20.421 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:19.725491+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.53:0/749271160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:35:20.421 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:19.725792+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:35:20.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:19.361473+0000 mgr.y (mgr.14535) 247 : cluster [DBG] pgmap v271: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:35:20.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:19.725491+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.53:0/749271160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:35:20.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:19.725792+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:35:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:20.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:20.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:20.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:20.555 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:19.361473+0000 mgr.y (mgr.14535) 247 : cluster [DBG] pgmap v271: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T19:35:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:19.725491+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.53:0/749271160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:35:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:19.725792+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:35:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:20.656 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:20.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:20.758 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:20.813 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:20.815 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:20.816 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:20.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:20.949 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:20.970 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:20.973 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:20.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:20.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-02-22T19:35:21.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:21.104 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:21.127 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:21.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:21.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:21.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:20.223432+0000 mon.a (mon.0) 947 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:21.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:20.230946+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:35:21.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:20.231018+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e156: 8 total, 8 up, 7 in 2024-02-22T19:35:21.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:21.176951+0000 mon.c (mon.2) 44 : audit [DBG] from='client.? 172.21.15.53:0/1323652900' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:21.257 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:20.223432+0000 mon.a (mon.0) 947 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:20.230946+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:35:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:20.231018+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e156: 8 total, 8 up, 7 in 2024-02-22T19:35:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:21.176951+0000 mon.c (mon.2) 44 : audit [DBG] from='client.? 172.21.15.53:0/1323652900' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:20.223432+0000 mon.a (mon.0) 947 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:20.230946+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:35:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:20.231018+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e156: 8 total, 8 up, 7 in 2024-02-22T19:35:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:21.176951+0000 mon.c (mon.2) 44 : audit [DBG] from='client.? 172.21.15.53:0/1323652900' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:35:21.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:35:21.937 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:21.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:21.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:22.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:20.841739+0000 osd.1 (osd.1) 85 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:35:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:20.843805+0000 osd.1 (osd.1) 86 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:35:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:21.290791+0000 osd.0 (osd.0) 105 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:35:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:21.292016+0000 osd.0 (osd.0) 106 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:35:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:21.362063+0000 mgr.y (mgr.14535) 248 : cluster [DBG] pgmap v273: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:35:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:20.841739+0000 osd.1 (osd.1) 85 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:35:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:20.843805+0000 osd.1 (osd.1) 86 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:35:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:21.290791+0000 osd.0 (osd.0) 105 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:35:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:21.292016+0000 osd.0 (osd.0) 106 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:35:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:21.362063+0000 mgr.y (mgr.14535) 248 : cluster [DBG] pgmap v273: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:35:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:20.841739+0000 osd.1 (osd.1) 85 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:35:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:20.843805+0000 osd.1 (osd.1) 86 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:35:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:21.290791+0000 osd.0 (osd.0) 105 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:35:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:21.292016+0000 osd.0 (osd.0) 106 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:35:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:21.362063+0000 mgr.y (mgr.14535) 248 : cluster [DBG] pgmap v273: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:35:22.619 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:22.622 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:22.623 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:22.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:22.753 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:23.076 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:21.800696+0000 osd.2 (osd.2) 66 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:21.803772+0000 osd.2 (osd.2) 67 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.249763+0000 osd.6 (osd.6) 57 : cluster [DBG] 4.a scrub starts 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.253370+0000 osd.6 (osd.6) 58 : cluster [DBG] 4.a scrub ok 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.257484+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.280414+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:35:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.512764+0000 osd.7 (osd.7) 101 : cluster [DBG] 3.e scrub starts 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.515818+0000 osd.7 (osd.7) 102 : cluster [DBG] 3.e scrub ok 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:21.800696+0000 osd.2 (osd.2) 66 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:21.803772+0000 osd.2 (osd.2) 67 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.249763+0000 osd.6 (osd.6) 57 : cluster [DBG] 4.a scrub starts 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.253370+0000 osd.6 (osd.6) 58 : cluster [DBG] 4.a scrub ok 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.257484+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.280414+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.512764+0000 osd.7 (osd.7) 101 : cluster [DBG] 3.e scrub starts 2024-02-22T19:35:23.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.515818+0000 osd.7 (osd.7) 102 : cluster [DBG] 3.e scrub ok 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:21.800696+0000 osd.2 (osd.2) 66 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:21.803772+0000 osd.2 (osd.2) 67 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.249763+0000 osd.6 (osd.6) 57 : cluster [DBG] 4.a scrub starts 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.253370+0000 osd.6 (osd.6) 58 : cluster [DBG] 4.a scrub ok 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.257484+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.280414+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.512764+0000 osd.7 (osd.7) 101 : cluster [DBG] 3.e scrub starts 2024-02-22T19:35:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.515818+0000 osd.7 (osd.7) 102 : cluster [DBG] 3.e scrub ok 2024-02-22T19:35:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:23.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:23.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:23.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:35:23.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:23.841 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:23.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:23.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:23.977 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:24.076 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:24.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:24.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:24.110 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20820 7.e 7:75479808:::benchmark_data_smithi053_107660_object20819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:35:24.117 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.13349083400001, 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0656461249999998, 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20820 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980810+0000", 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.211e-05 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980833+0000", 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8699999999999992e-06 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980840+0000", 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9620000000000001e-06 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980845+0000", 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084919699999999999 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981695+0000", 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000152538 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981847+0000", 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117994 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5", 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981965+0000", 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001797442 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.983763+0000", 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007165561 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990928+0000", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6160000000000008e-06 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990938+0000", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004499088 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995437+0000", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2568000000000001e-05 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995449+0000", 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509275339999999 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046377+0000", 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1163999999999997e-05 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046458+0000", 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0656461249999998 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.123 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20825 7.e 7:74b6997e:::benchmark_data_smithi053_107660_object20824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.13252371900001, 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0648078949999999, 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20825 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981778+0000", 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6081000000000006e-05 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981844+0000", 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7260000000000002e-06 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981852+0000", 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0959999999999997e-06 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981858+0000", 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088905199999999998 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.982747+0000", 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118694 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.982866+0000", 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014208600000000001 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5", 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.983008+0000", 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00131347 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.984321+0000", 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006647231 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990968+0000", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.109e-06 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990976+0000", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004509871 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995485+0000", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4160000000000004e-06 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995494+0000", 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0510047179999999 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046499+0000", 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8375999999999995e-05 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046587+0000", 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0648078949999999 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.129 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20834 7.e 7:73899197:::benchmark_data_smithi053_107660_object20833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.12652581399999, 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0674701959999999, 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20834 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987775+0000", 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0616999999999999e-05 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987796+0000", 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7879999999999995e-06 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987805+0000", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9380000000000003e-06 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987812+0000", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043811340000000001 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.992193+0000", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9510000000000001e-06 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.992203+0000", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003316238 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995519+0000", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0299999999999994e-06 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995527+0000", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0510881539999999 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046615+0000", 2024-02-22T19:35:24.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012597899999999999 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046741+0000", 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014336500000000001 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046885+0000", 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073188910000000001 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054204+0000", 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1601999999999997e-05 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054265+0000", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010688699999999999 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054372+0000", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2679000000000001e-05 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054435+0000", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081248500000000005 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055247+0000", 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0674701959999999 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20860 7.e 7:7d18546b:::benchmark_data_smithi053_107660_object20859:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.09005548100001, 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.034585796, 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20860 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024245+0000", 2024-02-22T19:35:24.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7005e-05 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024272+0000", 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3974999999999999e-05 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024286+0000", 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6139999999999997e-06 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024293+0000", 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6604000000000002e-05 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024349+0000", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6550000000000004e-06 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024357+0000", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.971194102 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995551+0000", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3649999999999998e-06 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995558+0000", 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521218619999999 2024-02-22T19:35:24.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047680+0000", 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114424 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047795+0000", 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210046 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048005+0000", 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062518670000000004 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054257+0000", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5546e-05 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054272+0000", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003673805 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057946+0000", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8497000000000005e-05 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058014+0000", 2024-02-22T19:35:24.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081864399999999999 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058833+0000", 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.034585796 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20873 7.e 7:70011a90:::benchmark_data_smithi053_107660_object20872:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.08633651299999, 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0315724149999999, 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20873 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027965+0000", 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3580000000000001e-05 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027988+0000", 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.577e-05 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028004+0000", 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028011+0000", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6941999999999999e-05 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028058+0000", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3840000000000002e-06 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028064+0000", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96751664000000004 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995581+0000", 2024-02-22T19:35:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3509999999999996e-06 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995588+0000", 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0532674879999999 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048856+0000", 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015597000000000001 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049012+0000", 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015027700000000001 2024-02-22T19:35:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049162+0000", 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051174610000000002 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054279+0000", 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.541e-05 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054335+0000", 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045385269999999997 2024-02-22T19:35:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058873+0000", 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5123000000000001e-05 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058919+0000", 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062019400000000002 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059539+0000", 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0315724149999999 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20878 7.e 7:7891943c:::benchmark_data_smithi053_107660_object20877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.083622703, 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0297337099999999, 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20878 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030678+0000", 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2857000000000002e-05 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030700+0000", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3713000000000001e-05 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030714+0000", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1090000000000003e-06 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030720+0000", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9595000000000002e-05 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030770+0000", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1749999999999994e-06 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030779+0000", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96483299300000003 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995612+0000", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5809999999999999e-06 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995620+0000", 2024-02-22T19:35:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0543208209999999 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049940+0000", 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7372000000000003e-05 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050028+0000", 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013467700000000001 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050162+0000", 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041241170000000001 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054287+0000", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010961230000000001 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055383+0000", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042013659999999998 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059584+0000", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8198000000000001e-05 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059642+0000", 2024-02-22T19:35:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077155300000000002 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060414+0000", 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0297337099999999 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20883 7.e 7:7c7f3c6f:::benchmark_data_smithi053_107660_object20882:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.07967251299999, 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265122409999998, 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20883 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034628+0000", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6835e-05 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034655+0000", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8521000000000001e-05 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034674+0000", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2910000000000002e-06 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034681+0000", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9109999999999994e-05 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034750+0000", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0144e-05 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034760+0000", 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96088248899999995 2024-02-22T19:35:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995643+0000", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995650+0000", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.055232344 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050883+0000", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016314899999999999 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051046+0000", 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000208643 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051255+0000", 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030399609999999999 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054295+0000", 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061688639999999996 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060463+0000", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0683000000000002e-05 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060524+0000", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1838e-05 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060586+0000", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055658699999999999 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061143+0000", 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265122409999998 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20886 7.e 7:75e1811d:::benchmark_data_smithi053_107660_object20885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.07832899900001, 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0257961170000001, 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20886 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035972+0000", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7784e-05 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036000+0000", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8953e-05 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036019+0000", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1119999999999996e-06 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036027+0000", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106254 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036133+0000", 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2412000000000001e-05 2024-02-22T19:35:24.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036146+0000", 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95952591799999998 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995672+0000", 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5299999999999999e-06 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995679+0000", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0562192079999999 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051898+0000", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000153024 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052051+0000", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195556 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052247+0000", 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049309180000000003 2024-02-22T19:35:24.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057178+0000", 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040298340000000004 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061208+0000", 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6218999999999995e-05 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061304+0000", 2024-02-22T19:35:24.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3900000000000002e-05 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061358+0000", 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041208899999999998 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061770+0000", 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0257961170000001 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110795 7.e 7:7fb91f79:::benchmark_data_smithi053_107660_object21588:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.084742+0000", 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 105.029560442, 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025483795, 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110795 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084742+0000", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084742+0000", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084741+0000", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1459999999999996e-06 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084749+0000", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2155e-05 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084761+0000", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5529999999999998e-06 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084768+0000", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024508709999999999 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.087219+0000", 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7408000000000001e-05 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.087306+0000", 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2807999999999996e-05 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.087399+0000", 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061312190000000003 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.093530+0000", 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065161530000000002 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.100046+0000", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0083518299999999993 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108398+0000", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6590000000000002e-06 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108408+0000", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.001729877 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110137+0000", 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8690000000000006e-05 2024-02-22T19:35:24.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110226+0000", 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025483795 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110799 7.e 7:726922b8:::benchmark_data_smithi053_107660_object21592:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.088641+0000", 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 105.025662279, 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021687464, 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110799 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088641+0000", 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088641+0000", 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088638+0000", 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8400000000000001e-06 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088647+0000", 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0780000000000002e-06 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088656+0000", 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3720000000000001e-06 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088663+0000", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001396865 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.090060+0000", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014967299999999999 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.090209+0000", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169808 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.090379+0000", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031592500000000002 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.093538+0000", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014899720999999999 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108438+0000", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1609999999999997e-06 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108445+0000", 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.001810976 2024-02-22T19:35:24.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110256+0000", 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1786999999999996e-05 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110328+0000", 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021687464 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110802 7.e 7:74556c42:::benchmark_data_smithi053_107660_object21595:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.097107+0000", 2024-02-22T19:35:24.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 105.017195587, 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0133390470000001, 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110802 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097107+0000", 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097107+0000", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097106+0000", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097114+0000", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2564e-05 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097127+0000", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4839999999999998e-06 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097134+0000", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5037999999999997e-05 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097189+0000", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013183699999999999 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097321+0000", 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013346899999999999 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097454+0000", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003854845 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.101309+0000", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071630269999999998 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108472+0000", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0369999999999997e-06 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108479+0000", 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0018937649999999 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110373+0000", 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3134999999999997e-05 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110446+0000", 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0133390470000001 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110814 7.e 7:740d9b1f:::benchmark_data_smithi053_107660_object21607:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.117657+0000", 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.99664561100001, 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027530931, 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110814 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117657+0000", 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117657+0000", 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117655+0000", 2024-02-22T19:35:24.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9629999999999998e-06 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117663+0000", 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2947e-05 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117676+0000", 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1865000000000001e-05 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117688+0000", 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046482799999999998 2024-02-22T19:35:24.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.118153+0000", 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3170000000000008e-06 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.118162+0000", 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99231844199999997 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110481+0000", 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012363900000000001 2024-02-22T19:35:24.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110604+0000", 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017102900000000001 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110775+0000", 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011035908000000001 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121811+0000", 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023248329000000002 2024-02-22T19:35:24.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145060+0000", 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1023999999999999e-05 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145101+0000", 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7902999999999999e-05 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145149+0000", 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9548000000000002e-05 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145188+0000", 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027530931 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110819 7.e 7:7eaeb1ee:::benchmark_data_smithi053_107660_object21612:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.124807+0000", 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.98949612600001, 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0205201960000001, 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110819 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124807+0000", 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124807+0000", 2024-02-22T19:35:24.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124805+0000", 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4290000000000001e-06 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124813+0000", 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1292e-05 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124825+0000", 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4409999999999998e-06 2024-02-22T19:35:24.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124831+0000", 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7509000000000002e-05 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124879+0000", 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124885+0000", 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98871795900000004 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.113603+0000", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7225999999999994e-05 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.113701+0000", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014236399999999999 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.113843+0000", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0079748839999999998 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121818+0000", 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023407692000000001 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145226+0000", 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0882e-05 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145256+0000", 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6516000000000001e-05 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145293+0000", 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3945000000000001e-05 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145327+0000", 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0205201960000001 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110820 7.e 7:754a5adc:::benchmark_data_smithi053_107660_object21613:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:35:24.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.989459344, 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026599126, 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110820 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.005e-06 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124846+0000", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124851+0000", 2024-02-22T19:35:24.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6149999999999996e-06 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124857+0000", 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7889000000000002e-05 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124905+0000", 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9429999999999997e-06 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124911+0000", 2024-02-22T19:35:24.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99109971699999999 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.116010+0000", 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013614500000000001 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.116147+0000", 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015206200000000001 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.116299+0000", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016877981 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133177+0000", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012185431 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145362+0000", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059805309999999999 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151343+0000", 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4162000000000001e-05 2024-02-22T19:35:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151397+0000", 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5803000000000002e-05 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151443+0000", 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026599126 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110830 7.e 7:7342ed68:::benchmark_data_smithi053_107660_object21623:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.138800+0000", 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.975502668, 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012783733, 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110830 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138800+0000", 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138800+0000", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138798+0000", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1530000000000003e-06 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138807+0000", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7183000000000001e-05 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138824+0000", 2024-02-22T19:35:24.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7870000000000004e-06 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138831+0000", 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2577999999999997e-05 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138884+0000", 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5100000000000004e-06 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138893+0000", 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98002727999999995 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.118920+0000", 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013081900000000001 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.119051+0000", 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000165923 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.119217+0000", 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013967171 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133184+0000", 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012208811 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145393+0000", 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061003960000000001 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151494+0000", 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6493999999999999e-05 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151540+0000", 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3776999999999998e-05 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151584+0000", 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012783733 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110847 7.e 7:7e4d7ad4:::benchmark_data_smithi053_107660_object21640:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.156297+0000", 2024-02-22T19:35:24.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.95800587700001, 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99545414099999996, 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110847 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156297+0000", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156297+0000", 2024-02-22T19:35:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156296+0000", 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7729999999999996e-06 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156301+0000", 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0149999999999999e-05 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156312+0000", 2024-02-22T19:35:24.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156317+0000", 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4360999999999997e-05 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156371+0000", 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156383+0000", 2024-02-22T19:35:24.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96546441100000002 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121847+0000", 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107951 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121955+0000", 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144988 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.122100+0000", 2024-02-22T19:35:24.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011091546000000001 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133192+0000", 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012232875000000001 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145425+0000", 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006235127 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151660+0000", 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7330999999999998e-05 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151707+0000", 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989000000000002e-05 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151751+0000", 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99545414099999996 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110896 7.e 7:73002236:::benchmark_data_smithi053_107660_object21689:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.212855+0000", 2024-02-22T19:35:24.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.901448286, 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.93903675200000003, 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110896 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212855+0000", 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212855+0000", 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212852+0000", 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4409999999999999e-06 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212862+0000", 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6711e-05 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212878+0000", 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6020000000000002e-06 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212886+0000", 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2688999999999996e-05 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212949+0000", 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1256e-05 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212960+0000", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.9117461 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.124706+0000", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000121069 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.124827+0000", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016258900000000001 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.124990+0000", 2024-02-22T19:35:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0082091390000000007 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133199+0000", 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012256107 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145455+0000", 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063481570000000001 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151803+0000", 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6461999999999997e-05 2024-02-22T19:35:24.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151850+0000", 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1643000000000001e-05 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151891+0000", 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.93903675200000003 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.221 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110920 7.e 7:76c1d686:::benchmark_data_smithi053_107660_object21713:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.235434+0000", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.87886926900001, 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91662422200000004, 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110920 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235434+0000", 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235434+0000", 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235431+0000", 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-06 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235440+0000", 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.366e-05 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235453+0000", 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4159999999999998e-06 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235461+0000", 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2850999999999998e-05 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235524+0000", 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1752e-05 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235536+0000", 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89218530900000004 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.127721+0000", 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012830799999999999 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.127849+0000", 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017058500000000001 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.128020+0000", 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017466145999999998 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145486+0000", 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051044749999999998 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.150590+0000", 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001387969 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151978+0000", 2024-02-22T19:35:24.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1754e-05 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152020+0000", 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7688999999999998e-05 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152058+0000", 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91662422200000004 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110928 7.e 7:7b4e694f:::benchmark_data_smithi053_107660_object21721:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.247486+0000", 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.866817289, 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90479611400000004, 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110928 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.229 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247486+0000", 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247486+0000", 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247484+0000", 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0610000000000001e-06 2024-02-22T19:35:24.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247490+0000", 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4266e-05 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247505+0000", 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8329999999999996e-06 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247511+0000", 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2525999999999998e-05 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247564+0000", 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0601000000000001e-05 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247575+0000", 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.88291269800000005 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.130487+0000", 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109404 2024-02-22T19:35:24.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.130597+0000", 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036100000000001 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.130737+0000", 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014779439 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145516+0000", 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050812640000000003 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.150598+0000", 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001600186 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152198+0000", 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4004000000000001e-05 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152242+0000", 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9711e-05 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152282+0000", 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90479611400000004 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110952 7.e 7:77e3b0a4:::benchmark_data_smithi053_107660_object21745:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.279954+0000", 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.834348558, 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87986089099999998, 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:24.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110952 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279954+0000", 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279954+0000", 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279952+0000", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2339999999999994e-06 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279961+0000", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3113e-05 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279974+0000", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1180000000000003e-06 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:24.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279981+0000", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2351e-05 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.280043+0000", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0951000000000001e-05 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.280054+0000", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85318869900000005 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133243+0000", 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114162 2024-02-22T19:35:24.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133357+0000", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4803999999999996e-05 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133452+0000", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012095134 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145547+0000", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050563459999999998 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:24.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.150603+0000", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091255810000000007 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.159729+0000", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6208999999999998e-05 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.159775+0000", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9975000000000003e-05 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.159815+0000", 2024-02-22T19:35:24.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87986089099999998 2024-02-22T19:35:24.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:24.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:24.241 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:35:24.241 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:24.295 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:24.298 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:24.299 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:24.373 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:35:24.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.831976+0000 osd.2 (osd.2) 68 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:22.835118+0000 osd.2 (osd.2) 69 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:23.234214+0000 osd.0 (osd.0) 107 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:23.243938+0000 osd.0 (osd.0) 108 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:23.362388+0000 mgr.y (mgr.14535) 249 : cluster [DBG] pgmap v274: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.831976+0000 osd.2 (osd.2) 68 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:22.835118+0000 osd.2 (osd.2) 69 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:23.234214+0000 osd.0 (osd.0) 107 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:35:24.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:23.243938+0000 osd.0 (osd.0) 108 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:35:24.401 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:23.362388+0000 mgr.y (mgr.14535) 249 : cluster [DBG] pgmap v274: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:35:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:24.450 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:24.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:24.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:24.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:24.557 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.831976+0000 osd.2 (osd.2) 68 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:35:24.557 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:22.835118+0000 osd.2 (osd.2) 69 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:35:24.557 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:23.234214+0000 osd.0 (osd.0) 107 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:35:24.557 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:23.243938+0000 osd.0 (osd.0) 108 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:35:24.557 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:23.362388+0000 mgr.y (mgr.14535) 249 : cluster [DBG] pgmap v274: 148 pgs: 10 creating+activating, 138 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:35:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:24.603 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:24.605 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:24.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:24.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:24.734 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:24.756 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:24.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:24.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:35:24.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:35:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:35:24.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:24.904 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:24.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:24.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:25.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:25.035 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:25.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:35:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:25.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:25.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:25.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:25.203 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:25.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:25.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:25.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:25.328 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:24.045842+0000 osd.4 (osd.4) 97 : cluster [DBG] 5.1c scrub starts 2024-02-22T19:35:25.328 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:24.048692+0000 osd.4 (osd.4) 98 : cluster [DBG] 5.1c scrub ok 2024-02-22T19:35:25.328 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:24.557034+0000 mgr.y (mgr.14535) 250 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:25.328 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:24.045842+0000 osd.4 (osd.4) 97 : cluster [DBG] 5.1c scrub starts 2024-02-22T19:35:25.328 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:24.048692+0000 osd.4 (osd.4) 98 : cluster [DBG] 5.1c scrub ok 2024-02-22T19:35:25.328 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:24.557034+0000 mgr.y (mgr.14535) 250 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:25.353 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:25.355 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:25.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:25.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:25.484 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:24.045842+0000 osd.4 (osd.4) 97 : cluster [DBG] 5.1c scrub starts 2024-02-22T19:35:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:24.048692+0000 osd.4 (osd.4) 98 : cluster [DBG] 5.1c scrub ok 2024-02-22T19:35:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:24.557034+0000 mgr.y (mgr.14535) 250 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:25.836 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:25.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:25.937 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:25.965 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:25.985 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:25.987 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:25.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:26.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:26.122 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:26.144 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:26.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:26.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:26.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:26.279 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:26.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:26.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:26.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:26.433 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:24.791574+0000 osd.2 (osd.2) 70 : cluster [DBG] 4.f scrub starts 2024-02-22T19:35:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:24.795023+0000 osd.2 (osd.2) 71 : cluster [DBG] 4.f scrub ok 2024-02-22T19:35:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:25.226285+0000 osd.6 (osd.6) 59 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:35:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:25.227181+0000 osd.6 (osd.6) 60 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:35:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:25.313344+0000 osd.3 (osd.3) 103 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:25.316831+0000 osd.3 (osd.3) 104 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:25.363038+0000 mgr.y (mgr.14535) 251 : cluster [DBG] pgmap v275: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:24.791574+0000 osd.2 (osd.2) 70 : cluster [DBG] 4.f scrub starts 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:24.795023+0000 osd.2 (osd.2) 71 : cluster [DBG] 4.f scrub ok 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:25.226285+0000 osd.6 (osd.6) 59 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:25.227181+0000 osd.6 (osd.6) 60 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:25.313344+0000 osd.3 (osd.3) 103 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:25.316831+0000 osd.3 (osd.3) 104 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:35:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:25.363038+0000 mgr.y (mgr.14535) 251 : cluster [DBG] pgmap v275: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:35:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:24.791574+0000 osd.2 (osd.2) 70 : cluster [DBG] 4.f scrub starts 2024-02-22T19:35:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:24.795023+0000 osd.2 (osd.2) 71 : cluster [DBG] 4.f scrub ok 2024-02-22T19:35:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:25.226285+0000 osd.6 (osd.6) 59 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:35:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:25.227181+0000 osd.6 (osd.6) 60 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:35:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:25.313344+0000 osd.3 (osd.3) 103 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:35:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:25.316831+0000 osd.3 (osd.3) 104 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:35:26.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:25.363038+0000 mgr.y (mgr.14535) 251 : cluster [DBG] pgmap v275: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:35:27.083 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:27.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:27.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:27.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:27.250 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:27.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:27.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:27.403 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:27.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:27.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:27.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:25.997803+0000 osd.4 (osd.4) 99 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:35:27.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:26.000684+0000 osd.4 (osd.4) 100 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:35:27.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:26.231375+0000 osd.0 (osd.0) 109 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:35:27.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:26.232440+0000 osd.0 (osd.0) 110 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:35:27.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:25.997803+0000 osd.4 (osd.4) 99 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:35:27.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:26.000684+0000 osd.4 (osd.4) 100 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:35:27.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:26.231375+0000 osd.0 (osd.0) 109 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:35:27.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:26.232440+0000 osd.0 (osd.0) 110 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:35:27.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:27.536 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:27.556 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:27.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:27.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:25.997803+0000 osd.4 (osd.4) 99 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:35:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:26.000684+0000 osd.4 (osd.4) 100 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:35:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:26.231375+0000 osd.0 (osd.0) 109 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:35:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:26.232440+0000 osd.0 (osd.0) 110 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:35:27.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:28.140 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:28.143 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:28.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:28.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:28.248 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:26.966162+0000 osd.4 (osd.4) 101 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:26.969821+0000 osd.4 (osd.4) 102 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:27.252965+0000 osd.0 (osd.0) 111 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:27.254978+0000 osd.0 (osd.0) 112 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:27.349658+0000 osd.3 (osd.3) 105 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:27.352542+0000 osd.3 (osd.3) 106 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:27.363453+0000 mgr.y (mgr.14535) 252 : cluster [DBG] pgmap v276: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:27.500784+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.53:0/2239643931' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:27.501155+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:26.966162+0000 osd.4 (osd.4) 101 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:26.969821+0000 osd.4 (osd.4) 102 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:27.252965+0000 osd.0 (osd.0) 111 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:35:28.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:27.254978+0000 osd.0 (osd.0) 112 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:35:28.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:27.349658+0000 osd.3 (osd.3) 105 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:35:28.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:27.352542+0000 osd.3 (osd.3) 106 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:35:28.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:27.363453+0000 mgr.y (mgr.14535) 252 : cluster [DBG] pgmap v276: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:28.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:27.500784+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.53:0/2239643931' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:35:28.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:27.501155+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:35:28.258 INFO:teuthology.orchestra.run.smithi053.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-02-22T19:35:28.278 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:28.299 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:28.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:28.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:28.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:28.432 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:28.584 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:28.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:26.966162+0000 osd.4 (osd.4) 101 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:26.969821+0000 osd.4 (osd.4) 102 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:27.252965+0000 osd.0 (osd.0) 111 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:27.254978+0000 osd.0 (osd.0) 112 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:27.349658+0000 osd.3 (osd.3) 105 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:27.352542+0000 osd.3 (osd.3) 106 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:35:28.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:27.363453+0000 mgr.y (mgr.14535) 252 : cluster [DBG] pgmap v276: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:28.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:27.500784+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.53:0/2239643931' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:35:28.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:27.501155+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:35:28.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:28.714 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:29.066 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:29.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:29.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:29.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:29.227 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:29.229 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:29.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:29.262 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:35:29.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:29.349 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:29] "GET /metrics HTTP/1.1" 200 175674 "" "Prometheus/2.33.4" 2024-02-22T19:35:29.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:28.254600+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:35:29.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:28.254682+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e157: 8 total, 8 up, 7 in 2024-02-22T19:35:29.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:28.260744+0000 osd.6 (osd.6) 61 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:28.262082+0000 osd.6 (osd.6) 62 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:29.083106+0000 mon.a (mon.0) 953 : audit [INF] from='client.? 172.21.15.53:0/2372908790' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:28.254600+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:28.254682+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e157: 8 total, 8 up, 7 in 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:28.260744+0000 osd.6 (osd.6) 61 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:28.262082+0000 osd.6 (osd.6) 62 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:35:29.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:29.083106+0000 mon.a (mon.0) 953 : audit [INF] from='client.? 172.21.15.53:0/2372908790' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:29.381 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:29.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:29.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:29.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:29.511 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:28.254600+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:35:29.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:28.254682+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e157: 8 total, 8 up, 7 in 2024-02-22T19:35:29.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:28.260744+0000 osd.6 (osd.6) 61 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:35:29.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:28.262082+0000 osd.6 (osd.6) 62 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:35:29.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:29.083106+0000 mon.a (mon.0) 953 : audit [INF] from='client.? 172.21.15.53:0/2372908790' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:29.533 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:29.536 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:29.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:29.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:29.669 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:29.722 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:29.730 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:29.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:29.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:29.860 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:30.113 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:30.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:30.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:30.241 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:30.441 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:28.846766+0000 osd.2 (osd.2) 72 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:28.850027+0000 osd.2 (osd.2) 73 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:29.240782+0000 osd.6 (osd.6) 63 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:29.242793+0000 osd.6 (osd.6) 64 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:29.252497+0000 mon.a (mon.0) 954 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:29.252540+0000 mon.a (mon.0) 955 : cluster [INF] Cluster is now healthy 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:29.259328+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.53:0/2372908790' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:29.259392+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e158: 8 total, 8 up, 7 in 2024-02-22T19:35:30.443 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:29.364106+0000 mgr.y (mgr.14535) 253 : cluster [DBG] pgmap v279: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:28.846766+0000 osd.2 (osd.2) 72 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:28.850027+0000 osd.2 (osd.2) 73 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:29.240782+0000 osd.6 (osd.6) 63 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:29.242793+0000 osd.6 (osd.6) 64 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:29.252497+0000 mon.a (mon.0) 954 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:29.252540+0000 mon.a (mon.0) 955 : cluster [INF] Cluster is now healthy 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:29.259328+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.53:0/2372908790' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:29.259392+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e158: 8 total, 8 up, 7 in 2024-02-22T19:35:30.444 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:29.364106+0000 mgr.y (mgr.14535) 253 : cluster [DBG] pgmap v279: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:30.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:30.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:30.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:30.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:28.846766+0000 osd.2 (osd.2) 72 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:28.850027+0000 osd.2 (osd.2) 73 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:29.240782+0000 osd.6 (osd.6) 63 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:29.242793+0000 osd.6 (osd.6) 64 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:29.252497+0000 mon.a (mon.0) 954 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:29.252540+0000 mon.a (mon.0) 955 : cluster [INF] Cluster is now healthy 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:29.259328+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.53:0/2372908790' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:29.259392+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e158: 8 total, 8 up, 7 in 2024-02-22T19:35:30.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:29.364106+0000 mgr.y (mgr.14535) 253 : cluster [DBG] pgmap v279: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:30.923 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:30.925 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:30.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:31.026 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:31.156 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:31.159 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:31.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:31.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:31.288 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:31.388 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:31.391 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:31.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:31.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:31.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:30.339465+0000 osd.0 (osd.0) 113 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:35:31.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:30.342855+0000 osd.0 (osd.0) 114 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:35:31.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:30.339465+0000 osd.0 (osd.0) 113 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:35:31.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:30.342855+0000 osd.0 (osd.0) 114 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:35:31.505 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:35:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:35:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-02-22T19:35:31.505 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:35:31.522 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:30.339465+0000 osd.0 (osd.0) 113 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:35:31.523 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:30.342855+0000 osd.0 (osd.0) 114 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:35:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:31.547 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:31.550 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:31.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:31.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:31.702 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:31.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:31.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:31.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:31.855 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:31.858 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:31.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:31.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:31.988 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:32.088 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:32.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:32.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:32.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:35:32.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:32.240 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:32.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:32.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:32.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:32.391 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:32.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:32.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:32.440 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:30.616959+0000 osd.7 (osd.7) 103 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:35:32.440 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:30.619961+0000 osd.7 (osd.7) 104 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:35:32.440 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:30.789512+0000 osd.1 (osd.1) 87 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:35:32.440 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:30.790801+0000 osd.1 (osd.1) 88 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.177006+0000 osd.6 (osd.6) 65 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.179886+0000 osd.6 (osd.6) 66 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.364444+0000 mgr.y (mgr.14535) 254 : cluster [DBG] pgmap v280: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:30.616959+0000 osd.7 (osd.7) 103 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:30.619961+0000 osd.7 (osd.7) 104 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:30.789512+0000 osd.1 (osd.1) 87 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:30.790801+0000 osd.1 (osd.1) 88 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.177006+0000 osd.6 (osd.6) 65 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.179886+0000 osd.6 (osd.6) 66 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:35:32.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.364444+0000 mgr.y (mgr.14535) 254 : cluster [DBG] pgmap v280: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:32.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:32.523 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:32.577 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:32.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:32.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:30.616959+0000 osd.7 (osd.7) 103 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:30.619961+0000 osd.7 (osd.7) 104 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:30.789512+0000 osd.1 (osd.1) 87 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:30.790801+0000 osd.1 (osd.1) 88 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.177006+0000 osd.6 (osd.6) 65 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.179886+0000 osd.6 (osd.6) 66 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:35:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.364444+0000 mgr.y (mgr.14535) 254 : cluster [DBG] pgmap v280: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:35:32.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:32.714 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:32.736 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:32.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:32.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:32.866 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:32.889 DEBUG:teuthology.orchestra.run.smithi053:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-02-22T19:35:32.890 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2024-02-22T19:35:32.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:35:32.892 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-02-22T19:35:32.893 INFO:teuthology.orchestra.run:waiting for 3000 2024-02-22T19:35:32.919 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:32.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:32.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:32.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:35:32.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:hints = 1 2024-02-22T19:35:32.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-02-22T19:35:32.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object prefix: benchmark_data_smithi053_130774 2024-02-22T19:35:32.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:35:32.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 0 0 0 0 0 0 - 0 2024-02-22T19:35:33.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:33.310 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:33.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:33.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:33.417 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:33.497 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:33.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:33.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.606219+0000 osd.7 (osd.7) 105 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:35:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.613898+0000 osd.7 (osd.7) 106 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:35:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.807438+0000 osd.2 (osd.2) 74 : cluster [DBG] 6.14 deep-scrub starts 2024-02-22T19:35:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:31.810349+0000 osd.2 (osd.2) 75 : cluster [DBG] 6.14 deep-scrub ok 2024-02-22T19:35:33.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:33.617 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:33.682 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.606219+0000 osd.7 (osd.7) 105 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:35:33.682 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.613898+0000 osd.7 (osd.7) 106 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:35:33.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.807438+0000 osd.2 (osd.2) 74 : cluster [DBG] 6.14 deep-scrub starts 2024-02-22T19:35:33.683 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:31.810349+0000 osd.2 (osd.2) 75 : cluster [DBG] 6.14 deep-scrub ok 2024-02-22T19:35:33.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.606219+0000 osd.7 (osd.7) 105 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:35:33.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.613898+0000 osd.7 (osd.7) 106 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:35:33.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.807438+0000 osd.2 (osd.2) 74 : cluster [DBG] 6.14 deep-scrub starts 2024-02-22T19:35:33.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:31.810349+0000 osd.2 (osd.2) 75 : cluster [DBG] 6.14 deep-scrub ok 2024-02-22T19:35:33.688 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:33.722 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:35:33.758 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:35:33.794 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:35:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:35:33.861 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:35:33.893 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:33.897 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:33.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:33.900 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:35:33.916 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:33.937 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:35:33.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 1 16 2210 2194 137.117 137.125 0.00719297 0.00725431 2024-02-22T19:35:33.970 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:35:33.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.982 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.982 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:35:33.999 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:34.038 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:34.063 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:34.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:34.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:34.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:34.206 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:34.231 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:34.234 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:34.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:32.618615+0000 osd.7 (osd.7) 107 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:32.621504+0000 osd.7 (osd.7) 108 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.269442+0000 osd.3 (osd.3) 107 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.275409+0000 osd.3 (osd.3) 108 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.365488+0000 mgr.y (mgr.14535) 255 : cluster [DBG] pgmap v281: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.404169+0000 osd.0 (osd.0) 115 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.417723+0000 osd.0 (osd.0) 116 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.472275+0000 osd.3 (osd.3) 109 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:34.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.472328+0000 osd.3 (osd.3) 110 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:34.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:33.690588+0000 mon.b (mon.1) 46 : audit [DBG] from='client.? 172.21.15.53:0/353353364' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:34.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:32.618615+0000 osd.7 (osd.7) 107 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:32.621504+0000 osd.7 (osd.7) 108 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.269442+0000 osd.3 (osd.3) 107 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.275409+0000 osd.3 (osd.3) 108 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.365488+0000 mgr.y (mgr.14535) 255 : cluster [DBG] pgmap v281: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.404169+0000 osd.0 (osd.0) 115 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.417723+0000 osd.0 (osd.0) 116 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:35:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.472275+0000 osd.3 (osd.3) 109 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.472328+0000 osd.3 (osd.3) 110 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:34.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:33.690588+0000 mon.b (mon.1) 46 : audit [DBG] from='client.? 172.21.15.53:0/353353364' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:34.392 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:34.556 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:32.618615+0000 osd.7 (osd.7) 107 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:35:34.556 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:32.621504+0000 osd.7 (osd.7) 108 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:35:34.556 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.269442+0000 osd.3 (osd.3) 107 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:35:34.556 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.275409+0000 osd.3 (osd.3) 108 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:35:34.556 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.365488+0000 mgr.y (mgr.14535) 255 : cluster [DBG] pgmap v281: 148 pgs: 148 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:35:34.556 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.404169+0000 osd.0 (osd.0) 115 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:35:34.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.417723+0000 osd.0 (osd.0) 116 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:35:34.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.472275+0000 osd.3 (osd.3) 109 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:34.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.472328+0000 osd.3 (osd.3) 110 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:34.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:33.690588+0000 mon.b (mon.1) 46 : audit [DBG] from='client.? 172.21.15.53:0/353353364' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:34.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:35:34.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:35:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:35:34.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 2 16 5046 5030 157.161 177.25 0.00365183 0.00635321 2024-02-22T19:35:35.359 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:35.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:35.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:35.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.657357+0000 osd.7 (osd.7) 109 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.660249+0000 osd.7 (osd.7) 110 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.967950+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.c scrub starts 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:33.971436+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.c scrub ok 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:34.285729+0000 mon.a (mon.0) 958 : cluster [DBG] osdmap e159: 8 total, 8 up, 7 in 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:34.570269+0000 mgr.y (mgr.14535) 256 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.062945+0000 osd.6 (osd.6) 67 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.657357+0000 osd.7 (osd.7) 109 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.660249+0000 osd.7 (osd.7) 110 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.967950+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.c scrub starts 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:33.971436+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.c scrub ok 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:34.285729+0000 mon.a (mon.0) 958 : cluster [DBG] osdmap e159: 8 total, 8 up, 7 in 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:34.570269+0000 mgr.y (mgr.14535) 256 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.062945+0000 osd.6 (osd.6) 67 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.657357+0000 osd.7 (osd.7) 109 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:35:35.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.660249+0000 osd.7 (osd.7) 110 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:35:35.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.967950+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.c scrub starts 2024-02-22T19:35:35.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:33.971436+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.c scrub ok 2024-02-22T19:35:35.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:34.285729+0000 mon.a (mon.0) 958 : cluster [DBG] osdmap e159: 8 total, 8 up, 7 in 2024-02-22T19:35:35.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:34.570269+0000 mgr.y (mgr.14535) 256 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:35.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.062945+0000 osd.6 (osd.6) 67 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:35:35.949 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 3 16 7807 7791 162.288 172.562 0.00420114 0.00615334 2024-02-22T19:35:36.421 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:36.429 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:36.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:36.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:36.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.065328+0000 osd.6 (osd.6) 68 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:35:36.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.368207+0000 mgr.y (mgr.14535) 257 : cluster [DBG] pgmap v283: 148 pgs: 148 active+clean; 196 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 1.4 KiB/s rd, 28 MiB/s wr, 442 op/s 2024-02-22T19:35:36.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.421565+0000 osd.0 (osd.0) 117 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:35:36.564 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.426291+0000 osd.0 (osd.0) 118 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:35:36.566 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:36.597 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:36.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:36.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:36.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.065328+0000 osd.6 (osd.6) 68 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:35:36.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.368207+0000 mgr.y (mgr.14535) 257 : cluster [DBG] pgmap v283: 148 pgs: 148 active+clean; 196 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 1.4 KiB/s rd, 28 MiB/s wr, 442 op/s 2024-02-22T19:35:36.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.421565+0000 osd.0 (osd.0) 117 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:35:36.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.426291+0000 osd.0 (osd.0) 118 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:35:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.065328+0000 osd.6 (osd.6) 68 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:35:36.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.368207+0000 mgr.y (mgr.14535) 257 : cluster [DBG] pgmap v283: 148 pgs: 148 active+clean; 196 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 1.4 KiB/s rd, 28 MiB/s wr, 442 op/s 2024-02-22T19:35:36.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.421565+0000 osd.0 (osd.0) 117 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:35:36.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.426291+0000 osd.0 (osd.0) 118 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:35:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:36.762 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:36.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:36.811 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:36.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:36.895 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:36.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:36.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:36.946 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 4 16 10425 10409 162.616 163.625 0.0062929 0.00614193 2024-02-22T19:35:37.027 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:37.080 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:37.108 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:37.113 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:37.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:37.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:37.249 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:37.273 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:37.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:37.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:37.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:37.429 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.852398+0000 osd.1 (osd.1) 89 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:35:37.429 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.855441+0000 osd.1 (osd.1) 90 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:35:37.429 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.927138+0000 osd.4 (osd.4) 105 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:35:37.429 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:35.930646+0000 osd.4 (osd.4) 106 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:36.334231+0000 osd.3 (osd.3) 111 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:36.338710+0000 osd.3 (osd.3) 112 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.852398+0000 osd.1 (osd.1) 89 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.855441+0000 osd.1 (osd.1) 90 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.927138+0000 osd.4 (osd.4) 105 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:35.930646+0000 osd.4 (osd.4) 106 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:36.334231+0000 osd.3 (osd.3) 111 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:35:37.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:36.338710+0000 osd.3 (osd.3) 112 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:35:37.431 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:37.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.852398+0000 osd.1 (osd.1) 89 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:35:37.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.855441+0000 osd.1 (osd.1) 90 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:35:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.927138+0000 osd.4 (osd.4) 105 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:35:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:35.930646+0000 osd.4 (osd.4) 106 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:35:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:36.334231+0000 osd.3 (osd.3) 111 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:35:37.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:36.338710+0000 osd.3 (osd.3) 112 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:35:37.795 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:37.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:37.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:37.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:37.947 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 5 16 12698 12682 158.488 142.062 0.0018046 0.00630117 2024-02-22T19:35:37.977 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:38.006 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:38.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:38.012 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:38.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:38.169 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:38.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:35:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:36.896921+0000 osd.4 (osd.4) 107 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:35:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:36.900542+0000 osd.4 (osd.4) 108 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:35:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.333551+0000 osd.3 (osd.3) 113 : cluster [DBG] 6.c scrub starts 2024-02-22T19:35:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.337243+0000 osd.3 (osd.3) 114 : cluster [DBG] 6.c scrub ok 2024-02-22T19:35:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.368616+0000 mgr.y (mgr.14535) 258 : cluster [DBG] pgmap v284: 148 pgs: 148 active+clean; 196 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 631 B/s rd, 24 MiB/s wr, 387 op/s 2024-02-22T19:35:38.739 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:38.742 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:38.743 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:38.748 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:36.896921+0000 osd.4 (osd.4) 107 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:35:38.748 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:36.900542+0000 osd.4 (osd.4) 108 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:35:38.748 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.333551+0000 osd.3 (osd.3) 113 : cluster [DBG] 6.c scrub starts 2024-02-22T19:35:38.748 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.337243+0000 osd.3 (osd.3) 114 : cluster [DBG] 6.c scrub ok 2024-02-22T19:35:38.749 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.368616+0000 mgr.y (mgr.14535) 258 : cluster [DBG] pgmap v284: 148 pgs: 148 active+clean; 196 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 631 B/s rd, 24 MiB/s wr, 387 op/s 2024-02-22T19:35:38.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:36.896921+0000 osd.4 (osd.4) 107 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:35:38.751 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:36.900542+0000 osd.4 (osd.4) 108 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:35:38.751 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.333551+0000 osd.3 (osd.3) 113 : cluster [DBG] 6.c scrub starts 2024-02-22T19:35:38.751 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.337243+0000 osd.3 (osd.3) 114 : cluster [DBG] 6.c scrub ok 2024-02-22T19:35:38.751 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.368616+0000 mgr.y (mgr.14535) 258 : cluster [DBG] pgmap v284: 148 pgs: 148 active+clean; 196 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 631 B/s rd, 24 MiB/s wr, 387 op/s 2024-02-22T19:35:38.837 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:35:38.838 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:35:38.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:38.903 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:38.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:38.950 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 6 16 14619 14603 152.081 120.062 0.00373447 0.00656928 2024-02-22T19:35:39.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:39.063 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:39.098 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:39.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:39.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:39.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:39.262 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:39.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:39.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:39.329 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.826508+0000 osd.2 (osd.2) 76 : cluster [DBG] 5.a scrub starts 2024-02-22T19:35:39.329 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.830998+0000 osd.2 (osd.2) 77 : cluster [DBG] 5.a scrub ok 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.884099+0000 osd.1 (osd.1) 91 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:37.886782+0000 osd.1 (osd.1) 92 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:38.327887+0000 osd.3 (osd.3) 115 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:38.331385+0000 osd.3 (osd.3) 116 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:38.335124+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.53:0/3074447436' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:38.335730+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:38.350189+0000 mon.a (mon.0) 960 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:39.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.826508+0000 osd.2 (osd.2) 76 : cluster [DBG] 5.a scrub starts 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.830998+0000 osd.2 (osd.2) 77 : cluster [DBG] 5.a scrub ok 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.884099+0000 osd.1 (osd.1) 91 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:37.886782+0000 osd.1 (osd.1) 92 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:38.327887+0000 osd.3 (osd.3) 115 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:38.331385+0000 osd.3 (osd.3) 116 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:38.335124+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.53:0/3074447436' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:38.335730+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:38.350189+0000 mon.a (mon.0) 960 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:39.331 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:35:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:39] "GET /metrics HTTP/1.1" 200 175744 "" "Prometheus/2.33.4" 2024-02-22T19:35:39.338 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:35:39.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:39.416 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.826508+0000 osd.2 (osd.2) 76 : cluster [DBG] 5.a scrub starts 2024-02-22T19:35:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.830998+0000 osd.2 (osd.2) 77 : cluster [DBG] 5.a scrub ok 2024-02-22T19:35:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.884099+0000 osd.1 (osd.1) 91 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:35:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:37.886782+0000 osd.1 (osd.1) 92 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:35:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:38.327887+0000 osd.3 (osd.3) 115 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:35:39.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:38.331385+0000 osd.3 (osd.3) 116 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:35:39.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:38.335124+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.53:0/3074447436' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:39.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:38.335730+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:35:39.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:38.350189+0000 mon.a (mon.0) 960 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:39.947 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 7 16 16811 16795 149.925 137 0.0129256 0.00666449 2024-02-22T19:35:40.087 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:40.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:40.095 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:40.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:40.231 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:40.257 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:40.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:40.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:40.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:40.417 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:38.911421+0000 osd.1 (osd.1) 93 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:35:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:38.917172+0000 osd.1 (osd.1) 94 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:35:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:39.312518+0000 mon.a (mon.0) 961 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:39.319515+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:35:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:39.319578+0000 mon.a (mon.0) 963 : cluster [DBG] osdmap e160: 8 total, 8 up, 7 in 2024-02-22T19:35:40.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:39.369650+0000 mgr.y (mgr.14535) 259 : cluster [DBG] pgmap v286: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-02-22T19:35:40.641 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:40.650 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:40.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:40.651 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:38.911421+0000 osd.1 (osd.1) 93 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:35:40.651 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:38.917172+0000 osd.1 (osd.1) 94 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:35:40.651 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:39.312518+0000 mon.a (mon.0) 961 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:40.651 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:39.319515+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:35:40.651 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:39.319578+0000 mon.a (mon.0) 963 : cluster [DBG] osdmap e160: 8 total, 8 up, 7 in 2024-02-22T19:35:40.651 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:39.369650+0000 mgr.y (mgr.14535) 259 : cluster [DBG] pgmap v286: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-02-22T19:35:40.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:38.911421+0000 osd.1 (osd.1) 93 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:35:40.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:38.917172+0000 osd.1 (osd.1) 94 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:35:40.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:39.312518+0000 mon.a (mon.0) 961 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:40.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:39.319515+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:35:40.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:39.319578+0000 mon.a (mon.0) 963 : cluster [DBG] osdmap e160: 8 total, 8 up, 7 in 2024-02-22T19:35:40.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:39.369650+0000 mgr.y (mgr.14535) 259 : cluster [DBG] pgmap v286: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-02-22T19:35:40.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:40.826 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:40.948 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 8 16 18717 18701 146.074 119.125 0.00841829 0.00683934 2024-02-22T19:35:41.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:35:41.256 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:35:41.272 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:35:41.295 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:41.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:41.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:41.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:41.498 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:40.818058+0000 mgr.y (mgr.14535) 260 : audit [DBG] from='client.24913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:35:41.500 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:41.500 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:40.818058+0000 mgr.y (mgr.14535) 260 : audit [DBG] from='client.24913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:35:41.569 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:41.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:41.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:41.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:41.713 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:40.818058+0000 mgr.y (mgr.14535) 260 : audit [DBG] from='client.24913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:35:41.717 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:41.740 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:41.744 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:41.744 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:41.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:41.879 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:41.909 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:41.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:41.950 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 9 16 20575 20559 142.715 116.125 0.00568646 0.00699963 2024-02-22T19:35:42.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:42.202 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:42.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:42.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:42.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:42.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:41.370417+0000 mgr.y (mgr.14535) 261 : cluster [DBG] pgmap v287: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-02-22T19:35:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:41.370417+0000 mgr.y (mgr.14535) 261 : cluster [DBG] pgmap v287: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-02-22T19:35:42.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:35:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:42.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:42.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:42.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:41.370417+0000 mgr.y (mgr.14535) 261 : cluster [DBG] pgmap v287: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-02-22T19:35:42.906 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:42.950 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 10 16 22349 22333 139.524 110.875 0.00607325 0.00716069 2024-02-22T19:35:43.180 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:43.186 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:43.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:43.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:43.335 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:43.951 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 11 16 24175 24159 137.21 114.125 0.00681151 0.00728098 2024-02-22T19:35:44.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:43.371792+0000 mgr.y (mgr.14535) 262 : cluster [DBG] pgmap v288: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 563 B/s rd, 100 MiB/s wr, 1.59k op/s 2024-02-22T19:35:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:43.371792+0000 mgr.y (mgr.14535) 262 : cluster [DBG] pgmap v288: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 563 B/s rd, 100 MiB/s wr, 1.59k op/s 2024-02-22T19:35:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:35:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:35:44.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:35:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:35:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:43.371792+0000 mgr.y (mgr.14535) 262 : cluster [DBG] pgmap v288: 148 pgs: 148 active+clean; 963 MiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 563 B/s rd, 100 MiB/s wr, 1.59k op/s 2024-02-22T19:35:44.951 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 12 16 26064 26048 135.614 118.062 0.00881337 0.00736845 2024-02-22T19:35:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:44.584383+0000 mgr.y (mgr.14535) 263 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:44.584383+0000 mgr.y (mgr.14535) 263 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:44.584383+0000 mgr.y (mgr.14535) 263 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:45.951 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 13 15 27889 27874 133.961 114.125 0.0254188 0.00745943 2024-02-22T19:35:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:45.372635+0000 mgr.y (mgr.14535) 264 : cluster [DBG] pgmap v289: 148 pgs: 148 active+clean; 1.5 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-02-22T19:35:46.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:45.372635+0000 mgr.y (mgr.14535) 264 : cluster [DBG] pgmap v289: 148 pgs: 148 active+clean; 1.5 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-02-22T19:35:46.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:45.372635+0000 mgr.y (mgr.14535) 264 : cluster [DBG] pgmap v289: 148 pgs: 148 active+clean; 1.5 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-02-22T19:35:46.951 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 14 16 29646 29630 132.231 109.75 0.00416533 0.00755578 2024-02-22T19:35:47.952 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 15 16 31318 31302 130.372 104.5 0.00655562 0.00766505 2024-02-22T19:35:47.978 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:47.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:47.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:48.009 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-02-22T19:35:39.369134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1009522340,"num_objects":15596,"num_object_clones":0,"num_object_copies":46788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15596,"num_whiteouts":0,"num_read":1105,"num_read_kb":857,"num_write":15832,"num_write_kb":985897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":2267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15907,"ondisk_log_size":15907,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":5007052,"kb_used_data":2848660,"kb_used_omap":0,"kb_used_meta":2158336,"kb_avail":651209012,"statfs":{"total":671965249536,"available":666838028288,"internally_reserved":0,"allocated":2917027840,"data_stored":2914227175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2210136064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009516544,"num_objects":15404,"num_object_clones":0,"num_object_copies":46212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15404,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":15404,"num_write_kb":985856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004866"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342610+0000","last_change":"2024-02-22T19:34:05.990011+0000","last_active":"2024-02-22T19:35:30.342610+0000","last_peered":"2024-02-22T19:35:30.342610+0000","last_clean":"2024-02-22T19:35:30.342610+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:35:30.342610+0000","last_undegraded":"2024-02-22T19:35:30.342610+0000","last_fullsized":"2024-02-22T19:35:30.342610+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:05.989944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:05.989944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":126,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260778+0000","last_change":"2024-02-22T19:33:58.781565+0000","last_active":"2024-02-22T19:35:29.260778+0000","last_peered":"2024-02-22T19:35:29.260778+0000","last_clean":"2024-02-22T19:35:29.260778+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:35:29.260778+0000","last_undegraded":"2024-02-22T19:35:29.260778+0000","last_fullsized":"2024-02-22T19:35:29.260778+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.781466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.781466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.286768+0000","last_change":"2024-02-22T19:34:44.286165+0000","last_active":"2024-02-22T19:35:34.286768+0000","last_peered":"2024-02-22T19:35:34.286768+0000","last_clean":"2024-02-22T19:35:34.286768+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:35:34.286768+0000","last_undegraded":"2024-02-22T19:35:34.286768+0000","last_fullsized":"2024-02-22T19:35:34.286768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.286059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.286059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":135,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342407+0000","last_change":"2024-02-22T19:33:53.940446+0000","last_active":"2024-02-22T19:35:30.342407+0000","last_peered":"2024-02-22T19:35:30.342407+0000","last_clean":"2024-02-22T19:35:30.342407+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:35:30.342407+0000","last_undegraded":"2024-02-22T19:35:30.342407+0000","last_fullsized":"2024-02-22T19:35:30.342407+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:53.940375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:53.940375+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":113,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.287477+0000","last_change":"2024-02-22T19:34:30.923797+0000","last_active":"2024-02-22T19:35:34.287477+0000","last_peered":"2024-02-22T19:35:34.287477+0000","last_clean":"2024-02-22T19:35:34.287477+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:35:34.287477+0000","last_undegraded":"2024-02-22T19:35:34.287477+0000","last_fullsized":"2024-02-22T19:35:34.287477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.923707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.923707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":117,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342868+0000","last_change":"2024-02-22T19:35:26.000779+0000","last_active":"2024-02-22T19:35:30.342868+0000","last_peered":"2024-02-22T19:35:30.342868+0000","last_clean":"2024-02-22T19:35:30.342868+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:35:30.342868+0000","last_undegraded":"2024-02-22T19:35:30.342868+0000","last_fullsized":"2024-02-22T19:35:30.342868+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.000689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.000689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":119,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.288827+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:35:34.288827+0000","last_peered":"2024-02-22T19:35:34.288827+0000","last_clean":"2024-02-22T19:35:34.288827+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:35:34.288827+0000","last_undegraded":"2024-02-22T19:35:34.288827+0000","last_fullsized":"2024-02-22T19:35:34.288827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.284421+0000","last_change":"2024-02-22T19:34:45.948451+0000","last_active":"2024-02-22T19:35:34.284421+0000","last_peered":"2024-02-22T19:35:34.284421+0000","last_clean":"2024-02-22T19:35:34.284421+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:35:34.284421+0000","last_undegraded":"2024-02-22T19:35:34.284421+0000","last_fullsized":"2024-02-22T19:35:34.284421+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:45.948358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:45.948358+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":119,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.286518+0000","last_change":"2024-02-22T19:35:23.244026+0000","last_active":"2024-02-22T19:35:34.286518+0000","last_peered":"2024-02-22T19:35:34.286518+0000","last_clean":"2024-02-22T19:35:34.286518+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:35:34.286518+0000","last_undegraded":"2024-02-22T19:35:34.286518+0000","last_fullsized":"2024-02-22T19:35:34.286518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:23.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:23.243945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":82,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.284890+0000","last_change":"2024-02-22T19:35:26.232511+0000","last_active":"2024-02-22T19:35:34.284890+0000","last_peered":"2024-02-22T19:35:34.284890+0000","last_clean":"2024-02-22T19:35:34.284890+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:35:34.284890+0000","last_undegraded":"2024-02-22T19:35:34.284890+0000","last_fullsized":"2024-02-22T19:35:34.284890+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.232444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.232444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":67,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293436+0000","last_change":"2024-02-22T19:35:22.835205+0000","last_active":"2024-02-22T19:35:34.293436+0000","last_peered":"2024-02-22T19:35:34.293436+0000","last_clean":"2024-02-22T19:35:34.293436+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:35:34.293436+0000","last_undegraded":"2024-02-22T19:35:34.293436+0000","last_fullsized":"2024-02-22T19:35:34.293436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.835123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.835123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":135,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342924+0000","last_change":"2024-02-22T19:33:52.938521+0000","last_active":"2024-02-22T19:35:30.342924+0000","last_peered":"2024-02-22T19:35:30.342924+0000","last_clean":"2024-02-22T19:35:30.342924+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:35:30.342924+0000","last_undegraded":"2024-02-22T19:35:30.342924+0000","last_fullsized":"2024-02-22T19:35:30.342924+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:33:52.938440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.938440+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":76,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260687+0000","last_change":"2024-02-22T19:34:02.783853+0000","last_active":"2024-02-22T19:35:29.260687+0000","last_peered":"2024-02-22T19:35:29.260687+0000","last_clean":"2024-02-22T19:35:29.260687+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:35:29.260687+0000","last_undegraded":"2024-02-22T19:35:29.260687+0000","last_fullsized":"2024-02-22T19:35:29.260687+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:34:02.783771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":115,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259461+0000","last_change":"2024-02-22T19:34:30.499933+0000","last_active":"2024-02-22T19:35:29.259461+0000","last_peered":"2024-02-22T19:35:29.259461+0000","last_clean":"2024-02-22T19:35:29.259461+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:35:29.259461+0000","last_undegraded":"2024-02-22T19:35:29.259461+0000","last_fullsized":"2024-02-22T19:35:29.259461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.499864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.499864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":88,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:35.930757+0000","last_change":"2024-02-22T19:35:35.930757+0000","last_active":"2024-02-22T19:35:35.930757+0000","last_peered":"2024-02-22T19:35:35.930757+0000","last_clean":"2024-02-22T19:35:35.930757+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:35:35.930757+0000","last_undegraded":"2024-02-22T19:35:35.930757+0000","last_fullsized":"2024-02-22T19:35:35.930757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:35:35.930654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.930654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":129,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343195+0000","last_change":"2024-02-22T19:34:46.118383+0000","last_active":"2024-02-22T19:35:30.343195+0000","last_peered":"2024-02-22T19:35:30.343195+0000","last_clean":"2024-02-22T19:35:30.343195+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:35:30.343195+0000","last_undegraded":"2024-02-22T19:35:30.343195+0000","last_fullsized":"2024-02-22T19:35:30.343195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:46.118282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.118282+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343526+0000","last_change":"2024-02-22T19:34:56.107642+0000","last_active":"2024-02-22T19:35:30.343526+0000","last_peered":"2024-02-22T19:35:30.343526+0000","last_clean":"2024-02-22T19:35:30.343526+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:35:30.343526+0000","last_undegraded":"2024-02-22T19:35:30.343526+0000","last_fullsized":"2024-02-22T19:35:30.343526+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:56.107558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:56.107558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":117,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259257+0000","last_change":"2024-02-22T19:34:31.025213+0000","last_active":"2024-02-22T19:35:29.259257+0000","last_peered":"2024-02-22T19:35:29.259257+0000","last_clean":"2024-02-22T19:35:29.259257+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:35:29.259257+0000","last_undegraded":"2024-02-22T19:35:29.259257+0000","last_fullsized":"2024-02-22T19:35:29.259257+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.025113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.025113+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":125,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285245+0000","last_change":"2024-02-22T19:34:53.279758+0000","last_active":"2024-02-22T19:35:34.285245+0000","last_peered":"2024-02-22T19:35:34.285245+0000","last_clean":"2024-02-22T19:35:34.285245+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:35:34.285245+0000","last_undegraded":"2024-02-22T19:35:34.285245+0000","last_fullsized":"2024-02-22T19:35:34.285245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:53.279662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:34:53.279662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293075+0000","last_change":"2024-02-22T19:34:40.861961+0000","last_active":"2024-02-22T19:35:34.293075+0000","last_peered":"2024-02-22T19:35:34.293075+0000","last_clean":"2024-02-22T19:35:34.293075+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:35:34.293075+0000","last_undegraded":"2024-02-22T19:35:34.293075+0000","last_fullsized":"2024-02-22T19:35:34.293075+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:40.861854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:40.861854+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":122,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.290186+0000","last_change":"2024-02-22T19:33:48.835132+0000","last_active":"2024-02-22T19:35:34.290186+0000","last_peered":"2024-02-22T19:35:34.290186+0000","last_clean":"2024-02-22T19:35:34.290186+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:35:34.290186+0000","last_undegraded":"2024-02-22T19:35:34.290186+0000","last_fullsized":"2024-02-22T19:35:34.290186+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.835030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.835030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":117,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343072+0000","last_change":"2024-02-22T19:35:24.048780+0000","last_active":"2024-02-22T19:35:30.343072+0000","last_peered":"2024-02-22T19:35:30.343072+0000","last_clean":"2024-02-22T19:35:30.343072+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:35:30.343072+0000","last_undegraded":"2024-02-22T19:35:30.343072+0000","last_fullsized":"2024-02-22T19:35:30.343072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:24.048696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.048696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":121,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342947+0000","last_change":"2024-02-22T19:34:30.007888+0000","last_active":"2024-02-22T19:35:30.342947+0000","last_peered":"2024-02-22T19:35:30.342947+0000","last_clean":"2024-02-22T19:35:30.342947+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:35:30.342947+0000","last_undegraded":"2024-02-22T19:35:30.342947+0000","last_fullsized":"2024-02-22T19:35:30.342947+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.007814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.007814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":145,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:36.338845+0000","last_change":"2024-02-22T19:35:36.338814+0000","last_active":"2024-02-22T19:35:36.338845+0000","last_peered":"2024-02-22T19:35:36.338845+0000","last_clean":"2024-02-22T19:35:36.338845+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:35:36.338845+0000","last_undegraded":"2024-02-22T19:35:36.338845+0000","last_fullsized":"2024-02-22T19:35:36.338845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":128,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259978+0000","last_change":"2024-02-22T19:34:03.590429+0000","last_active":"2024-02-22T19:35:29.259978+0000","last_peered":"2024-02-22T19:35:29.259978+0000","last_clean":"2024-02-22T19:35:29.259978+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:35:29.259978+0000","last_undegraded":"2024-02-22T19:35:29.259978+0000","last_fullsized":"2024-02-22T19:35:29.259978+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:34:03.590356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"137'9","reported_seq":215,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.225363+0000","last_change":"2024-02-22T19:35:28.262175+0000","last_active":"2024-02-22T19:35:34.225363+0000","last_peered":"2024-02-22T19:35:34.225363+0000","last_clean":"2024-02-22T19:35:34.225363+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:35:34.225363+0000","last_undegraded":"2024-02-22T19:35:34.225363+0000","last_fullsized":"2024-02-22T19:35:34.225363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:35:28.262089+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:35:28.262089+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":126,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.790903+0000","last_change":"2024-02-22T19:35:30.790879+0000","last_active":"2024-02-22T19:35:30.790903+0000","last_peered":"2024-02-22T19:35:30.790903+0000","last_clean":"2024-02-22T19:35:30.790903+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:35:30.790903+0000","last_undegraded":"2024-02-22T19:35:30.790903+0000","last_fullsized":"2024-02-22T19:35:30.790903+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.790805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.790805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":133,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285903+0000","last_change":"2024-02-22T19:34:01.123505+0000","last_active":"2024-02-22T19:35:34.285903+0000","last_peered":"2024-02-22T19:35:34.285903+0000","last_clean":"2024-02-22T19:35:34.285903+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:35:34.285903+0000","last_undegraded":"2024-02-22T19:35:34.285903+0000","last_fullsized":"2024-02-22T19:35:34.285903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:34:01.123405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.123405+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":126,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259384+0000","last_change":"2024-02-22T19:34:42.969287+0000","last_active":"2024-02-22T19:35:29.259384+0000","last_peered":"2024-02-22T19:35:29.259384+0000","last_clean":"2024-02-22T19:35:29.259384+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:35:29.259384+0000","last_undegraded":"2024-02-22T19:35:29.259384+0000","last_fullsized":"2024-02-22T19:35:29.259384+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.969189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.969189+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":116,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.288141+0000","last_change":"2024-02-22T19:34:33.122075+0000","last_active":"2024-02-22T19:35:34.288141+0000","last_peered":"2024-02-22T19:35:34.288141+0000","last_clean":"2024-02-22T19:35:34.288141+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:35:34.288141+0000","last_undegraded":"2024-02-22T19:35:34.288141+0000","last_fullsized":"2024-02-22T19:35:34.288141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:33.121981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.121981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":121,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285656+0000","last_change":"2024-02-22T19:34:58.165060+0000","last_active":"2024-02-22T19:35:34.285656+0000","last_peered":"2024-02-22T19:35:34.285656+0000","last_clean":"2024-02-22T19:35:34.285656+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:35:34.285656+0000","last_undegraded":"2024-02-22T19:35:34.285656+0000","last_fullsized":"2024-02-22T19:35:34.285656+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:34:58.164960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":193,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.753741+0000","last_change":"2024-02-22T19:35:29.242870+0000","last_active":"2024-02-22T19:35:29.753741+0000","last_peered":"2024-02-22T19:35:29.753741+0000","last_clean":"2024-02-22T19:35:29.753741+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:35:29.753741+0000","last_undegraded":"2024-02-22T19:35:29.753741+0000","last_fullsized":"2024-02-22T19:35:29.753741+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:35:29.242798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:29.242798+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":113,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259350+0000","last_change":"2024-02-22T19:34:34.514336+0000","last_active":"2024-02-22T19:35:29.259350+0000","last_peered":"2024-02-22T19:35:29.259350+0000","last_clean":"2024-02-22T19:35:29.259350+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:35:29.259350+0000","last_undegraded":"2024-02-22T19:35:29.259350+0000","last_fullsized":"2024-02-22T19:35:29.259350+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:34.514266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:34.514266+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":114,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.753264+0000","last_change":"2024-02-22T19:34:32.606492+0000","last_active":"2024-02-22T19:35:29.753264+0000","last_peered":"2024-02-22T19:35:29.753264+0000","last_clean":"2024-02-22T19:35:29.753264+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:35:29.753264+0000","last_undegraded":"2024-02-22T19:35:29.753264+0000","last_fullsized":"2024-02-22T19:35:29.753264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.606397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.606397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.286998+0000","last_change":"2024-02-22T19:35:21.292096+0000","last_active":"2024-02-22T19:35:34.286998+0000","last_peered":"2024-02-22T19:35:34.286998+0000","last_clean":"2024-02-22T19:35:34.286998+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:35:34.286998+0000","last_undegraded":"2024-02-22T19:35:34.286998+0000","last_fullsized":"2024-02-22T19:35:34.286998+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:21.292020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:21.292020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":123,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285735+0000","last_change":"2024-02-22T19:34:39.037075+0000","last_active":"2024-02-22T19:35:34.285735+0000","last_peered":"2024-02-22T19:35:34.285735+0000","last_clean":"2024-02-22T19:35:34.285735+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:35:34.285735+0000","last_undegraded":"2024-02-22T19:35:34.285735+0000","last_fullsized":"2024-02-22T19:35:34.285735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:34:39.036999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.036999+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":113,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.287682+0000","last_change":"2024-02-22T19:34:36.044624+0000","last_active":"2024-02-22T19:35:34.287682+0000","last_peered":"2024-02-22T19:35:34.287682+0000","last_clean":"2024-02-22T19:35:34.287682+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:35:34.287682+0000","last_undegraded":"2024-02-22T19:35:34.287682+0000","last_fullsized":"2024-02-22T19:35:34.287682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.044519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.044519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":120,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:33.660343+0000","last_change":"2024-02-22T19:35:33.660316+0000","last_active":"2024-02-22T19:35:33.660343+0000","last_peered":"2024-02-22T19:35:33.660343+0000","last_clean":"2024-02-22T19:35:33.660343+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:35:33.660343+0000","last_undegraded":"2024-02-22T19:35:33.660343+0000","last_fullsized":"2024-02-22T19:35:33.660343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.660254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.660254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":125,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:32.621616+0000","last_change":"2024-02-22T19:35:32.621589+0000","last_active":"2024-02-22T19:35:32.621616+0000","last_peered":"2024-02-22T19:35:32.621616+0000","last_clean":"2024-02-22T19:35:32.621616+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:35:32.621616+0000","last_undegraded":"2024-02-22T19:35:32.621616+0000","last_fullsized":"2024-02-22T19:35:32.621616+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:32.621508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:32.621508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.284360+0000","last_change":"2024-02-22T19:34:43.915568+0000","last_active":"2024-02-22T19:35:34.284360+0000","last_peered":"2024-02-22T19:35:34.284360+0000","last_clean":"2024-02-22T19:35:34.284360+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:35:34.284360+0000","last_undegraded":"2024-02-22T19:35:34.284360+0000","last_fullsized":"2024-02-22T19:35:34.284360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:43.915463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.915463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":123,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285636+0000","last_change":"2024-02-22T19:34:48.292233+0000","last_active":"2024-02-22T19:35:34.285636+0000","last_peered":"2024-02-22T19:35:34.285636+0000","last_clean":"2024-02-22T19:35:34.285636+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:35:34.285636+0000","last_undegraded":"2024-02-22T19:35:34.285636+0000","last_fullsized":"2024-02-22T19:35:34.285636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:48.292131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.292131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":115,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.287395+0000","last_change":"2024-02-22T19:34:41.961882+0000","last_active":"2024-02-22T19:35:34.287395+0000","last_peered":"2024-02-22T19:35:34.287395+0000","last_clean":"2024-02-22T19:35:34.287395+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:35:34.287395+0000","last_undegraded":"2024-02-22T19:35:34.287395+0000","last_fullsized":"2024-02-22T19:35:34.287395+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.961788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.961788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260034+0000","last_change":"2024-02-22T19:34:49.595355+0000","last_active":"2024-02-22T19:35:29.260034+0000","last_peered":"2024-02-22T19:35:29.260034+0000","last_clean":"2024-02-22T19:35:29.260034+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:35:29.260034+0000","last_undegraded":"2024-02-22T19:35:29.260034+0000","last_fullsized":"2024-02-22T19:35:29.260034+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:49.595253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:34:49.595253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":140,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:35.855558+0000","last_change":"2024-02-22T19:35:35.855558+0000","last_active":"2024-02-22T19:35:35.855558+0000","last_peered":"2024-02-22T19:35:35.855558+0000","last_clean":"2024-02-22T19:35:35.855558+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:35:35.855558+0000","last_undegraded":"2024-02-22T19:35:35.855558+0000","last_fullsized":"2024-02-22T19:35:35.855558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:35.855449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.855449+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":122,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.284213+0000","last_change":"2024-02-22T19:33:42.862634+0000","last_active":"2024-02-22T19:35:34.284213+0000","last_peered":"2024-02-22T19:35:34.284213+0000","last_clean":"2024-02-22T19:35:34.284213+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:35:34.284213+0000","last_undegraded":"2024-02-22T19:35:34.284213+0000","last_fullsized":"2024-02-22T19:35:34.284213+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:42.862563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.862563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":126,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260253+0000","last_change":"2024-02-22T19:34:44.906578+0000","last_active":"2024-02-22T19:35:29.260253+0000","last_peered":"2024-02-22T19:35:29.260253+0000","last_clean":"2024-02-22T19:35:29.260253+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:35:29.260253+0000","last_undegraded":"2024-02-22T19:35:29.260253+0000","last_fullsized":"2024-02-22T19:35:29.260253+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.906461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.906461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":121,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342799+0000","last_change":"2024-02-22T19:34:31.047792+0000","last_active":"2024-02-22T19:35:30.342799+0000","last_peered":"2024-02-22T19:35:30.342799+0000","last_clean":"2024-02-22T19:35:30.342799+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:35:30.342799+0000","last_undegraded":"2024-02-22T19:35:30.342799+0000","last_fullsized":"2024-02-22T19:35:30.342799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.047579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.047579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":138,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343142+0000","last_change":"2024-02-22T19:34:59.142089+0000","last_active":"2024-02-22T19:35:30.343142+0000","last_peered":"2024-02-22T19:35:30.343142+0000","last_clean":"2024-02-22T19:35:30.343142+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:35:30.343142+0000","last_undegraded":"2024-02-22T19:35:30.343142+0000","last_fullsized":"2024-02-22T19:35:30.343142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:59.141999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:59.141999+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":114,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285779+0000","last_change":"2024-02-22T19:34:29.079997+0000","last_active":"2024-02-22T19:35:34.285779+0000","last_peered":"2024-02-22T19:35:34.285779+0000","last_clean":"2024-02-22T19:35:34.285779+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:35:34.285779+0000","last_undegraded":"2024-02-22T19:35:34.285779+0000","last_fullsized":"2024-02-22T19:35:34.285779+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:29.079891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:29.079891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":146,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.289182+0000","last_change":"2024-02-22T19:34:57.925328+0000","last_active":"2024-02-22T19:35:34.289182+0000","last_peered":"2024-02-22T19:35:34.289182+0000","last_clean":"2024-02-22T19:35:34.289182+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:35:34.289182+0000","last_undegraded":"2024-02-22T19:35:34.289182+0000","last_fullsized":"2024-02-22T19:35:34.289182+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:57.925253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.925253+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":119,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259590+0000","last_change":"2024-02-22T19:34:42.438663+0000","last_active":"2024-02-22T19:35:29.259590+0000","last_peered":"2024-02-22T19:35:29.259590+0000","last_clean":"2024-02-22T19:35:29.259590+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:35:29.259590+0000","last_undegraded":"2024-02-22T19:35:29.259590+0000","last_fullsized":"2024-02-22T19:35:29.259590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.438557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.438557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.289394+0000","last_change":"2024-02-22T19:34:32.908841+0000","last_active":"2024-02-22T19:35:34.289394+0000","last_peered":"2024-02-22T19:35:34.289394+0000","last_clean":"2024-02-22T19:35:34.289394+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:35:34.289394+0000","last_undegraded":"2024-02-22T19:35:34.289394+0000","last_fullsized":"2024-02-22T19:35:34.289394+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:34:32.908721+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.908721+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343625+0000","last_change":"2024-02-22T19:33:58.835432+0000","last_active":"2024-02-22T19:35:30.343625+0000","last_peered":"2024-02-22T19:35:30.343625+0000","last_clean":"2024-02-22T19:35:30.343625+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:35:30.343625+0000","last_undegraded":"2024-02-22T19:35:30.343625+0000","last_fullsized":"2024-02-22T19:35:30.343625+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.835370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:16.057632+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.835370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":203,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.226601+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:35:39.226601+0000","last_peered":"2024-02-22T19:35:39.226601+0000","last_clean":"2024-02-22T19:35:39.226601+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:35:39.226601+0000","last_undegraded":"2024-02-22T19:35:39.226601+0000","last_fullsized":"2024-02-22T19:35:39.226601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":125,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285383+0000","last_change":"2024-02-22T19:34:41.254550+0000","last_active":"2024-02-22T19:35:34.285383+0000","last_peered":"2024-02-22T19:35:34.285383+0000","last_clean":"2024-02-22T19:35:34.285383+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:35:34.285383+0000","last_undegraded":"2024-02-22T19:35:34.285383+0000","last_fullsized":"2024-02-22T19:35:34.285383+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.254449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.254449+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":144,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260247+0000","last_change":"2024-02-22T19:34:47.527168+0000","last_active":"2024-02-22T19:35:29.260247+0000","last_peered":"2024-02-22T19:35:29.260247+0000","last_clean":"2024-02-22T19:35:29.260247+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:35:29.260247+0000","last_undegraded":"2024-02-22T19:35:29.260247+0000","last_fullsized":"2024-02-22T19:35:29.260247+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:34:47.527066+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.527066+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":121,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293338+0000","last_change":"2024-02-22T19:35:31.810419+0000","last_active":"2024-02-22T19:35:34.293338+0000","last_peered":"2024-02-22T19:35:34.293338+0000","last_clean":"2024-02-22T19:35:34.293338+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:35:34.293338+0000","last_undegraded":"2024-02-22T19:35:34.293338+0000","last_fullsized":"2024-02-22T19:35:34.293338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.810353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":110,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.290109+0000","last_change":"2024-02-22T19:31:01.012619+0000","last_active":"2024-02-22T19:35:34.290109+0000","last_peered":"2024-02-22T19:35:34.290109+0000","last_clean":"2024-02-22T19:35:34.290109+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:35:34.290109+0000","last_undegraded":"2024-02-22T19:35:34.290109+0000","last_fullsized":"2024-02-22T19:35:34.290109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:01.012492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:31:01.012492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.620082+0000","last_change":"2024-02-22T19:35:30.620049+0000","last_active":"2024-02-22T19:35:30.620082+0000","last_peered":"2024-02-22T19:35:30.620082+0000","last_clean":"2024-02-22T19:35:30.620082+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:35:30.620082+0000","last_undegraded":"2024-02-22T19:35:30.620082+0000","last_fullsized":"2024-02-22T19:35:30.620082+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.619969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":139,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285945+0000","last_change":"2024-02-22T19:35:27.255099+0000","last_active":"2024-02-22T19:35:34.285945+0000","last_peered":"2024-02-22T19:35:34.285945+0000","last_clean":"2024-02-22T19:35:34.285945+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:35:34.285945+0000","last_undegraded":"2024-02-22T19:35:34.285945+0000","last_fullsized":"2024-02-22T19:35:34.285945+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":118,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:31.614072+0000","last_change":"2024-02-22T19:35:31.614031+0000","last_active":"2024-02-22T19:35:31.614072+0000","last_peered":"2024-02-22T19:35:31.614072+0000","last_clean":"2024-02-22T19:35:31.614072+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:35:31.614072+0000","last_undegraded":"2024-02-22T19:35:31.614072+0000","last_fullsized":"2024-02-22T19:35:31.614072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.613909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.613909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":77,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.288955+0000","last_change":"2024-02-22T19:34:31.909139+0000","last_active":"2024-02-22T19:35:34.288955+0000","last_peered":"2024-02-22T19:35:34.288955+0000","last_clean":"2024-02-22T19:35:34.288955+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:35:34.288955+0000","last_undegraded":"2024-02-22T19:35:34.288955+0000","last_fullsized":"2024-02-22T19:35:34.288955+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.909036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.909036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":134,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.753010+0000","last_change":"2024-02-22T19:35:25.227250+0000","last_active":"2024-02-22T19:35:29.753010+0000","last_peered":"2024-02-22T19:35:29.753010+0000","last_clean":"2024-02-22T19:35:29.753010+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:35:29.753010+0000","last_undegraded":"2024-02-22T19:35:29.753010+0000","last_fullsized":"2024-02-22T19:35:29.753010+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:25.227185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.227185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":126,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.290263+0000","last_change":"2024-02-22T19:33:52.788152+0000","last_active":"2024-02-22T19:35:34.290263+0000","last_peered":"2024-02-22T19:35:34.290263+0000","last_clean":"2024-02-22T19:35:34.290263+0000","last_became_active":"2024-02-22T19:29:03.447245+0000","last_became_peered":"2024-02-22T19:29:03.447245+0000","last_unstale":"2024-02-22T19:35:34.290263+0000","last_undegraded":"2024-02-22T19:35:34.290263+0000","last_fullsized":"2024-02-22T19:35:34.290263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:33:52.788050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.788050+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"159'923","reported_seq":936,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.215671+0000","last_change":"2024-02-22T19:35:19.745999+0000","last_active":"2024-02-22T19:35:39.215671+0000","last_peered":"2024-02-22T19:35:39.215671+0000","last_clean":"2024-02-22T19:35:39.215671+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:35:39.215671+0000","last_undegraded":"2024-02-22T19:35:39.215671+0000","last_fullsized":"2024-02-22T19:35:39.215671+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60489728,"num_objects":923,"num_object_clones":0,"num_object_copies":2769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":923,"num_write_kb":59072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":81,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.753095+0000","last_change":"2024-02-22T19:34:53.494596+0000","last_active":"2024-02-22T19:35:29.753095+0000","last_peered":"2024-02-22T19:35:29.753095+0000","last_clean":"2024-02-22T19:35:29.753095+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:35:29.753095+0000","last_undegraded":"2024-02-22T19:35:29.753095+0000","last_fullsized":"2024-02-22T19:35:29.753095+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:53.494519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:53.494519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":134,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260340+0000","last_change":"2024-02-22T19:34:01.596593+0000","last_active":"2024-02-22T19:35:29.260340+0000","last_peered":"2024-02-22T19:35:29.260340+0000","last_clean":"2024-02-22T19:35:29.260340+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:35:29.260340+0000","last_undegraded":"2024-02-22T19:35:29.260340+0000","last_fullsized":"2024-02-22T19:35:29.260340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:34:01.596491+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.596491+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":129,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259467+0000","last_change":"2024-02-22T19:34:28.519793+0000","last_active":"2024-02-22T19:35:29.259467+0000","last_peered":"2024-02-22T19:35:29.259467+0000","last_clean":"2024-02-22T19:35:29.259467+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:35:29.259467+0000","last_undegraded":"2024-02-22T19:35:29.259467+0000","last_fullsized":"2024-02-22T19:35:29.259467+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:28.519724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.519724+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":203,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.281683+0000","last_change":"2024-02-22T19:34:39.483818+0000","last_active":"2024-02-22T19:35:34.281683+0000","last_peered":"2024-02-22T19:35:34.281683+0000","last_clean":"2024-02-22T19:35:34.281683+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:35:34.281683+0000","last_undegraded":"2024-02-22T19:35:34.281683+0000","last_fullsized":"2024-02-22T19:35:34.281683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:34:39.483740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.483740+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"159'998","reported_seq":1011,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.210841+0000","last_change":"2024-02-22T19:35:19.226102+0000","last_active":"2024-02-22T19:35:39.210841+0000","last_peered":"2024-02-22T19:35:39.210841+0000","last_clean":"2024-02-22T19:35:39.210841+0000","last_became_active":"2024-02-22T19:35:19.225793+0000","last_became_peered":"2024-02-22T19:35:19.225793+0000","last_unstale":"2024-02-22T19:35:39.210841+0000","last_undegraded":"2024-02-22T19:35:39.210841+0000","last_fullsized":"2024-02-22T19:35:39.210841+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":998,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65404928,"num_objects":998,"num_object_clones":0,"num_object_copies":2994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":998,"num_write_kb":63872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":108,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.287566+0000","last_change":"2024-02-22T19:31:06.066327+0000","last_active":"2024-02-22T19:35:34.287566+0000","last_peered":"2024-02-22T19:35:34.287566+0000","last_clean":"2024-02-22T19:35:34.287566+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:35:34.287566+0000","last_undegraded":"2024-02-22T19:35:34.287566+0000","last_fullsized":"2024-02-22T19:35:34.287566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:31:06.066247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:31:06.066247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260218+0000","last_change":"2024-02-22T19:35:22.515913+0000","last_active":"2024-02-22T19:35:29.260218+0000","last_peered":"2024-02-22T19:35:29.260218+0000","last_clean":"2024-02-22T19:35:29.260218+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:35:29.260218+0000","last_undegraded":"2024-02-22T19:35:29.260218+0000","last_fullsized":"2024-02-22T19:35:29.260218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.515824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.515824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342715+0000","last_change":"2024-02-22T19:34:33.123549+0000","last_active":"2024-02-22T19:35:30.342715+0000","last_peered":"2024-02-22T19:35:30.342715+0000","last_clean":"2024-02-22T19:35:30.342715+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:35:30.342715+0000","last_undegraded":"2024-02-22T19:35:30.342715+0000","last_fullsized":"2024-02-22T19:35:30.342715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:33.123452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.123452+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":118,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.292495+0000","last_change":"2024-02-22T19:34:28.806832+0000","last_active":"2024-02-22T19:35:34.292495+0000","last_peered":"2024-02-22T19:35:34.292495+0000","last_clean":"2024-02-22T19:35:34.292495+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:35:34.292495+0000","last_undegraded":"2024-02-22T19:35:34.292495+0000","last_fullsized":"2024-02-22T19:35:34.292495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:28.806726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.806726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"159'985","reported_seq":999,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.319318+0000","last_change":"2024-02-22T19:35:19.225144+0000","last_active":"2024-02-22T19:35:39.319318+0000","last_peered":"2024-02-22T19:35:39.319318+0000","last_clean":"2024-02-22T19:35:39.319318+0000","last_became_active":"2024-02-22T19:35:19.224823+0000","last_became_peered":"2024-02-22T19:35:19.224823+0000","last_unstale":"2024-02-22T19:35:39.319318+0000","last_undegraded":"2024-02-22T19:35:39.319318+0000","last_fullsized":"2024-02-22T19:35:39.319318+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":985,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64552960,"num_objects":985,"num_object_clones":0,"num_object_copies":2955,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":985,"num_write_kb":63040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":113,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259144+0000","last_change":"2024-02-22T19:34:43.434105+0000","last_active":"2024-02-22T19:35:29.259144+0000","last_peered":"2024-02-22T19:35:29.259144+0000","last_clean":"2024-02-22T19:35:29.259144+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:35:29.259144+0000","last_undegraded":"2024-02-22T19:35:29.259144+0000","last_fullsized":"2024-02-22T19:35:29.259144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:43.434030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.434030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.292962+0000","last_change":"2024-02-22T19:33:48.766810+0000","last_active":"2024-02-22T19:35:34.292962+0000","last_peered":"2024-02-22T19:35:34.292962+0000","last_clean":"2024-02-22T19:35:34.292962+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:35:34.292962+0000","last_undegraded":"2024-02-22T19:35:34.292962+0000","last_fullsized":"2024-02-22T19:35:34.292962+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.766706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.766706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":133,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260159+0000","last_change":"2024-02-22T19:34:46.476131+0000","last_active":"2024-02-22T19:35:29.260159+0000","last_peered":"2024-02-22T19:35:29.260159+0000","last_clean":"2024-02-22T19:35:29.260159+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:35:29.260159+0000","last_undegraded":"2024-02-22T19:35:29.260159+0000","last_fullsized":"2024-02-22T19:35:29.260159+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.476034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.476034+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":152,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.753339+0000","last_change":"2024-02-22T19:35:22.253473+0000","last_active":"2024-02-22T19:35:29.753339+0000","last_peered":"2024-02-22T19:35:29.753339+0000","last_clean":"2024-02-22T19:35:29.753339+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:35:29.753339+0000","last_undegraded":"2024-02-22T19:35:29.753339+0000","last_fullsized":"2024-02-22T19:35:29.753339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:22.253381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.253381+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.7","version":"159'967","reported_seq":980,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.285235+0000","last_change":"2024-02-22T19:35:19.227534+0000","last_active":"2024-02-22T19:35:39.285235+0000","last_peered":"2024-02-22T19:35:39.285235+0000","last_clean":"2024-02-22T19:35:39.285235+0000","last_became_active":"2024-02-22T19:35:19.227265+0000","last_became_peered":"2024-02-22T19:35:19.227265+0000","last_unstale":"2024-02-22T19:35:39.285235+0000","last_undegraded":"2024-02-22T19:35:39.285235+0000","last_fullsized":"2024-02-22T19:35:39.285235+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":967,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63373312,"num_objects":967,"num_object_clones":0,"num_object_copies":2901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":967,"num_write_kb":61888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":114,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285308+0000","last_change":"2024-02-22T19:34:43.230772+0000","last_active":"2024-02-22T19:35:34.285308+0000","last_peered":"2024-02-22T19:35:34.285308+0000","last_clean":"2024-02-22T19:35:34.285308+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:35:34.285308+0000","last_undegraded":"2024-02-22T19:35:34.285308+0000","last_fullsized":"2024-02-22T19:35:34.285308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:43.230673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.230673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":123,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:37.831134+0000","last_change":"2024-02-22T19:35:37.831080+0000","last_active":"2024-02-22T19:35:37.831134+0000","last_peered":"2024-02-22T19:35:37.831134+0000","last_clean":"2024-02-22T19:35:37.831134+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:35:37.831134+0000","last_undegraded":"2024-02-22T19:35:37.831134+0000","last_fullsized":"2024-02-22T19:35:37.831134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.831003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.831003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":77,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.289473+0000","last_change":"2024-02-22T19:34:04.892928+0000","last_active":"2024-02-22T19:35:34.289473+0000","last_peered":"2024-02-22T19:35:34.289473+0000","last_clean":"2024-02-22T19:35:34.289473+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:35:34.289473+0000","last_undegraded":"2024-02-22T19:35:34.289473+0000","last_fullsized":"2024-02-22T19:35:34.289473+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:04.892837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:04.892837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285378+0000","last_change":"2024-02-22T19:34:30.097726+0000","last_active":"2024-02-22T19:35:34.285378+0000","last_peered":"2024-02-22T19:35:34.285378+0000","last_clean":"2024-02-22T19:35:34.285378+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:35:34.285378+0000","last_undegraded":"2024-02-22T19:35:34.285378+0000","last_fullsized":"2024-02-22T19:35:34.285378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:34:30.097667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.097667+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"160'1007","reported_seq":1021,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.346234+0000","last_change":"2024-02-22T19:35:19.226141+0000","last_active":"2024-02-22T19:35:39.346234+0000","last_peered":"2024-02-22T19:35:39.346234+0000","last_clean":"2024-02-22T19:35:39.346234+0000","last_became_active":"2024-02-22T19:35:19.225860+0000","last_became_peered":"2024-02-22T19:35:19.225860+0000","last_unstale":"2024-02-22T19:35:39.346234+0000","last_undegraded":"2024-02-22T19:35:39.346234+0000","last_fullsized":"2024-02-22T19:35:39.346234+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":1007,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65994752,"num_objects":1007,"num_object_clones":0,"num_object_copies":3021,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":64448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":113,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.289099+0000","last_change":"2024-02-22T19:34:46.954016+0000","last_active":"2024-02-22T19:35:34.289099+0000","last_peered":"2024-02-22T19:35:34.289099+0000","last_clean":"2024-02-22T19:35:34.289099+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:35:34.289099+0000","last_undegraded":"2024-02-22T19:35:34.289099+0000","last_fullsized":"2024-02-22T19:35:34.289099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.953951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.953951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":76,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260030+0000","last_change":"2024-02-22T19:34:47.963364+0000","last_active":"2024-02-22T19:35:29.260030+0000","last_peered":"2024-02-22T19:35:29.260030+0000","last_clean":"2024-02-22T19:35:29.260030+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:35:29.260030+0000","last_undegraded":"2024-02-22T19:35:29.260030+0000","last_fullsized":"2024-02-22T19:35:29.260030+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:47.963267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.963267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342922+0000","last_change":"2024-02-22T19:34:32.095049+0000","last_active":"2024-02-22T19:35:30.342922+0000","last_peered":"2024-02-22T19:35:30.342922+0000","last_clean":"2024-02-22T19:35:30.342922+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:35:30.342922+0000","last_undegraded":"2024-02-22T19:35:30.342922+0000","last_fullsized":"2024-02-22T19:35:30.342922+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.094985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.094985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":252,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.291400+0000","last_change":"2024-02-22T19:35:20.843884+0000","last_active":"2024-02-22T19:35:34.291400+0000","last_peered":"2024-02-22T19:35:34.291400+0000","last_clean":"2024-02-22T19:35:34.291400+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:35:34.291400+0000","last_undegraded":"2024-02-22T19:35:34.291400+0000","last_fullsized":"2024-02-22T19:35:34.291400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:35:20.843809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:20.843809+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":116,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.287852+0000","last_change":"2024-02-22T19:33:58.185263+0000","last_active":"2024-02-22T19:35:34.287852+0000","last_peered":"2024-02-22T19:35:34.287852+0000","last_clean":"2024-02-22T19:35:34.287852+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:35:34.287852+0000","last_undegraded":"2024-02-22T19:35:34.287852+0000","last_fullsized":"2024-02-22T19:35:34.287852+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:58.185187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:58.185187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"160'957","reported_seq":971,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.346538+0000","last_change":"2024-02-22T19:35:19.225962+0000","last_active":"2024-02-22T19:35:39.346538+0000","last_peered":"2024-02-22T19:35:39.346538+0000","last_clean":"2024-02-22T19:35:39.346538+0000","last_became_active":"2024-02-22T19:35:19.225604+0000","last_became_peered":"2024-02-22T19:35:19.225604+0000","last_unstale":"2024-02-22T19:35:39.346538+0000","last_undegraded":"2024-02-22T19:35:39.346538+0000","last_fullsized":"2024-02-22T19:35:39.346538+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":957,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62717952,"num_objects":957,"num_object_clones":0,"num_object_copies":2871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":957,"num_write_kb":61248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":113,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260231+0000","last_change":"2024-02-22T19:34:38.529035+0000","last_active":"2024-02-22T19:35:29.260231+0000","last_peered":"2024-02-22T19:35:29.260231+0000","last_clean":"2024-02-22T19:35:29.260231+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:35:29.260231+0000","last_undegraded":"2024-02-22T19:35:29.260231+0000","last_fullsized":"2024-02-22T19:35:29.260231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:38.528933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.528933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":126,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:38.917238+0000","last_change":"2024-02-22T19:35:38.917238+0000","last_active":"2024-02-22T19:35:38.917238+0000","last_peered":"2024-02-22T19:35:38.917238+0000","last_clean":"2024-02-22T19:35:38.917238+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:35:38.917238+0000","last_undegraded":"2024-02-22T19:35:38.917238+0000","last_fullsized":"2024-02-22T19:35:38.917238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:38.917177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.917177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.292706+0000","last_change":"2024-02-22T19:35:28.850161+0000","last_active":"2024-02-22T19:35:34.292706+0000","last_peered":"2024-02-22T19:35:34.292706+0000","last_clean":"2024-02-22T19:35:34.292706+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:35:34.292706+0000","last_undegraded":"2024-02-22T19:35:34.292706+0000","last_fullsized":"2024-02-22T19:35:34.292706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:28.850036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:35:28.850036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":129,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293329+0000","last_change":"2024-02-22T19:33:59.950686+0000","last_active":"2024-02-22T19:35:34.293329+0000","last_peered":"2024-02-22T19:35:34.293329+0000","last_clean":"2024-02-22T19:35:34.293329+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:35:34.293329+0000","last_undegraded":"2024-02-22T19:35:34.293329+0000","last_fullsized":"2024-02-22T19:35:34.293329+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:33:59.950586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:59.950586+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":135,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293600+0000","last_change":"2024-02-22T19:35:21.803865+0000","last_active":"2024-02-22T19:35:34.293600+0000","last_peered":"2024-02-22T19:35:34.293600+0000","last_clean":"2024-02-22T19:35:34.293600+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:35:34.293600+0000","last_undegraded":"2024-02-22T19:35:34.293600+0000","last_fullsized":"2024-02-22T19:35:34.293600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:21.803776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:21.803776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"159'924","reported_seq":937,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.218390+0000","last_change":"2024-02-22T19:35:19.226967+0000","last_active":"2024-02-22T19:35:39.218390+0000","last_peered":"2024-02-22T19:35:39.218390+0000","last_clean":"2024-02-22T19:35:39.218390+0000","last_became_active":"2024-02-22T19:35:19.226638+0000","last_became_peered":"2024-02-22T19:35:19.226638+0000","last_unstale":"2024-02-22T19:35:39.218390+0000","last_undegraded":"2024-02-22T19:35:39.218390+0000","last_fullsized":"2024-02-22T19:35:39.218390+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":924,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60555264,"num_objects":924,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":924,"num_write_kb":59136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":77,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.290004+0000","last_change":"2024-02-22T19:35:22.280512+0000","last_active":"2024-02-22T19:35:34.290004+0000","last_peered":"2024-02-22T19:35:34.290004+0000","last_clean":"2024-02-22T19:35:34.290004+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:35:34.290004+0000","last_undegraded":"2024-02-22T19:35:34.290004+0000","last_fullsized":"2024-02-22T19:35:34.290004+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.280418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.280418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":127,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259302+0000","last_change":"2024-02-22T19:34:44.432542+0000","last_active":"2024-02-22T19:35:29.259302+0000","last_peered":"2024-02-22T19:35:29.259302+0000","last_clean":"2024-02-22T19:35:29.259302+0000","last_became_active":"2024-02-22T19:28:55.852728+0000","last_became_peered":"2024-02-22T19:28:55.852728+0000","last_unstale":"2024-02-22T19:35:29.259302+0000","last_undegraded":"2024-02-22T19:35:29.259302+0000","last_fullsized":"2024-02-22T19:35:29.259302+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":128,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285477+0000","last_change":"2024-02-22T19:34:50.967533+0000","last_active":"2024-02-22T19:35:34.285477+0000","last_peered":"2024-02-22T19:35:34.285477+0000","last_clean":"2024-02-22T19:35:34.285477+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:35:34.285477+0000","last_undegraded":"2024-02-22T19:35:34.285477+0000","last_fullsized":"2024-02-22T19:35:34.285477+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:50.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:50.967435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":150,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:35.065441+0000","last_change":"2024-02-22T19:35:35.065441+0000","last_active":"2024-02-22T19:35:35.065441+0000","last_peered":"2024-02-22T19:35:35.065441+0000","last_clean":"2024-02-22T19:35:35.065441+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:35:35.065441+0000","last_undegraded":"2024-02-22T19:35:35.065441+0000","last_fullsized":"2024-02-22T19:35:35.065441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:35.065332+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.065332+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259996+0000","last_change":"2024-02-22T19:34:31.534714+0000","last_active":"2024-02-22T19:35:29.259996+0000","last_peered":"2024-02-22T19:35:29.259996+0000","last_clean":"2024-02-22T19:35:29.259996+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:35:29.259996+0000","last_undegraded":"2024-02-22T19:35:29.259996+0000","last_fullsized":"2024-02-22T19:35:29.259996+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.534649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.534649+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"159'965","reported_seq":978,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.300386+0000","last_change":"2024-02-22T19:35:19.748247+0000","last_active":"2024-02-22T19:35:39.300386+0000","last_peered":"2024-02-22T19:35:39.300386+0000","last_clean":"2024-02-22T19:35:39.300386+0000","last_became_active":"2024-02-22T19:35:19.747933+0000","last_became_peered":"2024-02-22T19:35:19.747933+0000","last_unstale":"2024-02-22T19:35:39.300386+0000","last_undegraded":"2024-02-22T19:35:39.300386+0000","last_fullsized":"2024-02-22T19:35:39.300386+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":965,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63242240,"num_objects":965,"num_object_clones":0,"num_object_copies":2895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":965,"num_write_kb":61760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.258643+0000","last_change":"2024-02-22T19:34:38.060600+0000","last_active":"2024-02-22T19:35:29.258643+0000","last_peered":"2024-02-22T19:35:29.258643+0000","last_clean":"2024-02-22T19:35:29.258643+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:35:29.258643+0000","last_undegraded":"2024-02-22T19:35:29.258643+0000","last_fullsized":"2024-02-22T19:35:29.258643+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:38.060515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.060515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":125,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285061+0000","last_change":"2024-02-22T19:35:30.342964+0000","last_active":"2024-02-22T19:35:34.285061+0000","last_peered":"2024-02-22T19:35:34.285061+0000","last_clean":"2024-02-22T19:35:34.285061+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:35:34.285061+0000","last_undegraded":"2024-02-22T19:35:34.285061+0000","last_fullsized":"2024-02-22T19:35:34.285061+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.342864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.342864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":519,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:37.707142+0000","last_change":"2024-02-22T19:34:48.568447+0000","last_active":"2024-02-22T19:35:37.707142+0000","last_peered":"2024-02-22T19:35:37.707142+0000","last_clean":"2024-02-22T19:35:37.707142+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:35:37.707142+0000","last_undegraded":"2024-02-22T19:35:37.707142+0000","last_fullsized":"2024-02-22T19:35:37.707142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:48.568336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.568336+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":392,"num_read_kb":387,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285822+0000","last_change":"2024-02-22T19:34:38.184599+0000","last_active":"2024-02-22T19:35:34.285822+0000","last_peered":"2024-02-22T19:35:34.285822+0000","last_clean":"2024-02-22T19:35:34.285822+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:35:34.285822+0000","last_undegraded":"2024-02-22T19:35:34.285822+0000","last_fullsized":"2024-02-22T19:35:34.285822+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:38.184521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.184521+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":76,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259704+0000","last_change":"2024-02-22T19:34:00.713551+0000","last_active":"2024-02-22T19:35:29.259704+0000","last_peered":"2024-02-22T19:35:29.259704+0000","last_clean":"2024-02-22T19:35:29.259704+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:35:29.259704+0000","last_undegraded":"2024-02-22T19:35:29.259704+0000","last_fullsized":"2024-02-22T19:35:29.259704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:00.713486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:00.713486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"160'937","reported_seq":951,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.346751+0000","last_change":"2024-02-22T19:35:19.745745+0000","last_active":"2024-02-22T19:35:39.346751+0000","last_peered":"2024-02-22T19:35:39.346751+0000","last_clean":"2024-02-22T19:35:39.346751+0000","last_became_active":"2024-02-22T19:35:19.745488+0000","last_became_peered":"2024-02-22T19:35:19.745488+0000","last_unstale":"2024-02-22T19:35:39.346751+0000","last_undegraded":"2024-02-22T19:35:39.346751+0000","last_fullsized":"2024-02-22T19:35:39.346751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":937,"ondisk_log_size":937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61407232,"num_objects":937,"num_object_clones":0,"num_object_copies":2811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":59968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":115,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.260733+0000","last_change":"2024-02-22T19:34:36.003355+0000","last_active":"2024-02-22T19:35:29.260733+0000","last_peered":"2024-02-22T19:35:29.260733+0000","last_clean":"2024-02-22T19:35:29.260733+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:35:29.260733+0000","last_undegraded":"2024-02-22T19:35:29.260733+0000","last_fullsized":"2024-02-22T19:35:29.260733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.003297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.003297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":130,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.258842+0000","last_change":"2024-02-22T19:34:41.030962+0000","last_active":"2024-02-22T19:35:29.258842+0000","last_peered":"2024-02-22T19:35:29.258842+0000","last_clean":"2024-02-22T19:35:29.258842+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:35:29.258842+0000","last_undegraded":"2024-02-22T19:35:29.258842+0000","last_fullsized":"2024-02-22T19:35:29.258842+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.030860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.030860+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":186,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:35.426434+0000","last_change":"2024-02-22T19:35:35.426393+0000","last_active":"2024-02-22T19:35:35.426434+0000","last_peered":"2024-02-22T19:35:35.426434+0000","last_clean":"2024-02-22T19:35:35.426434+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:35:35.426434+0000","last_undegraded":"2024-02-22T19:35:35.426434+0000","last_fullsized":"2024-02-22T19:35:35.426434+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:35.426296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.426296+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":130,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:31.180001+0000","last_change":"2024-02-22T19:35:31.179969+0000","last_active":"2024-02-22T19:35:31.180001+0000","last_peered":"2024-02-22T19:35:31.180001+0000","last_clean":"2024-02-22T19:35:31.180001+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:35:31.180001+0000","last_undegraded":"2024-02-22T19:35:31.180001+0000","last_fullsized":"2024-02-22T19:35:31.180001+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.179891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.179891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"159'945","reported_seq":959,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.317420+0000","last_change":"2024-02-22T19:35:19.745825+0000","last_active":"2024-02-22T19:35:39.317420+0000","last_peered":"2024-02-22T19:35:39.317420+0000","last_clean":"2024-02-22T19:35:39.317420+0000","last_became_active":"2024-02-22T19:35:19.745522+0000","last_became_peered":"2024-02-22T19:35:19.745522+0000","last_unstale":"2024-02-22T19:35:39.317420+0000","last_undegraded":"2024-02-22T19:35:39.317420+0000","last_fullsized":"2024-02-22T19:35:39.317420+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":945,"ondisk_log_size":945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61931520,"num_objects":945,"num_object_clones":0,"num_object_copies":2835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":60480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":119,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.286192+0000","last_change":"2024-02-22T19:35:33.417797+0000","last_active":"2024-02-22T19:35:34.286192+0000","last_peered":"2024-02-22T19:35:34.286192+0000","last_clean":"2024-02-22T19:35:34.286192+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:35:34.286192+0000","last_undegraded":"2024-02-22T19:35:34.286192+0000","last_fullsized":"2024-02-22T19:35:34.286192+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":77,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.288524+0000","last_change":"2024-02-22T19:34:33.941965+0000","last_active":"2024-02-22T19:35:34.288524+0000","last_peered":"2024-02-22T19:35:34.288524+0000","last_clean":"2024-02-22T19:35:34.288524+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:35:34.288524+0000","last_undegraded":"2024-02-22T19:35:34.288524+0000","last_fullsized":"2024-02-22T19:35:34.288524+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:33.941905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.941905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":160,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:37.886857+0000","last_change":"2024-02-22T19:35:37.886857+0000","last_active":"2024-02-22T19:35:37.886857+0000","last_peered":"2024-02-22T19:35:37.886857+0000","last_clean":"2024-02-22T19:35:37.886857+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:35:37.886857+0000","last_undegraded":"2024-02-22T19:35:37.886857+0000","last_fullsized":"2024-02-22T19:35:37.886857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:37.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.886787+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":217,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.225859+0000","last_change":"2024-02-22T19:34:31.086582+0000","last_active":"2024-02-22T19:35:39.225859+0000","last_peered":"2024-02-22T19:35:39.225859+0000","last_clean":"2024-02-22T19:35:39.225859+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:35:39.225859+0000","last_undegraded":"2024-02-22T19:35:39.225859+0000","last_fullsized":"2024-02-22T19:35:39.225859+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:31.086518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.086518+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"159'1003","reported_seq":1016,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.223152+0000","last_change":"2024-02-22T19:35:19.746052+0000","last_active":"2024-02-22T19:35:39.223152+0000","last_peered":"2024-02-22T19:35:39.223152+0000","last_clean":"2024-02-22T19:35:39.223152+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:35:39.223152+0000","last_undegraded":"2024-02-22T19:35:39.223152+0000","last_fullsized":"2024-02-22T19:35:39.223152+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":1003,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65732608,"num_objects":1003,"num_object_clones":0,"num_object_copies":3009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1003,"num_write_kb":64192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":113,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259822+0000","last_change":"2024-02-22T19:34:32.514011+0000","last_active":"2024-02-22T19:35:29.259822+0000","last_peered":"2024-02-22T19:35:29.259822+0000","last_clean":"2024-02-22T19:35:29.259822+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:35:29.259822+0000","last_undegraded":"2024-02-22T19:35:29.259822+0000","last_fullsized":"2024-02-22T19:35:29.259822+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.513915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.513915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":120,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285861+0000","last_change":"2024-02-22T19:34:34.158347+0000","last_active":"2024-02-22T19:35:34.285861+0000","last_peered":"2024-02-22T19:35:34.285861+0000","last_clean":"2024-02-22T19:35:34.285861+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:35:34.285861+0000","last_undegraded":"2024-02-22T19:35:34.285861+0000","last_fullsized":"2024-02-22T19:35:34.285861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:34.158242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:34.158242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":172,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342569+0000","last_change":"2024-02-22T19:35:26.969937+0000","last_active":"2024-02-22T19:35:30.342569+0000","last_peered":"2024-02-22T19:35:30.342569+0000","last_clean":"2024-02-22T19:35:30.342569+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:35:30.342569+0000","last_undegraded":"2024-02-22T19:35:30.342569+0000","last_fullsized":"2024-02-22T19:35:30.342569+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:26.969827+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.969827+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":203,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.226688+0000","last_change":"2024-02-22T19:35:25.316903+0000","last_active":"2024-02-22T19:35:39.226688+0000","last_peered":"2024-02-22T19:35:39.226688+0000","last_clean":"2024-02-22T19:35:39.226688+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:35:39.226688+0000","last_undegraded":"2024-02-22T19:35:39.226688+0000","last_fullsized":"2024-02-22T19:35:39.226688+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"160'961","reported_seq":975,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.326466+0000","last_change":"2024-02-22T19:35:19.227135+0000","last_active":"2024-02-22T19:35:39.326466+0000","last_peered":"2024-02-22T19:35:39.326466+0000","last_clean":"2024-02-22T19:35:39.326466+0000","last_became_active":"2024-02-22T19:35:19.226763+0000","last_became_peered":"2024-02-22T19:35:19.226763+0000","last_unstale":"2024-02-22T19:35:39.326466+0000","last_undegraded":"2024-02-22T19:35:39.326466+0000","last_fullsized":"2024-02-22T19:35:39.326466+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":961,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62980096,"num_objects":961,"num_object_clones":0,"num_object_copies":2883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":61504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":115,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342749+0000","last_change":"2024-02-22T19:34:01.893979+0000","last_active":"2024-02-22T19:35:30.342749+0000","last_peered":"2024-02-22T19:35:30.342749+0000","last_clean":"2024-02-22T19:35:30.342749+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:35:30.342749+0000","last_undegraded":"2024-02-22T19:35:30.342749+0000","last_fullsized":"2024-02-22T19:35:30.342749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:01.893910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:01.893910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":119,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.289894+0000","last_change":"2024-02-22T19:34:40.000683+0000","last_active":"2024-02-22T19:35:34.289894+0000","last_peered":"2024-02-22T19:35:34.289894+0000","last_clean":"2024-02-22T19:35:34.289894+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:35:34.289894+0000","last_undegraded":"2024-02-22T19:35:34.289894+0000","last_fullsized":"2024-02-22T19:35:34.289894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:40.000583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:40.000583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":151,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285897+0000","last_change":"2024-02-22T19:33:54.749170+0000","last_active":"2024-02-22T19:35:34.285897+0000","last_peered":"2024-02-22T19:35:34.285897+0000","last_clean":"2024-02-22T19:35:34.285897+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:35:34.285897+0000","last_undegraded":"2024-02-22T19:35:34.285897+0000","last_fullsized":"2024-02-22T19:35:34.285897+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:33:54.749061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:54.749061+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":122,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:36.900633+0000","last_change":"2024-02-22T19:35:36.900633+0000","last_active":"2024-02-22T19:35:36.900633+0000","last_peered":"2024-02-22T19:35:36.900633+0000","last_clean":"2024-02-22T19:35:36.900633+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:35:36.900633+0000","last_undegraded":"2024-02-22T19:35:36.900633+0000","last_fullsized":"2024-02-22T19:35:36.900633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"159'972","reported_seq":985,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.287225+0000","last_change":"2024-02-22T19:35:19.745819+0000","last_active":"2024-02-22T19:35:39.287225+0000","last_peered":"2024-02-22T19:35:39.287225+0000","last_clean":"2024-02-22T19:35:39.287225+0000","last_became_active":"2024-02-22T19:35:19.745509+0000","last_became_peered":"2024-02-22T19:35:19.745509+0000","last_unstale":"2024-02-22T19:35:39.287225+0000","last_undegraded":"2024-02-22T19:35:39.287225+0000","last_fullsized":"2024-02-22T19:35:39.287225+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":972,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63700992,"num_objects":972,"num_object_clones":0,"num_object_copies":2916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":972,"num_write_kb":62208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":76,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.258976+0000","last_change":"2024-02-22T19:34:51.981539+0000","last_active":"2024-02-22T19:35:29.258976+0000","last_peered":"2024-02-22T19:35:29.258976+0000","last_clean":"2024-02-22T19:35:29.258976+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:35:29.258976+0000","last_undegraded":"2024-02-22T19:35:29.258976+0000","last_fullsized":"2024-02-22T19:35:29.258976+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:51.981444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:51.981444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":217,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.225634+0000","last_change":"2024-02-22T19:34:43.126256+0000","last_active":"2024-02-22T19:35:34.225634+0000","last_peered":"2024-02-22T19:35:34.225634+0000","last_clean":"2024-02-22T19:35:34.225634+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:35:34.225634+0000","last_undegraded":"2024-02-22T19:35:34.225634+0000","last_fullsized":"2024-02-22T19:35:34.225634+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":124,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.285813+0000","last_change":"2024-02-22T19:35:27.352610+0000","last_active":"2024-02-22T19:35:34.285813+0000","last_peered":"2024-02-22T19:35:34.285813+0000","last_clean":"2024-02-22T19:35:34.285813+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:35:34.285813+0000","last_undegraded":"2024-02-22T19:35:34.285813+0000","last_fullsized":"2024-02-22T19:35:34.285813+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:27.352545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.352545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":117,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293401+0000","last_change":"2024-02-22T19:35:24.795107+0000","last_active":"2024-02-22T19:35:34.293401+0000","last_peered":"2024-02-22T19:35:34.293401+0000","last_clean":"2024-02-22T19:35:34.293401+0000","last_became_active":"2024-02-22T19:30:24.414334+0000","last_became_peered":"2024-02-22T19:30:24.414334+0000","last_unstale":"2024-02-22T19:35:34.293401+0000","last_undegraded":"2024-02-22T19:35:34.293401+0000","last_fullsized":"2024-02-22T19:35:34.293401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:35:24.795027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.795027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"159'938","reported_seq":951,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.212380+0000","last_change":"2024-02-22T19:35:19.225232+0000","last_active":"2024-02-22T19:35:39.212380+0000","last_peered":"2024-02-22T19:35:39.212380+0000","last_clean":"2024-02-22T19:35:39.212380+0000","last_became_active":"2024-02-22T19:35:19.224941+0000","last_became_peered":"2024-02-22T19:35:19.224941+0000","last_unstale":"2024-02-22T19:35:39.212380+0000","last_undegraded":"2024-02-22T19:35:39.212380+0000","last_fullsized":"2024-02-22T19:35:39.212380+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":938,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61472768,"num_objects":938,"num_object_clones":0,"num_object_copies":2814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":938,"num_write_kb":60032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":122,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:37.337381+0000","last_change":"2024-02-22T19:35:37.337349+0000","last_active":"2024-02-22T19:35:37.337381+0000","last_peered":"2024-02-22T19:35:37.337381+0000","last_clean":"2024-02-22T19:35:37.337381+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:35:37.337381+0000","last_undegraded":"2024-02-22T19:35:37.337381+0000","last_fullsized":"2024-02-22T19:35:37.337381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":76,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343595+0000","last_change":"2024-02-22T19:34:39.082294+0000","last_active":"2024-02-22T19:35:30.343595+0000","last_peered":"2024-02-22T19:35:30.343595+0000","last_clean":"2024-02-22T19:35:30.343595+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:35:30.343595+0000","last_undegraded":"2024-02-22T19:35:30.343595+0000","last_fullsized":"2024-02-22T19:35:30.343595+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:39.082190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.082190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":133,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.342407+0000","last_change":"2024-02-22T19:33:48.898875+0000","last_active":"2024-02-22T19:35:30.342407+0000","last_peered":"2024-02-22T19:35:30.342407+0000","last_clean":"2024-02-22T19:35:30.342407+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:35:30.342407+0000","last_undegraded":"2024-02-22T19:35:30.342407+0000","last_fullsized":"2024-02-22T19:35:30.342407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:48.898801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:33:48.898801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":138,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343495+0000","last_change":"2024-02-22T19:33:51.953460+0000","last_active":"2024-02-22T19:35:30.343495+0000","last_peered":"2024-02-22T19:35:30.343495+0000","last_clean":"2024-02-22T19:35:30.343495+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:35:30.343495+0000","last_undegraded":"2024-02-22T19:35:30.343495+0000","last_fullsized":"2024-02-22T19:35:30.343495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:33:51.953387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:33:51.953387+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1","version":"159'997","reported_seq":1010,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.292170+0000","last_change":"2024-02-22T19:35:19.745821+0000","last_active":"2024-02-22T19:35:39.292170+0000","last_peered":"2024-02-22T19:35:39.292170+0000","last_clean":"2024-02-22T19:35:39.292170+0000","last_became_active":"2024-02-22T19:35:19.745498+0000","last_became_peered":"2024-02-22T19:35:19.745498+0000","last_unstale":"2024-02-22T19:35:39.292170+0000","last_undegraded":"2024-02-22T19:35:39.292170+0000","last_fullsized":"2024-02-22T19:35:39.292170+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":997,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65339392,"num_objects":997,"num_object_clones":0,"num_object_copies":2991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":63808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":116,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.293248+0000","last_change":"2024-02-22T19:33:52.892034+0000","last_active":"2024-02-22T19:35:34.293248+0000","last_peered":"2024-02-22T19:35:34.293248+0000","last_clean":"2024-02-22T19:35:34.293248+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:35:34.293248+0000","last_undegraded":"2024-02-22T19:35:34.293248+0000","last_fullsized":"2024-02-22T19:35:34.293248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:52.891927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:33:52.891927+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":117,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.259069+0000","last_change":"2024-02-22T19:33:57.772428+0000","last_active":"2024-02-22T19:35:29.259069+0000","last_peered":"2024-02-22T19:35:29.259069+0000","last_clean":"2024-02-22T19:35:29.259069+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:35:29.259069+0000","last_undegraded":"2024-02-22T19:35:29.259069+0000","last_fullsized":"2024-02-22T19:35:29.259069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:33:57.772326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:57.772326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":123,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:29.753125+0000","last_change":"2024-02-22T19:34:55.459973+0000","last_active":"2024-02-22T19:35:29.753125+0000","last_peered":"2024-02-22T19:35:29.753125+0000","last_clean":"2024-02-22T19:35:29.753125+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:35:29.753125+0000","last_undegraded":"2024-02-22T19:35:29.753125+0000","last_fullsized":"2024-02-22T19:35:29.753125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:55.459912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:55.459912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":147,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343376+0000","last_change":"2024-02-22T19:34:57.145803+0000","last_active":"2024-02-22T19:35:30.343376+0000","last_peered":"2024-02-22T19:35:30.343376+0000","last_clean":"2024-02-22T19:35:30.343376+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:35:30.343376+0000","last_undegraded":"2024-02-22T19:35:30.343376+0000","last_fullsized":"2024-02-22T19:35:30.343376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:34:57.145716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.145716+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.0","version":"160'925","reported_seq":939,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.334956+0000","last_change":"2024-02-22T19:35:19.229848+0000","last_active":"2024-02-22T19:35:39.334956+0000","last_peered":"2024-02-22T19:35:39.334956+0000","last_clean":"2024-02-22T19:35:39.334956+0000","last_became_active":"2024-02-22T19:35:19.229525+0000","last_became_peered":"2024-02-22T19:35:19.229525+0000","last_unstale":"2024-02-22T19:35:39.334956+0000","last_undegraded":"2024-02-22T19:35:39.334956+0000","last_fullsized":"2024-02-22T19:35:39.334956+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":925,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60620800,"num_objects":925,"num_object_clones":0,"num_object_copies":2775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":925,"num_write_kb":59200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":119,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:30.343638+0000","last_change":"2024-02-22T19:34:54.019363+0000","last_active":"2024-02-22T19:35:30.343638+0000","last_peered":"2024-02-22T19:35:30.343638+0000","last_clean":"2024-02-22T19:35:30.343638+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:35:30.343638+0000","last_undegraded":"2024-02-22T19:35:30.343638+0000","last_fullsized":"2024-02-22T19:35:30.343638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:34:54.019270+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:34:54.019270+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":219,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:39.228282+0000","last_change":"2024-02-22T19:33:42.757859+0000","last_active":"2024-02-22T19:35:39.228282+0000","last_peered":"2024-02-22T19:35:39.228282+0000","last_clean":"2024-02-22T19:35:39.228282+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:35:39.228282+0000","last_undegraded":"2024-02-22T19:35:39.228282+0000","last_fullsized":"2024-02-22T19:35:39.228282+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:33:42.757755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:33:42.757755+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":128,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-02-22T19:35:34.287315+0000","last_change":"2024-02-22T19:34:28.943926+0000","last_active":"2024-02-22T19:35:34.287315+0000","last_peered":"2024-02-22T19:35:34.287315+0000","last_clean":"2024-02-22T19:35:34.287315+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:35:34.287315+0000","last_undegraded":"2024-02-22T19:35:34.287315+0000","last_fullsized":"2024-02-22T19:35:34.287315+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:34:28.943835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.943835+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":134,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-02-22T19:35:33.971582+0000","last_change":"2024-02-22T19:35:33.971551+0000","last_active":"2024-02-22T19:35:33.971582+0000","last_peered":"2024-02-22T19:35:33.971582+0000","last_clean":"2024-02-22T19:35:33.971582+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:35:33.971582+0000","last_undegraded":"2024-02-22T19:35:33.971582+0000","last_fullsized":"2024-02-22T19:35:33.971582+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:33.971446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.971446+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":1009516544,"num_objects":15404,"num_object_clones":0,"num_object_copies":46212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15404,"num_write_kb":985856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3028025344,"data_stored":3028025344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15404,"ondisk_log_size":15404,"up":48,"acting":48,"num_store_stats":7},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":392,"num_read_kb":387,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266771,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":600212,"kb_used_data":290700,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93144940,"statfs":{"total":95995035648,"available":95380418560,"internally_reserved":0,"allocated":297676800,"data_stored":297275689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.879,"15min":1.0089999999999999},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":7.0700000000000003,"5min":42.597999999999999,"15min":42.597999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":2.3620000000000001,"5min":1.744,"15min":0.93400000000000005},"min":{"1min":0.216,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":9.8870000000000005,"5min":42.155999999999999,"15min":42.155999999999999},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":1.6359999999999999,"15min":0.93000000000000005},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":5.6609999999999996,"5min":38.188000000000002,"15min":38.188000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":2.077,"5min":1.899,"15min":0.995},"min":{"1min":0.33200000000000002,"5min":0.219,"15min":0.219},"max":{"1min":9.7520000000000007,"5min":37.938000000000002,"15min":37.938000000000002},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":1.389,"15min":0.83599999999999997},"min":{"1min":0.28899999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.4510000000000001,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.377},{"interface":"front","average":{"1min":1.306,"5min":1.6890000000000001,"15min":0.96299999999999997},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.3010000000000002,"5min":41.722000000000001,"15min":41.722000000000001},"last":0.873}]},{"osd":3,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.7110000000000001,"15min":0.97199999999999998},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.5309999999999997,"5min":41.457000000000001,"15min":41.457000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.5529999999999999,"15min":0.93500000000000005},"min":{"1min":0.375,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.806,"5min":41.311999999999998,"15min":41.311999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.51500000000000001,"15min":0.47699999999999998},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.427,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.70399999999999996,"5min":3.4780000000000002,"15min":3.4780000000000002},"last":0.34300000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78300000000000003,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.58899999999999997,"15min":0.53100000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.3959999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.36199999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.71999999999999997,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":4.7160000000000002,"5min":12.199999999999999,"15min":12.199999999999999},"last":0.31},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.63100000000000001,"15min":0.57099999999999995},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76900000000000002,"5min":12.067,"15min":12.067},"last":0.47399999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528408,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":728568,"kb_used_data":419056,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93016584,"statfs":{"total":95995035648,"available":95248982016,"internally_reserved":0,"allocated":429113344,"data_stored":428739973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.901,"5min":1.2789999999999999,"15min":0.77200000000000002},"min":{"1min":0.223,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":6.9939999999999998,"5min":17.885000000000002,"15min":17.885000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.034,"5min":1.216,"15min":0.748},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.9849999999999999,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":0.96299999999999997,"15min":0.69199999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.8090000000000002,"5min":11.301,"15min":11.301},"last":1.921},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.94499999999999995,"15min":0.66300000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.9580000000000002,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.288,"15min":0.78600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":5.9950000000000001,"5min":12.859999999999999,"15min":12.859999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.03,"15min":0.71099999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.6339999999999999,"15min":0.90500000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.4950000000000001,"5min":38.762,"15min":38.762},"last":0.38400000000000001},{"interface":"front","average":{"1min":2.0880000000000001,"5min":2.0939999999999999,"15min":1.0680000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7070000000000007,"5min":35.612000000000002,"15min":35.612000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.26000000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.27,"5min":1.583,"15min":1.583},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.436,"15min":0.42699999999999999},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.67500000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.442},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.056,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.442},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Feb 22 19:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.61699999999999999,"15min":0.58399999999999996},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.5139999999999998,"5min":3.593,"15min":3.593},"last":0.82199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":713320,"kb_used_data":403808,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93031832,"statfs":{"total":95995035648,"available":95264595968,"internally_reserved":0,"allocated":413499392,"data_stored":413078245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:49 2024","interfaces":[{"interface":"back","average":{"1min":2.9870000000000001,"5min":1.986,"15min":0.89100000000000001},"min":{"1min":0.33000000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":9.9459999999999997,"5min":15.613,"15min":15.613},"last":0.38500000000000001},{"interface":"front","average":{"1min":2.665,"5min":2.008,"15min":0.91700000000000004},"min":{"1min":0.30599999999999999,"5min":0.25900000000000001,"15min":0.23899999999999999},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":2.3039999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:35:05 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":2.327,"15min":1.038},"min":{"1min":0.32200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":9.9700000000000006,"5min":16.503,"15min":16.503},"last":1.952},{"interface":"front","average":{"1min":2.2450000000000001,"5min":2.1110000000000002,"15min":0.96799999999999997},"min":{"1min":0.218,"5min":0.192,"15min":0.192},"max":{"1min":11.284000000000001,"5min":13.461,"15min":13.461},"last":2.0779999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:35:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":2.5659999999999998,"15min":1.155},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":8.4900000000000002,"5min":25.145,"15min":25.145},"last":4.1429999999999998},{"interface":"front","average":{"1min":1.1899999999999999,"5min":2.5169999999999999,"15min":1.175},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":5.4850000000000003,"5min":22.25,"15min":22.25},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:34:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":0.92300000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.1449999999999996,"5min":15.542999999999999,"15min":15.542999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.2390000000000001,"15min":0.79300000000000004},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":4.8499999999999996,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Feb 22 19:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.581,"15min":0.92500000000000004},"min":{"1min":0.371,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":8.2189999999999994,"5min":15.587999999999999,"15min":15.587999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.389,"5min":1.4299999999999999,"15min":0.85699999999999998},"min":{"1min":0.31900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.6790000000000003,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":3.1629999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.1830000000000001,"15min":0.75700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.8730000000000002,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5209999999999999,"15min":0.90100000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":8.1880000000000006,"5min":15.651999999999999,"15min":15.651999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Feb 22 19:35:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.431,"15min":0.82699999999999996},"min":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.7050000000000001,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":3.1299999999999999},{"interface":"front","average":{"1min":1.718,"5min":1.3169999999999999,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.316,"15min":0.316},"max":{"1min":4.8879999999999999,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":0.54300000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":740508,"kb_used_data":435092,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93004644,"statfs":{"total":95995035648,"available":95236755456,"internally_reserved":0,"allocated":445534208,"data_stored":445132017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:34:52 2024","interfaces":[{"interface":"back","average":{"1min":3.2829999999999999,"5min":1.853,"15min":0.85499999999999998},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":12.755000000000001,"5min":12.755000000000001,"15min":12.755000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.976,"5min":1.6439999999999999,"15min":0.78900000000000003},"min":{"1min":0.24199999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":7.4989999999999997,"5min":13.013999999999999,"15min":13.013999999999999},"last":5.9560000000000004}]},{"osd":2,"last update":"Thu Feb 22 19:35:15 2024","interfaces":[{"interface":"back","average":{"1min":2.9420000000000002,"5min":2.3919999999999999,"15min":1.032},"min":{"1min":0.29799999999999999,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":7.4390000000000001,"5min":19.927,"15min":19.927},"last":0.56299999999999994},{"interface":"front","average":{"1min":2.246,"5min":2.2690000000000001,"15min":1.006},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":8.2390000000000008,"5min":17.907,"15min":17.907},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.8540000000000001,"15min":0.89900000000000002},"min":{"1min":0.28799999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.1550000000000002,"5min":13.06,"15min":13.06},"last":1.145},{"interface":"front","average":{"1min":2.1890000000000001,"5min":2.4980000000000002,"15min":1.125},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":6.7939999999999996,"5min":22.936,"15min":22.936},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:34:52 2024","interfaces":[{"interface":"back","average":{"1min":2.2040000000000002,"5min":1.226,"15min":0.78900000000000003},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":7.0069999999999997,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":1.0629999999999999},{"interface":"front","average":{"1min":3.2519999999999998,"5min":1.9690000000000001,"15min":1.0449999999999999},"min":{"1min":0.39600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.4809999999999999,"5min":13.874000000000001,"15min":13.874000000000001},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:35:20 2024","interfaces":[{"interface":"back","average":{"1min":2.5390000000000001,"5min":2.0139999999999998,"15min":1.044},"min":{"1min":0.46500000000000002,"5min":0.33500000000000002,"15min":0.33100000000000002},"max":{"1min":7.5110000000000001,"5min":13.903,"15min":13.903},"last":0.52300000000000002},{"interface":"front","average":{"1min":2.0179999999999998,"5min":1.3080000000000001,"15min":0.81399999999999995},"min":{"1min":0.32100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0380000000000003,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.343,"15min":0.83799999999999997},"min":{"1min":0.40999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.0830000000000002,"5min":7.0830000000000002,"15min":7.0830000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":0.83499999999999996},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.1219999999999999,"5min":7.5410000000000004,"15min":7.5410000000000004},"last":1.2450000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:35:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0449999999999999,"5min":1.3180000000000001,"15min":0.82199999999999995},"min":{"1min":0.42899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":7.0620000000000003,"5min":7.0620000000000003,"15min":7.0620000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.3220000000000001,"15min":0.83399999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":7.423,"5min":7.423,"15min":7.423},"last":1.3180000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":711752,"kb_used_data":402240,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93033400,"statfs":{"total":95995035648,"available":95266201600,"internally_reserved":0,"allocated":411893760,"data_stored":411504233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.6879999999999999,"15min":0.80800000000000005},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":7.181,"5min":16.356999999999999,"15min":16.356999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":2.0219999999999998,"5min":2.1139999999999999,"15min":0.94999999999999996},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":16.545000000000002,"5min":18.059999999999999,"15min":18.059999999999999},"last":1.6439999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:35:07 2024","interfaces":[{"interface":"back","average":{"1min":2.4390000000000001,"5min":1.8720000000000001,"15min":0.878},"min":{"1min":0.23599999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":9.9030000000000005,"5min":10.183,"15min":10.183},"last":1.784},{"interface":"front","average":{"1min":2.2229999999999999,"5min":1.4770000000000001,"15min":0.76700000000000002},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.193},"max":{"1min":9.7810000000000006,"5min":12.393000000000001,"15min":12.393000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:34:36 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":2.0750000000000002,"15min":1.0009999999999999},"min":{"1min":0.36299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":12.656000000000001,"5min":30.071000000000002,"15min":30.071000000000002},"last":1.1990000000000001},{"interface":"front","average":{"1min":2.3490000000000002,"5min":2.3700000000000001,"15min":1.087},"min":{"1min":0.29299999999999998,"5min":0.192,"15min":0.192},"max":{"1min":10.223000000000001,"5min":32.119,"15min":32.119},"last":1.6719999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:34:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4910000000000001,"5min":1.52,"15min":0.88600000000000001},"min":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.3890000000000002,"5min":6.3890000000000002,"15min":6.3890000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.331,"5min":0.93500000000000005,"15min":0.70299999999999996},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":5.516,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.46700000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.97899999999999998,"15min":0.73999999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.2869999999999999,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.5700000000000001,"15min":0.92000000000000004},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.3520000000000003,"5min":6.4320000000000004,"15min":6.4320000000000004},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:34:40 2024","interfaces":[{"interface":"back","average":{"1min":2.4060000000000001,"5min":1.5189999999999999,"15min":0.90900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4109999999999996,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":0.627},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.0249999999999999,"15min":0.76200000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5309999999999997,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:35:07 2024","interfaces":[{"interface":"back","average":{"1min":2.589,"5min":1.6559999999999999,"15min":0.93799999999999994},"min":{"1min":0.41399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":9.8510000000000009,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":0.755},{"interface":"front","average":{"1min":1.579,"5min":1.089,"15min":0.75900000000000001},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":9.2810000000000006,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.66200000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149797,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":855068,"kb_used_data":549652,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92890084,"statfs":{"total":95995035648,"available":95119446016,"internally_reserved":0,"allocated":562843648,"data_stored":562435051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":2.2290000000000001,"15min":1.028},"min":{"1min":0.19400000000000001,"5min":0.193,"15min":0.193},"max":{"1min":8.9049999999999994,"5min":12.929,"15min":12.929},"last":3.1829999999999998},{"interface":"front","average":{"1min":1.875,"5min":2.5779999999999998,"15min":1.1739999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":6.7119999999999997,"5min":14.831,"15min":14.831},"last":1.095}]},{"osd":1,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":2.5539999999999998,"15min":1.1579999999999999},"min":{"1min":0.246,"5min":0.22,"15min":0.22},"max":{"1min":5.1580000000000004,"5min":18.259,"15min":18.259},"last":2.9039999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.9410000000000001,"15min":0.94599999999999995},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.6139999999999999,"5min":13.154,"15min":13.154},"last":2.431}]},{"osd":2,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":2.4209999999999998,"15min":1.1299999999999999},"min":{"1min":0.29699999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.3810000000000002,"5min":16.271000000000001,"15min":16.271000000000001},"last":2.2709999999999999},{"interface":"front","average":{"1min":1.345,"5min":2.3149999999999999,"15min":1.0820000000000001},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":8.7759999999999998,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.9099999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:34:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2400000000000002,"5min":1.7470000000000001,"15min":0.997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":13.680999999999999,"5min":13.680999999999999,"15min":13.680999999999999},"last":1.127},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.5209999999999999,"15min":0.91100000000000003},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":9.8940000000000001,"5min":11.366,"15min":11.366},"last":1.218}]},{"osd":5,"last update":"Thu Feb 22 19:35:20 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.5860000000000001,"15min":0.92000000000000004},"min":{"1min":0.38100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.1970000000000001,"5min":12.879,"15min":12.879},"last":2.8079999999999998},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.7989999999999999,"15min":0.98499999999999999},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.2789999999999999,"5min":13.707000000000001,"15min":13.707000000000001},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7030000000000001,"5min":1.5860000000000001,"15min":0.92000000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.4199999999999999,"5min":12.898999999999999,"15min":12.898999999999999},"last":2.7719999999999998},{"interface":"front","average":{"1min":2.6589999999999998,"5min":1.861,"15min":1.054},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":13.803000000000001,"5min":13.803000000000001,"15min":13.803000000000001},"last":1.1830000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:34:58 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.613,"15min":0.92100000000000004},"min":{"1min":0.42799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":9.9250000000000007,"5min":11.348000000000001,"15min":11.348000000000001},"last":2.2389999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.625,"15min":0.94699999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":4.7590000000000003,"5min":12.846,"15min":12.846},"last":2.8359999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":657624,"kb_used_data":348112,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93087528,"statfs":{"total":95995035648,"available":95321628672,"internally_reserved":0,"allocated":356466688,"data_stored":356061967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:34:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":1.5089999999999999,"15min":0.84699999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":5.8920000000000003,"5min":24.009,"15min":24.009},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.482,"15min":0.85499999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":8.2219999999999995,"5min":24.263000000000002,"15min":24.263000000000002},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Feb 22 19:34:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7030000000000001,"5min":1.204,"15min":0.76600000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":4.3579999999999997,"5min":6.3029999999999999,"15min":6.3029999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.0189999999999999,"15min":0.67800000000000005},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":5.1479999999999997,"5min":12.445,"15min":12.445},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Feb 22 19:34:54 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.276,"15min":0.80200000000000005},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.4569999999999999,"5min":23.082000000000001,"15min":23.082000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.72,"5min":1.5580000000000001,"15min":0.879},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":4.577,"5min":23.138999999999999,"15min":23.138999999999999},"last":3.2149999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:34:54 2024","interfaces":[{"interface":"back","average":{"1min":2.6819999999999999,"5min":1.6539999999999999,"15min":0.91800000000000004},"min":{"1min":0.432,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":13.476000000000001,"5min":13.476000000000001,"15min":13.476000000000001},"last":0.996},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.306,"15min":0.79100000000000004},"min":{"1min":0.29199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.3840000000000003,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.2470000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.42799999999999999},"min":{"1min":0.27300000000000002,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":1.877,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.51800000000000002,"15min":0.44800000000000001},"min":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.46899999999999997},"min":{"1min":0.25900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.0499999999999998,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.48099999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.456,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.32600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.96399999999999997,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.64100000000000001}]}]},{"osd":5,"up_from":0,"seq":167503724637,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444989440,"data_stored":444989440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":439156736,"data_stored":439156736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444334080,"data_stored":444334080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":565772288,"data_stored":565772288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":369950720,"data_stored":369950720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":441516032,"data_stored":441516032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":322306048,"data_stored":322306048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:35:48.016 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-02-22T19:35:48.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set device_health_metrics pg_num 11 2024-02-22T19:35:48.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:48.120 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:48.148 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:48.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:48.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:48.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:47.373023+0000 mgr.y (mgr.14535) 265 : cluster [DBG] pgmap v290: 148 pgs: 148 active+clean; 1.5 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-02-22T19:35:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:47.373023+0000 mgr.y (mgr.14535) 265 : cluster [DBG] pgmap v290: 148 pgs: 148 active+clean; 1.5 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-02-22T19:35:48.326 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:47.373023+0000 mgr.y (mgr.14535) 265 : cluster [DBG] pgmap v290: 148 pgs: 148 active+clean; 1.5 GiB data, 6.4 GiB used, 619 GiB / 626 GiB avail; 1023 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-02-22T19:35:48.955 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 16 16 32962 32946 128.626 102.75 0.0117795 0.00776829 2024-02-22T19:35:49.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:35:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:49] "GET /metrics HTTP/1.1" 200 175744 "" "Prometheus/2.33.4" 2024-02-22T19:35:49.819 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:49.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:49.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:49.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:35:49.955 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 17 16 34618 34602 127.148 103.5 0.00750844 0.00785984 2024-02-22T19:35:49.957 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:35:49.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:49.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:35:50.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:50.145 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:50.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:50.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:50.170 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:50.170 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:50.233 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:50.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:50.277 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:50.280 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:35:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:50.306 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:50.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:50.315 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:35:50.344 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:35:50.372 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:35:50.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:50.431 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:35:50.442 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:50.466 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:50.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:50.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:50.473 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:35:50.504 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:35:50.546 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:35:50.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:49.373995+0000 mgr.y (mgr.14535) 266 : cluster [DBG] pgmap v291: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1018 B/s rd, 146 MiB/s wr, 2.33k op/s 2024-02-22T19:35:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:50.220409+0000 mon.b (mon.1) 48 : audit [DBG] from='client.? 172.21.15.53:0/2448007446' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:50.649 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:50.731 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:49.373995+0000 mgr.y (mgr.14535) 266 : cluster [DBG] pgmap v291: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1018 B/s rd, 146 MiB/s wr, 2.33k op/s 2024-02-22T19:35:50.731 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:50.220409+0000 mon.b (mon.1) 48 : audit [DBG] from='client.? 172.21.15.53:0/2448007446' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:50.731 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:49.373995+0000 mgr.y (mgr.14535) 266 : cluster [DBG] pgmap v291: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1018 B/s rd, 146 MiB/s wr, 2.33k op/s 2024-02-22T19:35:50.731 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:50.220409+0000 mon.b (mon.1) 48 : audit [DBG] from='client.? 172.21.15.53:0/2448007446' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:35:50.955 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 18 16 35936 35920 124.66 82.375 0.00729514 0.00801623 2024-02-22T19:35:51.955 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 19 16 37503 37487 123.254 97.9375 0.0113829 0.00810761 2024-02-22T19:35:52.134 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:52.144 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:52.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:52.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:52.312 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:52.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:51.374786+0000 mgr.y (mgr.14535) 267 : cluster [DBG] pgmap v292: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:35:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:51.374786+0000 mgr.y (mgr.14535) 267 : cluster [DBG] pgmap v292: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:35:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:51.374786+0000 mgr.y (mgr.14535) 267 : cluster [DBG] pgmap v292: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:35:52.957 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:35:52.957800+0000 min lat: 0.00121627 max lat: 0.0749631 avg lat: 0.00819814 2024-02-22T19:35:52.957 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:35:52.957 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 20 16 39040 39024 121.884 96.0625 0.0137182 0.00819814 2024-02-22T19:35:52.990 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:52.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:52.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:53.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:53.133 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:53.160 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:53.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:53.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:53.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:53.297 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:53.328 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:53.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:53.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:35:53.500 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:53.593 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:35:53.600 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:53.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:35:53.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:53.763 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:53.796 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:53.803 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:53.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:53.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:53.959 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 21 16 40449 40433 120.267 88.0625 0.0148075 0.00830802 2024-02-22T19:35:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:54.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:53.073520+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:54.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:53.073561+0000 osd.2 (osd.2) 79 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:54.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:53.350539+0000 mon.a (mon.0) 964 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:53.378426+0000 mgr.y (mgr.14535) 268 : cluster [DBG] pgmap v293: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:53.449848+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e161: 8 total, 8 up, 7 in 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:53.073520+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:53.073561+0000 osd.2 (osd.2) 79 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:53.350539+0000 mon.a (mon.0) 964 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:53.378426+0000 mgr.y (mgr.14535) 268 : cluster [DBG] pgmap v293: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:35:54.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:53.449848+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e161: 8 total, 8 up, 7 in 2024-02-22T19:35:54.637 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20875 7.a 7:5368db82:::benchmark_data_smithi053_107660_object20874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 245.597606947, 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001, 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20875 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030654+0000", 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5442000000000001e-05 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1599e-05 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030691+0000", 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8590000000000001e-06 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030697+0000", 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5996999999999998e-05 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030763+0000", 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1030000000000002e-06 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030771+0000", 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96976369299999998 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000535+0000", 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3981999999999999e-05 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000549+0000", 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449743229999999 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045523+0000", 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015328299999999999 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045676+0000", 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167651 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045844+0000", 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003101946 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048946+0000", 2024-02-22T19:35:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029887199999999998 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051935+0000", 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057814199999999998 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057716+0000", 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5281000000000001e-05 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057771+0000", 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092475900000000004 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058696+0000", 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:109865 7.a 7:5217f792:::benchmark_data_smithi053_107660_object20658:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:37.881949+0000", 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 136.74631400199999, 2024-02-22T19:35:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068392611000000006, 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 109865 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.881949+0000", 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.881949+0000", 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.881946+0000", 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0175e-05 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.881957+0000", 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.34e-05 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.881970+0000", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3980000000000002e-06 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.881977+0000", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049971099999999999 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.886974+0000", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011592299999999999 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for rw locks", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.887090+0000", 2024-02-22T19:35:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041565471 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.928656+0000", 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0697000000000003e-05 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.928737+0000", 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015400799999999999 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.928891+0000", 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025586049999999998 2024-02-22T19:35:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.931449+0000", 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0068367660000000002 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.938286+0000", 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01181979 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.950106+0000", 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4556999999999999e-05 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.950170+0000", 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017090200000000001 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:37.950341+0000", 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068392611000000006 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111852 7.a 7:5162ddfb:::benchmark_data_smithi053_107660_object22645:head [delete] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-22T19:35:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.51944534399999, 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.030160867, 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111852 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:35:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108815+0000", 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8550000000000008e-06 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108824+0000", 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3647e-05 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108837+0000", 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9809999999999997e-06 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108844+0000", 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9802000000000001e-05 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108904+0000", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013143000000000001 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.109036+0000", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016598399999999999 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.109202+0000", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012752025 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121954+0000", 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1360000000000002e-06 2024-02-22T19:35:54.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121960+0000", 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016965211 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.138925+0000", 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3065000000000001e-05 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.138978+0000", 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.030160867 2024-02-22T19:35:54.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111854 7.a 7:5a0cbb42:::benchmark_data_smithi053_107660_object22647:head [delete] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.51446972700001, 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531219629999999, 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111854 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.657 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113791+0000", 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3090000000000003e-06 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113797+0000", 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8690000000000004e-06 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113805+0000", 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6439999999999997e-06 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113810+0000", 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109718 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113920+0000", 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125108 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.114045+0000", 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016840599999999999 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.114214+0000", 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077607329999999997 2024-02-22T19:35:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121974+0000", 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6069999999999999e-06 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121978+0000", 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170384509999999 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139016+0000", 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100129 2024-02-22T19:35:54.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139116+0000", 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027745477000000001 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166862+0000", 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2849000000000002e-05 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166915+0000", 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531219629999999 2024-02-22T19:35:54.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111874 7.a 7:51c8a40d:::benchmark_data_smithi053_107660_object22667:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.49513216400001, 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0339878849999999, 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111874 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133128+0000", 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7869999999999996e-06 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133138+0000", 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2543e-05 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133150+0000", 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8499999999999994e-06 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133158+0000", 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019007130000000001 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.135059+0000", 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2880000000000004e-06 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.135066+0000", 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0040841840000001 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139150+0000", 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011695400000000001 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139267+0000", 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014209699999999999 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139409+0000", 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011988443999999999 2024-02-22T19:35:54.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151398+0000", 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015560497 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166958+0000", 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1764000000000002e-05 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167000+0000", 2024-02-22T19:35:54.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4150000000000003e-05 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167034+0000", 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4145000000000002e-05 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167118+0000", 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0339878849999999 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111911 7.a 7:53f6a651:::benchmark_data_smithi053_107660_object22704:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.43930172500001, 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97843876299999999, 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111911 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188959+0000", 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188967+0000", 2024-02-22T19:35:54.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1484000000000001e-05 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188978+0000", 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7379999999999997e-06 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188985+0000", 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00214072 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.191126+0000", 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1008e-05 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.191137+0000", 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95036631800000004 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141503+0000", 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001082 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141611+0000", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015199899999999999 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141763+0000", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096414059999999999 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151405+0000", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015767331999999998 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167172+0000", 2024-02-22T19:35:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9851000000000003e-05 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167272+0000", 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2466999999999999e-05 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167304+0000", 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5302000000000003e-05 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167400+0000", 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97843876299999999 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.675 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111925 7.a 7:55d29d8b:::benchmark_data_smithi053_107660_object22718:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.431024837, 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97039658799999995, 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111925 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197237+0000", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.563e-06 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197241+0000", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3770000000000004e-06 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197246+0000", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197250+0000", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2380999999999997e-05 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197322+0000", 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.268e-06 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197332+0000", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94598222200000004 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143314+0000", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013365499999999999 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143448+0000", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014185900000000001 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143589+0000", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078197079999999999 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151409+0000", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015815124 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167224+0000", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027372900000000001 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167498+0000", 2024-02-22T19:35:54.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5688000000000001e-05 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167534+0000", 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010068699999999999 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167634+0000", 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97039658799999995 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111926 7.a 7:5f8a638d:::benchmark_data_smithi053_107660_object22719:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.43100138, 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97061990799999998, 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111926 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5790000000000002e-06 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197263+0000", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197268+0000", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6339999999999999e-06 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197272+0000", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0308000000000004e-05 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197352+0000", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.28e-06 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197358+0000", 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94796157400000003 2024-02-22T19:35:54.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145320+0000", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012371600000000001 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145444+0000", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158498 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145602+0000", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095829260000000003 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155185+0000", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012273795000000001 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167459+0000", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028113900000000002 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167740+0000", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4212999999999998e-05 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167784+0000", 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6879000000000005e-05 2024-02-22T19:35:54.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167881+0000", 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97061990799999998 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111975 7.a 7:5962aaaf:::benchmark_data_smithi053_107660_object22768:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.37148995999999, 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91135001299999996, 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111975 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256772+0000", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7850000000000002e-06 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256776+0000", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0140000000000001e-06 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256783+0000", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9890000000000003e-06 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256787+0000", 2024-02-22T19:35:54.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6700999999999998e-05 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256843+0000", 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8880000000000003e-06 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256852+0000", 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89037246800000003 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147225+0000", 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011598499999999999 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147341+0000", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013154 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147472+0000", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077203439999999996 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155193+0000", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012737145 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167930+0000", 2024-02-22T19:35:54.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0123999999999997e-05 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167970+0000", 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3705000000000003e-05 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168014+0000", 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108915 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168123+0000", 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91135001299999996 2024-02-22T19:35:54.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112025 7.a 7:5114742b:::benchmark_data_smithi053_107660_object22818:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.313366519, 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85347167700000004, 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112025 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314895+0000", 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5490000000000001e-06 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314902+0000", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531e-05 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314913+0000", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1970000000000004e-06 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314918+0000", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1729999999999993e-05 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314980+0000", 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1537e-05 2024-02-22T19:35:54.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314991+0000", 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.83431657000000004 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149308+0000", 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011637900000000001 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149424+0000", 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000147207 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149571+0000", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011278806000000001 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160850+0000", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073124599999999998 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168162+0000", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2563000000000004e-05 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168225+0000", 2024-02-22T19:35:54.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5136999999999998e-05 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168270+0000", 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7520000000000001e-05 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168368+0000", 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85347167700000004 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112063 7.a 7:5ff85e4d:::benchmark_data_smithi053_107660_object22856:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:35:54.698 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.27130498, 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.811635777, 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112063 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356956+0000", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356962+0000", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8179999999999993e-06 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356971+0000", 2024-02-22T19:35:54.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4229999999999996e-06 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356977+0000", 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1233000000000001e-05 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.357038+0000", 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.357048+0000", 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.79440474900000002 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151452+0000", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012225300000000001 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151575+0000", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8306999999999993e-05 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151663+0000", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091947409999999993 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160858+0000", 2024-02-22T19:35:54.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075599589999999998 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168418+0000", 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2265999999999998e-05 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168460+0000", 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0454999999999998e-05 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168500+0000", 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2958e-05 2024-02-22T19:35:54.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168593+0000", 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.811635777 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112096 7.a 7:5442534a:::benchmark_data_smithi053_107660_object22889:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.236960256, 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.777506645, 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112096 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.313e-06 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391306+0000", 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0849999999999999e-06 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391313+0000", 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0760000000000002e-06 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391318+0000", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1353999999999994e-05 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391399+0000", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0243e-05 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391410+0000", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.76178757200000002 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153197+0000", 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109708 2024-02-22T19:35:54.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153307+0000", 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013573900000000001 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153443+0000", 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011010321999999999 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164453+0000", 2024-02-22T19:35:54.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041854379999999997 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168638+0000", 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0509999999999997e-05 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168679+0000", 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8803000000000001e-05 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168718+0000", 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1186000000000004e-05 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168809+0000", 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.777506645 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112098 7.a 7:519628cb:::benchmark_data_smithi053_107660_object22891:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.23502114999999, 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.77578508499999999, 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112098 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393240+0000", 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1999999999999997e-06 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393247+0000", 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5688999999999998e-05 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393263+0000", 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-06 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393270+0000", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4311000000000001e-05 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393334+0000", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3839999999999997e-06 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393342+0000", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.76188229600000001 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155225+0000", 2024-02-22T19:35:54.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5840999999999993e-05 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155310+0000", 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5882000000000002e-05 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155396+0000", 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0090645329999999996 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164461+0000", 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043933849999999997 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168854+0000", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0268000000000001e-05 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168895+0000", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9428000000000003e-05 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168934+0000", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2540000000000005e-05 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169026+0000", 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.77578508499999999 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112130 7.a 7:5e909bd4:::benchmark_data_smithi053_107660_object22923:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.19647022300001, 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73752859400000004, 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112130 2024-02-22T19:35:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431791+0000", 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.251e-06 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431797+0000", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0682999999999999e-05 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431808+0000", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3590000000000002e-06 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431816+0000", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5444999999999994e-05 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431881+0000", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0912e-05 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431892+0000", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.72501750099999995 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.156909+0000", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111295 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.157021+0000", 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7911999999999994e-05 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.157119+0000", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073499309999999997 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164468+0000", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046038349999999997 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169072+0000", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105296 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169178+0000", 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4647999999999997e-05 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169222+0000", 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8640000000000004e-05 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169321+0000", 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73752859400000004 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.721 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112203 7.a 7:5c4dbf81:::benchmark_data_smithi053_107660_object22996:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.11201627200001, 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65325745000000002, 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112203 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516245+0000", 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516253+0000", 2024-02-22T19:35:54.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7668000000000001e-05 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516270+0000", 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6799999999999993e-06 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516278+0000", 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8060000000000003e-05 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516336+0000", 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6299999999999993e-06 2024-02-22T19:35:54.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516346+0000", 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64245662999999997 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.158802+0000", 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103798 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.158906+0000", 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129794 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.159036+0000", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071273559999999996 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166163+0000", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029783499999999998 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169142+0000", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022555399999999999 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169367+0000", 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2079000000000001e-05 2024-02-22T19:35:54.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169409+0000", 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4319000000000003e-05 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169504+0000", 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65325745000000002 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112210 7.a 7:56d9e136:::benchmark_data_smithi053_107660_object23003:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:35:54.727 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.101811204, 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64327634499999997, 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112210 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526450+0000", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2809999999999997e-06 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526456+0000", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0599e-05 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526467+0000", 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4539999999999998e-06 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526472+0000", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6777000000000003e-05 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526519+0000", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3770000000000001e-06 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526526+0000", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63437577199999995 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160901+0000", 2024-02-22T19:35:54.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106143 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.161007+0000", 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109844 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.161117+0000", 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050523790000000001 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166170+0000", 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003381743 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169551+0000", 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1298999999999999e-05 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169593+0000", 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0349000000000003e-05 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.733 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169633+0000", 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4598000000000005e-05 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169728+0000", 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64327634499999997 2024-02-22T19:35:54.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112222 7.a 7:55f40a1f:::benchmark_data_smithi053_107660_object23015:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.550201+0000", 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 133.078061873, 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.61974588799999997, 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112222 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550201+0000", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550201+0000", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550199+0000", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.092e-06 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550206+0000", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.275e-05 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550219+0000", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6590000000000004e-06 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550225+0000", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2351000000000001e-05 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550277+0000", 2024-02-22T19:35:54.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.799e-06 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550286+0000", 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.61258154899999995 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.162867+0000", 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010998899999999999 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.162977+0000", 2024-02-22T19:35:54.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7453000000000004e-05 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.163075+0000", 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033242739999999999 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166399+0000", 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033724670000000001 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169772+0000", 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0144000000000001e-05 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169812+0000", 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2228999999999998e-05 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169854+0000", 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2577000000000005e-05 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169947+0000", 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.61974588799999997 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112313 7.a 7:5a054abe:::benchmark_data_smithi053_107660_object23106:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.811504+0000", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.81675829599999, 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.35866692999999999, 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112313 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811504+0000", 2024-02-22T19:35:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811504+0000", 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811503+0000", 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9540000000000004e-06 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811511+0000", 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1779e-05 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811522+0000", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6200000000000004e-06 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811528+0000", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7555000000000001e-05 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811576+0000", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0870000000000001e-06 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:35:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811583+0000", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.35293079100000002 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164513+0000", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010971699999999999 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164623+0000", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015673400000000001 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164780+0000", 2024-02-22T19:35:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020604970000000001 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166840+0000", 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003159944 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170000+0000", 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5516999999999997e-05 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170056+0000", 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0157000000000002e-05 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170096+0000", 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5141000000000004e-05 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170171+0000", 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.35866692999999999 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:114123 7.3 7:cb5a238a:::benchmark_data_smithi053_107660_object24916:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150214+0000", 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.478048556, 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066204064000000007, 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114123 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150214+0000", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150214+0000", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150212+0000", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5990000000000002e-06 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150221+0000", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0383e-05 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150231+0000", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9410000000000004e-06 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150237+0000", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7024000000000002e-05 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150284+0000", 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029884460000000001 2024-02-22T19:35:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.153273+0000", 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015062699999999999 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.153423+0000", 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042010270000000004 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.157624+0000", 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011881420000000001 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158812+0000", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057520868000000003 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.216333+0000", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8092e-05 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.216381+0000", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6637999999999997e-05 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.216418+0000", 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066204064000000007 2024-02-22T19:35:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:114135 7.3 7:cff05426:::benchmark_data_smithi053_107660_object24928:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.158929+0000", 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.46933397999999, 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064737410999999995, 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:35:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114135 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158929+0000", 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158929+0000", 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:35:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158928+0000", 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8149999999999996e-06 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158933+0000", 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6540000000000004e-06 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158938+0000", 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2259999999999999e-06 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.158942+0000", 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054540957000000001 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213483+0000", 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124239 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213608+0000", 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014612300000000001 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.213754+0000", 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029662080000000001 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.216720+0000", 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046049399999999999 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.217180+0000", 2024-02-22T19:35:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063923239999999996 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.223573+0000", 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1514000000000002e-05 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.223624+0000", 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1687000000000003e-05 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:35:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.223666+0000", 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064737410999999995 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:35:54.757 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:35:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:53.073520+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:53.073561+0000 osd.2 (osd.2) 79 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:53.350539+0000 mon.a (mon.0) 964 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:35:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:53.378426+0000 mgr.y (mgr.14535) 268 : cluster [DBG] pgmap v293: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:35:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:53.449848+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e161: 8 total, 8 up, 7 in 2024-02-22T19:35:54.959 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 22 16 41758 41742 118.516 81.8125 0.00768141 0.00843244 2024-02-22T19:35:54.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:35:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:35:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:35:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 1 pg_num to 11 2024-02-22T19:35:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:54.599294+0000 mgr.y (mgr.14535) 269 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:54.612933+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.53:0/1130775500' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]: dispatch 2024-02-22T19:35:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:54.613605+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]: dispatch 2024-02-22T19:35:55.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:54.599294+0000 mgr.y (mgr.14535) 269 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:55.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:54.612933+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.53:0/1130775500' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]: dispatch 2024-02-22T19:35:55.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:54.613605+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]: dispatch 2024-02-22T19:35:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:54.599294+0000 mgr.y (mgr.14535) 269 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:35:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:54.612933+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.53:0/1130775500' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]: dispatch 2024-02-22T19:35:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:54.613605+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]: dispatch 2024-02-22T19:35:55.959 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 23 16 43115 43099 117.051 84.8125 0.0101174 0.00853748 2024-02-22T19:35:56.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:55.377532+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]': finished 2024-02-22T19:35:56.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:55.377601+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e162: 8 total, 8 up, 7 in 2024-02-22T19:35:56.410 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:55.378846+0000 mgr.y (mgr.14535) 270 : cluster [DBG] pgmap v296: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2k op/s 2024-02-22T19:35:56.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:55.380591+0000 mon.a (mon.0) 969 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-02-22T19:35:56.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:56.391807+0000 mon.a (mon.0) 970 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num_actual", "val": "11"}]': finished 2024-02-22T19:35:56.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:56.391865+0000 mon.a (mon.0) 971 : cluster [DBG] osdmap e163: 8 total, 8 up, 7 in 2024-02-22T19:35:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:56.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:56.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:56.697 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:55.377532+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]': finished 2024-02-22T19:35:56.697 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:55.377601+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e162: 8 total, 8 up, 7 in 2024-02-22T19:35:56.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:55.378846+0000 mgr.y (mgr.14535) 270 : cluster [DBG] pgmap v296: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2k op/s 2024-02-22T19:35:56.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:55.380591+0000 mon.a (mon.0) 969 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-02-22T19:35:56.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:56.391807+0000 mon.a (mon.0) 970 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num_actual", "val": "11"}]': finished 2024-02-22T19:35:56.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:56.391865+0000 mon.a (mon.0) 971 : cluster [DBG] osdmap e163: 8 total, 8 up, 7 in 2024-02-22T19:35:56.798 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:35:56.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:55.377532+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num", "val": "11"}]': finished 2024-02-22T19:35:56.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:55.377601+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e162: 8 total, 8 up, 7 in 2024-02-22T19:35:56.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:55.378846+0000 mgr.y (mgr.14535) 270 : cluster [DBG] pgmap v296: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2k op/s 2024-02-22T19:35:56.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:55.380591+0000 mon.a (mon.0) 969 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-02-22T19:35:56.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:56.391807+0000 mon.a (mon.0) 970 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pg_num_actual", "val": "11"}]': finished 2024-02-22T19:35:56.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:56.391865+0000 mon.a (mon.0) 971 : cluster [DBG] osdmap e163: 8 total, 8 up, 7 in 2024-02-22T19:35:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:35:56.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:56.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:35:56.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:35:56.961 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 24 16 44483 44467 115.736 85.5 0.00806977 0.00863463 2024-02-22T19:35:56.984 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:57.008 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:35:57.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:57.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:35:57.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:35:57.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:57.451 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:57.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:35:57.482 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:35:57.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:57.614 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:57.685 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:57.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:57.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:57.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:35:57.823 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:35:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:57.965 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 25 16 45693 45677 114.114 75.625 0.0218418 0.00875687 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:57.177414+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.53:0/3289805123' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:57.179450+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:57.379440+0000 mgr.y (mgr.14535) 271 : cluster [DBG] pgmap v298: 158 pgs: 10 unknown, 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 852 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:57.382811+0000 mon.a (mon.0) 973 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:57.398376+0000 mon.a (mon.0) 974 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:57.407955+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:35:57.408093+0000 mon.a (mon.0) 976 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:57.408153+0000 mon.a (mon.0) 977 : cluster [DBG] osdmap e164: 8 total, 8 up, 7 in 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:57.418765+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:57.418829+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:57.177414+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.53:0/3289805123' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:57.179450+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:57.379440+0000 mgr.y (mgr.14535) 271 : cluster [DBG] pgmap v298: 158 pgs: 10 unknown, 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 852 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:35:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:57.382811+0000 mon.a (mon.0) 973 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-22T19:35:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:57.398376+0000 mon.a (mon.0) 974 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:57.407955+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:58.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:35:57.408093+0000 mon.a (mon.0) 976 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-22T19:35:58.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:57.408153+0000 mon.a (mon.0) 977 : cluster [DBG] osdmap e164: 8 total, 8 up, 7 in 2024-02-22T19:35:58.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:57.418765+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:58.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:57.418829+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:57.177414+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.53:0/3289805123' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:57.179450+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:57.379440+0000 mgr.y (mgr.14535) 271 : cluster [DBG] pgmap v298: 158 pgs: 10 unknown, 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 852 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:57.382811+0000 mon.a (mon.0) 973 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:57.398376+0000 mon.a (mon.0) 974 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:57.407955+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:35:57.408093+0000 mon.a (mon.0) 976 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:57.408153+0000 mon.a (mon.0) 977 : cluster [DBG] osdmap e164: 8 total, 8 up, 7 in 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:57.418765+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:35:58.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:57.418829+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:35:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:35:58.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:58.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:35:58.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:58.791 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:58.911 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:58.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:58.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:58.965 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 26 15 47036 47021 112.948 84 0.0115706 0.00884789 2024-02-22T19:35:59.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:35:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:35:59.324 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:59.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:35:59.403 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:35:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:35:59] "GET /metrics HTTP/1.1" 200 175763 "" "Prometheus/2.33.4" 2024-02-22T19:35:59.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:35:59.494 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:59.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:35:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:58.408661+0000 mon.a (mon.0) 978 : cluster [DBG] osdmap e165: 8 total, 8 up, 7 in 2024-02-22T19:35:59.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:35:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:58.408661+0000 mon.a (mon.0) 978 : cluster [DBG] osdmap e165: 8 total, 8 up, 7 in 2024-02-22T19:35:59.741 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:35:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:58.408661+0000 mon.a (mon.0) 978 : cluster [DBG] osdmap e165: 8 total, 8 up, 7 in 2024-02-22T19:35:59.760 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:35:59.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:59.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:35:59.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:35:59.905 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:35:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:35:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:35:59.905 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:35:59.913 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:35:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:35:59.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:35:59.961 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:35:59.965 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 27 16 48352 48336 111.809 82.1875 0.00932111 0.00893903 2024-02-22T19:36:00.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:00.132 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:36:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:00.166 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:00.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:00.167 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:36:00.195 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:36:00.237 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:36:00.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:00.279 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:36:00.305 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:00.310 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:36:00.342 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:36:00.379 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:36:00.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:36:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:35:59.380796+0000 mgr.y (mgr.14535) 272 : cluster [DBG] pgmap v301: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.66k op/s 2024-02-22T19:36:00.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:00.093722+0000 mon.c (mon.2) 46 : audit [DBG] from='client.? 172.21.15.53:0/2454095815' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:00.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:36:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:00.660 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:00.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:00.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:35:59.380796+0000 mgr.y (mgr.14535) 272 : cluster [DBG] pgmap v301: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.66k op/s 2024-02-22T19:36:00.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:00.093722+0000 mon.c (mon.2) 46 : audit [DBG] from='client.? 172.21.15.53:0/2454095815' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:00.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:35:59.380796+0000 mgr.y (mgr.14535) 272 : cluster [DBG] pgmap v301: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.66k op/s 2024-02-22T19:36:00.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:00.093722+0000 mon.c (mon.2) 46 : audit [DBG] from='client.? 172.21.15.53:0/2454095815' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:00.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:00.797 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:00.826 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:00.827 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:00.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:00.967 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 28 16 49569 49553 110.526 76.0625 0.0108179 0.009042 2024-02-22T19:36:00.986 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:01.022 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:01.254 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:01.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:01.324 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:01.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:01.390 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:01.420 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:01.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:01.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:01.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:01.584 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:01.615 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:36:01.616 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:36:01.616 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-02-22T19:36:01.616 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-22T19:36:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:01.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:01.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:01.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:01.806 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:01.888 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:01.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:01.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:01.967 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 29 16 50711 50695 109.176 71.375 0.00299797 0.00915387 2024-02-22T19:36:01.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:02.030 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:02.076 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:02.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:02.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:02.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:02.424 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:02.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:02.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:02.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:02.585 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:02.694 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:01.382012+0000 mgr.y (mgr.14535) 273 : cluster [DBG] pgmap v302: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-02-22T19:36:02.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:01.382012+0000 mgr.y (mgr.14535) 273 : cluster [DBG] pgmap v302: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-02-22T19:36:02.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:01.382012+0000 mgr.y (mgr.14535) 273 : cluster [DBG] pgmap v302: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-02-22T19:36:02.848 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:02.861 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:02.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:02.898 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:02.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:02.969 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 30 16 51954 51938 108.121 77.6875 0.00602182 0.00924376 2024-02-22T19:36:03.004 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:36:03.004 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:36:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:03.196 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:03.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:03.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:03.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:36:03.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:03.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:36:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:03.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:03.813 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:03.913 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:03.969 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 31 16 53008 52992 106.759 65.875 0.00848168 0.00936008 2024-02-22T19:36:03.972 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:04.042 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:04.256 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:04.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:04.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:04.427 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:04.503 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:36:04.529 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:03.382836+0000 mgr.y (mgr.14535) 274 : cluster [DBG] pgmap v303: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 732 B/s rd, 59 MiB/s wr, 949 op/s 2024-02-22T19:36:04.529 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:03.691840+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:04.529 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:03.691900+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:04.529 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:03.935068+0000 mon.a (mon.0) 979 : audit [INF] from='client.? 172.21.15.53:0/1246650393' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:36:04.530 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:03.382836+0000 mgr.y (mgr.14535) 274 : cluster [DBG] pgmap v303: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 732 B/s rd, 59 MiB/s wr, 949 op/s 2024-02-22T19:36:04.530 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:03.691840+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:04.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:03.691900+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:04.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:03.935068+0000 mon.a (mon.0) 979 : audit [INF] from='client.? 172.21.15.53:0/1246650393' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:36:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:03.382836+0000 mgr.y (mgr.14535) 274 : cluster [DBG] pgmap v303: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 732 B/s rd, 59 MiB/s wr, 949 op/s 2024-02-22T19:36:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:03.691840+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:03.691900+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:03.935068+0000 mon.a (mon.0) 979 : audit [INF] from='client.? 172.21.15.53:0/1246650393' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:36:04.801 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:04.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:04.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:04.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:04.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:36:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:36:04.943 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:36:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:36:04.947 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:04.969 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 32 16 54200 54184 105.752 74.5 0.009617 0.00945101 2024-02-22T19:36:05.047 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:05.409 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:05.416 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:05.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:05.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.450956+0000 mon.a (mon.0) 980 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:05.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:04.470570+0000 mon.a (mon.0) 981 : audit [INF] from='client.? 172.21.15.53:0/1246650393' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:36:05.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.470625+0000 mon.a (mon.0) 982 : cluster [DBG] osdmap e166: 8 total, 8 up, 7 in 2024-02-22T19:36:05.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.540756+0000 osd.0 (osd.0) 119 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:36:05.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.546696+0000 osd.0 (osd.0) 120 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:36:05.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:04.609393+0000 mgr.y (mgr.14535) 275 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:05.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:05.640 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:05.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:05.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.450956+0000 mon.a (mon.0) 980 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:05.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:04.470570+0000 mon.a (mon.0) 981 : audit [INF] from='client.? 172.21.15.53:0/1246650393' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:36:05.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.470625+0000 mon.a (mon.0) 982 : cluster [DBG] osdmap e166: 8 total, 8 up, 7 in 2024-02-22T19:36:05.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.540756+0000 osd.0 (osd.0) 119 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:36:05.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.546696+0000 osd.0 (osd.0) 120 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:36:05.701 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:04.609393+0000 mgr.y (mgr.14535) 275 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:05.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.450956+0000 mon.a (mon.0) 980 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:04.470570+0000 mon.a (mon.0) 981 : audit [INF] from='client.? 172.21.15.53:0/1246650393' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:36:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.470625+0000 mon.a (mon.0) 982 : cluster [DBG] osdmap e166: 8 total, 8 up, 7 in 2024-02-22T19:36:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.540756+0000 osd.0 (osd.0) 119 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:36:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.546696+0000 osd.0 (osd.0) 120 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:36:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:04.609393+0000 mgr.y (mgr.14535) 275 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:05.781 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:05.970 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 33 16 55428 55412 104.873 76.75 0.00746257 0.00953026 2024-02-22T19:36:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:05.995 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:05.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:06.013 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:36:06.013 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:36:06.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:06.160 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:06.258 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:06.265 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:06.265 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:06.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:06.405 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:06.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.638970+0000 osd.7 (osd.7) 111 : cluster [DBG] 3.f scrub starts 2024-02-22T19:36:06.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.642351+0000 osd.7 (osd.7) 112 : cluster [DBG] 3.f scrub ok 2024-02-22T19:36:06.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.920855+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.d scrub starts 2024-02-22T19:36:06.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:04.927060+0000 osd.2 (osd.2) 81 : cluster [DBG] 5.d scrub ok 2024-02-22T19:36:06.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:05.174170+0000 osd.6 (osd.6) 71 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:36:06.490 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:05.177289+0000 osd.6 (osd.6) 72 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:36:06.490 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:05.383245+0000 mgr.y (mgr.14535) 276 : cluster [DBG] pgmap v305: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.3 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.638970+0000 osd.7 (osd.7) 111 : cluster [DBG] 3.f scrub starts 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.642351+0000 osd.7 (osd.7) 112 : cluster [DBG] 3.f scrub ok 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.920855+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.d scrub starts 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:04.927060+0000 osd.2 (osd.2) 81 : cluster [DBG] 5.d scrub ok 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:05.174170+0000 osd.6 (osd.6) 71 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:05.177289+0000 osd.6 (osd.6) 72 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:36:06.675 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:05.383245+0000 mgr.y (mgr.14535) 276 : cluster [DBG] pgmap v305: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.3 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-02-22T19:36:06.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 166, 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-02-22T19:22:59.746063+0000", 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-02-22T19:36:04.450735+0000", 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-02-22T19:28:53.274045+0000", 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-02-22T19:31:48.970507+0000", 2024-02-22T19:36:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5804032, 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 18, 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 8, 2024-02-22T19:36:06.723 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "pacific", 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "device_health_metrics", 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:26:54.342689+0000", 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.724 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 11, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 11, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 11, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 11, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 11, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.725 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "164", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "163", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.727 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:36:06.728 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_devicehealth": {} 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "iscsi", 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.555543+0000", 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 3, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 3, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 3, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 3, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 3, 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.730 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "59", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.733 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".rgw.root", 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.986190+0000", 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.734 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.735 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "58", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.736 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.737 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.738 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 4, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.log", 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:01.494051+0000", 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:36:06.739 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "60", 2024-02-22T19:36:06.740 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.741 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.742 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 5, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.control", 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:03.518927+0000", 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.743 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.744 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "62", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.745 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.746 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.747 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 6, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.meta", 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:05.596341+0000", 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.748 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "64", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:36:06.749 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.750 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.751 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_bias": 4 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 8, 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "unique_pool_1", 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:35:18.161650+0000", 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:36:06.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 16, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 16, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 16, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 16, 2024-02-22T19:36:06.753 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 16, 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "157", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:36:06.754 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:36:06.755 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:36:06.756 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "rados": {} 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab", 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.757 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 8, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.758 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.760 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/162271081", 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6802/162271081", 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6804/162271081", 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6803/162271081", 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2", 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-02-22T19:36:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6805/2973403044", 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6806/2973403044", 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6808/2973403044", 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6807/2973403044", 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.765 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83", 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 18, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.766 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.768 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/936980935", 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6810/936980935", 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6812/936980935", 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6811/936980935", 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993", 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.769 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 26, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.771 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6813/3878043918", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6814/3878043918", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6816/3878043918", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6815/3878043918", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.772 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211", 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 32, 2024-02-22T19:36:06.773 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6800", 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6801", 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6803", 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6802", 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6800/2373040976", 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6801/2373040976", 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6803/2373040976", 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6802/2373040976", 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.776 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07", 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 39, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 93, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.777 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6804", 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6805", 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6807", 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6806", 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:36:06.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6804/2413553331", 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6805/2413553331", 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6807/2413553331", 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6806/2413553331", 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e", 2024-02-22T19:36:06.780 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 45, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6808", 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.781 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6809", 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.782 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6811", 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6810", 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6808/3043274732", 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6809/3043274732", 2024-02-22T19:36:06.783 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6811/3043274732", 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6810/3043274732", 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a", 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:36:06.784 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 53, 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 164, 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6812", 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6813", 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6815", 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:36:06.786 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6814", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6812/2605082260", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6813/2605082260", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6815/2605082260", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6814/2605082260", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:36:06.787 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:34:54.279840+0000", 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:36:06.788 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:35:57.418832+0000", 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:35:53.073563+0000", 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:35:33.472333+0000", 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.790 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:35:06.102652+0000", 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:34:58.011730+0000", 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:36:06.791 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:36:03.691902+0000", 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:36:06.792 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:35:01.285990+0000", 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [ 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.1", 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.793 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.1d", 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 4, 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.794 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.7", 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.b", 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.f", 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": { 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4161439976": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2578736337": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:36:06.796 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/762516580": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3676085495": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3516945073": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/828421682": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4127851809": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/1906929179": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2640420245": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/2854906057": "2024-02-23T19:23:32.485117+0000" 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-02-22T19:36:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "ruleset-failure-domain": "osd", 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-02-22T19:36:06.798 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-02-22T19:36:06.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:06.799 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:36:06.799 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.638970+0000 osd.7 (osd.7) 111 : cluster [DBG] 3.f scrub starts 2024-02-22T19:36:06.800 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.642351+0000 osd.7 (osd.7) 112 : cluster [DBG] 3.f scrub ok 2024-02-22T19:36:06.800 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.920855+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.d scrub starts 2024-02-22T19:36:06.800 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:04.927060+0000 osd.2 (osd.2) 81 : cluster [DBG] 5.d scrub ok 2024-02-22T19:36:06.800 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:05.174170+0000 osd.6 (osd.6) 71 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:36:06.800 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:05.177289+0000 osd.6 (osd.6) 72 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:36:06.800 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:05.383245+0000 mgr.y (mgr.14535) 276 : cluster [DBG] pgmap v305: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.3 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-02-22T19:36:06.800 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:06.801 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:06.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:06.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:06.969 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 34 16 56540 56524 103.833 69.5 0.00619822 0.00962557 2024-02-22T19:36:07.148 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:07.184 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:07.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:07.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:07.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:07.498 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:06.713332+0000 mon.b (mon.1) 51 : audit [DBG] from='client.? 172.21.15.53:0/3984625869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:36:07.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:36:07.738 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:07.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:07.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:06.713332+0000 mon.b (mon.1) 51 : audit [DBG] from='client.? 172.21.15.53:0/3984625869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:36:07.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:06.713332+0000 mon.b (mon.1) 51 : audit [DBG] from='client.? 172.21.15.53:0/3984625869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:36:07.885 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:07.970 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 35 16 57586 57570 102.734 65.375 0.01079 0.00972934 2024-02-22T19:36:08.318 INFO:tasks.thrashosds.thrasher:j is {'epoch': 166, 'fsid': 'b6783888-d1b7-11ee-95bf-87774f69a715', 'created': '2024-02-22T19:22:59.746063+0000', 'modified': '2024-02-22T19:36:04.450735+0000', 'last_up_change': '2024-02-22T19:28:53.274045+0000', 'last_in_change': '2024-02-22T19:31:48.970507+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '163', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-02-22T19:35:18.161650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '4326a76a-6144-4859-9b40-5ff69edbbcab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6801', 'nonce': 162271081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 162271081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 162271081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6803', 'nonce': 162271081}]}, 'public_addr': '172.21.15.53:6801/162271081', 'cluster_addr': '172.21.15.53:6802/162271081', 'heartbeat_back_addr': '172.21.15.53:6804/162271081', 'heartbeat_front_addr': '172.21.15.53:6803/162271081', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6805', 'nonce': 2973403044}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2973403044}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2973403044}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6807', 'nonce': 2973403044}]}, 'public_addr': '172.21.15.53:6805/2973403044', 'cluster_addr': '172.21.15.53:6806/2973403044', 'heartbeat_back_addr': '172.21.15.53:6808/2973403044', 'heartbeat_front_addr': '172.21.15.53:6807/2973403044', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6809', 'nonce': 936980935}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 936980935}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 936980935}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6811', 'nonce': 936980935}]}, 'public_addr': '172.21.15.53:6809/936980935', 'cluster_addr': '172.21.15.53:6810/936980935', 'heartbeat_back_addr': '172.21.15.53:6812/936980935', 'heartbeat_front_addr': '172.21.15.53:6811/936980935', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3f285320-aa28-4fc1-b1c7-456c9aecd993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6813', 'nonce': 3878043918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 3878043918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 3878043918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6815', 'nonce': 3878043918}]}, 'public_addr': '172.21.15.53:6813/3878043918', 'cluster_addr': '172.21.15.53:6814/3878043918', 'heartbeat_back_addr': '172.21.15.53:6816/3878043918', 'heartbeat_front_addr': '172.21.15.53:6815/3878043918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '773e8080-1028-48fe-8f6e-ed6beca60211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2373040976}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6801', 'nonce': 2373040976}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 2373040976}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2373040976}]}, 'public_addr': '172.21.15.64:6800/2373040976', 'cluster_addr': '172.21.15.64:6801/2373040976', 'heartbeat_back_addr': '172.21.15.64:6803/2373040976', 'heartbeat_front_addr': '172.21.15.64:6802/2373040976', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6804', 'nonce': 2413553331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6805', 'nonce': 2413553331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 2413553331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2413553331}]}, 'public_addr': '172.21.15.64:6804/2413553331', 'cluster_addr': '172.21.15.64:6805/2413553331', 'heartbeat_back_addr': '172.21.15.64:6807/2413553331', 'heartbeat_front_addr': '172.21.15.64:6806/2413553331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6808', 'nonce': 3043274732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6809', 'nonce': 3043274732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 3043274732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 3043274732}]}, 'public_addr': '172.21.15.64:6808/3043274732', 'cluster_addr': '172.21.15.64:6809/3043274732', 'heartbeat_back_addr': '172.21.15.64:6811/3043274732', 'heartbeat_front_addr': '172.21.15.64:6810/3043274732', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'cde36201-cb77-4967-9402-6fdbe633b40a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6812', 'nonce': 2605082260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6813', 'nonce': 2605082260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6815', 'nonce': 2605082260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 2605082260}]}, 'public_addr': '172.21.15.64:6812/2605082260', 'cluster_addr': '172.21.15.64:6813/2605082260', 'heartbeat_back_addr': '172.21.15.64:6815/2605082260', 'heartbeat_front_addr': '172.21.15.64:6814/2605082260', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:34:54.279840+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:35:57.418832+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:35:53.073563+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:35:33.472333+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:35:06.102652+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:34:58.011730+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:36:03.691902+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:35:01.285990+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.1', 'osds': [5, 3, 4]}], 'pg_upmap_items': [{'pgid': '3.1d', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.53:0/4161439976': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/2578736337': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/762516580': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:6800/3676085495': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:6800/3516945073': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/828421682': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/4127851809': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/1906929179': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:0/2640420245': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:6800/2854906057': '2024-02-23T19:23:32.485117+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-22T19:36:08.318 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.f 2024-02-22T19:36:08.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd rm-pg-upmap-items 4.f 2024-02-22T19:36:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:08.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:08.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:08.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:08.598 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:08.653 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:08.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:06.571603+0000 osd.7 (osd.7) 113 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:36:08.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:06.575856+0000 osd.7 (osd.7) 114 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:36:08.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.160326+0000 osd.6 (osd.6) 73 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:36:08.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.161521+0000 osd.6 (osd.6) 74 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:36:08.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.383657+0000 mgr.y (mgr.14535) 277 : cluster [DBG] pgmap v306: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 87 MiB/s wr, 1.40k op/s 2024-02-22T19:36:08.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.587898+0000 osd.0 (osd.0) 121 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:36:08.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.590502+0000 osd.0 (osd.0) 122 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:36:08.657 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:08.662 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:08.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:08.678 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:36:08.710 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:36:08.754 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:36:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:06.571603+0000 osd.7 (osd.7) 113 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:36:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:06.575856+0000 osd.7 (osd.7) 114 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:36:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.160326+0000 osd.6 (osd.6) 73 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:36:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.161521+0000 osd.6 (osd.6) 74 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.383657+0000 mgr.y (mgr.14535) 277 : cluster [DBG] pgmap v306: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 87 MiB/s wr, 1.40k op/s 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.587898+0000 osd.0 (osd.0) 121 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.590502+0000 osd.0 (osd.0) 122 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:06.571603+0000 osd.7 (osd.7) 113 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:06.575856+0000 osd.7 (osd.7) 114 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.160326+0000 osd.6 (osd.6) 73 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.161521+0000 osd.6 (osd.6) 74 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.383657+0000 mgr.y (mgr.14535) 277 : cluster [DBG] pgmap v306: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 87 MiB/s wr, 1.40k op/s 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.587898+0000 osd.0 (osd.0) 121 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:36:08.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.590502+0000 osd.0 (osd.0) 122 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:36:08.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:08.801 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:08.803 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:36:08.844 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:36:08.888 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:36:08.925 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:36:08.929 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:08.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:08.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:08.955 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:08.955 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:36:08.955 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:36:08.956 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:08.967 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:36:08.970 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 36 16 58616 58600 101.668 64.375 0.0157476 0.00983055 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:36:08.980 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:36:09.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:09.067 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:09.198 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:09.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:09.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:09.350 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:09.548 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.620220+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:36:09.548 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:07.626194+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:36:09.551 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.620220+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:36:09.552 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:07.626194+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:36:09.552 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:08.633290+0000 mon.c (mon.2) 47 : audit [DBG] from='client.? 172.21.15.53:0/1570982687' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:09.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.620220+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:36:09.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:07.626194+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:36:09.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:08.633290+0000 mon.c (mon.2) 47 : audit [DBG] from='client.? 172.21.15.53:0/1570982687' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:09.554 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:36:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:09] "GET /metrics HTTP/1.1" 200 175796 "" "Prometheus/2.33.4" 2024-02-22T19:36:09.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:36:09.633 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:09.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:09.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:09.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:09.799 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:08.633290+0000 mon.c (mon.2) 47 : audit [DBG] from='client.? 172.21.15.53:0/1570982687' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:09.970 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 37 16 59738 59722 100.816 70.125 0.00837208 0.00991424 2024-02-22T19:36:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:10.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:10.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:10.290 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:10.360 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:10.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:10.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:10.516 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:08.661395+0000 osd.7 (osd.7) 117 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:36:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:08.682996+0000 osd.7 (osd.7) 118 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:36:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:08.706303+0000 osd.4 (osd.4) 109 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:36:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:08.709159+0000 osd.4 (osd.4) 110 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:36:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:08.875252+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:36:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:08.908567+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:36:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.200750+0000 osd.3 (osd.3) 117 : cluster [DBG] 5.17 scrub starts 2024-02-22T19:36:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.204722+0000 osd.3 (osd.3) 118 : cluster [DBG] 5.17 scrub ok 2024-02-22T19:36:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.385994+0000 mgr.y (mgr.14535) 278 : cluster [DBG] pgmap v307: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-02-22T19:36:10.972 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 38 16 60848 60832 99.9838 69.375 0.0651171 0.00999023 2024-02-22T19:36:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:08.661395+0000 osd.7 (osd.7) 117 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:08.682996+0000 osd.7 (osd.7) 118 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:08.706303+0000 osd.4 (osd.4) 109 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:08.709159+0000 osd.4 (osd.4) 110 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:08.875252+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:08.908567+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.200750+0000 osd.3 (osd.3) 117 : cluster [DBG] 5.17 scrub starts 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.204722+0000 osd.3 (osd.3) 118 : cluster [DBG] 5.17 scrub ok 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.385994+0000 mgr.y (mgr.14535) 278 : cluster [DBG] pgmap v307: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:08.661395+0000 osd.7 (osd.7) 117 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:08.682996+0000 osd.7 (osd.7) 118 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:08.706303+0000 osd.4 (osd.4) 109 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:36:11.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:08.709159+0000 osd.4 (osd.4) 110 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:36:11.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:08.875252+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:36:11.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:08.908567+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:36:11.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.200750+0000 osd.3 (osd.3) 117 : cluster [DBG] 5.17 scrub starts 2024-02-22T19:36:11.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.204722+0000 osd.3 (osd.3) 118 : cluster [DBG] 5.17 scrub ok 2024-02-22T19:36:11.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.385994+0000 mgr.y (mgr.14535) 278 : cluster [DBG] pgmap v307: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-02-22T19:36:11.296 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:11.301 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:11.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:11.402 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:11.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:11.574 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:11.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:11.739 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.619864+0000 osd.7 (osd.7) 119 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:36:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.623516+0000 osd.7 (osd.7) 120 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:36:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.880463+0000 osd.1 (osd.1) 99 : cluster [DBG] 5.c scrub starts 2024-02-22T19:36:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:09.885990+0000 osd.1 (osd.1) 100 : cluster [DBG] 5.c scrub ok 2024-02-22T19:36:11.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.619864+0000 osd.7 (osd.7) 119 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:36:11.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.623516+0000 osd.7 (osd.7) 120 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:36:11.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.880463+0000 osd.1 (osd.1) 99 : cluster [DBG] 5.c scrub starts 2024-02-22T19:36:11.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:09.885990+0000 osd.1 (osd.1) 100 : cluster [DBG] 5.c scrub ok 2024-02-22T19:36:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:11.900 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.619864+0000 osd.7 (osd.7) 119 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:36:11.900 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.623516+0000 osd.7 (osd.7) 120 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:36:11.900 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.880463+0000 osd.1 (osd.1) 99 : cluster [DBG] 5.c scrub starts 2024-02-22T19:36:11.900 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:09.885990+0000 osd.1 (osd.1) 100 : cluster [DBG] 5.c scrub ok 2024-02-22T19:36:11.903 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:11.904 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:11.972 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 39 16 61943 61927 99.1752 68.4375 0.0216241 0.0100778 2024-02-22T19:36:12.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:12.037 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:12.399 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:12.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:12.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:12.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:12.542 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:12.626 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:10.673776+0000 osd.4 (osd.4) 111 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:10.678483+0000 osd.4 (osd.4) 112 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:10.966435+0000 osd.2 (osd.2) 82 : cluster [DBG] 6.f scrub starts 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.024566+0000 osd.2 (osd.2) 83 : cluster [DBG] 6.f scrub ok 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.183952+0000 osd.3 (osd.3) 119 : cluster [DBG] 6.b scrub starts 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.224638+0000 osd.3 (osd.3) 120 : cluster [DBG] 6.b scrub ok 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.387931+0000 mgr.y (mgr.14535) 279 : cluster [DBG] pgmap v308: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.532328+0000 osd.7 (osd.7) 121 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.554374+0000 osd.7 (osd.7) 122 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:12.312483+0000 osd.7 (osd.7) 123 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:12.312524+0000 osd.7 (osd.7) 124 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:12.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:12.505017+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.53:0/2742383242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:12.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:12.505411+0000 mon.a (mon.0) 983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:12.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:12.509105+0000 osd.7 (osd.7) 125 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:36:12.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:12.513641+0000 osd.7 (osd.7) 126 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:36:12.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:12.632 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:12.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:12.931 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:36:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:36:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:36:12.931 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:12.976 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:36:12.977247+0000 min lat: 0.00121627 max lat: 0.0749631 avg lat: 0.0101629 2024-02-22T19:36:12.976 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:36:12.976 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 40 16 62998 62982 98.3349 65.9375 0.0159148 0.0101629 2024-02-22T19:36:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:10.673776+0000 osd.4 (osd.4) 111 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:36:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:10.678483+0000 osd.4 (osd.4) 112 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:36:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:10.966435+0000 osd.2 (osd.2) 82 : cluster [DBG] 6.f scrub starts 2024-02-22T19:36:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.024566+0000 osd.2 (osd.2) 83 : cluster [DBG] 6.f scrub ok 2024-02-22T19:36:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.183952+0000 osd.3 (osd.3) 119 : cluster [DBG] 6.b scrub starts 2024-02-22T19:36:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.224638+0000 osd.3 (osd.3) 120 : cluster [DBG] 6.b scrub ok 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.387931+0000 mgr.y (mgr.14535) 279 : cluster [DBG] pgmap v308: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.532328+0000 osd.7 (osd.7) 121 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.554374+0000 osd.7 (osd.7) 122 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:12.312483+0000 osd.7 (osd.7) 123 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:12.312524+0000 osd.7 (osd.7) 124 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:12.505017+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.53:0/2742383242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:12.505411+0000 mon.a (mon.0) 983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:12.509105+0000 osd.7 (osd.7) 125 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:12.513641+0000 osd.7 (osd.7) 126 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:10.673776+0000 osd.4 (osd.4) 111 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:10.678483+0000 osd.4 (osd.4) 112 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:10.966435+0000 osd.2 (osd.2) 82 : cluster [DBG] 6.f scrub starts 2024-02-22T19:36:13.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.024566+0000 osd.2 (osd.2) 83 : cluster [DBG] 6.f scrub ok 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.183952+0000 osd.3 (osd.3) 119 : cluster [DBG] 6.b scrub starts 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.224638+0000 osd.3 (osd.3) 120 : cluster [DBG] 6.b scrub ok 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.387931+0000 mgr.y (mgr.14535) 279 : cluster [DBG] pgmap v308: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.532328+0000 osd.7 (osd.7) 121 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.554374+0000 osd.7 (osd.7) 122 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:12.312483+0000 osd.7 (osd.7) 123 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:12.312524+0000 osd.7 (osd.7) 124 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:12.505017+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.53:0/2742383242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:12.505411+0000 mon.a (mon.0) 983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:12.509105+0000 osd.7 (osd.7) 125 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:36:13.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:12.513641+0000 osd.7 (osd.7) 126 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:36:13.141 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:36:13.166 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:13.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:13.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:13.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:13.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:36:13.311 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:13.579 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.686114+0000 osd.4 (osd.4) 113 : cluster [DBG] 4.e scrub starts 2024-02-22T19:36:13.579 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:11.687518+0000 osd.4 (osd.4) 114 : cluster [DBG] 4.e scrub ok 2024-02-22T19:36:13.579 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:13.101604+0000 mon.a (mon.0) 984 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:36:13.579 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:13.101637+0000 mon.a (mon.0) 985 : cluster [INF] Cluster is now healthy 2024-02-22T19:36:13.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:13.106846+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:36:13.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:13.106937+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e167: 8 total, 8 up, 7 in 2024-02-22T19:36:13.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:36:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:13.668 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:13.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:13.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:13.829 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:13.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.686114+0000 osd.4 (osd.4) 113 : cluster [DBG] 4.e scrub starts 2024-02-22T19:36:13.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:11.687518+0000 osd.4 (osd.4) 114 : cluster [DBG] 4.e scrub ok 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:13.101604+0000 mon.a (mon.0) 984 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:13.101637+0000 mon.a (mon.0) 985 : cluster [INF] Cluster is now healthy 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:13.106846+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:13.106937+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e167: 8 total, 8 up, 7 in 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.686114+0000 osd.4 (osd.4) 113 : cluster [DBG] 4.e scrub starts 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:11.687518+0000 osd.4 (osd.4) 114 : cluster [DBG] 4.e scrub ok 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:13.101604+0000 mon.a (mon.0) 984 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:13.101637+0000 mon.a (mon.0) 985 : cluster [INF] Cluster is now healthy 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:13.106846+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:36:13.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:13.106937+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e167: 8 total, 8 up, 7 in 2024-02-22T19:36:13.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:13.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:13.983 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 41 16 64018 64002 97.4872 63.75 0.0145093 0.0102524 2024-02-22T19:36:14.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:14.078 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:14.435 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:14.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:14.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:14.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:14.621 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:14.665 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:14.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:14.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:13.388546+0000 mgr.y (mgr.14535) 280 : cluster [DBG] pgmap v310: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 573 B/s rd, 39 MiB/s wr, 624 op/s 2024-02-22T19:36:14.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:14.429701+0000 mon.a (mon.0) 988 : audit [INF] from='client.? 172.21.15.53:0/3231520892' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.f"}]: dispatch 2024-02-22T19:36:14.674 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:13.388546+0000 mgr.y (mgr.14535) 280 : cluster [DBG] pgmap v310: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 573 B/s rd, 39 MiB/s wr, 624 op/s 2024-02-22T19:36:14.674 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:14.429701+0000 mon.a (mon.0) 988 : audit [INF] from='client.? 172.21.15.53:0/3231520892' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.f"}]: dispatch 2024-02-22T19:36:14.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:14.735 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:14.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:14.816 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:13.388546+0000 mgr.y (mgr.14535) 280 : cluster [DBG] pgmap v310: 158 pgs: 158 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 573 B/s rd, 39 MiB/s wr, 624 op/s 2024-02-22T19:36:14.816 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:14.429701+0000 mon.a (mon.0) 988 : audit [INF] from='client.? 172.21.15.53:0/3231520892' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.f"}]: dispatch 2024-02-22T19:36:14.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:36:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:36:14.817 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:36:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:36:14.819 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:14.901 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:14.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:14.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:14.979 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 42 16 65028 65012 96.6691 63.125 0.0113769 0.010339 2024-02-22T19:36:15.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:15.074 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:15.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:15.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:15.152 INFO:teuthology.orchestra.run.smithi053.stderr:clear 4.f pg_upmap_items mapping 2024-02-22T19:36:15.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:15.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:15.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:15.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:15.542 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:13.834157+0000 osd.1 (osd.1) 101 : cluster [DBG] 5.19 deep-scrub starts 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:13.836973+0000 osd.1 (osd.1) 102 : cluster [DBG] 5.19 deep-scrub ok 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:14.021538+0000 osd.2 (osd.2) 84 : cluster [DBG] 5.b deep-scrub starts 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:14.029381+0000 osd.2 (osd.2) 85 : cluster [DBG] 5.b deep-scrub ok 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:14.222375+0000 osd.6 (osd.6) 75 : cluster [DBG] 1.6 deep-scrub starts 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:14.230361+0000 osd.6 (osd.6) 76 : cluster [DBG] 1.6 deep-scrub ok 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:14.620708+0000 mgr.y (mgr.14535) 281 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:15.114572+0000 mon.a (mon.0) 989 : audit [INF] from='client.? 172.21.15.53:0/3231520892' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.f"}]': finished 2024-02-22T19:36:15.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.114629+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e168: 8 total, 8 up, 7 in 2024-02-22T19:36:15.962 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:15.962 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:36:15.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:36:15.962 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19162 7.0 7:049100ab:::benchmark_data_smithi053_107660_object19161:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.132188+0000", 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.82767708, 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012938493000000001, 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19162 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132188+0000", 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132188+0000", 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132186+0000", 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0939999999999999e-05 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132217+0000", 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0973e-05 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132228+0000", 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2610000000000001e-06 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132234+0000", 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1192999999999997e-05 2024-02-22T19:36:15.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132275+0000", 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0587999999999994e-05 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132365+0000", 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100864 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132466+0000", 2024-02-22T19:36:15.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00148449 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133951+0000", 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032656590000000002 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.137216+0000", 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0077983369999999998 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.145015+0000", 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4248999999999999e-05 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.145099+0000", 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7319999999999999e-05 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.145126+0000", 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012938493000000001 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:15.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19170 7.0 7:099f3196:::benchmark_data_smithi053_107660_object19169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.132770+0000", 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.82709490299999, 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013757603, 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19170 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132770+0000", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132770+0000", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132769+0000", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7450999999999998e-05 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132806+0000", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4518999999999999e-05 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132821+0000", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3330000000000002e-06 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.132827+0000", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00037433600000000002 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133202+0000", 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115468 2024-02-22T19:36:15.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133317+0000", 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015316 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.133470+0000", 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00062185599999999995 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.134092+0000", 2024-02-22T19:36:15.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003163646 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.137256+0000", 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0092206400000000004 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.146476+0000", 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9346e-05 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.146506+0000", 2024-02-22T19:36:15.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1888000000000001e-05 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.146528+0000", 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013757603 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19192 7.0 7:0c92f274:::benchmark_data_smithi053_107660_object19191:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.148536+0000", 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.81132860700001, 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006757702, 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.974 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19192 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148536+0000", 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148536+0000", 2024-02-22T19:36:15.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148534+0000", 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1286000000000003e-05 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148586+0000", 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7541e-05 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148603+0000", 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2600000000000005e-06 2024-02-22T19:36:15.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148611+0000", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102261 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148714+0000", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000128057 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.148842+0000", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000163614 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.149005+0000", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015786750000000001 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.150584+0000", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018755429999999999 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.152460+0000", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027746709999999998 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.155234+0000", 2024-02-22T19:36:15.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4789999999999997e-05 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.155269+0000", 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5043000000000001e-05 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.155294+0000", 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006757702 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19339 7.0 7:03559344:::benchmark_data_smithi053_107660_object19338:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.980 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.243068+0000", 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.71679733299999, 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014996083, 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19339 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243068+0000", 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243068+0000", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243066+0000", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7571000000000003e-05 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243103+0000", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1447000000000001e-05 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243115+0000", 2024-02-22T19:36:15.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4299999999999997e-06 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243120+0000", 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3835000000000002e-05 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243174+0000", 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100698 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243275+0000", 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000101266 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.243376+0000", 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00076734500000000001 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.244144+0000", 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081373880000000006 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.252281+0000", 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056537219999999999 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.257935+0000", 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5027000000000001e-05 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.257980+0000", 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4048000000000006e-05 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.258064+0000", 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014996083 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.985 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19495 7.0 7:03bef8f6:::benchmark_data_smithi053_107660_object19494:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.332319+0000", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.62754638799998, 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010829564999999999, 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19495 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332319+0000", 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332319+0000", 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332317+0000", 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3709999999999999e-05 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332371+0000", 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0648999999999999e-05 2024-02-22T19:36:15.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332382+0000", 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3790000000000003e-06 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332388+0000", 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1232000000000001e-05 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332429+0000", 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014674600000000001 2024-02-22T19:36:15.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332576+0000", 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015044 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.332727+0000", 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00071417799999999999 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.333441+0000", 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024283830000000001 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.335869+0000", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0071718609999999999 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.343041+0000", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3146999999999997e-05 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.343104+0000", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4029000000000001e-05 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.343148+0000", 2024-02-22T19:36:15.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010829564999999999 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19582 7.0 7:04cf3c1f:::benchmark_data_smithi053_107660_object19581:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.382535+0000", 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.577330184, 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070217949999999999, 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19582 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.991 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382535+0000", 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382535+0000", 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382532+0000", 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8909000000000001e-05 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382591+0000", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7160000000000003e-05 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382628+0000", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2578999999999999e-05 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382641+0000", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018372399999999999 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382825+0000", 2024-02-22T19:36:15.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015238099999999999 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382977+0000", 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017174399999999999 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383149+0000", 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014893879999999999 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.384638+0000", 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00052996900000000003 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.385168+0000", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043303209999999998 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.389498+0000", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3080000000000002e-05 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.389532+0000", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5000000000000001e-05 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.389557+0000", 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070217949999999999 2024-02-22T19:36:15.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19587 7.0 7:0712d9c9:::benchmark_data_smithi053_107660_object19586:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.382667+0000", 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.57719823600002, 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083673589999999996, 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19587 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.996 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382667+0000", 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382667+0000", 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382665+0000", 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1635999999999999e-05 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382707+0000", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0740000000000001e-05 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382718+0000", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0060000000000003e-06 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.382725+0000", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0010469699999999999 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383772+0000", 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124206 2024-02-22T19:36:15.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383896+0000", 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7546999999999994e-05 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.383983+0000", 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00075819799999999997 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.384742+0000", 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000459751 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:15.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.385201+0000", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005775662 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.390977+0000", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0908999999999998e-05 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.391008+0000", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6064e-05 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.391034+0000", 2024-02-22T19:36:16.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083673589999999996 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19598 7.0 7:04b15c7c:::benchmark_data_smithi053_107660_object19597:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.393641+0000", 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.56622356399998, 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089943990000000001, 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19598 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.001 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393641+0000", 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393641+0000", 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393640+0000", 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1282e-05 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393701+0000", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3458000000000001e-05 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393714+0000", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.418e-06 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393722+0000", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1810000000000001e-05 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393764+0000", 2024-02-22T19:36:16.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6761000000000006e-05 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393840+0000", 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110919 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.393951+0000", 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001293137 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.395244+0000", 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002559221 2024-02-22T19:36:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.397804+0000", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004777051 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.402581+0000", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3751000000000003e-05 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.402614+0000", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1347000000000001e-05 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.402636+0000", 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089943990000000001 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19665 7.0 7:0fd8bd0c:::benchmark_data_smithi053_107660_object19664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.432947+0000", 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.526918217, 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093526809999999998, 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19665 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.432947+0000", 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.432947+0000", 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.432945+0000", 2024-02-22T19:36:16.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1642000000000002e-05 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433026+0000", 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5486e-05 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433052+0000", 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5729999999999998e-06 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433057+0000", 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8538999999999997e-05 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433106+0000", 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8690000000000006e-05 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433195+0000", 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104849 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.433299+0000", 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00090149700000000002 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.434201+0000", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002709595 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.436910+0000", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052803090000000004 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.442191+0000", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4300000000000004e-05 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.442255+0000", 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4342000000000003e-05 2024-02-22T19:36:16.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.442299+0000", 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093526809999999998 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19707 7.0 7:0b2b3dfb:::benchmark_data_smithi053_107660_object19706:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.454436+0000", 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.50542881799998, 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0079920819999999993, 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.011 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19707 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454436+0000", 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454436+0000", 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454435+0000", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0632e-05 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454476+0000", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8410000000000006e-06 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454485+0000", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1099999999999996e-06 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454489+0000", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4212000000000003e-05 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454533+0000", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123671 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454657+0000", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000175818 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.454832+0000", 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00075767599999999999 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.455590+0000", 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061248409999999998 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.461715+0000", 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060546099999999995 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.462320+0000", 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9639000000000003e-05 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.462380+0000", 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8124000000000002e-05 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.462428+0000", 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0079920819999999993 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:19721 7.0 7:0e922d33:::benchmark_data_smithi053_107660_object19720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.466702+0000", 2024-02-22T19:36:16.016 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.49316319899998, 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090205489999999992, 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 19721 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466702+0000", 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466702+0000", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466700+0000", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1017999999999998e-05 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466751+0000", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2935999999999999e-05 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466764+0000", 2024-02-22T19:36:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.725e-06 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466770+0000", 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6407999999999999e-05 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466827+0000", 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016757999999999999 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.466994+0000", 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018296600000000001 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.467177+0000", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001057987 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.468235+0000", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064652999999999998 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.468882+0000", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067415519999999996 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.475623+0000", 2024-02-22T19:36:16.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7846000000000001e-05 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.475681+0000", 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0992999999999999e-05 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.475722+0000", 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090205489999999992 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20011 7.0 7:05e9e5be:::benchmark_data_smithi053_107660_object20010:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.021 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.617516+0000", 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.34234889800001, 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081888140000000009, 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20011 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617516+0000", 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617516+0000", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999967 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617513+0000", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0525999999999997e-05 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617563+0000", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5023000000000001e-05 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617588+0000", 2024-02-22T19:36:16.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2320000000000001e-05 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617601+0000", 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5715000000000001e-05 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617697+0000", 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3256000000000004e-05 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617780+0000", 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000175458 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.617955+0000", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00093616999999999995 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.618891+0000", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056099899999999996 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.624501+0000", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001148114 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.625649+0000", 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2036000000000003e-05 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.625682+0000", 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3323e-05 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.625705+0000", 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081888140000000009 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20051 7.0 7:02db49fd:::benchmark_data_smithi053_107660_object20050:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.639182+0000", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.320682852, 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0088055890000000008, 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20051 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639182+0000", 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639182+0000", 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639181+0000", 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5690000000000003e-05 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639247+0000", 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2836e-05 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639260+0000", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9449999999999999e-06 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.639268+0000", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001110703 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.640379+0000", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000132364 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.640512+0000", 2024-02-22T19:36:16.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123743 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.640635+0000", 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00068257599999999995 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.641318+0000", 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001252967 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.642571+0000", 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053208530000000004 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.647892+0000", 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4314000000000002e-05 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.647946+0000", 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1668999999999997e-05 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.647988+0000", 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0088055890000000008 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20366 7.0 7:0c62a8f1:::benchmark_data_smithi053_107660_object20365:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.780949+0000", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.17891573499998, 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073489389999999996, 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20366 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.780949+0000", 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.780949+0000", 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.780947+0000", 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3261999999999999e-05 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781000+0000", 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5065999999999999e-05 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781015+0000", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.233e-06 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781022+0000", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8088999999999999e-05 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781080+0000", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016692500000000001 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781247+0000", 2024-02-22T19:36:16.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9491e-05 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781347+0000", 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001557938 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782905+0000", 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000526221 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.783431+0000", 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004772933 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788204+0000", 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1786000000000002e-05 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788256+0000", 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2366000000000001e-05 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788298+0000", 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073489389999999996 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20367 7.0 7:090e9347:::benchmark_data_smithi053_107660_object20366:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.781154+0000", 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.17871060900001, 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072720479999999997, 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20367 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781154+0000", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781154+0000", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781153+0000", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9843999999999998e-05 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781213+0000", 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2939999999999994e-06 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781223+0000", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1700000000000002e-06 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.781229+0000", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00079298900000000002 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782022+0000", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000112934 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782135+0000", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016562200000000001 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.782300+0000", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001390929 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.783691+0000", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026650099999999999 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.783958+0000", 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043807660000000003 2024-02-22T19:36:16.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788338+0000", 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8968999999999998e-05 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788387+0000", 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8951e-05 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.788426+0000", 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072720479999999997 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20451 7.0 7:00a00ce1:::benchmark_data_smithi053_107660_object20450:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.817018+0000", 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.14284692699999, 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087138210000000001, 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20451 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817018+0000", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817018+0000", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817016+0000", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3357999999999999e-05 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817079+0000", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5642e-05 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817095+0000", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7360000000000001e-06 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817103+0000", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2648999999999997e-05 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817155+0000", 2024-02-22T19:36:16.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013609900000000001 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817292+0000", 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015985500000000001 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.817451+0000", 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00083612599999999997 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.818287+0000", 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020579639999999998 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.820345+0000", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053291049999999998 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.825675+0000", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3754000000000002e-05 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.825708+0000", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.353e-05 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.825732+0000", 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087138210000000001 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20519 7.0 7:0f3a20e9:::benchmark_data_smithi053_107660_object20518:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.845539+0000", 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.11432569200002, 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084815829999999991, 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20519 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845539+0000", 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845539+0000", 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845538+0000", 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3948999999999994e-05 2024-02-22T19:36:16.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845602+0000", 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3067999999999999e-05 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845615+0000", 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0070000000000003e-06 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.845621+0000", 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014366870000000001 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847058+0000", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102033 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847160+0000", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014512800000000001 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847305+0000", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064053899999999997 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.847945+0000", 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017828989999999999 2024-02-22T19:36:16.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.849728+0000", 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042388959999999998 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.853967+0000", 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1578999999999997e-05 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.853999+0000", 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2167999999999998e-05 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.854021+0000", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084815829999999991 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20643 7.0 7:0874e533:::benchmark_data_smithi053_107660_object20642:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.897311+0000", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.06255390500002, 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0077622109999999998, 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20643 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897311+0000", 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897311+0000", 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897310+0000", 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9523000000000003e-05 2024-02-22T19:36:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897399+0000", 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1929e-05 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897411+0000", 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897417+0000", 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3964999999999998e-05 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897451+0000", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012626900000000001 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897577+0000", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000129837 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.897707+0000", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00096261199999999995 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.898670+0000", 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016538670000000001 2024-02-22T19:36:16.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.900324+0000", 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046556480000000001 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.904979+0000", 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5241000000000002e-05 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.905034+0000", 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8747e-05 2024-02-22T19:36:16.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.905073+0000", 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0077622109999999998 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20650 7.0 7:055267f9:::benchmark_data_smithi053_107660_object20649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.05750562700001, 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073057419999999996, 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20650 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902359+0000", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5485000000000002e-05 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902414+0000", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.144e-06 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902420+0000", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.558e-06 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.902426+0000", 2024-02-22T19:36:16.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00071791599999999998 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.903144+0000", 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013517 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.903279+0000", 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017210399999999999 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.903451+0000", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000651338 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.904102+0000", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012351790000000001 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.905337+0000", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042757710000000003 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.909613+0000", 2024-02-22T19:36:16.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1090000000000002e-05 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.909644+0000", 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0721999999999999e-05 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.909665+0000", 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073057419999999996 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:20673 7.0 7:0c980cb3:::benchmark_data_smithi053_107660_object20672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:48.915300+0000", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 267.04456486999999, 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0068715149999999999, 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 20673 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915300+0000", 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915300+0000", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915299+0000", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8996000000000003e-05 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915348+0000", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2449000000000001e-05 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915360+0000", 2024-02-22T19:36:16.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3000000000000001e-06 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915366+0000", 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0315000000000002e-05 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915416+0000", 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6032999999999994e-05 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915502+0000", 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100528 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.915602+0000", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00079250499999999999 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.916395+0000", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033575110000000001 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.919752+0000", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023228369999999999 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:16.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.922075+0000", 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4342999999999998e-05 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.922130+0000", 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1965999999999998e-05 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:48.922172+0000", 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0068715149999999999 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:16.067 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:16.068 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 43 16 66064 66048 95.9229 64.75 0.0133686 0.0104202 2024-02-22T19:36:16.068 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:13.834157+0000 osd.1 (osd.1) 101 : cluster [DBG] 5.19 deep-scrub starts 2024-02-22T19:36:16.068 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:13.836973+0000 osd.1 (osd.1) 102 : cluster [DBG] 5.19 deep-scrub ok 2024-02-22T19:36:16.068 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:14.021538+0000 osd.2 (osd.2) 84 : cluster [DBG] 5.b deep-scrub starts 2024-02-22T19:36:16.068 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:14.029381+0000 osd.2 (osd.2) 85 : cluster [DBG] 5.b deep-scrub ok 2024-02-22T19:36:16.068 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:14.222375+0000 osd.6 (osd.6) 75 : cluster [DBG] 1.6 deep-scrub starts 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:14.230361+0000 osd.6 (osd.6) 76 : cluster [DBG] 1.6 deep-scrub ok 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:14.620708+0000 mgr.y (mgr.14535) 281 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:15.114572+0000 mon.a (mon.0) 989 : audit [INF] from='client.? 172.21.15.53:0/3231520892' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.f"}]': finished 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.114629+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e168: 8 total, 8 up, 7 in 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:13.834157+0000 osd.1 (osd.1) 101 : cluster [DBG] 5.19 deep-scrub starts 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:13.836973+0000 osd.1 (osd.1) 102 : cluster [DBG] 5.19 deep-scrub ok 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:14.021538+0000 osd.2 (osd.2) 84 : cluster [DBG] 5.b deep-scrub starts 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:14.029381+0000 osd.2 (osd.2) 85 : cluster [DBG] 5.b deep-scrub ok 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:14.222375+0000 osd.6 (osd.6) 75 : cluster [DBG] 1.6 deep-scrub starts 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:14.230361+0000 osd.6 (osd.6) 76 : cluster [DBG] 1.6 deep-scrub ok 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:14.620708+0000 mgr.y (mgr.14535) 281 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:15.114572+0000 mon.a (mon.0) 989 : audit [INF] from='client.? 172.21.15.53:0/3231520892' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.f"}]': finished 2024-02-22T19:36:16.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.114629+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e168: 8 total, 8 up, 7 in 2024-02-22T19:36:16.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:14.792033+0000 osd.4 (osd.4) 115 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:14.806064+0000 osd.4 (osd.4) 116 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.042815+0000 osd.2 (osd.2) 86 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.051944+0000 osd.2 (osd.2) 87 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.388943+0000 mgr.y (mgr.14535) 282 : cluster [DBG] pgmap v312: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.684681+0000 osd.0 (osd.0) 123 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.699315+0000 osd.0 (osd.0) 124 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:16.132224+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e169: 8 total, 8 up, 7 in 2024-02-22T19:36:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:16.433982+0000 mon.a (mon.0) 992 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:36:16.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:36:16.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:14.792033+0000 osd.4 (osd.4) 115 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:36:16.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:14.806064+0000 osd.4 (osd.4) 116 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:36:16.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.042815+0000 osd.2 (osd.2) 86 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:36:16.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.051944+0000 osd.2 (osd.2) 87 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:36:16.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.388943+0000 mgr.y (mgr.14535) 282 : cluster [DBG] pgmap v312: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:36:16.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.684681+0000 osd.0 (osd.0) 123 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:36:16.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.699315+0000 osd.0 (osd.0) 124 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:36:16.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:16.132224+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e169: 8 total, 8 up, 7 in 2024-02-22T19:36:16.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:16.433982+0000 mon.a (mon.0) 992 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:36:16.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:14.792033+0000 osd.4 (osd.4) 115 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:36:16.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:14.806064+0000 osd.4 (osd.4) 116 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:36:16.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.042815+0000 osd.2 (osd.2) 86 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:36:16.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.051944+0000 osd.2 (osd.2) 87 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:36:16.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.388943+0000 mgr.y (mgr.14535) 282 : cluster [DBG] pgmap v312: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:36:16.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.684681+0000 osd.0 (osd.0) 123 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:36:16.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.699315+0000 osd.0 (osd.0) 124 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:36:16.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:16.132224+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e169: 8 total, 8 up, 7 in 2024-02-22T19:36:16.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:16.433982+0000 mon.a (mon.0) 992 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:36:16.982 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 44 16 67008 66992 95.0803 59 0.0128372 0.010512 2024-02-22T19:36:17.059 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:17.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.901552+0000 osd.1 (osd.1) 103 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:36:17.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:15.908707+0000 osd.1 (osd.1) 104 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:36:17.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:16.542782+0000 osd.5 (osd.5) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:17.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:16.542834+0000 osd.5 (osd.5) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:17.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:17.109366+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e170: 8 total, 8 up, 7 in 2024-02-22T19:36:17.983 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 45 16 68075 68059 94.4497 66.6875 0.026491 0.0105828 2024-02-22T19:36:18.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.901552+0000 osd.1 (osd.1) 103 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:36:18.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:15.908707+0000 osd.1 (osd.1) 104 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:36:18.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:16.542782+0000 osd.5 (osd.5) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:18.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:16.542834+0000 osd.5 (osd.5) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:18.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:17.109366+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e170: 8 total, 8 up, 7 in 2024-02-22T19:36:18.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.901552+0000 osd.1 (osd.1) 103 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:36:18.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:15.908707+0000 osd.1 (osd.1) 104 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:36:18.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:16.542782+0000 osd.5 (osd.5) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:18.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:16.542834+0000 osd.5 (osd.5) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:18.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:17.109366+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e170: 8 total, 8 up, 7 in 2024-02-22T19:36:18.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:17.138520+0000 osd.6 (osd.6) 77 : cluster [DBG] 6.a scrub starts 2024-02-22T19:36:18.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:17.151288+0000 osd.6 (osd.6) 78 : cluster [DBG] 6.a scrub ok 2024-02-22T19:36:18.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:17.391221+0000 mgr.y (mgr.14535) 283 : cluster [DBG] pgmap v315: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-02-22T19:36:18.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:17.655346+0000 osd.0 (osd.0) 125 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:36:18.700 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:17.690291+0000 osd.0 (osd.0) 126 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:36:18.983 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 46 16 69133 69117 93.8342 66.125 0.0169548 0.0106517 2024-02-22T19:36:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:17.138520+0000 osd.6 (osd.6) 77 : cluster [DBG] 6.a scrub starts 2024-02-22T19:36:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:17.151288+0000 osd.6 (osd.6) 78 : cluster [DBG] 6.a scrub ok 2024-02-22T19:36:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:17.391221+0000 mgr.y (mgr.14535) 283 : cluster [DBG] pgmap v315: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-02-22T19:36:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:17.655346+0000 osd.0 (osd.0) 125 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:36:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:17.690291+0000 osd.0 (osd.0) 126 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:36:19.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:17.138520+0000 osd.6 (osd.6) 77 : cluster [DBG] 6.a scrub starts 2024-02-22T19:36:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:17.151288+0000 osd.6 (osd.6) 78 : cluster [DBG] 6.a scrub ok 2024-02-22T19:36:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:17.391221+0000 mgr.y (mgr.14535) 283 : cluster [DBG] pgmap v315: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-02-22T19:36:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:17.655346+0000 osd.0 (osd.0) 125 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:36:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:17.690291+0000 osd.0 (osd.0) 126 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:69187 8.9 8:9659213c:::benchmark_data_smithi053_130774_object69186:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:19.032916+0000", 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042373050000000002, 2024-02-22T19:36:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042685609999999997, 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 69187 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.032916+0000", 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.032916+0000", 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.032914+0000", 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6932999999999999e-05 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.032971+0000", 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0801999999999999e-05 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.032982+0000", 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7779999999999998e-06 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.032987+0000", 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7549999999999997e-05 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033035+0000", 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012427799999999999 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033159+0000", 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016704799999999999 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033326+0000", 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00040999999999999999 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:69189 8.9 8:9da3de55:::benchmark_data_smithi053_130774_object69188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2024-02-22T19:36:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:19.033522+0000", 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0036314210000000001, 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037527950000000002, 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 69189 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033522+0000", 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033522+0000", 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033521+0000", 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6283000000000001e-05 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033597+0000", 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4229999999999994e-06 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033606+0000", 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0889999999999996e-06 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.033613+0000", 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001946063 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.035559+0000", 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120649 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.035679+0000", 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015467000000000001 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:19.035834+0000", 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002312143 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 2 2024-02-22T19:36:19.045 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:18.191134+0000 osd.0 (osd.0) 127 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:18.191191+0000 osd.0 (osd.0) 128 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:18.615960+0000 osd.0 (osd.0) 129 : cluster [DBG] 4.b scrub starts 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:18.621739+0000 osd.0 (osd.0) 130 : cluster [DBG] 4.b scrub ok 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.121177+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e171: 8 total, 8 up, 7 in 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.508123+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.1a scrub starts 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.511042+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.1a scrub ok 2024-02-22T19:36:19.612 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:18.191134+0000 osd.0 (osd.0) 127 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:18.191191+0000 osd.0 (osd.0) 128 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:18.615960+0000 osd.0 (osd.0) 129 : cluster [DBG] 4.b scrub starts 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:18.621739+0000 osd.0 (osd.0) 130 : cluster [DBG] 4.b scrub ok 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.121177+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e171: 8 total, 8 up, 7 in 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.508123+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.1a scrub starts 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.511042+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.1a scrub ok 2024-02-22T19:36:19.613 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:36:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:19] "GET /metrics HTTP/1.1" 200 175796 "" "Prometheus/2.33.4" 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:18.191134+0000 osd.0 (osd.0) 127 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:18.191191+0000 osd.0 (osd.0) 128 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:18.615960+0000 osd.0 (osd.0) 129 : cluster [DBG] 4.b scrub starts 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:18.621739+0000 osd.0 (osd.0) 130 : cluster [DBG] 4.b scrub ok 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.121177+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e171: 8 total, 8 up, 7 in 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.508123+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.1a scrub starts 2024-02-22T19:36:19.721 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.511042+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.1a scrub ok 2024-02-22T19:36:19.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:36:19.984 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 47 16 70176 70160 93.223 65.1875 0.0127897 0.0107211 2024-02-22T19:36:20.112 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:20.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:18.887670+0000 osd.1 (osd.1) 105 : cluster [DBG] 5.1d scrub starts 2024-02-22T19:36:20.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:18.889419+0000 osd.1 (osd.1) 106 : cluster [DBG] 5.1d scrub ok 2024-02-22T19:36:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.029979+0000 osd.2 (osd.2) 88 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:36:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.039453+0000 osd.2 (osd.2) 89 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:36:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.218315+0000 osd.3 (osd.3) 121 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:36:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.229807+0000 osd.3 (osd.3) 122 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:36:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.394011+0000 mgr.y (mgr.14535) 284 : cluster [DBG] pgmap v317: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 25 B/s, 1 objects/s recovering 2024-02-22T19:36:20.987 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 48 16 71243 71227 92.6641 66.6875 0.0206248 0.0107864 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:18.887670+0000 osd.1 (osd.1) 105 : cluster [DBG] 5.1d scrub starts 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:18.889419+0000 osd.1 (osd.1) 106 : cluster [DBG] 5.1d scrub ok 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.029979+0000 osd.2 (osd.2) 88 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.039453+0000 osd.2 (osd.2) 89 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.218315+0000 osd.3 (osd.3) 121 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.229807+0000 osd.3 (osd.3) 122 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:36:21.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.394011+0000 mgr.y (mgr.14535) 284 : cluster [DBG] pgmap v317: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 25 B/s, 1 objects/s recovering 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:18.887670+0000 osd.1 (osd.1) 105 : cluster [DBG] 5.1d scrub starts 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:18.889419+0000 osd.1 (osd.1) 106 : cluster [DBG] 5.1d scrub ok 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.029979+0000 osd.2 (osd.2) 88 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.039453+0000 osd.2 (osd.2) 89 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.218315+0000 osd.3 (osd.3) 121 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.229807+0000 osd.3 (osd.3) 122 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:36:21.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.394011+0000 mgr.y (mgr.14535) 284 : cluster [DBG] pgmap v317: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 25 B/s, 1 objects/s recovering 2024-02-22T19:36:21.921 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:21.921 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:36:21.921 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:36:21.921 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:36:21.921 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:21.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.853094+0000 osd.1 (osd.1) 107 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:36:21.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:19.861914+0000 osd.1 (osd.1) 108 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:36:21.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:20.062599+0000 osd.2 (osd.2) 90 : cluster [DBG] 1.a scrub starts 2024-02-22T19:36:21.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:20.071496+0000 osd.2 (osd.2) 91 : cluster [DBG] 1.a scrub ok 2024-02-22T19:36:21.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:20.550076+0000 osd.7 (osd.7) 129 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:36:21.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:20.553547+0000 osd.7 (osd.7) 130 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:36:21.988 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 49 16 72239 72223 92.0416 62.25 0.0244259 0.0108597 2024-02-22T19:36:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.853094+0000 osd.1 (osd.1) 107 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:36:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:19.861914+0000 osd.1 (osd.1) 108 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:36:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:20.062599+0000 osd.2 (osd.2) 90 : cluster [DBG] 1.a scrub starts 2024-02-22T19:36:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:20.071496+0000 osd.2 (osd.2) 91 : cluster [DBG] 1.a scrub ok 2024-02-22T19:36:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:20.550076+0000 osd.7 (osd.7) 129 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:36:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:20.553547+0000 osd.7 (osd.7) 130 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:36:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.853094+0000 osd.1 (osd.1) 107 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:36:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:19.861914+0000 osd.1 (osd.1) 108 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:36:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:20.062599+0000 osd.2 (osd.2) 90 : cluster [DBG] 1.a scrub starts 2024-02-22T19:36:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:20.071496+0000 osd.2 (osd.2) 91 : cluster [DBG] 1.a scrub ok 2024-02-22T19:36:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:20.550076+0000 osd.7 (osd.7) 129 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:36:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:20.553547+0000 osd.7 (osd.7) 130 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:36:22.294 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:22.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:22.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:22.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:22.499 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:22.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:22.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:22.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:36:22.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:20.733685+0000 osd.4 (osd.4) 117 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:36:22.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:20.739744+0000 osd.4 (osd.4) 118 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:36:22.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:21.394400+0000 mgr.y (mgr.14535) 285 : cluster [DBG] pgmap v318: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 53 MiB/s wr, 853 op/s; 18 B/s, 1 objects/s recovering 2024-02-22T19:36:22.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:22.113194+0000 mon.a (mon.0) 995 : cluster [DBG] osdmap e172: 8 total, 8 up, 7 in 2024-02-22T19:36:22.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:22.553811+0000 mon.a (mon.0) 996 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:36:22.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:22.558173+0000 mon.a (mon.0) 997 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:36:22.641 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:22.564682+0000 mon.a (mon.0) 998 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:22.641 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:20.733685+0000 osd.4 (osd.4) 117 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:36:22.641 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:20.739744+0000 osd.4 (osd.4) 118 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:36:22.641 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:21.394400+0000 mgr.y (mgr.14535) 285 : cluster [DBG] pgmap v318: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 53 MiB/s wr, 853 op/s; 18 B/s, 1 objects/s recovering 2024-02-22T19:36:22.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:22.748 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:20.733685+0000 osd.4 (osd.4) 117 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:20.739744+0000 osd.4 (osd.4) 118 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:21.394400+0000 mgr.y (mgr.14535) 285 : cluster [DBG] pgmap v318: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 53 MiB/s wr, 853 op/s; 18 B/s, 1 objects/s recovering 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:22.113194+0000 mon.a (mon.0) 995 : cluster [DBG] osdmap e172: 8 total, 8 up, 7 in 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:22.553811+0000 mon.a (mon.0) 996 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:22.558173+0000 mon.a (mon.0) 997 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:36:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:22.564682+0000 mon.a (mon.0) 998 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:22.988 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 50 15 73189 73174 91.3899 59.4375 0.0195905 0.0109386 2024-02-22T19:36:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:22.113194+0000 mon.a (mon.0) 995 : cluster [DBG] osdmap e172: 8 total, 8 up, 7 in 2024-02-22T19:36:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:22.553811+0000 mon.a (mon.0) 996 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:36:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:22.558173+0000 mon.a (mon.0) 997 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:36:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:22.564682+0000 mon.a (mon.0) 998 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:23.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:21.719927+0000 osd.4 (osd.4) 119 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:36:23.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:21.735506+0000 osd.4 (osd.4) 120 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:36:23.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:22.675814+0000 osd.0 (osd.0) 131 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:36:23.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:22.679481+0000 osd.0 (osd.0) 132 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:36:23.989 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 51 15 74091 74076 90.7037 56.375 0.0126972 0.0110203 2024-02-22T19:36:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:21.719927+0000 osd.4 (osd.4) 119 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:36:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:21.735506+0000 osd.4 (osd.4) 120 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:36:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:22.675814+0000 osd.0 (osd.0) 131 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:36:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:22.679481+0000 osd.0 (osd.0) 132 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:36:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:23.351088+0000 mon.a (mon.0) 999 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:23.351088+0000 mon.a (mon.0) 999 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:21.719927+0000 osd.4 (osd.4) 119 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:36:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:21.735506+0000 osd.4 (osd.4) 120 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:36:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:22.675814+0000 osd.0 (osd.0) 131 : cluster [DBG] 5.1e scrub starts 2024-02-22T19:36:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:22.679481+0000 osd.0 (osd.0) 132 : cluster [DBG] 5.1e scrub ok 2024-02-22T19:36:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:23.351088+0000 mon.a (mon.0) 999 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:24.504 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:24.510 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:24.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:24.575 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:24.611 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:24.645 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:24.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.012436+0000 osd.2 (osd.2) 92 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:36:24.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.022670+0000 osd.2 (osd.2) 93 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:36:24.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.394804+0000 mgr.y (mgr.14535) 286 : cluster [DBG] pgmap v320: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 814 B/s rd, 51 MiB/s wr, 815 op/s; 17 B/s, 1 objects/s recovering 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.567925+0000 osd.7 (osd.7) 131 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.598749+0000 osd.7 (osd.7) 132 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.657145+0000 osd.0 (osd.0) 133 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.658951+0000 osd.0 (osd.0) 134 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:24.367120+0000 mon.a (mon.0) 1000 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.012436+0000 osd.2 (osd.2) 92 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.022670+0000 osd.2 (osd.2) 93 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.394804+0000 mgr.y (mgr.14535) 286 : cluster [DBG] pgmap v320: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 814 B/s rd, 51 MiB/s wr, 815 op/s; 17 B/s, 1 objects/s recovering 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.567925+0000 osd.7 (osd.7) 131 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.598749+0000 osd.7 (osd.7) 132 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.657145+0000 osd.0 (osd.0) 133 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.658951+0000 osd.0 (osd.0) 134 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:36:24.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:24.367120+0000 mon.a (mon.0) 1000 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-02-22T19:36:24.989 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 52 16 75046 75030 90.1064 59.625 0.0103432 0.0110933 2024-02-22T19:36:25.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:36:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:36:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.012436+0000 osd.2 (osd.2) 92 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:36:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.022670+0000 osd.2 (osd.2) 93 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:36:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.394804+0000 mgr.y (mgr.14535) 286 : cluster [DBG] pgmap v320: 158 pgs: 158 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 814 B/s rd, 51 MiB/s wr, 815 op/s; 17 B/s, 1 objects/s recovering 2024-02-22T19:36:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.567925+0000 osd.7 (osd.7) 131 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:36:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.598749+0000 osd.7 (osd.7) 132 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:36:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.657145+0000 osd.0 (osd.0) 133 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:36:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.658951+0000 osd.0 (osd.0) 134 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:36:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:24.367120+0000 mon.a (mon.0) 1000 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-02-22T19:36:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:25.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:25.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:36:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:36:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:36:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:36:25.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:36:25.437 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:36:25.482 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:36:25.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:25.523 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:36:25.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.527 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.527 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:36:25.540 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:25.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:25.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:25.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:25.697 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:25.777 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:25.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:25.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:25.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:25.951 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.896894+0000 osd.1 (osd.1) 109 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:36:25.951 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:23.901504+0000 osd.1 (osd.1) 110 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:36:25.951 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:24.003562+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.1c deep-scrub starts 2024-02-22T19:36:25.951 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:24.021101+0000 osd.2 (osd.2) 95 : cluster [DBG] 4.1c deep-scrub ok 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:24.592562+0000 osd.7 (osd.7) 133 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:24.596519+0000 osd.7 (osd.7) 134 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:24.631166+0000 mgr.y (mgr.14535) 287 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:24.836786+0000 osd.4 (osd.4) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:24.836858+0000 osd.4 (osd.4) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:25.236030+0000 mon.b (mon.1) 53 : audit [DBG] from='client.? 172.21.15.53:0/3241964670' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:25.360105+0000 mon.a (mon.0) 1001 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-02-22T19:36:25.952 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:25.360148+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e173: 8 total, 8 up, 7 in 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.896894+0000 osd.1 (osd.1) 109 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:23.901504+0000 osd.1 (osd.1) 110 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:24.003562+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.1c deep-scrub starts 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:24.021101+0000 osd.2 (osd.2) 95 : cluster [DBG] 4.1c deep-scrub ok 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:24.592562+0000 osd.7 (osd.7) 133 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:24.596519+0000 osd.7 (osd.7) 134 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:36:25.955 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:24.631166+0000 mgr.y (mgr.14535) 287 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:25.956 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:24.836786+0000 osd.4 (osd.4) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:25.956 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:24.836858+0000 osd.4 (osd.4) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:25.956 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:25.236030+0000 mon.b (mon.1) 53 : audit [DBG] from='client.? 172.21.15.53:0/3241964670' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:25.956 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:25.360105+0000 mon.a (mon.0) 1001 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-02-22T19:36:25.956 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:25.360148+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e173: 8 total, 8 up, 7 in 2024-02-22T19:36:25.960 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:25.992 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 53 16 76003 75987 89.5294 59.8125 0.0158677 0.0111645 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.896894+0000 osd.1 (osd.1) 109 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:23.901504+0000 osd.1 (osd.1) 110 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:24.003562+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.1c deep-scrub starts 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:24.021101+0000 osd.2 (osd.2) 95 : cluster [DBG] 4.1c deep-scrub ok 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:24.592562+0000 osd.7 (osd.7) 133 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:24.596519+0000 osd.7 (osd.7) 134 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:36:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:24.631166+0000 mgr.y (mgr.14535) 287 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:26.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:24.836786+0000 osd.4 (osd.4) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:26.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:24.836858+0000 osd.4 (osd.4) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:26.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:25.236030+0000 mon.b (mon.1) 53 : audit [DBG] from='client.? 172.21.15.53:0/3241964670' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:26.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:25.360105+0000 mon.a (mon.0) 1001 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-02-22T19:36:26.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:25.360148+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e173: 8 total, 8 up, 7 in 2024-02-22T19:36:26.192 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:26.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:26.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:26.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:26.363 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:26.420 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:26.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:26.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:26.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:26.595 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:26.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:25.285984+0000 osd.3 (osd.3) 123 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:36:26.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:25.296477+0000 osd.3 (osd.3) 124 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:36:26.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:25.395810+0000 mgr.y (mgr.14535) 288 : cluster [DBG] pgmap v322: 158 pgs: 158 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.61k op/s; 17 B/s, 1 objects/s recovering 2024-02-22T19:36:26.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:25.566858+0000 osd.0 (osd.0) 135 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:36:26.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:25.576846+0000 osd.0 (osd.0) 136 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:36:26.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:26.363928+0000 mon.a (mon.0) 1003 : cluster [DBG] osdmap e174: 8 total, 8 up, 7 in 2024-02-22T19:36:26.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:25.285984+0000 osd.3 (osd.3) 123 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:36:26.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:25.296477+0000 osd.3 (osd.3) 124 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:36:26.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:25.395810+0000 mgr.y (mgr.14535) 288 : cluster [DBG] pgmap v322: 158 pgs: 158 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.61k op/s; 17 B/s, 1 objects/s recovering 2024-02-22T19:36:26.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:25.566858+0000 osd.0 (osd.0) 135 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:36:26.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:25.576846+0000 osd.0 (osd.0) 136 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:36:26.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:26.363928+0000 mon.a (mon.0) 1003 : cluster [DBG] osdmap e174: 8 total, 8 up, 7 in 2024-02-22T19:36:26.922 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:25.285984+0000 osd.3 (osd.3) 123 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:36:26.923 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:25.296477+0000 osd.3 (osd.3) 124 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:36:26.923 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:25.395810+0000 mgr.y (mgr.14535) 288 : cluster [DBG] pgmap v322: 158 pgs: 158 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.61k op/s; 17 B/s, 1 objects/s recovering 2024-02-22T19:36:26.923 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:25.566858+0000 osd.0 (osd.0) 135 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:36:26.923 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:25.576846+0000 osd.0 (osd.0) 136 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:36:26.923 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:26.363928+0000 mon.a (mon.0) 1003 : cluster [DBG] osdmap e174: 8 total, 8 up, 7 in 2024-02-22T19:36:26.996 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 54 16 77031 77015 89.0551 64.25 0.0153838 0.0112248 2024-02-22T19:36:27.250 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:36:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:36:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:27.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:27.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:27.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:27.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:26.277276+0000 osd.3 (osd.3) 125 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:36:27.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:26.284455+0000 osd.3 (osd.3) 126 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:36:27.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:26.726321+0000 osd.4 (osd.4) 123 : cluster [DBG] 6.17 deep-scrub starts 2024-02-22T19:36:27.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:26.761219+0000 osd.4 (osd.4) 124 : cluster [DBG] 6.17 deep-scrub ok 2024-02-22T19:36:27.672 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:27.112668+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e175: 8 total, 8 up, 7 in 2024-02-22T19:36:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:26.277276+0000 osd.3 (osd.3) 125 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:36:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:26.284455+0000 osd.3 (osd.3) 126 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:36:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:26.726321+0000 osd.4 (osd.4) 123 : cluster [DBG] 6.17 deep-scrub starts 2024-02-22T19:36:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:26.761219+0000 osd.4 (osd.4) 124 : cluster [DBG] 6.17 deep-scrub ok 2024-02-22T19:36:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:27.112668+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e175: 8 total, 8 up, 7 in 2024-02-22T19:36:27.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:36:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:27.991 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:27.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:27.996 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 55 16 78041 78025 88.5838 63.125 0.024167 0.0112836 2024-02-22T19:36:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:26.277276+0000 osd.3 (osd.3) 125 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:36:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:26.284455+0000 osd.3 (osd.3) 126 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:36:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:26.726321+0000 osd.4 (osd.4) 123 : cluster [DBG] 6.17 deep-scrub starts 2024-02-22T19:36:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:26.761219+0000 osd.4 (osd.4) 124 : cluster [DBG] 6.17 deep-scrub ok 2024-02-22T19:36:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:27.112668+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e175: 8 total, 8 up, 7 in 2024-02-22T19:36:28.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:28.162 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:28.215 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:28.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:28.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:28.318 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:28.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:28.622 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:28.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:28.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:28.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:28.786 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:28.822 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:28.826 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:28.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:28.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:36:28.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:27.396242+0000 mgr.y (mgr.14535) 289 : cluster [DBG] pgmap v325: 158 pgs: 158 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 969 B/s rd, 59 MiB/s wr, 938 op/s 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:27.591784+0000 osd.7 (osd.7) 135 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:27.599287+0000 osd.7 (osd.7) 136 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:27.681710+0000 osd.4 (osd.4) 125 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:27.685413+0000 osd.4 (osd.4) 126 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:28.370563+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.53:0/3691133895' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:36:28.963 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:28.371080+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:36:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:27.396242+0000 mgr.y (mgr.14535) 289 : cluster [DBG] pgmap v325: 158 pgs: 158 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 969 B/s rd, 59 MiB/s wr, 938 op/s 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:27.591784+0000 osd.7 (osd.7) 135 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:27.599287+0000 osd.7 (osd.7) 136 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:27.681710+0000 osd.4 (osd.4) 125 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:27.685413+0000 osd.4 (osd.4) 126 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:28.370563+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.53:0/3691133895' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:28.371080+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:27.396242+0000 mgr.y (mgr.14535) 289 : cluster [DBG] pgmap v325: 158 pgs: 158 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 969 B/s rd, 59 MiB/s wr, 938 op/s 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:27.591784+0000 osd.7 (osd.7) 135 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:36:28.975 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:27.599287+0000 osd.7 (osd.7) 136 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:36:28.976 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:27.681710+0000 osd.4 (osd.4) 125 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:36:28.976 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:27.685413+0000 osd.4 (osd.4) 126 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:36:28.976 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:28.370563+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.53:0/3691133895' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:36:28.976 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:28.371080+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:36:28.997 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 56 16 79030 79014 88.1059 61.8125 0.0186608 0.0113453 2024-02-22T19:36:29.184 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:36:29.280 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:29.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:29.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:29.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:29.427 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:29.435 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:36:29.435 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:36:29.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:36:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:29.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:29.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:29.624 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:29] "GET /metrics HTTP/1.1" 200 175793 "" "Prometheus/2.33.4" 2024-02-22T19:36:29.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:28.172612+0000 osd.6 (osd.6) 79 : cluster [DBG] 3.a scrub starts 2024-02-22T19:36:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:28.180293+0000 osd.6 (osd.6) 80 : cluster [DBG] 3.a scrub ok 2024-02-22T19:36:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:28.285397+0000 osd.3 (osd.3) 127 : cluster [DBG] 6.1f deep-scrub starts 2024-02-22T19:36:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:28.335742+0000 osd.3 (osd.3) 128 : cluster [DBG] 6.1f deep-scrub ok 2024-02-22T19:36:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:29.129402+0000 mon.a (mon.0) 1006 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:29.140784+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:36:29.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:29.140856+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e176: 8 total, 8 up, 7 in 2024-02-22T19:36:30.001 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 57 16 80008 79992 87.6299 61.125 0.00744763 0.0114069 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:28.172612+0000 osd.6 (osd.6) 79 : cluster [DBG] 3.a scrub starts 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:28.180293+0000 osd.6 (osd.6) 80 : cluster [DBG] 3.a scrub ok 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:28.285397+0000 osd.3 (osd.3) 127 : cluster [DBG] 6.1f deep-scrub starts 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:28.335742+0000 osd.3 (osd.3) 128 : cluster [DBG] 6.1f deep-scrub ok 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:29.129402+0000 mon.a (mon.0) 1006 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:29.140784+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:36:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:29.140856+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e176: 8 total, 8 up, 7 in 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:28.172612+0000 osd.6 (osd.6) 79 : cluster [DBG] 3.a scrub starts 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:28.180293+0000 osd.6 (osd.6) 80 : cluster [DBG] 3.a scrub ok 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:28.285397+0000 osd.3 (osd.3) 127 : cluster [DBG] 6.1f deep-scrub starts 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:28.335742+0000 osd.3 (osd.3) 128 : cluster [DBG] 6.1f deep-scrub ok 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:29.129402+0000 mon.a (mon.0) 1006 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:29.140784+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:36:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:29.140856+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e176: 8 total, 8 up, 7 in 2024-02-22T19:36:30.102 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:30.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:30.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:30.185 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:36:30.185 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:36:30.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:30.248 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:30.270 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:30.280 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:30.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:30.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:30.562 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:30.852 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:29.397017+0000 mgr.y (mgr.14535) 290 : cluster [DBG] pgmap v327: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 918 op/s 2024-02-22T19:36:30.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:30.863 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:30.867 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:29.397017+0000 mgr.y (mgr.14535) 290 : cluster [DBG] pgmap v327: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 918 op/s 2024-02-22T19:36:30.867 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:29.397017+0000 mgr.y (mgr.14535) 290 : cluster [DBG] pgmap v327: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 918 op/s 2024-02-22T19:36:30.884 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:36:30.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:30.999 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 58 16 80999 80983 87.1872 61.9375 0.014023 0.0114654 2024-02-22T19:36:31.014 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:31.048 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:31.052 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:31.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:31.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:31.198 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:31.229 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:31.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:31.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:31.317 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:31.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:31.393 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:31.488 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:31.493 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:31.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:31.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:31.638 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:31.671 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:31.677 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:31.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:31.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:31.833 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:31.999 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 59 16 82034 82018 86.806 64.6875 0.00934064 0.0115147 2024-02-22T19:36:32.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:36:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:32.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:32.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:32.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:32.406 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:32.536 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:32.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:32.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:32.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:32.680 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:32.821 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:32.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:32.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:32.937 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:31.400543+0000 mgr.y (mgr.14535) 291 : cluster [DBG] pgmap v328: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 852 B/s rd, 51 MiB/s wr, 820 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:32.937 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:32.118152+0000 mon.a (mon.0) 1009 : cluster [DBG] osdmap e177: 8 total, 8 up, 7 in 2024-02-22T19:36:32.980 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:31.400543+0000 mgr.y (mgr.14535) 291 : cluster [DBG] pgmap v328: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 852 B/s rd, 51 MiB/s wr, 820 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:32.980 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:32.118152+0000 mon.a (mon.0) 1009 : cluster [DBG] osdmap e177: 8 total, 8 up, 7 in 2024-02-22T19:36:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:31.400543+0000 mgr.y (mgr.14535) 291 : cluster [DBG] pgmap v328: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 852 B/s rd, 51 MiB/s wr, 820 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:32.118152+0000 mon.a (mon.0) 1009 : cluster [DBG] osdmap e177: 8 total, 8 up, 7 in 2024-02-22T19:36:32.993 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:33.005 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:36:33.005646+0000 min lat: 0.00121627 max lat: 0.0881957 avg lat: 0.0115785 2024-02-22T19:36:33.005 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:36:33.005 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 60 16 82971 82955 86.327 58.5625 0.0183375 0.0115785 2024-02-22T19:36:33.251 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:36:33.251 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:36:33.254 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:36:33.273 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:36:33.801 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:33.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:33.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:33.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:33.950 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:33.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:33.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:34.005 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 61 15 83935 83920 85.9003 60.3125 0.021312 0.0116368 2024-02-22T19:36:34.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:36:34.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:34.178 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:34.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:34.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:34.345 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:34.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:34.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:34.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:34.497 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-02-22T19:36:32.119274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5197862564,"num_objects":79505,"num_object_clones":0,"num_object_copies":238515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79505,"num_whiteouts":0,"num_read":1155,"num_read_kb":907,"num_write":79741,"num_write_kb":5076073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70623,"ondisk_log_size":70623,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":475,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":17289368,"kb_used_data":15155552,"kb_used_omap":0,"kb_used_meta":2133760,"kb_avail":638926696,"statfs":{"total":671965249536,"available":654260936704,"internally_reserved":0,"allocated":15519285248,"data_stored":15515870561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2184970240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":322568192,"num_objects":4922,"num_object_clones":0,"num_object_copies":14766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4922,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4922,"num_write_kb":315008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.754487"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130447+0000","last_change":"2024-02-22T19:36:14.806137+0000","last_active":"2024-02-22T19:36:27.130447+0000","last_peered":"2024-02-22T19:36:27.130447+0000","last_clean":"2024-02-22T19:36:27.130447+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:36:27.130447+0000","last_undegraded":"2024-02-22T19:36:27.130447+0000","last_fullsized":"2024-02-22T19:36:27.130447+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.806069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.806069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":148,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121104+0000","last_change":"2024-02-22T19:36:13.837113+0000","last_active":"2024-02-22T19:36:27.121104+0000","last_peered":"2024-02-22T19:36:27.121104+0000","last_clean":"2024-02-22T19:36:27.121104+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:36:27.121104+0000","last_undegraded":"2024-02-22T19:36:27.121104+0000","last_fullsized":"2024-02-22T19:36:27.121104+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:13.836981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:13.836981+0000","last_clean_scrub_stamp":"2024-02-22T19:36:13.836981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":150,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.135736+0000","last_change":"2024-02-22T19:34:44.286165+0000","last_active":"2024-02-22T19:36:27.135736+0000","last_peered":"2024-02-22T19:36:27.135736+0000","last_clean":"2024-02-22T19:36:27.135736+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:36:27.135736+0000","last_undegraded":"2024-02-22T19:36:27.135736+0000","last_fullsized":"2024-02-22T19:36:27.135736+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.286059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.286059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130148+0000","last_change":"2024-02-22T19:36:10.678582+0000","last_active":"2024-02-22T19:36:27.130148+0000","last_peered":"2024-02-22T19:36:27.130148+0000","last_clean":"2024-02-22T19:36:27.130148+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:36:27.130148+0000","last_undegraded":"2024-02-22T19:36:27.130148+0000","last_fullsized":"2024-02-22T19:36:27.130148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:10.678491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:10.678491+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":129,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132266+0000","last_change":"2024-02-22T19:34:30.923797+0000","last_active":"2024-02-22T19:36:27.132266+0000","last_peered":"2024-02-22T19:36:27.132266+0000","last_clean":"2024-02-22T19:36:27.132266+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:36:27.132266+0000","last_undegraded":"2024-02-22T19:36:27.132266+0000","last_fullsized":"2024-02-22T19:36:27.132266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.923707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.923707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":134,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130233+0000","last_change":"2024-02-22T19:35:26.000779+0000","last_active":"2024-02-22T19:36:27.130233+0000","last_peered":"2024-02-22T19:36:27.130233+0000","last_clean":"2024-02-22T19:36:27.130233+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:36:27.130233+0000","last_undegraded":"2024-02-22T19:36:27.130233+0000","last_fullsized":"2024-02-22T19:36:27.130233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.000689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.000689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130143+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:36:27.130143+0000","last_peered":"2024-02-22T19:36:27.130143+0000","last_clean":"2024-02-22T19:36:27.130143+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:36:27.130143+0000","last_undegraded":"2024-02-22T19:36:27.130143+0000","last_fullsized":"2024-02-22T19:36:27.130143+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":158,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129550+0000","last_change":"2024-02-22T19:34:45.948451+0000","last_active":"2024-02-22T19:36:27.129550+0000","last_peered":"2024-02-22T19:36:27.129550+0000","last_clean":"2024-02-22T19:36:27.129550+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:36:27.129550+0000","last_undegraded":"2024-02-22T19:36:27.129550+0000","last_fullsized":"2024-02-22T19:36:27.129550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:45.948358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:45.948358+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132963+0000","last_change":"2024-02-22T19:35:23.244026+0000","last_active":"2024-02-22T19:36:27.132963+0000","last_peered":"2024-02-22T19:36:27.132963+0000","last_clean":"2024-02-22T19:36:27.132963+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:36:27.132963+0000","last_undegraded":"2024-02-22T19:36:27.132963+0000","last_fullsized":"2024-02-22T19:36:27.132963+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:23.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:23.243945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":98,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133025+0000","last_change":"2024-02-22T19:35:26.232511+0000","last_active":"2024-02-22T19:36:27.133025+0000","last_peered":"2024-02-22T19:36:27.133025+0000","last_clean":"2024-02-22T19:36:27.133025+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:36:27.133025+0000","last_undegraded":"2024-02-22T19:36:27.133025+0000","last_fullsized":"2024-02-22T19:36:27.133025+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.232444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.232444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":83,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122088+0000","last_change":"2024-02-22T19:35:22.835205+0000","last_active":"2024-02-22T19:36:27.122088+0000","last_peered":"2024-02-22T19:36:27.122088+0000","last_clean":"2024-02-22T19:36:27.122088+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:36:27.122088+0000","last_undegraded":"2024-02-22T19:36:27.122088+0000","last_fullsized":"2024-02-22T19:36:27.122088+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.835123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.835123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130545+0000","last_change":"2024-02-22T19:36:21.735619+0000","last_active":"2024-02-22T19:36:27.130545+0000","last_peered":"2024-02-22T19:36:27.130545+0000","last_clean":"2024-02-22T19:36:27.130545+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:36:27.130545+0000","last_undegraded":"2024-02-22T19:36:27.130545+0000","last_fullsized":"2024-02-22T19:36:27.130545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:21.735513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:21.735513+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":98,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.120944+0000","last_change":"2024-02-22T19:36:08.908709+0000","last_active":"2024-02-22T19:36:27.120944+0000","last_peered":"2024-02-22T19:36:27.120944+0000","last_clean":"2024-02-22T19:36:27.120944+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:36:27.120944+0000","last_undegraded":"2024-02-22T19:36:27.120944+0000","last_fullsized":"2024-02-22T19:36:27.120944+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.908571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.908571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123252+0000","last_change":"2024-02-22T19:36:19.511120+0000","last_active":"2024-02-22T19:36:27.123252+0000","last_peered":"2024-02-22T19:36:27.123252+0000","last_clean":"2024-02-22T19:36:27.123252+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:36:27.123252+0000","last_undegraded":"2024-02-22T19:36:27.123252+0000","last_fullsized":"2024-02-22T19:36:27.123252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.511049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.511049+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":105,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130883+0000","last_change":"2024-02-22T19:35:35.930757+0000","last_active":"2024-02-22T19:36:27.130883+0000","last_peered":"2024-02-22T19:36:27.130883+0000","last_clean":"2024-02-22T19:36:27.130883+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:36:27.130883+0000","last_undegraded":"2024-02-22T19:36:27.130883+0000","last_fullsized":"2024-02-22T19:36:27.130883+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:35:35.930654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.930654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131550+0000","last_change":"2024-02-22T19:34:46.118383+0000","last_active":"2024-02-22T19:36:27.131550+0000","last_peered":"2024-02-22T19:36:27.131550+0000","last_clean":"2024-02-22T19:36:27.131550+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:36:27.131550+0000","last_undegraded":"2024-02-22T19:36:27.131550+0000","last_fullsized":"2024-02-22T19:36:27.131550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:46.118282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.118282+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133208+0000","last_change":"2024-02-22T19:34:56.107642+0000","last_active":"2024-02-22T19:36:27.133208+0000","last_peered":"2024-02-22T19:36:27.133208+0000","last_clean":"2024-02-22T19:36:27.133208+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:36:27.133208+0000","last_undegraded":"2024-02-22T19:36:27.133208+0000","last_fullsized":"2024-02-22T19:36:27.133208+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:56.107558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:56.107558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.125275+0000","last_change":"2024-02-22T19:36:18.889514+0000","last_active":"2024-02-22T19:36:27.125275+0000","last_peered":"2024-02-22T19:36:27.125275+0000","last_clean":"2024-02-22T19:36:27.125275+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:36:27.125275+0000","last_undegraded":"2024-02-22T19:36:27.125275+0000","last_fullsized":"2024-02-22T19:36:27.125275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:18.889425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.889425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133331+0000","last_change":"2024-02-22T19:34:53.279758+0000","last_active":"2024-02-22T19:36:27.133331+0000","last_peered":"2024-02-22T19:36:27.133331+0000","last_clean":"2024-02-22T19:36:27.133331+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:36:27.133331+0000","last_undegraded":"2024-02-22T19:36:27.133331+0000","last_fullsized":"2024-02-22T19:36:27.133331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:53.279662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:34:53.279662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"172'22","reported_seq":175,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.120116+0000","last_change":"2024-02-22T19:36:24.021213+0000","last_active":"2024-02-22T19:36:27.120116+0000","last_peered":"2024-02-22T19:36:27.120116+0000","last_clean":"2024-02-22T19:36:27.120116+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:36:27.120116+0000","last_undegraded":"2024-02-22T19:36:27.120116+0000","last_fullsized":"2024-02-22T19:36:27.120116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'22","last_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.021111+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:28.335868+0000","last_change":"2024-02-22T19:36:28.335837+0000","last_active":"2024-02-22T19:36:28.335868+0000","last_peered":"2024-02-22T19:36:28.335868+0000","last_clean":"2024-02-22T19:36:28.335868+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:36:28.335868+0000","last_undegraded":"2024-02-22T19:36:28.335868+0000","last_fullsized":"2024-02-22T19:36:28.335868+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":134,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131212+0000","last_change":"2024-02-22T19:35:24.048780+0000","last_active":"2024-02-22T19:36:27.131212+0000","last_peered":"2024-02-22T19:36:27.131212+0000","last_clean":"2024-02-22T19:36:27.131212+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:36:27.131212+0000","last_undegraded":"2024-02-22T19:36:27.131212+0000","last_fullsized":"2024-02-22T19:36:27.131212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:24.048696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.048696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":138,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130603+0000","last_change":"2024-02-22T19:34:30.007888+0000","last_active":"2024-02-22T19:36:27.130603+0000","last_peered":"2024-02-22T19:36:27.130603+0000","last_clean":"2024-02-22T19:36:27.130603+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:36:27.130603+0000","last_undegraded":"2024-02-22T19:36:27.130603+0000","last_fullsized":"2024-02-22T19:36:27.130603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.007814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.007814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":161,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130171+0000","last_change":"2024-02-22T19:35:36.338814+0000","last_active":"2024-02-22T19:36:27.130171+0000","last_peered":"2024-02-22T19:36:27.130171+0000","last_clean":"2024-02-22T19:36:27.130171+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:36:27.130171+0000","last_undegraded":"2024-02-22T19:36:27.130171+0000","last_fullsized":"2024-02-22T19:36:27.130171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":150,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123360+0000","last_change":"2024-02-22T19:36:06.575957+0000","last_active":"2024-02-22T19:36:27.123360+0000","last_peered":"2024-02-22T19:36:27.123360+0000","last_clean":"2024-02-22T19:36:27.123360+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:36:27.123360+0000","last_undegraded":"2024-02-22T19:36:27.123360+0000","last_fullsized":"2024-02-22T19:36:27.123360+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:36:06.575860+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:36:06.575860+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"137'9","reported_seq":243,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.153064+0000","last_change":"2024-02-22T19:35:28.262175+0000","last_active":"2024-02-22T19:36:29.153064+0000","last_peered":"2024-02-22T19:36:29.153064+0000","last_clean":"2024-02-22T19:36:29.153064+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:36:29.153064+0000","last_undegraded":"2024-02-22T19:36:29.153064+0000","last_fullsized":"2024-02-22T19:36:29.153064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:35:28.262089+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:35:28.262089+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121182+0000","last_change":"2024-02-22T19:35:30.790879+0000","last_active":"2024-02-22T19:36:27.121182+0000","last_peered":"2024-02-22T19:36:27.121182+0000","last_clean":"2024-02-22T19:36:27.121182+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:36:27.121182+0000","last_undegraded":"2024-02-22T19:36:27.121182+0000","last_fullsized":"2024-02-22T19:36:27.121182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.790805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.790805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":154,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.136516+0000","last_change":"2024-02-22T19:36:04.546799+0000","last_active":"2024-02-22T19:36:27.136516+0000","last_peered":"2024-02-22T19:36:27.136516+0000","last_clean":"2024-02-22T19:36:27.136516+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:36:27.136516+0000","last_undegraded":"2024-02-22T19:36:27.136516+0000","last_fullsized":"2024-02-22T19:36:27.136516+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:36:04.546703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.546703+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.125401+0000","last_change":"2024-02-22T19:34:42.969287+0000","last_active":"2024-02-22T19:36:27.125401+0000","last_peered":"2024-02-22T19:36:27.125401+0000","last_clean":"2024-02-22T19:36:27.125401+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:36:27.125401+0000","last_undegraded":"2024-02-22T19:36:27.125401+0000","last_fullsized":"2024-02-22T19:36:27.125401+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.969189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.969189+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.136718+0000","last_change":"2024-02-22T19:36:22.679579+0000","last_active":"2024-02-22T19:36:27.136718+0000","last_peered":"2024-02-22T19:36:27.136718+0000","last_clean":"2024-02-22T19:36:27.136718+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:36:27.136718+0000","last_undegraded":"2024-02-22T19:36:27.136718+0000","last_fullsized":"2024-02-22T19:36:27.136718+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:22.679488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:22.679488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132498+0000","last_change":"2024-02-22T19:34:58.165060+0000","last_active":"2024-02-22T19:36:27.132498+0000","last_peered":"2024-02-22T19:36:27.132498+0000","last_clean":"2024-02-22T19:36:27.132498+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:36:27.132498+0000","last_undegraded":"2024-02-22T19:36:27.132498+0000","last_fullsized":"2024-02-22T19:36:27.132498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:34:58.164960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":210,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.119109+0000","last_change":"2024-02-22T19:35:29.242870+0000","last_active":"2024-02-22T19:36:27.119109+0000","last_peered":"2024-02-22T19:36:27.119109+0000","last_clean":"2024-02-22T19:36:27.119109+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:36:27.119109+0000","last_undegraded":"2024-02-22T19:36:27.119109+0000","last_fullsized":"2024-02-22T19:36:27.119109+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:35:29.242798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:29.242798+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123436+0000","last_change":"2024-02-22T19:36:07.626299+0000","last_active":"2024-02-22T19:36:27.123436+0000","last_peered":"2024-02-22T19:36:27.123436+0000","last_clean":"2024-02-22T19:36:27.123436+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:36:27.123436+0000","last_undegraded":"2024-02-22T19:36:27.123436+0000","last_fullsized":"2024-02-22T19:36:27.123436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.626201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.626201+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":136,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122479+0000","last_change":"2024-02-22T19:36:05.177403+0000","last_active":"2024-02-22T19:36:27.122479+0000","last_peered":"2024-02-22T19:36:27.122479+0000","last_clean":"2024-02-22T19:36:27.122479+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:36:27.122479+0000","last_undegraded":"2024-02-22T19:36:27.122479+0000","last_fullsized":"2024-02-22T19:36:27.122479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:05.177297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:05.177297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":150,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.135934+0000","last_change":"2024-02-22T19:35:21.292096+0000","last_active":"2024-02-22T19:36:27.135934+0000","last_peered":"2024-02-22T19:36:27.135934+0000","last_clean":"2024-02-22T19:36:27.135934+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:36:27.135934+0000","last_undegraded":"2024-02-22T19:36:27.135934+0000","last_fullsized":"2024-02-22T19:36:27.135934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:21.292020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:21.292020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132580+0000","last_change":"2024-02-22T19:34:39.037075+0000","last_active":"2024-02-22T19:36:27.132580+0000","last_peered":"2024-02-22T19:36:27.132580+0000","last_clean":"2024-02-22T19:36:27.132580+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:36:27.132580+0000","last_undegraded":"2024-02-22T19:36:27.132580+0000","last_fullsized":"2024-02-22T19:36:27.132580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:34:39.036999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.036999+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":129,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130136+0000","last_change":"2024-02-22T19:34:36.044624+0000","last_active":"2024-02-22T19:36:27.130136+0000","last_peered":"2024-02-22T19:36:27.130136+0000","last_clean":"2024-02-22T19:36:27.130136+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:36:27.130136+0000","last_undegraded":"2024-02-22T19:36:27.130136+0000","last_fullsized":"2024-02-22T19:36:27.130136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.044519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.044519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122332+0000","last_change":"2024-02-22T19:35:33.660316+0000","last_active":"2024-02-22T19:36:27.122332+0000","last_peered":"2024-02-22T19:36:27.122332+0000","last_clean":"2024-02-22T19:36:27.122332+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:36:27.122332+0000","last_undegraded":"2024-02-22T19:36:27.122332+0000","last_fullsized":"2024-02-22T19:36:27.122332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.660254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.660254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":142,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122698+0000","last_change":"2024-02-22T19:35:32.621589+0000","last_active":"2024-02-22T19:36:27.122698+0000","last_peered":"2024-02-22T19:36:27.122698+0000","last_clean":"2024-02-22T19:36:27.122698+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:36:27.122698+0000","last_undegraded":"2024-02-22T19:36:27.122698+0000","last_fullsized":"2024-02-22T19:36:27.122698+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:32.621508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:32.621508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":152,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129455+0000","last_change":"2024-02-22T19:34:43.915568+0000","last_active":"2024-02-22T19:36:27.129455+0000","last_peered":"2024-02-22T19:36:27.129455+0000","last_clean":"2024-02-22T19:36:27.129455+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:36:27.129455+0000","last_undegraded":"2024-02-22T19:36:27.129455+0000","last_fullsized":"2024-02-22T19:36:27.129455+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:43.915463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.915463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132583+0000","last_change":"2024-02-22T19:34:48.292233+0000","last_active":"2024-02-22T19:36:27.132583+0000","last_peered":"2024-02-22T19:36:27.132583+0000","last_clean":"2024-02-22T19:36:27.132583+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:36:27.132583+0000","last_undegraded":"2024-02-22T19:36:27.132583+0000","last_fullsized":"2024-02-22T19:36:27.132583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:48.292131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.292131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":131,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129947+0000","last_change":"2024-02-22T19:34:41.961882+0000","last_active":"2024-02-22T19:36:27.129947+0000","last_peered":"2024-02-22T19:36:27.129947+0000","last_clean":"2024-02-22T19:36:27.129947+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:36:27.129947+0000","last_undegraded":"2024-02-22T19:36:27.129947+0000","last_fullsized":"2024-02-22T19:36:27.129947+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.961788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.961788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122795+0000","last_change":"2024-02-22T19:34:49.595355+0000","last_active":"2024-02-22T19:36:27.122795+0000","last_peered":"2024-02-22T19:36:27.122795+0000","last_clean":"2024-02-22T19:36:27.122795+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:36:27.122795+0000","last_undegraded":"2024-02-22T19:36:27.122795+0000","last_fullsized":"2024-02-22T19:36:27.122795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:49.595253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:34:49.595253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122211+0000","last_change":"2024-02-22T19:35:35.855558+0000","last_active":"2024-02-22T19:36:27.122211+0000","last_peered":"2024-02-22T19:36:27.122211+0000","last_clean":"2024-02-22T19:36:27.122211+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:36:27.122211+0000","last_undegraded":"2024-02-22T19:36:27.122211+0000","last_fullsized":"2024-02-22T19:36:27.122211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:35.855449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.855449+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129221+0000","last_change":"2024-02-22T19:36:19.229925+0000","last_active":"2024-02-22T19:36:27.129221+0000","last_peered":"2024-02-22T19:36:27.129221+0000","last_clean":"2024-02-22T19:36:27.129221+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:36:27.129221+0000","last_undegraded":"2024-02-22T19:36:27.129221+0000","last_fullsized":"2024-02-22T19:36:27.129221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.229816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.229816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122040+0000","last_change":"2024-02-22T19:34:44.906578+0000","last_active":"2024-02-22T19:36:27.122040+0000","last_peered":"2024-02-22T19:36:27.122040+0000","last_clean":"2024-02-22T19:36:27.122040+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:36:27.122040+0000","last_undegraded":"2024-02-22T19:36:27.122040+0000","last_fullsized":"2024-02-22T19:36:27.122040+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.906461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.906461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.685541+0000","last_change":"2024-02-22T19:36:27.685508+0000","last_active":"2024-02-22T19:36:27.685541+0000","last_peered":"2024-02-22T19:36:27.685541+0000","last_clean":"2024-02-22T19:36:27.685541+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:36:27.685541+0000","last_undegraded":"2024-02-22T19:36:27.685541+0000","last_fullsized":"2024-02-22T19:36:27.685541+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.685418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.685418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":155,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131353+0000","last_change":"2024-02-22T19:34:59.142089+0000","last_active":"2024-02-22T19:36:27.131353+0000","last_peered":"2024-02-22T19:36:27.131353+0000","last_clean":"2024-02-22T19:36:27.131353+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:36:27.131353+0000","last_undegraded":"2024-02-22T19:36:27.131353+0000","last_fullsized":"2024-02-22T19:36:27.131353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:59.141999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:59.141999+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.136425+0000","last_change":"2024-02-22T19:36:15.699409+0000","last_active":"2024-02-22T19:36:27.136425+0000","last_peered":"2024-02-22T19:36:27.136425+0000","last_clean":"2024-02-22T19:36:27.136425+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:36:27.136425+0000","last_undegraded":"2024-02-22T19:36:27.136425+0000","last_fullsized":"2024-02-22T19:36:27.136425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.699319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.699319+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.150976+0000","last_change":"2024-02-22T19:34:57.925328+0000","last_active":"2024-02-22T19:36:29.150976+0000","last_peered":"2024-02-22T19:36:29.150976+0000","last_clean":"2024-02-22T19:36:29.150976+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:36:29.150976+0000","last_undegraded":"2024-02-22T19:36:29.150976+0000","last_fullsized":"2024-02-22T19:36:29.150976+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:57.925253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.925253+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":136,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122920+0000","last_change":"2024-02-22T19:34:42.438663+0000","last_active":"2024-02-22T19:36:27.122920+0000","last_peered":"2024-02-22T19:36:27.122920+0000","last_clean":"2024-02-22T19:36:27.122920+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:36:27.122920+0000","last_undegraded":"2024-02-22T19:36:27.122920+0000","last_fullsized":"2024-02-22T19:36:27.122920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.438557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.438557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":153,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131466+0000","last_change":"2024-02-22T19:34:32.908841+0000","last_active":"2024-02-22T19:36:27.131466+0000","last_peered":"2024-02-22T19:36:27.131466+0000","last_clean":"2024-02-22T19:36:27.131466+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:36:27.131466+0000","last_undegraded":"2024-02-22T19:36:27.131466+0000","last_fullsized":"2024-02-22T19:36:27.131466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:34:32.908721+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.908721+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133400+0000","last_change":"2024-02-22T19:36:26.761319+0000","last_active":"2024-02-22T19:36:27.133400+0000","last_peered":"2024-02-22T19:36:27.133400+0000","last_clean":"2024-02-22T19:36:27.133400+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:36:27.133400+0000","last_undegraded":"2024-02-22T19:36:27.133400+0000","last_fullsized":"2024-02-22T19:36:27.133400+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.761226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":229,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.153617+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:36:29.153617+0000","last_peered":"2024-02-22T19:36:29.153617+0000","last_clean":"2024-02-22T19:36:29.153617+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:36:29.153617+0000","last_undegraded":"2024-02-22T19:36:29.153617+0000","last_fullsized":"2024-02-22T19:36:29.153617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133293+0000","last_change":"2024-02-22T19:34:41.254550+0000","last_active":"2024-02-22T19:36:27.133293+0000","last_peered":"2024-02-22T19:36:27.133293+0000","last_clean":"2024-02-22T19:36:27.133293+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:36:27.133293+0000","last_undegraded":"2024-02-22T19:36:27.133293+0000","last_fullsized":"2024-02-22T19:36:27.133293+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.254449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.254449+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":161,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123350+0000","last_change":"2024-02-22T19:34:47.527168+0000","last_active":"2024-02-22T19:36:27.123350+0000","last_peered":"2024-02-22T19:36:27.123350+0000","last_clean":"2024-02-22T19:36:27.123350+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:36:27.123350+0000","last_undegraded":"2024-02-22T19:36:27.123350+0000","last_fullsized":"2024-02-22T19:36:27.123350+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:34:47.527066+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.527066+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123008+0000","last_change":"2024-02-22T19:35:31.810419+0000","last_active":"2024-02-22T19:36:27.123008+0000","last_peered":"2024-02-22T19:36:27.123008+0000","last_clean":"2024-02-22T19:36:27.123008+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:36:27.123008+0000","last_undegraded":"2024-02-22T19:36:27.123008+0000","last_fullsized":"2024-02-22T19:36:27.123008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.810353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":131,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130922+0000","last_change":"2024-02-22T19:36:09.204798+0000","last_active":"2024-02-22T19:36:27.130922+0000","last_peered":"2024-02-22T19:36:27.130922+0000","last_clean":"2024-02-22T19:36:27.130922+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:36:27.130922+0000","last_undegraded":"2024-02-22T19:36:27.130922+0000","last_fullsized":"2024-02-22T19:36:27.130922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.204726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.204726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122462+0000","last_change":"2024-02-22T19:35:30.620049+0000","last_active":"2024-02-22T19:36:27.122462+0000","last_peered":"2024-02-22T19:36:27.122462+0000","last_clean":"2024-02-22T19:36:27.122462+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:36:27.122462+0000","last_undegraded":"2024-02-22T19:36:27.122462+0000","last_fullsized":"2024-02-22T19:36:27.122462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.619969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":155,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.136601+0000","last_change":"2024-02-22T19:35:27.255099+0000","last_active":"2024-02-22T19:36:27.136601+0000","last_peered":"2024-02-22T19:36:27.136601+0000","last_clean":"2024-02-22T19:36:27.136601+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:36:27.136601+0000","last_undegraded":"2024-02-22T19:36:27.136601+0000","last_fullsized":"2024-02-22T19:36:27.136601+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122619+0000","last_change":"2024-02-22T19:35:31.614031+0000","last_active":"2024-02-22T19:36:27.122619+0000","last_peered":"2024-02-22T19:36:27.122619+0000","last_clean":"2024-02-22T19:36:27.122619+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:36:27.122619+0000","last_undegraded":"2024-02-22T19:36:27.122619+0000","last_fullsized":"2024-02-22T19:36:27.122619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.613909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.613909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130267+0000","last_change":"2024-02-22T19:34:31.909139+0000","last_active":"2024-02-22T19:36:27.130267+0000","last_peered":"2024-02-22T19:36:27.130267+0000","last_clean":"2024-02-22T19:36:27.130267+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:36:27.130267+0000","last_undegraded":"2024-02-22T19:36:27.130267+0000","last_fullsized":"2024-02-22T19:36:27.130267+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.909036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.909036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":151,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.117863+0000","last_change":"2024-02-22T19:35:25.227250+0000","last_active":"2024-02-22T19:36:27.117863+0000","last_peered":"2024-02-22T19:36:27.117863+0000","last_clean":"2024-02-22T19:36:27.117863+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:36:27.117863+0000","last_undegraded":"2024-02-22T19:36:27.117863+0000","last_fullsized":"2024-02-22T19:36:27.117863+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:25.227185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.227185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":164,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131062+0000","last_change":"2024-02-22T19:36:26.647888+0000","last_active":"2024-02-22T19:36:27.131062+0000","last_peered":"2024-02-22T19:36:27.131062+0000","last_clean":"2024-02-22T19:36:27.131062+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:36:27.131062+0000","last_undegraded":"2024-02-22T19:36:27.131062+0000","last_fullsized":"2024-02-22T19:36:27.131062+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:36:25.296484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.296484+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"176'4828","reported_seq":4858,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.180540+0000","last_change":"2024-02-22T19:35:19.745999+0000","last_active":"2024-02-22T19:36:29.180540+0000","last_peered":"2024-02-22T19:36:29.180540+0000","last_clean":"2024-02-22T19:36:29.180540+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:36:29.180540+0000","last_undegraded":"2024-02-22T19:36:29.180540+0000","last_fullsized":"2024-02-22T19:36:29.180540+0000","mapping_epoch":154,"log_start":"159'700","ondisk_log_start":"159'700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4128,"ondisk_log_size":4128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":316407808,"num_objects":4828,"num_object_clones":0,"num_object_copies":14484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4828,"num_write_kb":308992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":103,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.118406+0000","last_change":"2024-02-22T19:36:17.151389+0000","last_active":"2024-02-22T19:36:27.118406+0000","last_peered":"2024-02-22T19:36:27.118406+0000","last_clean":"2024-02-22T19:36:27.118406+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:36:27.118406+0000","last_undegraded":"2024-02-22T19:36:27.118406+0000","last_fullsized":"2024-02-22T19:36:27.118406+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.151294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.151294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":156,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123277+0000","last_change":"2024-02-22T19:36:04.642477+0000","last_active":"2024-02-22T19:36:27.123277+0000","last_peered":"2024-02-22T19:36:27.123277+0000","last_clean":"2024-02-22T19:36:27.123277+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:36:27.123277+0000","last_undegraded":"2024-02-22T19:36:27.123277+0000","last_fullsized":"2024-02-22T19:36:27.123277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:36:04.642361+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.642361+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":151,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122204+0000","last_change":"2024-02-22T19:36:09.623594+0000","last_active":"2024-02-22T19:36:27.122204+0000","last_peered":"2024-02-22T19:36:27.122204+0000","last_clean":"2024-02-22T19:36:27.122204+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:36:27.122204+0000","last_undegraded":"2024-02-22T19:36:27.122204+0000","last_fullsized":"2024-02-22T19:36:27.122204+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:36:09.623521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.623521+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":229,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122337+0000","last_change":"2024-02-22T19:34:39.483818+0000","last_active":"2024-02-22T19:36:27.122337+0000","last_peered":"2024-02-22T19:36:27.122337+0000","last_clean":"2024-02-22T19:36:27.122337+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:36:27.122337+0000","last_undegraded":"2024-02-22T19:36:27.122337+0000","last_fullsized":"2024-02-22T19:36:27.122337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:34:39.483740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.483740+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"176'5029","reported_seq":5059,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.231345+0000","last_change":"2024-02-22T19:35:19.226102+0000","last_active":"2024-02-22T19:36:29.231345+0000","last_peered":"2024-02-22T19:36:29.231345+0000","last_clean":"2024-02-22T19:36:29.231345+0000","last_became_active":"2024-02-22T19:35:19.225793+0000","last_became_peered":"2024-02-22T19:35:19.225793+0000","last_unstale":"2024-02-22T19:36:29.231345+0000","last_undegraded":"2024-02-22T19:36:29.231345+0000","last_fullsized":"2024-02-22T19:36:29.231345+0000","mapping_epoch":154,"log_start":"159'400","ondisk_log_start":"159'400","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":329580544,"num_objects":5029,"num_object_clones":0,"num_object_copies":15087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5029,"num_write_kb":321856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":129,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130042+0000","last_change":"2024-02-22T19:36:11.224751+0000","last_active":"2024-02-22T19:36:27.130042+0000","last_peered":"2024-02-22T19:36:27.130042+0000","last_clean":"2024-02-22T19:36:27.130042+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:36:27.130042+0000","last_undegraded":"2024-02-22T19:36:27.130042+0000","last_fullsized":"2024-02-22T19:36:27.130042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.224646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.224646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122784+0000","last_change":"2024-02-22T19:35:22.515913+0000","last_active":"2024-02-22T19:36:27.122784+0000","last_peered":"2024-02-22T19:36:27.122784+0000","last_clean":"2024-02-22T19:36:27.122784+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:36:27.122784+0000","last_undegraded":"2024-02-22T19:36:27.122784+0000","last_fullsized":"2024-02-22T19:36:27.122784+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.515824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.515824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":159,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130741+0000","last_change":"2024-02-22T19:34:33.123549+0000","last_active":"2024-02-22T19:36:27.130741+0000","last_peered":"2024-02-22T19:36:27.130741+0000","last_clean":"2024-02-22T19:36:27.130741+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:36:27.130741+0000","last_undegraded":"2024-02-22T19:36:27.130741+0000","last_fullsized":"2024-02-22T19:36:27.130741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:33.123452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.123452+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.124093+0000","last_change":"2024-02-22T19:36:19.039536+0000","last_active":"2024-02-22T19:36:27.124093+0000","last_peered":"2024-02-22T19:36:27.124093+0000","last_clean":"2024-02-22T19:36:27.124093+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:36:27.124093+0000","last_undegraded":"2024-02-22T19:36:27.124093+0000","last_fullsized":"2024-02-22T19:36:27.124093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.039457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.039457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"176'5054","reported_seq":5084,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.322225+0000","last_change":"2024-02-22T19:35:19.225144+0000","last_active":"2024-02-22T19:36:29.322225+0000","last_peered":"2024-02-22T19:36:29.322225+0000","last_clean":"2024-02-22T19:36:29.322225+0000","last_became_active":"2024-02-22T19:35:19.224823+0000","last_became_peered":"2024-02-22T19:35:19.224823+0000","last_unstale":"2024-02-22T19:36:29.322225+0000","last_undegraded":"2024-02-22T19:36:29.322225+0000","last_fullsized":"2024-02-22T19:36:29.322225+0000","mapping_epoch":154,"log_start":"159'300","ondisk_log_start":"159'300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4754,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":331218944,"num_objects":5054,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5054,"num_write_kb":323456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":130,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121920+0000","last_change":"2024-02-22T19:34:43.434105+0000","last_active":"2024-02-22T19:36:27.121920+0000","last_peered":"2024-02-22T19:36:27.121920+0000","last_clean":"2024-02-22T19:36:27.121920+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:36:27.121920+0000","last_undegraded":"2024-02-22T19:36:27.121920+0000","last_fullsized":"2024-02-22T19:36:27.121920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:43.434030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.434030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":148,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.120972+0000","last_change":"2024-02-22T19:36:14.029499+0000","last_active":"2024-02-22T19:36:27.120972+0000","last_peered":"2024-02-22T19:36:27.120972+0000","last_clean":"2024-02-22T19:36:27.120972+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:36:27.120972+0000","last_undegraded":"2024-02-22T19:36:27.120972+0000","last_fullsized":"2024-02-22T19:36:27.120972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.029389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":150,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123255+0000","last_change":"2024-02-22T19:34:46.476131+0000","last_active":"2024-02-22T19:36:27.123255+0000","last_peered":"2024-02-22T19:36:27.123255+0000","last_clean":"2024-02-22T19:36:27.123255+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:36:27.123255+0000","last_undegraded":"2024-02-22T19:36:27.123255+0000","last_fullsized":"2024-02-22T19:36:27.123255+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.476034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.476034+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":169,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.118592+0000","last_change":"2024-02-22T19:35:22.253473+0000","last_active":"2024-02-22T19:36:27.118592+0000","last_peered":"2024-02-22T19:36:27.118592+0000","last_clean":"2024-02-22T19:36:27.118592+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:36:27.118592+0000","last_undegraded":"2024-02-22T19:36:27.118592+0000","last_fullsized":"2024-02-22T19:36:27.118592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:22.253381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.253381+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.7","version":"176'5039","reported_seq":5069,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.265214+0000","last_change":"2024-02-22T19:35:19.227534+0000","last_active":"2024-02-22T19:36:29.265214+0000","last_peered":"2024-02-22T19:36:29.265214+0000","last_clean":"2024-02-22T19:36:29.265214+0000","last_became_active":"2024-02-22T19:35:19.227265+0000","last_became_peered":"2024-02-22T19:35:19.227265+0000","last_unstale":"2024-02-22T19:36:29.265214+0000","last_undegraded":"2024-02-22T19:36:29.265214+0000","last_fullsized":"2024-02-22T19:36:29.265214+0000","mapping_epoch":154,"log_start":"159'500","ondisk_log_start":"159'500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4539,"ondisk_log_size":4539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330235904,"num_objects":5039,"num_object_clones":0,"num_object_copies":15117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5039,"num_write_kb":322496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.135861+0000","last_change":"2024-02-22T19:36:17.690399+0000","last_active":"2024-02-22T19:36:27.135861+0000","last_peered":"2024-02-22T19:36:27.135861+0000","last_clean":"2024-02-22T19:36:27.135861+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:36:27.135861+0000","last_undegraded":"2024-02-22T19:36:27.135861+0000","last_fullsized":"2024-02-22T19:36:27.135861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.690298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.690298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123400+0000","last_change":"2024-02-22T19:35:37.831080+0000","last_active":"2024-02-22T19:36:27.123400+0000","last_peered":"2024-02-22T19:36:27.123400+0000","last_clean":"2024-02-22T19:36:27.123400+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:36:27.123400+0000","last_undegraded":"2024-02-22T19:36:27.123400+0000","last_fullsized":"2024-02-22T19:36:27.123400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.831003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.831003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131534+0000","last_change":"2024-02-22T19:34:04.892928+0000","last_active":"2024-02-22T19:36:27.131534+0000","last_peered":"2024-02-22T19:36:27.131534+0000","last_clean":"2024-02-22T19:36:27.131534+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:36:27.131534+0000","last_undegraded":"2024-02-22T19:36:27.131534+0000","last_fullsized":"2024-02-22T19:36:27.131534+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:04.892837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:04.892837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":174,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133476+0000","last_change":"2024-02-22T19:36:18.621857+0000","last_active":"2024-02-22T19:36:27.133476+0000","last_peered":"2024-02-22T19:36:27.133476+0000","last_clean":"2024-02-22T19:36:27.133476+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:36:27.133476+0000","last_undegraded":"2024-02-22T19:36:27.133476+0000","last_fullsized":"2024-02-22T19:36:27.133476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:18.621747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.621747+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132517+0000","last_change":"2024-02-22T19:36:25.576957+0000","last_active":"2024-02-22T19:36:27.132517+0000","last_peered":"2024-02-22T19:36:27.132517+0000","last_clean":"2024-02-22T19:36:27.132517+0000","last_became_active":"2024-02-22T19:35:58.432364+0000","last_became_peered":"2024-02-22T19:35:58.432364+0000","last_unstale":"2024-02-22T19:36:27.132517+0000","last_undegraded":"2024-02-22T19:36:27.132517+0000","last_fullsized":"2024-02-22T19:36:27.132517+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"176'5102","reported_seq":5132,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.313843+0000","last_change":"2024-02-22T19:35:19.226141+0000","last_active":"2024-02-22T19:36:29.313843+0000","last_peered":"2024-02-22T19:36:29.313843+0000","last_clean":"2024-02-22T19:36:29.313843+0000","last_became_active":"2024-02-22T19:35:19.225860+0000","last_became_peered":"2024-02-22T19:35:19.225860+0000","last_unstale":"2024-02-22T19:36:29.313843+0000","last_undegraded":"2024-02-22T19:36:29.313843+0000","last_fullsized":"2024-02-22T19:36:29.313843+0000","mapping_epoch":154,"log_start":"160'1200","ondisk_log_start":"160'1200","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3902,"ondisk_log_size":3902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":334364672,"num_objects":5102,"num_object_clones":0,"num_object_copies":15306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5102,"num_write_kb":326528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":129,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130389+0000","last_change":"2024-02-22T19:34:46.954016+0000","last_active":"2024-02-22T19:36:27.130389+0000","last_peered":"2024-02-22T19:36:27.130389+0000","last_clean":"2024-02-22T19:36:27.130389+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:36:27.130389+0000","last_undegraded":"2024-02-22T19:36:27.130389+0000","last_fullsized":"2024-02-22T19:36:27.130389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.953951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.953951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121872+0000","last_change":"2024-02-22T19:34:47.963364+0000","last_active":"2024-02-22T19:36:27.121872+0000","last_peered":"2024-02-22T19:36:27.121872+0000","last_clean":"2024-02-22T19:36:27.121872+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:36:27.121872+0000","last_undegraded":"2024-02-22T19:36:27.121872+0000","last_fullsized":"2024-02-22T19:36:27.121872+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:47.963267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.963267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":138,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131214+0000","last_change":"2024-02-22T19:34:32.095049+0000","last_active":"2024-02-22T19:36:27.131214+0000","last_peered":"2024-02-22T19:36:27.131214+0000","last_clean":"2024-02-22T19:36:27.131214+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:36:27.131214+0000","last_undegraded":"2024-02-22T19:36:27.131214+0000","last_fullsized":"2024-02-22T19:36:27.131214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.094985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.094985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":279,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.154226+0000","last_change":"2024-02-22T19:35:20.843884+0000","last_active":"2024-02-22T19:36:29.154226+0000","last_peered":"2024-02-22T19:36:29.154226+0000","last_clean":"2024-02-22T19:36:29.154226+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:36:29.154226+0000","last_undegraded":"2024-02-22T19:36:29.154226+0000","last_fullsized":"2024-02-22T19:36:29.154226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:35:20.843809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:20.843809+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.135422+0000","last_change":"2024-02-22T19:36:07.590613+0000","last_active":"2024-02-22T19:36:27.135422+0000","last_peered":"2024-02-22T19:36:27.135422+0000","last_clean":"2024-02-22T19:36:27.135422+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:36:27.135422+0000","last_undegraded":"2024-02-22T19:36:27.135422+0000","last_fullsized":"2024-02-22T19:36:27.135422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.590509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.590509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":158,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123340+0000","last_change":"2024-02-22T19:36:24.596597+0000","last_active":"2024-02-22T19:36:27.123340+0000","last_peered":"2024-02-22T19:36:27.123340+0000","last_clean":"2024-02-22T19:36:27.123340+0000","last_became_active":"2024-02-22T19:35:58.429347+0000","last_became_peered":"2024-02-22T19:35:58.429347+0000","last_unstale":"2024-02-22T19:36:27.123340+0000","last_undegraded":"2024-02-22T19:36:27.123340+0000","last_fullsized":"2024-02-22T19:36:27.123340+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:24.596531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.596531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"176'4746","reported_seq":4776,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.386401+0000","last_change":"2024-02-22T19:35:19.225962+0000","last_active":"2024-02-22T19:36:29.386401+0000","last_peered":"2024-02-22T19:36:29.386401+0000","last_clean":"2024-02-22T19:36:29.386401+0000","last_became_active":"2024-02-22T19:35:19.225604+0000","last_became_peered":"2024-02-22T19:35:19.225604+0000","last_unstale":"2024-02-22T19:36:29.386401+0000","last_undegraded":"2024-02-22T19:36:29.386401+0000","last_fullsized":"2024-02-22T19:36:29.386401+0000","mapping_epoch":154,"log_start":"159'800","ondisk_log_start":"159'800","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3946,"ondisk_log_size":3946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":311033856,"num_objects":4746,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4746,"num_write_kb":303744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123408+0000","last_change":"2024-02-22T19:36:08.683152+0000","last_active":"2024-02-22T19:36:27.123408+0000","last_peered":"2024-02-22T19:36:27.123408+0000","last_clean":"2024-02-22T19:36:27.123408+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:36:27.123408+0000","last_undegraded":"2024-02-22T19:36:27.123408+0000","last_fullsized":"2024-02-22T19:36:27.123408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.683006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.683006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121473+0000","last_change":"2024-02-22T19:35:38.917238+0000","last_active":"2024-02-22T19:36:27.121473+0000","last_peered":"2024-02-22T19:36:27.121473+0000","last_clean":"2024-02-22T19:36:27.121473+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:36:27.121473+0000","last_undegraded":"2024-02-22T19:36:27.121473+0000","last_fullsized":"2024-02-22T19:36:27.121473+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:38.917177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.917177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.120398+0000","last_change":"2024-02-22T19:35:28.850161+0000","last_active":"2024-02-22T19:36:27.120398+0000","last_peered":"2024-02-22T19:36:27.120398+0000","last_clean":"2024-02-22T19:36:27.120398+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:36:27.120398+0000","last_undegraded":"2024-02-22T19:36:27.120398+0000","last_fullsized":"2024-02-22T19:36:27.120398+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:28.850036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:35:28.850036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":150,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121366+0000","last_change":"2024-02-22T19:36:15.052075+0000","last_active":"2024-02-22T19:36:27.121366+0000","last_peered":"2024-02-22T19:36:27.121366+0000","last_clean":"2024-02-22T19:36:27.121366+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:36:27.121366+0000","last_undegraded":"2024-02-22T19:36:27.121366+0000","last_fullsized":"2024-02-22T19:36:27.121366+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:36:15.051953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.051953+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":151,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122572+0000","last_change":"2024-02-22T19:35:21.803865+0000","last_active":"2024-02-22T19:36:27.122572+0000","last_peered":"2024-02-22T19:36:27.122572+0000","last_clean":"2024-02-22T19:36:27.122572+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:36:27.122572+0000","last_undegraded":"2024-02-22T19:36:27.122572+0000","last_fullsized":"2024-02-22T19:36:27.122572+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:21.803776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:21.803776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"176'4956","reported_seq":4985,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.229851+0000","last_change":"2024-02-22T19:35:19.226967+0000","last_active":"2024-02-22T19:36:29.229851+0000","last_peered":"2024-02-22T19:36:29.229851+0000","last_clean":"2024-02-22T19:36:29.229851+0000","last_became_active":"2024-02-22T19:35:19.226638+0000","last_became_peered":"2024-02-22T19:35:19.226638+0000","last_unstale":"2024-02-22T19:36:29.229851+0000","last_undegraded":"2024-02-22T19:36:29.229851+0000","last_fullsized":"2024-02-22T19:36:29.229851+0000","mapping_epoch":154,"log_start":"159'300","ondisk_log_start":"159'300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4656,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324796416,"num_objects":4956,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4956,"num_write_kb":317184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130858+0000","last_change":"2024-02-22T19:35:22.280512+0000","last_active":"2024-02-22T19:36:27.130858+0000","last_peered":"2024-02-22T19:36:27.130858+0000","last_clean":"2024-02-22T19:36:27.130858+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:36:27.130858+0000","last_undegraded":"2024-02-22T19:36:27.130858+0000","last_fullsized":"2024-02-22T19:36:27.130858+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.280418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.280418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":158,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122077+0000","last_change":"2024-02-22T19:36:23.598862+0000","last_active":"2024-02-22T19:36:27.122077+0000","last_peered":"2024-02-22T19:36:27.122077+0000","last_clean":"2024-02-22T19:36:27.122077+0000","last_became_active":"2024-02-22T19:35:57.424387+0000","last_became_peered":"2024-02-22T19:35:57.424387+0000","last_unstale":"2024-02-22T19:36:27.122077+0000","last_undegraded":"2024-02-22T19:36:27.122077+0000","last_fullsized":"2024-02-22T19:36:27.122077+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.598758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.598758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":144,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129531+0000","last_change":"2024-02-22T19:34:50.967533+0000","last_active":"2024-02-22T19:36:27.129531+0000","last_peered":"2024-02-22T19:36:27.129531+0000","last_clean":"2024-02-22T19:36:27.129531+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:36:27.129531+0000","last_undegraded":"2024-02-22T19:36:27.129531+0000","last_fullsized":"2024-02-22T19:36:27.129531+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:50.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:50.967435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":167,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.119739+0000","last_change":"2024-02-22T19:35:35.065441+0000","last_active":"2024-02-22T19:36:27.119739+0000","last_peered":"2024-02-22T19:36:27.119739+0000","last_clean":"2024-02-22T19:36:27.119739+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:36:27.119739+0000","last_undegraded":"2024-02-22T19:36:27.119739+0000","last_fullsized":"2024-02-22T19:36:27.119739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:35.065332+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.065332+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123452+0000","last_change":"2024-02-22T19:36:12.513753+0000","last_active":"2024-02-22T19:36:27.123452+0000","last_peered":"2024-02-22T19:36:27.123452+0000","last_clean":"2024-02-22T19:36:27.123452+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:36:27.123452+0000","last_undegraded":"2024-02-22T19:36:27.123452+0000","last_fullsized":"2024-02-22T19:36:27.123452+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:12.513648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:12.513648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":21,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122314+0000","last_change":"2024-02-22T19:35:58.418691+0000","last_active":"2024-02-22T19:36:27.122314+0000","last_peered":"2024-02-22T19:36:27.122314+0000","last_clean":"2024-02-22T19:36:27.122314+0000","last_became_active":"2024-02-22T19:35:58.418408+0000","last_became_peered":"2024-02-22T19:35:58.418408+0000","last_unstale":"2024-02-22T19:36:27.122314+0000","last_undegraded":"2024-02-22T19:36:27.122314+0000","last_fullsized":"2024-02-22T19:36:27.122314+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"176'4961","reported_seq":4991,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.279875+0000","last_change":"2024-02-22T19:35:19.748247+0000","last_active":"2024-02-22T19:36:29.279875+0000","last_peered":"2024-02-22T19:36:29.279875+0000","last_clean":"2024-02-22T19:36:29.279875+0000","last_became_active":"2024-02-22T19:35:19.747933+0000","last_became_peered":"2024-02-22T19:35:19.747933+0000","last_unstale":"2024-02-22T19:36:29.279875+0000","last_undegraded":"2024-02-22T19:36:29.279875+0000","last_fullsized":"2024-02-22T19:36:29.279875+0000","mapping_epoch":154,"log_start":"159'400","ondisk_log_start":"159'400","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4561,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":325124096,"num_objects":4961,"num_object_clones":0,"num_object_copies":14883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4961,"num_write_kb":317504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.125248+0000","last_change":"2024-02-22T19:36:23.901614+0000","last_active":"2024-02-22T19:36:27.125248+0000","last_peered":"2024-02-22T19:36:27.125248+0000","last_clean":"2024-02-22T19:36:27.125248+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:36:27.125248+0000","last_undegraded":"2024-02-22T19:36:27.125248+0000","last_fullsized":"2024-02-22T19:36:27.125248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.901512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.901512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.135630+0000","last_change":"2024-02-22T19:35:30.342964+0000","last_active":"2024-02-22T19:36:27.135630+0000","last_peered":"2024-02-22T19:36:27.135630+0000","last_clean":"2024-02-22T19:36:27.135630+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:36:27.135630+0000","last_undegraded":"2024-02-22T19:36:27.135630+0000","last_fullsized":"2024-02-22T19:36:27.135630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.342864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.342864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":591,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:28.764196+0000","last_change":"2024-02-22T19:36:20.553666+0000","last_active":"2024-02-22T19:36:28.764196+0000","last_peered":"2024-02-22T19:36:28.764196+0000","last_clean":"2024-02-22T19:36:28.764196+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:36:28.764196+0000","last_undegraded":"2024-02-22T19:36:28.764196+0000","last_fullsized":"2024-02-22T19:36:28.764196+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:36:20.553554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.553554+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":442,"num_read_kb":437,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":152,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.136440+0000","last_change":"2024-02-22T19:34:38.184599+0000","last_active":"2024-02-22T19:36:27.136440+0000","last_peered":"2024-02-22T19:36:27.136440+0000","last_clean":"2024-02-22T19:36:27.136440+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:36:27.136440+0000","last_undegraded":"2024-02-22T19:36:27.136440+0000","last_fullsized":"2024-02-22T19:36:27.136440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:38.184521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.184521+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":98,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.127722+0000","last_change":"2024-02-22T19:36:15.908805+0000","last_active":"2024-02-22T19:36:27.127722+0000","last_peered":"2024-02-22T19:36:27.127722+0000","last_clean":"2024-02-22T19:36:27.127722+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:36:27.127722+0000","last_undegraded":"2024-02-22T19:36:27.127722+0000","last_fullsized":"2024-02-22T19:36:27.127722+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.908714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.908714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.119739+0000","last_change":"2024-02-22T19:36:14.230430+0000","last_active":"2024-02-22T19:36:27.119739+0000","last_peered":"2024-02-22T19:36:27.119739+0000","last_clean":"2024-02-22T19:36:27.119739+0000","last_became_active":"2024-02-22T19:35:58.429215+0000","last_became_peered":"2024-02-22T19:35:58.429215+0000","last_unstale":"2024-02-22T19:36:27.119739+0000","last_undegraded":"2024-02-22T19:36:27.119739+0000","last_fullsized":"2024-02-22T19:36:27.119739+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.230365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"176'4906","reported_seq":4936,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.346202+0000","last_change":"2024-02-22T19:35:19.745745+0000","last_active":"2024-02-22T19:36:29.346202+0000","last_peered":"2024-02-22T19:36:29.346202+0000","last_clean":"2024-02-22T19:36:29.346202+0000","last_became_active":"2024-02-22T19:35:19.745488+0000","last_became_peered":"2024-02-22T19:35:19.745488+0000","last_unstale":"2024-02-22T19:36:29.346202+0000","last_undegraded":"2024-02-22T19:36:29.346202+0000","last_fullsized":"2024-02-22T19:36:29.346202+0000","mapping_epoch":154,"log_start":"160'1000","ondisk_log_start":"160'1000","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3906,"ondisk_log_size":3906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":321519616,"num_objects":4906,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4906,"num_write_kb":313984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":132,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121020+0000","last_change":"2024-02-22T19:34:36.003355+0000","last_active":"2024-02-22T19:36:27.121020+0000","last_peered":"2024-02-22T19:36:27.121020+0000","last_clean":"2024-02-22T19:36:27.121020+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:36:27.121020+0000","last_undegraded":"2024-02-22T19:36:27.121020+0000","last_fullsized":"2024-02-22T19:36:27.121020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.003297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.003297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":152,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.127109+0000","last_change":"2024-02-22T19:36:19.862015+0000","last_active":"2024-02-22T19:36:27.127109+0000","last_peered":"2024-02-22T19:36:27.127109+0000","last_clean":"2024-02-22T19:36:27.127109+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:36:27.127109+0000","last_undegraded":"2024-02-22T19:36:27.127109+0000","last_fullsized":"2024-02-22T19:36:27.127109+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.861921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.861921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":202,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133253+0000","last_change":"2024-02-22T19:35:35.426393+0000","last_active":"2024-02-22T19:36:27.133253+0000","last_peered":"2024-02-22T19:36:27.133253+0000","last_clean":"2024-02-22T19:36:27.133253+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:36:27.133253+0000","last_undegraded":"2024-02-22T19:36:27.133253+0000","last_fullsized":"2024-02-22T19:36:27.133253+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:35.426296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.426296+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":147,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.118666+0000","last_change":"2024-02-22T19:35:31.179969+0000","last_active":"2024-02-22T19:36:27.118666+0000","last_peered":"2024-02-22T19:36:27.118666+0000","last_clean":"2024-02-22T19:36:27.118666+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:36:27.118666+0000","last_undegraded":"2024-02-22T19:36:27.118666+0000","last_fullsized":"2024-02-22T19:36:27.118666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.179891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.179891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.118782+0000","last_change":"2024-02-22T19:36:07.161771+0000","last_active":"2024-02-22T19:36:27.118782+0000","last_peered":"2024-02-22T19:36:27.118782+0000","last_clean":"2024-02-22T19:36:27.118782+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:36:27.118782+0000","last_undegraded":"2024-02-22T19:36:27.118782+0000","last_fullsized":"2024-02-22T19:36:27.118782+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.161538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.161538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"176'4989","reported_seq":5019,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.335058+0000","last_change":"2024-02-22T19:35:19.745825+0000","last_active":"2024-02-22T19:36:29.335058+0000","last_peered":"2024-02-22T19:36:29.335058+0000","last_clean":"2024-02-22T19:36:29.335058+0000","last_became_active":"2024-02-22T19:35:19.745522+0000","last_became_peered":"2024-02-22T19:35:19.745522+0000","last_unstale":"2024-02-22T19:36:29.335058+0000","last_undegraded":"2024-02-22T19:36:29.335058+0000","last_fullsized":"2024-02-22T19:36:29.335058+0000","mapping_epoch":154,"log_start":"159'300","ondisk_log_start":"159'300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4689,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":326959104,"num_objects":4989,"num_object_clones":0,"num_object_copies":14967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":319296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132664+0000","last_change":"2024-02-22T19:35:33.417797+0000","last_active":"2024-02-22T19:36:27.132664+0000","last_peered":"2024-02-22T19:36:27.132664+0000","last_clean":"2024-02-22T19:36:27.132664+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:36:27.132664+0000","last_undegraded":"2024-02-22T19:36:27.132664+0000","last_fullsized":"2024-02-22T19:36:27.132664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129670+0000","last_change":"2024-02-22T19:34:33.941965+0000","last_active":"2024-02-22T19:36:27.129670+0000","last_peered":"2024-02-22T19:36:27.129670+0000","last_clean":"2024-02-22T19:36:27.129670+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:36:27.129670+0000","last_undegraded":"2024-02-22T19:36:27.129670+0000","last_fullsized":"2024-02-22T19:36:27.129670+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:33.941905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.941905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":177,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.122127+0000","last_change":"2024-02-22T19:35:37.886857+0000","last_active":"2024-02-22T19:36:27.122127+0000","last_peered":"2024-02-22T19:36:27.122127+0000","last_clean":"2024-02-22T19:36:27.122127+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:36:27.122127+0000","last_undegraded":"2024-02-22T19:36:27.122127+0000","last_fullsized":"2024-02-22T19:36:27.122127+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:37.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.886787+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":248,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.154623+0000","last_change":"2024-02-22T19:36:23.659056+0000","last_active":"2024-02-22T19:36:29.154623+0000","last_peered":"2024-02-22T19:36:29.154623+0000","last_clean":"2024-02-22T19:36:29.154623+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:36:29.154623+0000","last_undegraded":"2024-02-22T19:36:29.154623+0000","last_fullsized":"2024-02-22T19:36:29.154623+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:36:23.658957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.658957+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131162+0000","last_change":"2024-02-22T19:35:58.422975+0000","last_active":"2024-02-22T19:36:27.131162+0000","last_peered":"2024-02-22T19:36:27.131162+0000","last_clean":"2024-02-22T19:36:27.131162+0000","last_became_active":"2024-02-22T19:35:58.422662+0000","last_became_peered":"2024-02-22T19:35:58.422662+0000","last_unstale":"2024-02-22T19:36:27.131162+0000","last_undegraded":"2024-02-22T19:36:27.131162+0000","last_fullsized":"2024-02-22T19:36:27.131162+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"176'5044","reported_seq":5074,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.221307+0000","last_change":"2024-02-22T19:35:19.746052+0000","last_active":"2024-02-22T19:36:29.221307+0000","last_peered":"2024-02-22T19:36:29.221307+0000","last_clean":"2024-02-22T19:36:29.221307+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:36:29.221307+0000","last_undegraded":"2024-02-22T19:36:29.221307+0000","last_fullsized":"2024-02-22T19:36:29.221307+0000","mapping_epoch":154,"log_start":"159'500","ondisk_log_start":"159'500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4544,"ondisk_log_size":4544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330563584,"num_objects":5044,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5044,"num_write_kb":322816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123288+0000","last_change":"2024-02-22T19:36:11.554490+0000","last_active":"2024-02-22T19:36:27.123288+0000","last_peered":"2024-02-22T19:36:27.123288+0000","last_clean":"2024-02-22T19:36:27.123288+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:36:27.123288+0000","last_undegraded":"2024-02-22T19:36:27.123288+0000","last_fullsized":"2024-02-22T19:36:27.123288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.554382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.554382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":136,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.136543+0000","last_change":"2024-02-22T19:34:34.158347+0000","last_active":"2024-02-22T19:36:27.136543+0000","last_peered":"2024-02-22T19:36:27.136543+0000","last_clean":"2024-02-22T19:36:27.136543+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:36:27.136543+0000","last_undegraded":"2024-02-22T19:36:27.136543+0000","last_fullsized":"2024-02-22T19:36:27.136543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:34.158242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:34.158242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":189,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130538+0000","last_change":"2024-02-22T19:35:26.969937+0000","last_active":"2024-02-22T19:36:27.130538+0000","last_peered":"2024-02-22T19:36:27.130538+0000","last_clean":"2024-02-22T19:36:27.130538+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:36:27.130538+0000","last_undegraded":"2024-02-22T19:36:27.130538+0000","last_fullsized":"2024-02-22T19:36:27.130538+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:26.969827+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.969827+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":229,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.149424+0000","last_change":"2024-02-22T19:35:25.316903+0000","last_active":"2024-02-22T19:36:29.149424+0000","last_peered":"2024-02-22T19:36:29.149424+0000","last_clean":"2024-02-22T19:36:29.149424+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:36:29.149424+0000","last_undegraded":"2024-02-22T19:36:29.149424+0000","last_fullsized":"2024-02-22T19:36:29.149424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130922+0000","last_change":"2024-02-22T19:35:58.417513+0000","last_active":"2024-02-22T19:36:27.130922+0000","last_peered":"2024-02-22T19:36:27.130922+0000","last_clean":"2024-02-22T19:36:27.130922+0000","last_became_active":"2024-02-22T19:35:58.417154+0000","last_became_peered":"2024-02-22T19:35:58.417154+0000","last_unstale":"2024-02-22T19:36:27.130922+0000","last_undegraded":"2024-02-22T19:36:27.130922+0000","last_fullsized":"2024-02-22T19:36:27.130922+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"176'4993","reported_seq":5023,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.327008+0000","last_change":"2024-02-22T19:35:19.227135+0000","last_active":"2024-02-22T19:36:29.327008+0000","last_peered":"2024-02-22T19:36:29.327008+0000","last_clean":"2024-02-22T19:36:29.327008+0000","last_became_active":"2024-02-22T19:35:19.226763+0000","last_became_peered":"2024-02-22T19:35:19.226763+0000","last_unstale":"2024-02-22T19:36:29.327008+0000","last_undegraded":"2024-02-22T19:36:29.327008+0000","last_fullsized":"2024-02-22T19:36:29.327008+0000","mapping_epoch":154,"log_start":"159'300","ondisk_log_start":"159'300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4693,"ondisk_log_size":4693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":327221248,"num_objects":4993,"num_object_clones":0,"num_object_copies":14979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4993,"num_write_kb":319552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130439+0000","last_change":"2024-02-22T19:36:20.739814+0000","last_active":"2024-02-22T19:36:27.130439+0000","last_peered":"2024-02-22T19:36:27.130439+0000","last_clean":"2024-02-22T19:36:27.130439+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:36:27.130439+0000","last_undegraded":"2024-02-22T19:36:27.130439+0000","last_fullsized":"2024-02-22T19:36:27.130439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":135,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130735+0000","last_change":"2024-02-22T19:34:40.000683+0000","last_active":"2024-02-22T19:36:27.130735+0000","last_peered":"2024-02-22T19:36:27.130735+0000","last_clean":"2024-02-22T19:36:27.130735+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:36:27.130735+0000","last_undegraded":"2024-02-22T19:36:27.130735+0000","last_fullsized":"2024-02-22T19:36:27.130735+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:40.000583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:40.000583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":172,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132744+0000","last_change":"2024-02-22T19:36:26.284565+0000","last_active":"2024-02-22T19:36:27.132744+0000","last_peered":"2024-02-22T19:36:27.132744+0000","last_clean":"2024-02-22T19:36:27.132744+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:36:27.132744+0000","last_undegraded":"2024-02-22T19:36:27.132744+0000","last_fullsized":"2024-02-22T19:36:27.132744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130311+0000","last_change":"2024-02-22T19:35:36.900633+0000","last_active":"2024-02-22T19:36:27.130311+0000","last_peered":"2024-02-22T19:36:27.130311+0000","last_clean":"2024-02-22T19:36:27.130311+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:36:27.130311+0000","last_undegraded":"2024-02-22T19:36:27.130311+0000","last_fullsized":"2024-02-22T19:36:27.130311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":26,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131309+0000","last_change":"2024-02-22T19:36:20.071572+0000","last_active":"2024-02-22T19:36:27.131309+0000","last_peered":"2024-02-22T19:36:27.131309+0000","last_clean":"2024-02-22T19:36:27.131309+0000","last_became_active":"2024-02-22T19:35:58.418647+0000","last_became_peered":"2024-02-22T19:35:58.418647+0000","last_unstale":"2024-02-22T19:36:27.131309+0000","last_undegraded":"2024-02-22T19:36:27.131309+0000","last_fullsized":"2024-02-22T19:36:27.131309+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.071500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.071500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"176'4848","reported_seq":4877,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.326671+0000","last_change":"2024-02-22T19:35:19.745819+0000","last_active":"2024-02-22T19:36:29.326671+0000","last_peered":"2024-02-22T19:36:29.326671+0000","last_clean":"2024-02-22T19:36:29.326671+0000","last_became_active":"2024-02-22T19:35:19.745509+0000","last_became_peered":"2024-02-22T19:35:19.745509+0000","last_unstale":"2024-02-22T19:36:29.326671+0000","last_undegraded":"2024-02-22T19:36:29.326671+0000","last_fullsized":"2024-02-22T19:36:29.326671+0000","mapping_epoch":154,"log_start":"159'300","ondisk_log_start":"159'300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4548,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":317718528,"num_objects":4848,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4848,"num_write_kb":310272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121070+0000","last_change":"2024-02-22T19:34:51.981539+0000","last_active":"2024-02-22T19:36:27.121070+0000","last_peered":"2024-02-22T19:36:27.121070+0000","last_clean":"2024-02-22T19:36:27.121070+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:36:27.121070+0000","last_undegraded":"2024-02-22T19:36:27.121070+0000","last_fullsized":"2024-02-22T19:36:27.121070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:51.981444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:51.981444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":244,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133206+0000","last_change":"2024-02-22T19:34:43.126256+0000","last_active":"2024-02-22T19:36:27.133206+0000","last_peered":"2024-02-22T19:36:27.133206+0000","last_clean":"2024-02-22T19:36:27.133206+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:36:27.133206+0000","last_undegraded":"2024-02-22T19:36:27.133206+0000","last_fullsized":"2024-02-22T19:36:27.133206+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.132662+0000","last_change":"2024-02-22T19:35:27.352610+0000","last_active":"2024-02-22T19:36:27.132662+0000","last_peered":"2024-02-22T19:36:27.132662+0000","last_clean":"2024-02-22T19:36:27.132662+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:36:27.132662+0000","last_undegraded":"2024-02-22T19:36:27.132662+0000","last_fullsized":"2024-02-22T19:36:27.132662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:27.352545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.352545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":33,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.128361+0000","last_change":"2024-02-22T19:36:16.249180+0000","last_active":"2024-02-22T19:36:27.128361+0000","last_peered":"2024-02-22T19:36:27.128361+0000","last_clean":"2024-02-22T19:36:27.128361+0000","last_became_active":"2024-02-22T19:36:16.154859+0000","last_became_peered":"2024-02-22T19:36:16.154859+0000","last_unstale":"2024-02-22T19:36:27.128361+0000","last_undegraded":"2024-02-22T19:36:27.128361+0000","last_fullsized":"2024-02-22T19:36:27.128361+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:35:24.795027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.795027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"176'4904","reported_seq":4934,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.166682+0000","last_change":"2024-02-22T19:35:19.225232+0000","last_active":"2024-02-22T19:36:29.166682+0000","last_peered":"2024-02-22T19:36:29.166682+0000","last_clean":"2024-02-22T19:36:29.166682+0000","last_became_active":"2024-02-22T19:35:19.224941+0000","last_became_peered":"2024-02-22T19:35:19.224941+0000","last_unstale":"2024-02-22T19:36:29.166682+0000","last_undegraded":"2024-02-22T19:36:29.166682+0000","last_fullsized":"2024-02-22T19:36:29.166682+0000","mapping_epoch":154,"log_start":"159'700","ondisk_log_start":"159'700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4204,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":321388544,"num_objects":4904,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4904,"num_write_kb":313856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":138,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131394+0000","last_change":"2024-02-22T19:35:37.337349+0000","last_active":"2024-02-22T19:36:27.131394+0000","last_peered":"2024-02-22T19:36:27.131394+0000","last_clean":"2024-02-22T19:36:27.131394+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:36:27.131394+0000","last_undegraded":"2024-02-22T19:36:27.131394+0000","last_fullsized":"2024-02-22T19:36:27.131394+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":93,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133340+0000","last_change":"2024-02-22T19:34:39.082294+0000","last_active":"2024-02-22T19:36:27.133340+0000","last_peered":"2024-02-22T19:36:27.133340+0000","last_clean":"2024-02-22T19:36:27.133340+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:36:27.133340+0000","last_undegraded":"2024-02-22T19:36:27.133340+0000","last_fullsized":"2024-02-22T19:36:27.133340+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:39.082190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.082190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":155,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.130390+0000","last_change":"2024-02-22T19:36:08.709226+0000","last_active":"2024-02-22T19:36:27.130390+0000","last_peered":"2024-02-22T19:36:27.130390+0000","last_clean":"2024-02-22T19:36:27.130390+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:36:27.130390+0000","last_undegraded":"2024-02-22T19:36:27.130390+0000","last_fullsized":"2024-02-22T19:36:27.130390+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.709163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.709163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":160,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133151+0000","last_change":"2024-02-22T19:36:11.687594+0000","last_active":"2024-02-22T19:36:27.133151+0000","last_peered":"2024-02-22T19:36:27.133151+0000","last_clean":"2024-02-22T19:36:27.133151+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:36:27.133151+0000","last_undegraded":"2024-02-22T19:36:27.133151+0000","last_fullsized":"2024-02-22T19:36:27.133151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:36:11.687522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.687522+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":26,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.123892+0000","last_change":"2024-02-22T19:36:23.022761+0000","last_active":"2024-02-22T19:36:27.123892+0000","last_peered":"2024-02-22T19:36:27.123892+0000","last_clean":"2024-02-22T19:36:27.123892+0000","last_became_active":"2024-02-22T19:35:58.422620+0000","last_became_peered":"2024-02-22T19:35:58.422620+0000","last_unstale":"2024-02-22T19:36:27.123892+0000","last_undegraded":"2024-02-22T19:36:27.123892+0000","last_fullsized":"2024-02-22T19:36:27.123892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.022678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.022678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"176'4993","reported_seq":5023,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.284518+0000","last_change":"2024-02-22T19:35:19.745821+0000","last_active":"2024-02-22T19:36:29.284518+0000","last_peered":"2024-02-22T19:36:29.284518+0000","last_clean":"2024-02-22T19:36:29.284518+0000","last_became_active":"2024-02-22T19:35:19.745498+0000","last_became_peered":"2024-02-22T19:35:19.745498+0000","last_unstale":"2024-02-22T19:36:29.284518+0000","last_undegraded":"2024-02-22T19:36:29.284518+0000","last_fullsized":"2024-02-22T19:36:29.284518+0000","mapping_epoch":154,"log_start":"159'500","ondisk_log_start":"159'500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4493,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":327221248,"num_objects":4993,"num_object_clones":0,"num_object_copies":14979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4993,"num_write_kb":319552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":137,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.121245+0000","last_change":"2024-02-22T19:36:11.024673+0000","last_active":"2024-02-22T19:36:27.121245+0000","last_peered":"2024-02-22T19:36:27.121245+0000","last_clean":"2024-02-22T19:36:27.121245+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:36:27.121245+0000","last_undegraded":"2024-02-22T19:36:27.121245+0000","last_fullsized":"2024-02-22T19:36:27.121245+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.024573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.024573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.127208+0000","last_change":"2024-02-22T19:36:09.886117+0000","last_active":"2024-02-22T19:36:27.127208+0000","last_peered":"2024-02-22T19:36:27.127208+0000","last_clean":"2024-02-22T19:36:27.127208+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:36:27.127208+0000","last_undegraded":"2024-02-22T19:36:27.127208+0000","last_fullsized":"2024-02-22T19:36:27.127208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.885998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.885998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:28.180413+0000","last_change":"2024-02-22T19:36:28.180370+0000","last_active":"2024-02-22T19:36:28.180413+0000","last_peered":"2024-02-22T19:36:28.180413+0000","last_clean":"2024-02-22T19:36:28.180413+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:36:28.180413+0000","last_undegraded":"2024-02-22T19:36:28.180413+0000","last_fullsized":"2024-02-22T19:36:28.180413+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.180297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.180297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":164,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131333+0000","last_change":"2024-02-22T19:34:57.145803+0000","last_active":"2024-02-22T19:36:27.131333+0000","last_peered":"2024-02-22T19:36:27.131333+0000","last_clean":"2024-02-22T19:36:27.131333+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:36:27.131333+0000","last_undegraded":"2024-02-22T19:36:27.131333+0000","last_fullsized":"2024-02-22T19:36:27.131333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:34:57.145716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.145716+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":158,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.599426+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:36:27.599426+0000","last_peered":"2024-02-22T19:36:27.599426+0000","last_clean":"2024-02-22T19:36:27.599426+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:36:27.599426+0000","last_undegraded":"2024-02-22T19:36:27.599426+0000","last_fullsized":"2024-02-22T19:36:27.599426+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"176'4921","reported_seq":4950,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.394171+0000","last_change":"2024-02-22T19:35:19.229848+0000","last_active":"2024-02-22T19:36:29.394171+0000","last_peered":"2024-02-22T19:36:29.394171+0000","last_clean":"2024-02-22T19:36:29.394171+0000","last_became_active":"2024-02-22T19:35:19.229525+0000","last_became_peered":"2024-02-22T19:35:19.229525+0000","last_unstale":"2024-02-22T19:36:29.394171+0000","last_undegraded":"2024-02-22T19:36:29.394171+0000","last_fullsized":"2024-02-22T19:36:29.394171+0000","mapping_epoch":154,"log_start":"160'1000","ondisk_log_start":"160'1000","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3921,"ondisk_log_size":3921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":322502656,"num_objects":4921,"num_object_clones":0,"num_object_copies":14763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4921,"num_write_kb":314944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":136,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.133272+0000","last_change":"2024-02-22T19:34:54.019363+0000","last_active":"2024-02-22T19:36:27.133272+0000","last_peered":"2024-02-22T19:36:27.133272+0000","last_clean":"2024-02-22T19:36:27.133272+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:36:27.133272+0000","last_undegraded":"2024-02-22T19:36:27.133272+0000","last_fullsized":"2024-02-22T19:36:27.133272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:34:54.019270+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:34:54.019270+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":250,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-02-22T19:36:29.157493+0000","last_change":"2024-02-22T19:36:04.927145+0000","last_active":"2024-02-22T19:36:29.157493+0000","last_peered":"2024-02-22T19:36:29.157493+0000","last_clean":"2024-02-22T19:36:29.157493+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:36:29.157493+0000","last_undegraded":"2024-02-22T19:36:29.157493+0000","last_fullsized":"2024-02-22T19:36:29.157493+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:36:04.927065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.927065+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":144,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.129928+0000","last_change":"2024-02-22T19:34:28.943926+0000","last_active":"2024-02-22T19:36:27.129928+0000","last_peered":"2024-02-22T19:36:27.129928+0000","last_clean":"2024-02-22T19:36:27.129928+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:36:27.129928+0000","last_undegraded":"2024-02-22T19:36:27.129928+0000","last_fullsized":"2024-02-22T19:36:27.129928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:34:28.943835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.943835+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":151,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-02-22T19:36:27.131140+0000","last_change":"2024-02-22T19:35:33.971551+0000","last_active":"2024-02-22T19:36:27.131140+0000","last_peered":"2024-02-22T19:36:27.131140+0000","last_clean":"2024-02-22T19:36:27.131140+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:36:27.131140+0000","last_undegraded":"2024-02-22T19:36:27.131140+0000","last_fullsized":"2024-02-22T19:36:27.131140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:33.971446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.971446+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5197856768,"num_objects":79313,"num_object_clones":0,"num_object_copies":237939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79313,"num_write_kb":5076032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15595536384,"data_stored":15595536384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70113,"ondisk_log_size":70113,"up":48,"acting":48,"num_store_stats":7},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":442,"num_read_kb":437,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":7},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1650,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266781,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1906004,"kb_used_data":1600588,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91839148,"statfs":{"total":95995035648,"available":94043287552,"internally_reserved":0,"allocated":1639002112,"data_stored":1638514073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":1.9419999999999999,"15min":1.0229999999999999},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.0739999999999998,"5min":42.597999999999999,"15min":42.597999999999999},"last":1.6950000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.8400000000000001,"15min":0.96299999999999997},"min":{"1min":0.255,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.8659999999999997,"5min":42.155999999999999,"15min":42.155999999999999},"last":2.1440000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.796,"15min":0.97699999999999998},"min":{"1min":0.28199999999999997,"5min":0.248,"15min":0.248},"max":{"1min":5.4260000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":2.302},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.9710000000000001,"15min":1.016},"min":{"1min":0.28699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":4.2190000000000003,"5min":37.938000000000002,"15min":37.938000000000002},"last":2.5230000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.4870000000000001,"15min":0.86099999999999999},"min":{"1min":0.32100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.165,"5min":41.640999999999998,"15min":41.640999999999998},"last":1.595},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.764,"15min":0.98099999999999998},"min":{"1min":0.26400000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.3650000000000002,"5min":41.722000000000001,"15min":41.722000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.802,"15min":0.99399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.9009999999999998,"5min":41.457000000000001,"15min":41.457000000000001},"last":2.8420000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.651,"15min":0.96099999999999997},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6699999999999999,"5min":41.311999999999998,"15min":41.311999999999998},"last":0.436}]},{"osd":4,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54700000000000004,"15min":0.48699999999999999},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.393,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.499,"15min":0.47299999999999998},"min":{"1min":0.33400000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.8819999999999999,"5min":3.4780000000000002,"15min":3.4780000000000002},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.091,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60799999999999998,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0900000000000001,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77100000000000002,"15min":0.60599999999999998},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1259999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":1.891},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.825,"5min":12.067,"15min":12.067},"last":0.54200000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528418,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2527364,"kb_used_data":2226044,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91217788,"statfs":{"total":95995035648,"available":93407014912,"internally_reserved":0,"allocated":2279469056,"data_stored":2279003231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4059999999999999,"15min":0.81200000000000006},"min":{"1min":0.26200000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":5.8010000000000002,"5min":17.885000000000002,"15min":17.885000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":1.246,"15min":0.75800000000000001},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.5470000000000002,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.75}]},{"osd":1,"last update":"Thu Feb 22 19:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.0369999999999999,"15min":0.71399999999999997},"min":{"1min":0.29999999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.4220000000000002,"5min":11.301,"15min":11.301},"last":1.123},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0529999999999999,"15min":0.69799999999999995},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.3799999999999999,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4470000000000001,"15min":0.83799999999999997},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":6.0049999999999999,"5min":12.859999999999999,"15min":12.859999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.1080000000000001,"15min":0.73399999999999999},"min":{"1min":0.40200000000000002,"5min":0.25,"15min":0.25},"max":{"1min":3.6840000000000002,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.308}]},{"osd":3,"last update":"Thu Feb 22 19:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":1.7,"15min":0.92600000000000005},"min":{"1min":0.35799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.0350000000000001,"5min":38.762,"15min":38.762},"last":2.2989999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":2.1509999999999998,"15min":1.0860000000000001},"min":{"1min":0.438,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.3370000000000002,"5min":35.612000000000002,"15min":35.612000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Feb 22 19:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.28699999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71399999999999997,"5min":1.583,"15min":1.583},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.433,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.81100000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.318}]},{"osd":5,"last update":"Thu Feb 22 19:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83399999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54000000000000004,"15min":0.46300000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.59199999999999997,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.0129999999999999,"5min":3.488,"15min":3.488},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.67800000000000005,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5600000000000001,"5min":3.593,"15min":3.593},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2494124,"kb_used_data":2188708,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91251028,"statfs":{"total":95995035648,"available":93441052672,"internally_reserved":0,"allocated":2241236992,"data_stored":2240724931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":2.0590000000000002,"15min":0.91700000000000004},"min":{"1min":0.20899999999999999,"5min":0.20699999999999999,"15min":0.20499999999999999},"max":{"1min":5.3550000000000004,"5min":15.613,"15min":15.613},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":2.1880000000000002,"15min":0.97499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23899999999999999},"max":{"1min":7.3620000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":6.0599999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:36:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":2.4729999999999999,"15min":1.091},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":5.29,"5min":16.503,"15min":16.503},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.0169999999999999,"5min":2.2320000000000002,"15min":1.01},"min":{"1min":0.20200000000000001,"5min":0.192,"15min":0.192},"max":{"1min":2.859,"5min":13.461,"15min":13.461},"last":5.6319999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:35:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":2.5659999999999998,"15min":1.155},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":8.4900000000000002,"5min":25.145,"15min":25.145},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":2.5169999999999999,"15min":1.175},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":5.4850000000000003,"5min":22.25,"15min":22.25},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":1.591,"15min":0.93200000000000005},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.669,"5min":15.542999999999999,"15min":15.542999999999999},"last":2.5569999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.4159999999999999,"15min":0.84599999999999997},"min":{"1min":0.32500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":8.5500000000000007,"5min":8.5500000000000007,"15min":8.5500000000000007},"last":1.3440000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.6930000000000001,"15min":0.95999999999999996},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.5389999999999997,"5min":15.587999999999999,"15min":15.587999999999999},"last":2.3180000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.5600000000000001,"15min":0.90100000000000002},"min":{"1min":0.34799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.2930000000000001,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":5.4359999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:35:46 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3620000000000001,"15min":0.81200000000000006},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":8.6189999999999998,"5min":8.6189999999999998,"15min":8.6189999999999998},"last":3.4849999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.615,"15min":0.92900000000000005},"min":{"1min":0.38400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":5.4290000000000003,"5min":15.651999999999999,"15min":15.651999999999999},"last":2.625}]},{"osd":7,"last update":"Thu Feb 22 19:36:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.538,"15min":0.86499999999999999},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.1299999999999999,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":5.407},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.5389999999999999,"15min":0.877},"min":{"1min":0.45900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":8.5869999999999997,"5min":8.5869999999999997,"15min":8.5869999999999997},"last":3.52}]}]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2524944,"kb_used_data":2219528,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91220208,"statfs":{"total":95995035648,"available":93409492992,"internally_reserved":0,"allocated":2272796672,"data_stored":2272313079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":1.946,"15min":0.88300000000000001},"min":{"1min":0.245,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":3.5590000000000002,"5min":12.755000000000001,"15min":12.755000000000001},"last":0.153},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.774,"15min":0.83299999999999996},"min":{"1min":0.255,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":5.9560000000000004,"5min":13.013999999999999,"15min":13.013999999999999},"last":3.9350000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:36:17 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":2.5590000000000002,"15min":1.0880000000000001},"min":{"1min":0.25800000000000001,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":4.3700000000000001,"5min":19.927,"15min":19.927},"last":3.8860000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":2.3719999999999999,"15min":1.0409999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.8889999999999998,"5min":17.907,"15min":17.907},"last":0.19700000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.8540000000000001,"15min":0.89900000000000002},"min":{"1min":0.28799999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.1550000000000002,"5min":13.06,"15min":13.06},"last":1.5800000000000001},{"interface":"front","average":{"1min":2.1890000000000001,"5min":2.4980000000000002,"15min":1.125},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":6.7939999999999996,"5min":22.936,"15min":22.936},"last":3.7799999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":1.2989999999999999,"15min":0.80700000000000005},"min":{"1min":0.45900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.419,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":0.224},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.992,"15min":1.044},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89200000000000002,"5min":13.874000000000001,"15min":13.874000000000001},"last":3.8490000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:36:26 2024","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":2.274,"15min":1.129},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33100000000000002},"max":{"1min":8.8369999999999997,"5min":13.903,"15min":13.903},"last":3.9180000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.3640000000000001,"15min":0.83199999999999996},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.5230000000000001,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.30499999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.343,"15min":0.83799999999999997},"min":{"1min":0.40999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.0830000000000002,"5min":7.0830000000000002,"15min":7.0830000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":0.83499999999999996},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.1219999999999999,"5min":7.5410000000000004,"15min":7.5410000000000004},"last":1.6419999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.381,"15min":0.84099999999999997},"min":{"1min":0.41999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":3.492,"5min":7.0620000000000003,"15min":7.0620000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.45,"15min":0.873},"min":{"1min":0.42899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.4139999999999997,"5min":7.423,"15min":7.423},"last":1.6779999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2520068,"kb_used_data":2206460,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":91225084,"statfs":{"total":95995035648,"available":93414486016,"internally_reserved":0,"allocated":2259415040,"data_stored":2258942571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":1.756,"15min":0.83199999999999996},"min":{"1min":0.249,"5min":0.183,"15min":0.183},"max":{"1min":2.2989999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.357,"5min":2.3100000000000001,"15min":1.0169999999999999},"min":{"1min":0.30299999999999999,"5min":0.191,"15min":0.191},"max":{"1min":9.5530000000000008,"5min":18.059999999999999,"15min":18.059999999999999},"last":0.21099999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:36:07 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":2.024,"15min":0.92700000000000005},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":3.4830000000000001,"5min":10.183,"15min":10.183},"last":9.5370000000000008},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.577,"15min":0.80000000000000004},"min":{"1min":0.25600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":6.4329999999999998,"5min":12.393000000000001,"15min":12.393000000000001},"last":1.8440000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":2.149,"15min":1.0229999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":3.9550000000000001,"5min":30.071000000000002,"15min":30.071000000000002},"last":1.726},{"interface":"front","average":{"1min":1.0740000000000001,"5min":2.5059999999999998,"15min":1.1279999999999999},"min":{"1min":0.315,"5min":0.192,"15min":0.192},"max":{"1min":6.2610000000000001,"5min":32.119,"15min":32.119},"last":4.2009999999999996}]},{"osd":4,"last update":"Thu Feb 22 19:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.575,"15min":0.90500000000000003},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.2080000000000002,"5min":6.3890000000000002,"15min":6.3890000000000002},"last":1.405},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.94699999999999995,"15min":0.70599999999999996},"min":{"1min":0.28100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":2.1219999999999999,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":1.752}]},{"osd":5,"last update":"Thu Feb 22 19:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.0049999999999999,"15min":0.749},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.895,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.6639999999999999,"15min":0.95099999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.2410000000000001,"5min":6.4320000000000004,"15min":6.4320000000000004},"last":1.5049999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.6539999999999999,"15min":0.95399999999999996},"min":{"1min":0.40899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":5.1150000000000002,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":1.478},{"interface":"front","average":{"1min":0.60899999999999999,"5min":1.024,"15min":0.76000000000000001},"min":{"1min":0.35599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.542,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":1.7989999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:36:07 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.73,"15min":0.96699999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.3530000000000002,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.4419999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":1.117,"15min":0.77100000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.173,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":1.776}]}]},{"osd":3,"up_from":26,"seq":111669149807,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3149096,"kb_used_data":2847776,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":90596056,"statfs":{"total":95995035648,"available":92770361344,"internally_reserved":0,"allocated":2916122624,"data_stored":2915621573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":2.2290000000000001,"15min":1.028},"min":{"1min":0.19400000000000001,"5min":0.193,"15min":0.193},"max":{"1min":8.9049999999999994,"5min":12.929,"15min":12.929},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.875,"5min":2.5779999999999998,"15min":1.1739999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":6.7119999999999997,"5min":14.831,"15min":14.831},"last":1.373}]},{"osd":1,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":2.5539999999999998,"15min":1.1579999999999999},"min":{"1min":0.246,"5min":0.22,"15min":0.22},"max":{"1min":5.1580000000000004,"5min":18.259,"15min":18.259},"last":4.5529999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.9410000000000001,"15min":0.94599999999999995},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.6139999999999999,"5min":13.154,"15min":13.154},"last":0.214}]},{"osd":2,"last update":"Thu Feb 22 19:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":2.4209999999999998,"15min":1.1299999999999999},"min":{"1min":0.29699999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.3810000000000002,"5min":16.271000000000001,"15min":16.271000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":1.345,"5min":2.3149999999999999,"15min":1.0820000000000001},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":8.7759999999999998,"5min":15.143000000000001,"15min":15.143000000000001},"last":0.253}]},{"osd":4,"last update":"Thu Feb 22 19:35:54 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.853,"15min":1.026},"min":{"1min":0.432,"5min":0.249,"15min":0.249},"max":{"1min":4.0780000000000003,"5min":13.680999999999999,"15min":13.680999999999999},"last":1.431},{"interface":"front","average":{"1min":0.67800000000000005,"5min":1.55,"15min":0.91600000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3440000000000001,"5min":11.366,"15min":11.366},"last":0.27800000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:36:21 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.752,"15min":0.97099999999999997},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":8.0259999999999998,"5min":12.879,"15min":12.879},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.395,"5min":1.9650000000000001,"15min":1.0389999999999999},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.1070000000000002,"5min":13.707000000000001,"15min":13.707000000000001},"last":1.4670000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.647,"15min":0.93899999999999995},"min":{"1min":0.44600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.7719999999999998,"5min":12.898999999999999,"15min":12.898999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.9550000000000001,"15min":1.0780000000000001},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":4.0350000000000001,"5min":13.803000000000001,"15min":13.803000000000001},"last":1.498}]},{"osd":7,"last update":"Thu Feb 22 19:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.6779999999999999,"15min":0.94199999999999995},"min":{"1min":0.443,"5min":0.317,"15min":0.317},"max":{"1min":3.8250000000000002,"5min":11.348000000000001,"15min":11.348000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.6879999999999999,"15min":0.96699999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.8359999999999999,"5min":12.846,"15min":12.846},"last":0.433}]}]},{"osd":4,"up_from":32,"seq":137438953581,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2167768,"kb_used_data":1866448,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91577384,"statfs":{"total":95995035648,"available":93775241216,"internally_reserved":0,"allocated":1911242752,"data_stored":1910751103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.5229999999999999,"15min":0.84999999999999998},"min":{"1min":0.246,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.94699999999999995,"5min":24.009,"15min":24.009},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":1.5249999999999999,"15min":0.86599999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.8480000000000001,"5min":24.263000000000002,"15min":24.263000000000002},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":1.2789999999999999,"15min":0.78600000000000003},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":2.9500000000000002,"5min":6.3029999999999999,"15min":6.3029999999999999},"last":1.871},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0649999999999999,"15min":0.69399999999999995},"min":{"1min":0.29499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.5939999999999999,"5min":12.445,"15min":12.445},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.3009999999999999,"15min":0.80600000000000005},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.4219999999999999,"5min":23.082000000000001,"15min":23.082000000000001},"last":0.501},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.627,"15min":0.90000000000000002},"min":{"1min":0.28199999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.2149999999999999,"5min":23.138999999999999,"15min":23.138999999999999},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.7,"15min":0.93100000000000005},"min":{"1min":0.29599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.4450000000000001,"5min":13.476000000000001,"15min":13.476000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":1.3280000000000001,"15min":0.79600000000000004},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Feb 22 19:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.55300000000000005,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.29399999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.50700000000000001,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.5469999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.48799999999999999},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.4530000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.437}]},{"osd":7,"last update":"Thu Feb 22 19:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96599999999999997,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.58499999999999996}]}]},{"osd":5,"up_from":0,"seq":167503724647,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2294677504,"data_stored":2294677504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2254962688,"data_stored":2254962688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2267611136,"data_stored":2267611136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2920284160,"data_stored":2920284160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1926168576,"data_stored":1926168576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2279800832,"data_stored":2279800832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1652031488,"data_stored":1652031488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:36:34.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:36:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.633 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:36:34.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:33.229969+0000 mgr.y (mgr.14535) 292 : audit [DBG] from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:36:34.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:33.402465+0000 mgr.y (mgr.14535) 293 : cluster [DBG] pgmap v330: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 814 B/s rd, 49 MiB/s wr, 783 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:34.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:34.222280+0000 mon.a (mon.0) 1010 : audit [DBG] from='client.? 172.21.15.53:0/1169503008' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:34.842 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:34.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:34.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:34.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:36:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:36:34.941 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:33.229969+0000 mgr.y (mgr.14535) 292 : audit [DBG] from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:36:34.942 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:33.402465+0000 mgr.y (mgr.14535) 293 : cluster [DBG] pgmap v330: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 814 B/s rd, 49 MiB/s wr, 783 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:34.942 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:34.222280+0000 mon.a (mon.0) 1010 : audit [DBG] from='client.? 172.21.15.53:0/1169503008' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:34.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:36:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:36:34.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:34.986 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:33.229969+0000 mgr.y (mgr.14535) 292 : audit [DBG] from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:36:34.986 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:33.402465+0000 mgr.y (mgr.14535) 293 : cluster [DBG] pgmap v330: 158 pgs: 158 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 814 B/s rd, 49 MiB/s wr, 783 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:34.986 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:34.222280+0000 mon.a (mon.0) 1010 : audit [DBG] from='client.? 172.21.15.53:0/1169503008' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:35.011 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 62 16 84913 84897 85.4925 61.0625 0.026617 0.0116917 2024-02-22T19:36:35.021 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:35.403 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:35.410 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:35.411 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:35.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:35.630 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:35.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:35.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:35.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:35.780 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:35.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:34.642374+0000 mgr.y (mgr.14535) 294 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:35.938 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:35.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:35.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:35.978 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:34.642374+0000 mgr.y (mgr.14535) 294 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:35.979 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:34.642374+0000 mgr.y (mgr.14535) 294 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:36.011 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 63 16 85939 85923 85.1535 64.125 0.0118163 0.0117384 2024-02-22T19:36:36.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:36.085 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:36.189 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:36.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:36.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:47829 7.5 7:a6121554:::benchmark_data_smithi053_107660_object47828:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:32:18.785923+0000", 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 257.47518644199999, 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039849519, 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 47829 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.785923+0000", 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.785923+0000", 2024-02-22T19:36:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.785922+0000", 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5612000000000002e-05 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.785977+0000", 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4549000000000001e-05 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.785992+0000", 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5789999999999998e-06 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.785999+0000", 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9398999999999999e-05 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.786049+0000", 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6960999999999995e-05 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.786146+0000", 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9933999999999996e-05 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.786246+0000", 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0013888049999999999 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.787634+0000", 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0080942940000000001 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.795729+0000", 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029947168999999999 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.825676+0000", 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0186000000000003e-05 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.825736+0000", 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6523000000000003e-05 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:18.825773+0000", 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039849519 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:75614 7.5 7:ab8bc097:::benchmark_data_smithi053_107660_object75613:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e118)", 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:32:51.861615+0000", 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 224.39949448900001, 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046122623000000001, 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 75614 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.861615+0000", 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.861615+0000", 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.861614+0000", 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4669999999999998e-05 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.861649+0000", 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2149999999999998e-06 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.861655+0000", 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1750000000000001e-06 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.861659+0000", 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021639559999999999 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.863823+0000", 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000150214 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.863973+0000", 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018156800000000001 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.864154+0000", 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016575100000000001 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.865812+0000", 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022382776 2024-02-22T19:36:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.888195+0000", 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019469663000000002 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.907664+0000", 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6409999999999998e-05 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.907711+0000", 2024-02-22T19:36:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7078999999999999e-05 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:32:51.907738+0000", 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046122623000000001 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:91486 7.5 7:ad4ca408:::benchmark_data_smithi053_107660_object2279:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:13.626730+0000", 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 202.63437944500001, 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03903417, 2024-02-22T19:36:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 91486 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626730+0000", 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626730+0000", 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626729+0000", 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4249999999999997e-06 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626735+0000", 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7059999999999999e-06 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626744+0000", 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8870000000000003e-06 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626751+0000", 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0238000000000001e-05 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626781+0000", 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8232000000000007e-05 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626850+0000", 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2168000000000003e-05 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626932+0000", 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032321899999999998 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.630164+0000", 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001796707 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.631961+0000", 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033709441999999999 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.665670+0000", 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4261000000000001e-05 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.665724+0000", 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9844e-05 2024-02-22T19:36:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.665764+0000", 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03903417 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:91490 7.5 7:afd2998a:::benchmark_data_smithi053_107660_object2283:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:13.626762+0000", 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 202.63434755599999, 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045319953000000003, 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 91490 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626762+0000", 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626762+0000", 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626761+0000", 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.704e-06 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626765+0000", 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9160000000000004e-06 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626772+0000", 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6779999999999999e-06 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.626777+0000", 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017274580000000001 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.628504+0000", 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4613999999999999e-05 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.628599+0000", 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1889000000000002e-05 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.628681+0000", 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016205080000000001 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.630301+0000", 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093233480000000004 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.639625+0000", 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032352804999999998 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.671977+0000", 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2757999999999999e-05 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.672030+0000", 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1853e-05 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.672082+0000", 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045319953000000003 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:91723 7.5 7:a34f091a:::benchmark_data_smithi053_107660_object2516:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:13.890256+0000", 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 202.370853678, 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.041931049999999997, 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 91723 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890256+0000", 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890256+0000", 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890254+0000", 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0639999999999996e-06 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890262+0000", 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1786e-05 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890273+0000", 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6259999999999997e-06 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890280+0000", 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2342999999999998e-05 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890322+0000", 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6481000000000002e-05 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890399+0000", 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7905999999999995e-05 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.890487+0000", 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022364630000000002 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.892723+0000", 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066439250000000002 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.899367+0000", 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032758250000000003 2024-02-22T19:36:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.932125+0000", 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7024000000000003e-05 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.932162+0000", 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4635999999999999e-05 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:13.932187+0000", 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.041931049999999997 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:92138 7.5 7:a5fd529b:::benchmark_data_smithi053_107660_object2931:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:14.374300+0000", 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 201.88680999499999, 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047796109000000003, 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 92138 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374300+0000", 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374300+0000", 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374298+0000", 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5490000000000003e-06 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374305+0000", 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1841999999999999e-05 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374317+0000", 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8670000000000002e-06 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374321+0000", 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4938000000000002e-05 2024-02-22T19:36:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374376+0000", 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012243200000000001 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374499+0000", 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5427999999999995e-05 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.374594+0000", 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018739570000000001 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.376468+0000", 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.044679120000000003 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.421147+0000", 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00083337199999999998 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.421981+0000", 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1940999999999994e-05 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.422053+0000", 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3053999999999998e-05 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:14.422096+0000", 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047796109000000003 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:96615 7.5 7:a2c18e8f:::benchmark_data_smithi053_107660_object7408:head [delete] snapc 0=[] ondisk+write+known_if_redirected e125)", 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:20.101659+0000", 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 196.159450703, 2024-02-22T19:36:36.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039923646, 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 96615 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101659+0000", 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101659+0000", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101657+0000", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2439999999999996e-06 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101665+0000", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3135e-05 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101678+0000", 2024-02-22T19:36:36.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5919999999999997e-06 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101684+0000", 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3470999999999997e-05 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101738+0000", 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116674 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101855+0000", 2024-02-22T19:36:36.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010349999999999999 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.101958+0000", 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018559360000000001 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.103814+0000", 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012369389999999999 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.116183+0000", 2024-02-22T19:36:36.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025297997999999999 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.141481+0000", 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8309e-05 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.141540+0000", 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2867999999999997e-05 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:20.141583+0000", 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039923646 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:97995 7.5 7:a0b01839:::benchmark_data_smithi053_107660_object8788:head [delete] snapc 0=[] ondisk+write+known_if_redirected e126)", 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:21.961679+0000", 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 194.299430266, 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.043008782000000002, 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 97995 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961679+0000", 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961679+0000", 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961678+0000", 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1509999999999998e-06 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961685+0000", 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1348999999999999e-05 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961696+0000", 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9290000000000001e-06 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961703+0000", 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9018999999999998e-05 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961772+0000", 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3919999999999996e-05 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961856+0000", 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4233000000000004e-05 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.961950+0000", 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002040399 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.963991+0000", 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013543917000000001 2024-02-22T19:36:36.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:21.977535+0000", 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027067151000000001 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:22.004602+0000", 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9919000000000001e-05 2024-02-22T19:36:36.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:22.004652+0000", 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6225e-05 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:22.004688+0000", 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.043008782000000002 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:100554 7.5 7:a886ea9b:::benchmark_data_smithi053_107660_object11347:head [delete] snapc 0=[] ondisk+write+known_if_redirected e127)", 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:25.409102+0000", 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 190.85200744100001, 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047513821999999997, 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 100554 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409102+0000", 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409102+0000", 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409101+0000", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8169999999999998e-06 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409106+0000", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5869999999999993e-06 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409114+0000", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0629999999999999e-06 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409118+0000", 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2960999999999997e-05 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409161+0000", 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6446000000000006e-05 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409238+0000", 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6171e-05 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.409314+0000", 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019446909999999999 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.411259+0000", 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022599945 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.433859+0000", 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022677157999999999 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.456536+0000", 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2617999999999998e-05 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.456578+0000", 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7515999999999998e-05 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:25.456616+0000", 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047513821999999997 2024-02-22T19:36:36.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:107686 7.5 7:a1608fef:::benchmark_data_smithi053_107660_object18479:head [delete] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:35.036221+0000", 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 181.22488871300001, 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045468735000000003, 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 107686 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.319 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.036221+0000", 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.036221+0000", 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.036220+0000", 2024-02-22T19:36:36.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1860000000000001e-06 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.036223+0000", 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4819999999999997e-06 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.036231+0000", 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.036235+0000", 2024-02-22T19:36:36.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029011330000000002 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.039136+0000", 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6224999999999994e-05 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.039232+0000", 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4661999999999997e-05 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.039317+0000", 2024-02-22T19:36:36.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001782215 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.041099+0000", 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032207339999999998 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.044320+0000", 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037284853 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.081605+0000", 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3655000000000001e-05 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.081648+0000", 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1257999999999997e-05 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:35.081690+0000", 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045468735000000003 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:114129 7.5 7:a0f0a8d6:::benchmark_data_smithi053_107660_object24922:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:44.152145+0000", 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 172.10896410300001, 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068586619000000001, 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 114129 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152145+0000", 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152145+0000", 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152144+0000", 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9630000000000004e-06 2024-02-22T19:36:36.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152153+0000", 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4742999999999999e-05 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152167+0000", 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1679999999999998e-06 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152175+0000", 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9959000000000001e-05 2024-02-22T19:36:36.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152224+0000", 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011922900000000001 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152344+0000", 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012733699999999999 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.152471+0000", 2024-02-22T19:36:36.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021722690000000001 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.154643+0000", 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0013053769999999999 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.155949+0000", 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064685651999999996 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.220634+0000", 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3953999999999999e-05 2024-02-22T19:36:36.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.220688+0000", 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3794000000000001e-05 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:44.220732+0000", 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068586619000000001 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.330 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:114887 7.5 7:a6452c84:::benchmark_data_smithi053_107660_object25680:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:45.004195+0000", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 171.256914456, 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.048711723999999998, 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 114887 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004195+0000", 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004195+0000", 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004194+0000", 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004201+0000", 2024-02-22T19:36:36.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1793999999999999e-05 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004213+0000", 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6039999999999998e-06 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004219+0000", 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9799999999999998e-05 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004269+0000", 2024-02-22T19:36:36.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012224800000000001 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004391+0000", 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014169700000000001 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004533+0000", 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037765099999999998 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.008310+0000", 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00109245 2024-02-22T19:36:36.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.009402+0000", 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.043422684000000003 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.052825+0000", 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1871e-05 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.052857+0000", 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0194000000000001e-05 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.052907+0000", 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.048711723999999998 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24670.0:114888 7.5 7:a88bf192:::benchmark_data_smithi053_107660_object25681:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:33:45.004527+0000", 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 171.256582941, 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.049551525999999999, 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24670", 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 114888 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004527+0000", 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004527+0000", 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004526+0000", 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6119999999999998e-06 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004531+0000", 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2565000000000001e-05 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004544+0000", 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1059999999999996e-06 2024-02-22T19:36:36.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.004550+0000", 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001847102 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.006397+0000", 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7900999999999997e-05 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.006485+0000", 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013053899999999999 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.006616+0000", 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019479390000000001 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.008564+0000", 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000935533 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.009499+0000", 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.044512786999999998 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.054012+0000", 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7351e-05 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.054039+0000", 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8983000000000003e-05 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:33:45.054078+0000", 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.049551525999999999 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73682 8.d 8:b1bd8515:::benchmark_data_smithi053_130774_object73681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.556683+0000", 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.704426598, 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045056706000000002, 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73682 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556683+0000", 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556683+0000", 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556682+0000", 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4894999999999997e-05 2024-02-22T19:36:36.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556717+0000", 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1169999999999998e-06 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556724+0000", 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9300000000000002e-06 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556729+0000", 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005079821 2024-02-22T19:36:36.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.561809+0000", 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124843 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.561934+0000", 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000181769 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562116+0000", 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065577459999999997 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568673+0000", 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8644e-05 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568692+0000", 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032956592 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601649+0000", 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3634999999999997e-05 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601702+0000", 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7423999999999999e-05 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601740+0000", 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045056706000000002 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73684 8.d 8:b8cd64a5:::benchmark_data_smithi053_130774_object73683:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.556735+0000", 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.704374156, 2024-02-22T19:36:36.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045125606999999998, 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73684 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556735+0000", 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556735+0000", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556735+0000", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8186999999999998e-05 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556783+0000", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7749999999999993e-06 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556791+0000", 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7829999999999998e-06 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556795+0000", 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072034229999999996 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.563999+0000", 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105886 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.564105+0000", 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2998999999999995e-05 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.564198+0000", 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044987500000000001 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568697+0000", 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029494999000000001 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.598192+0000", 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003589446 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601781+0000", 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3287000000000001e-05 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601824+0000", 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6779000000000001e-05 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601861+0000", 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045125606999999998 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73685 8.d 8:b5e81d28:::benchmark_data_smithi053_130774_object73684:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.556802+0000", 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.704307828999999, 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046316504000000001, 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73685 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556802+0000", 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556802+0000", 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556801+0000", 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1344999999999999e-05 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556832+0000", 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6050000000000003e-06 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556839+0000", 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0100000000000001e-06 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.556845+0000", 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.009344336 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.566189+0000", 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107905 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.566297+0000", 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000158061 2024-02-22T19:36:36.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.566455+0000", 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038345979999999998 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.570290+0000", 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027945213999999999 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.598235+0000", 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047945110000000004 2024-02-22T19:36:36.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.603030+0000", 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2432000000000002e-05 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.603082+0000", 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6139000000000001e-05 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.603118+0000", 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046316504000000001 2024-02-22T19:36:36.358 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73687 8.d 8:b636a457:::benchmark_data_smithi053_130774_object73686:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.562040+0000", 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.699069694, 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.044963428, 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73687 2024-02-22T19:36:36.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562040+0000", 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562040+0000", 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562038+0000", 2024-02-22T19:36:36.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011116 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562149+0000", 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0892e-05 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562160+0000", 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0009999999999998e-06 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.562166+0000", 2024-02-22T19:36:36.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066135309999999998 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568779+0000", 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5054000000000003e-05 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568854+0000", 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010288100000000001 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568957+0000", 2024-02-22T19:36:36.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029242154999999999 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.598199+0000", 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3071999999999994e-05 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.598272+0000", 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0086299389999999997 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.606902+0000", 2024-02-22T19:36:36.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6814999999999997e-05 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.606959+0000", 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4258000000000002e-05 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.607003+0000", 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.044963428 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73695 8.d 8:bae88725:::benchmark_data_smithi053_130774_object73694:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.568221+0000", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.692888570000001, 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.041427183999999999, 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73695 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568221+0000", 2024-02-22T19:36:36.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568221+0000", 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568219+0000", 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4119000000000002e-05 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568263+0000", 2024-02-22T19:36:36.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1959e-05 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568275+0000", 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4749999999999998e-06 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568282+0000", 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020358479999999998 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.570317+0000", 2024-02-22T19:36:36.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8553000000000006e-05 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.570386+0000", 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025794996000000001 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.596181+0000", 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054365459999999996 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601618+0000", 2024-02-22T19:36:36.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000265479 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.601883+0000", 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076736189999999996 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.609557+0000", 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3464000000000002e-05 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.609610+0000", 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8090000000000003e-05 2024-02-22T19:36:36.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.609648+0000", 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.041427183999999999 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73697 8.d 8:be7be42e:::benchmark_data_smithi053_130774_object73696:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.568509+0000", 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.692600906999999, 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.041284833, 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73697 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568509+0000", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568509+0000", 2024-02-22T19:36:36.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568508+0000", 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6833999999999998e-05 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568565+0000", 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7530000000000007e-06 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568573+0000", 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8889999999999999e-06 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.568579+0000", 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030195469999999999 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.598775+0000", 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017306600000000001 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.598948+0000", 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018354600000000001 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.599131+0000", 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002917542 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.602049+0000", 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7671999999999997e-05 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.602137+0000", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0075624159999999998 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.609699+0000", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7783e-05 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.609747+0000", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6740999999999999e-05 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.609794+0000", 2024-02-22T19:36:36.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.041284833 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:74558 8.d 8:beb86f28:::benchmark_data_smithi053_130774_object74557:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:24.490996+0000", 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 11.770113297, 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039413047, 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:36:36.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 74558 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.490996+0000", 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.490996+0000", 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.490996+0000", 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6452999999999999e-05 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.491022+0000", 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0599999999999998e-06 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.491027+0000", 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0259999999999999e-06 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.491030+0000", 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024197950000000002 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.493450+0000", 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3290999999999999e-05 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.493533+0000", 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104706 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.493638+0000", 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021100979999999999 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.495748+0000", 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8189000000000001e-05 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.495806+0000", 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034522125000000001 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.530328+0000", 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6987000000000003e-05 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.530375+0000", 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3864e-05 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:24.530409+0000", 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039413047 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.381 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:36:36.382 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:36:36.382 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:36.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:36.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:35.412884+0000 mgr.y (mgr.14535) 295 : cluster [DBG] pgmap v331: 158 pgs: 158 active+clean; 5.1 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.23k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:36.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:35.412884+0000 mgr.y (mgr.14535) 295 : cluster [DBG] pgmap v331: 158 pgs: 158 active+clean; 5.1 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.23k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:36.834 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:36.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:36.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:35.412884+0000 mgr.y (mgr.14535) 295 : cluster [DBG] pgmap v331: 158 pgs: 158 active+clean; 5.1 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.23k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:36.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:36:36.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:36.978 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:37.011 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 64 16 86967 86951 84.8264 64.25 0.0139615 0.0117834 2024-02-22T19:36:37.013 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:37.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:37.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:37.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:37.258 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:37.399 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:37.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:37.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:37.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:36:37.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:37.545 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:37.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:37.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:37.616 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:37.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:37.705 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:37.728 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:37.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:37.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:37.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:37.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:37.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:38.012 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 65 16 87756 87740 84.2796 49.3125 0.0138163 0.0118603 2024-02-22T19:36:38.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:38.085 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:38.116 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:38.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:38.151 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:36:38.188 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:37.413976+0000 mgr.y (mgr.14535) 296 : cluster [DBG] pgmap v332: 158 pgs: 158 active+clean; 5.1 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 74 MiB/s wr, 1.19k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:38.188 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:37.927844+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.53:0/2587003838' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:38.188 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:37.413976+0000 mgr.y (mgr.14535) 296 : cluster [DBG] pgmap v332: 158 pgs: 158 active+clean; 5.1 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 74 MiB/s wr, 1.19k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:38.188 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:37.927844+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.53:0/2587003838' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:38.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:38.286 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:38.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:38.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:37.413976+0000 mgr.y (mgr.14535) 296 : cluster [DBG] pgmap v332: 158 pgs: 158 active+clean; 5.1 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 74 MiB/s wr, 1.19k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:36:38.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:37.927844+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.53:0/2587003838' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:36:38.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:38.448 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:38.554 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:38.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:38.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:38.727 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:38.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:38.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:38.895 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:39.012 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 66 16 88754 88738 83.9477 62.375 0.00387828 0.0119083 2024-02-22T19:36:39.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:38.120647+0000 mon.a (mon.0) 1012 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:39.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:38.132305+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? 172.21.15.53:0/2587003838' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:36:39.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:38.132359+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e178: 8 total, 8 up, 7 in 2024-02-22T19:36:39.135 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:38.359201+0000 mon.a (mon.0) 1015 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:39.374 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:39.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:39.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:39.393 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:39.401 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:39] "GET /metrics HTTP/1.1" 200 175789 "" "Prometheus/2.33.4" 2024-02-22T19:36:39.401 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:38.120647+0000 mon.a (mon.0) 1012 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:39.402 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:38.132305+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? 172.21.15.53:0/2587003838' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:36:39.402 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:38.132359+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e178: 8 total, 8 up, 7 in 2024-02-22T19:36:39.402 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:38.359201+0000 mon.a (mon.0) 1015 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:39.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:39.505 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:36:39.506 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:36:39.506 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-02-22T19:36:39.506 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:36:39.506 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:36:39.507 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-02-22T19:36:39.534 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:38.120647+0000 mon.a (mon.0) 1012 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:39.535 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:38.132305+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? 172.21.15.53:0/2587003838' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:36:39.535 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:38.132359+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e178: 8 total, 8 up, 7 in 2024-02-22T19:36:39.535 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:38.359201+0000 mon.a (mon.0) 1015 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:36:39.547 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:39.867 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:36:40.015 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 67 16 89784 89768 83.6531 64.375 0.0170548 0.0119495 2024-02-22T19:36:40.119 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:40.124 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:40.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:40.154 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:39.416477+0000 mgr.y (mgr.14535) 297 : cluster [DBG] pgmap v334: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:36:40.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:40.265 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:40.342 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:40.345 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:40.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:39.416477+0000 mgr.y (mgr.14535) 297 : cluster [DBG] pgmap v334: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:36:40.405 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:39.416477+0000 mgr.y (mgr.14535) 297 : cluster [DBG] pgmap v334: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:36:40.447 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:40.913 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:40.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:40.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:41.015 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 68 16 90768 90752 83.3269 61.5 0.00766619 0.0119962 2024-02-22T19:36:41.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:41.019 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:90476 8.1 e178/154 8:81174e7e:::benchmark_data_smithi053_130774_object90475:head v 178'5683, mlcod=178'5680)", 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:40.742187+0000", 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.006305667, 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006334653, 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742187+0000", 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742187+0000", 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742186+0000", 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1444999999999999e-05 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742208+0000", 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.19e-06 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742214+0000", 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8659999999999999e-06 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742218+0000", 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6317999999999998e-05 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742284+0000", 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9097999999999999e-05 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.742303+0000", 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011648499999999999 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:90478 8.9 e178/154 8:9156aee7:::benchmark_data_smithi053_130774_object90477:head v 178'5695, mlcod=178'5693)", 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:40.745127+0000", 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0033649019999999999, 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003464286, 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:41.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745127+0000", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745127+0000", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745127+0000", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010001900000000001 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745227+0000", 2024-02-22T19:36:41.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3733e-05 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745240+0000", 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.81e-06 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745247+0000", 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745283+0000", 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4419e-05 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:41.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:40.745298+0000", 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017040600000000001 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 2 2024-02-22T19:36:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:36:41.062 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:41.090 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:41.097 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:41.097 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:41.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:41.258 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:41.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:41.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:41.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:36:41.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:41.440 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:41.520 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:41.523 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:41.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:41.624 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:41.662 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:36:41.662 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-22T19:36:41.662 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:36:41.695 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:41.984 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:41.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:41.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:42.016 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 69 16 91740 91724 82.9994 60.75 0.0209803 0.0120429 2024-02-22T19:36:42.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:42.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:42.354 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:42.413 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:36:42.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:41.416825+0000 mgr.y (mgr.14535) 298 : cluster [DBG] pgmap v335: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 66 MiB/s wr, 1.05k op/s 2024-02-22T19:36:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:41.517783+0000 osd.3 (osd.3) 129 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:41.517837+0000 osd.3 (osd.3) 130 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:42.121555+0000 mon.a (mon.0) 1016 : cluster [DBG] osdmap e179: 8 total, 8 up, 7 in 2024-02-22T19:36:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:41.416825+0000 mgr.y (mgr.14535) 298 : cluster [DBG] pgmap v335: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 66 MiB/s wr, 1.05k op/s 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:41.517783+0000 osd.3 (osd.3) 129 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:41.517837+0000 osd.3 (osd.3) 130 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:42.121555+0000 mon.a (mon.0) 1016 : cluster [DBG] osdmap e179: 8 total, 8 up, 7 in 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:41.416825+0000 mgr.y (mgr.14535) 298 : cluster [DBG] pgmap v335: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 66 MiB/s wr, 1.05k op/s 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:41.517783+0000 osd.3 (osd.3) 129 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:41.517837+0000 osd.3 (osd.3) 130 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:36:42.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:42.121555+0000 mon.a (mon.0) 1016 : cluster [DBG] osdmap e179: 8 total, 8 up, 7 in 2024-02-22T19:36:42.688 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:42.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:42.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:42.844 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:42.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:42.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:42.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:43.006 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:43.024 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 70 16 92658 92642 82.6239 57.375 0.0211224 0.0120962 2024-02-22T19:36:43.031 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:43.113 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:36:43.148 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:36:43.188 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:36:43.226 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:36:43.263 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:36:43.284 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:43.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:43.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:43.312 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:36:43.359 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:36:43.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:43.392 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:36:43.401 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.401 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.401 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.401 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.401 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.402 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.402 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.402 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:36:43.479 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:43.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:43.066642+0000 mon.b (mon.1) 55 : audit [DBG] from='client.? 172.21.15.53:0/2400984792' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:43.554 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:43.066642+0000 mon.b (mon.1) 55 : audit [DBG] from='client.? 172.21.15.53:0/2400984792' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:43.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:43.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:43.066642+0000 mon.b (mon.1) 55 : audit [DBG] from='client.? 172.21.15.53:0/2400984792' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:43.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:43.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:43.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:44.024 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 71 15 93622 93607 82.3098 60.3125 0.0116062 0.0121451 2024-02-22T19:36:44.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:44.099 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:44.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:44.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:44.121 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:44.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:44.237 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:44.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:44.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:44.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:44.432 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:44.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:44.567 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:44.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:44.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:44.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:36:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:36:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:43.417181+0000 mgr.y (mgr.14535) 299 : cluster [DBG] pgmap v337: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 38 MiB/s wr, 606 op/s 2024-02-22T19:36:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:36:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:36:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:43.417181+0000 mgr.y (mgr.14535) 299 : cluster [DBG] pgmap v337: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 38 MiB/s wr, 606 op/s 2024-02-22T19:36:45.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:43.417181+0000 mgr.y (mgr.14535) 299 : cluster [DBG] pgmap v337: 158 pgs: 158 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 38 MiB/s wr, 606 op/s 2024-02-22T19:36:45.024 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 72 16 94618 94602 82.0304 62.1875 0.0109623 0.0121853 2024-02-22T19:36:45.410 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:45.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:45.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:45.514 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:45.578 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:45.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:44.652230+0000 mgr.y (mgr.14535) 300 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:44.652230+0000 mgr.y (mgr.14535) 300 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:44.652230+0000 mgr.y (mgr.14535) 300 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:46.027 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 73 16 95635 95619 81.7751 63.5625 0.0181269 0.0122242 2024-02-22T19:36:46.466 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:46.473 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:46.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:46.574 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:46.631 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:46.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:45.417894+0000 mgr.y (mgr.14535) 301 : cluster [DBG] pgmap v338: 158 pgs: 158 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-02-22T19:36:46.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:45.417894+0000 mgr.y (mgr.14535) 301 : cluster [DBG] pgmap v338: 158 pgs: 158 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-02-22T19:36:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:45.417894+0000 mgr.y (mgr.14535) 301 : cluster [DBG] pgmap v338: 158 pgs: 158 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-02-22T19:36:47.027 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 74 16 96590 96574 81.4768 59.6875 0.0159422 0.0122692 2024-02-22T19:36:47.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:36:47.376 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:36:47.376 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:36:47.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:36:47.531 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:47.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:47.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:47.560 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:36:47.599 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:36:47.599 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:36:47.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:47.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:47.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:47.806 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:47.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:46.638586+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.53:0/156581841' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:36:47.871 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:46.638586+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.53:0/156581841' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:36:47.871 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:46.638586+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.53:0/156581841' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:36:47.881 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:48.016 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:48.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:48.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:48.029 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 75 16 97452 97436 81.1078 53.875 0.0124829 0.012324 2024-02-22T19:36:48.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:48.163 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:48.187 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:48.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:48.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:48.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:48.325 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:48.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:48.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:48.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:48.525 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:47.423028+0000 mgr.y (mgr.14535) 302 : cluster [DBG] pgmap v339: 158 pgs: 158 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:36:48.525 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:47.520903+0000 mon.a (mon.0) 1018 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:48.525 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:47.534033+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? 172.21.15.53:0/156581841' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:36:48.526 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:47.534106+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e180: 8 total, 8 up, 7 in 2024-02-22T19:36:48.602 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:48.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:48.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:48.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:48.784 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:48.788 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:48.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:48.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:48.942 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:47.423028+0000 mgr.y (mgr.14535) 302 : cluster [DBG] pgmap v339: 158 pgs: 158 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:47.520903+0000 mon.a (mon.0) 1018 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:47.534033+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? 172.21.15.53:0/156581841' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:47.534106+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e180: 8 total, 8 up, 7 in 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:47.423028+0000 mgr.y (mgr.14535) 302 : cluster [DBG] pgmap v339: 158 pgs: 158 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:47.520903+0000 mon.a (mon.0) 1018 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:47.534033+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? 172.21.15.53:0/156581841' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:36:48.943 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:47.534106+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e180: 8 total, 8 up, 7 in 2024-02-22T19:36:48.946 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:49.028 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 76 16 98447 98431 80.8591 62.1875 0.00607121 0.0123626 2024-02-22T19:36:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:49.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:49.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:49.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:36:49.172 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:49.227 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:49.348 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:49.352 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:49.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:49.453 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:49.501 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:49] "GET /metrics HTTP/1.1" 200 175789 "" "Prometheus/2.33.4" 2024-02-22T19:36:49.503 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:49.548 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:49.551 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:49.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:49.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:49.721 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:49.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:49.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:47.989597+0000 osd.1 (osd.1) 111 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:47.993322+0000 osd.1 (osd.1) 112 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:48.453213+0000 osd.7 (osd.7) 137 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:48.456655+0000 osd.7 (osd.7) 138 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:48.679782+0000 osd.0 (osd.0) 137 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:48.680720+0000 osd.0 (osd.0) 138 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:36:49.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:47.989597+0000 osd.1 (osd.1) 111 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:36:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:47.993322+0000 osd.1 (osd.1) 112 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:36:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:48.453213+0000 osd.7 (osd.7) 137 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:36:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:48.456655+0000 osd.7 (osd.7) 138 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:36:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:48.679782+0000 osd.0 (osd.0) 137 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:36:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:48.680720+0000 osd.0 (osd.0) 138 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:36:49.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:47.989597+0000 osd.1 (osd.1) 111 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:47.993322+0000 osd.1 (osd.1) 112 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:48.453213+0000 osd.7 (osd.7) 137 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:48.456655+0000 osd.7 (osd.7) 138 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:48.679782+0000 osd.0 (osd.0) 137 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:48.680720+0000 osd.0 (osd.0) 138 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:36:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:49.970 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:49.971 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:49.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:49.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:50.034 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 77 16 99310 99294 80.5042 53.9375 0.0231892 0.0124174 2024-02-22T19:36:50.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:50.137 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:50.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:50.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:50.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:50.542 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:48.966141+0000 osd.1 (osd.1) 113 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:36:50.543 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:48.994995+0000 osd.1 (osd.1) 114 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:36:50.543 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:49.137607+0000 osd.2 (osd.2) 96 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:36:50.619 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:50.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:36:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:49.141848+0000 osd.2 (osd.2) 97 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:36:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:49.412430+0000 osd.6 (osd.6) 81 : cluster [DBG] 8.9 scrub starts 2024-02-22T19:36:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:49.424896+0000 mgr.y (mgr.14535) 303 : cluster [DBG] pgmap v341: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:36:50.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:48.966141+0000 osd.1 (osd.1) 113 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:48.994995+0000 osd.1 (osd.1) 114 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:49.137607+0000 osd.2 (osd.2) 96 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:49.141848+0000 osd.2 (osd.2) 97 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:49.412430+0000 osd.6 (osd.6) 81 : cluster [DBG] 8.9 scrub starts 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:49.424896+0000 mgr.y (mgr.14535) 303 : cluster [DBG] pgmap v341: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:48.966141+0000 osd.1 (osd.1) 113 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:48.994995+0000 osd.1 (osd.1) 114 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:36:50.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:49.137607+0000 osd.2 (osd.2) 96 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:36:50.936 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:49.141848+0000 osd.2 (osd.2) 97 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:36:50.936 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:49.412430+0000 osd.6 (osd.6) 81 : cluster [DBG] 8.9 scrub starts 2024-02-22T19:36:50.936 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:49.424896+0000 mgr.y (mgr.14535) 303 : cluster [DBG] pgmap v341: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:36:51.034 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 78 15 100250 100235 80.2263 58.8125 0.0313887 0.0124602 2024-02-22T19:36:51.594 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:51.598 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:51.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:51.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:51.764 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:51.996 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:51.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:52.035 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 79 16 101236 101220 79.9889 61.5625 0.0134081 0.0124976 2024-02-22T19:36:52.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:52.189 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:52.250 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:36:52.318 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:36:52.352 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:52.360 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:36:52.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:52.378 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:52.394 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:36:52.429 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:36:52.471 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:36:52.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:52.506 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:36:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:52.544 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:36:52.556 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:52.559 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:52.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.569 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:36:52.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:52.730 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:52.791 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:52.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:52.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:51.425339+0000 mgr.y (mgr.14535) 304 : cluster [DBG] pgmap v342: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.1 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-02-22T19:36:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:52.129704+0000 mon.a (mon.0) 1021 : cluster [DBG] osdmap e181: 8 total, 8 up, 7 in 2024-02-22T19:36:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:52.186012+0000 mon.a (mon.0) 1022 : audit [DBG] from='client.? 172.21.15.53:0/2893639511' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:52.460795+0000 osd.7 (osd.7) 139 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:36:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:52.463767+0000 osd.7 (osd.7) 140 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:36:52.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:52.937 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:52.961 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:52.964 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:52.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:51.425339+0000 mgr.y (mgr.14535) 304 : cluster [DBG] pgmap v342: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.1 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:52.129704+0000 mon.a (mon.0) 1021 : cluster [DBG] osdmap e181: 8 total, 8 up, 7 in 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:52.186012+0000 mon.a (mon.0) 1022 : audit [DBG] from='client.? 172.21.15.53:0/2893639511' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:52.460795+0000 osd.7 (osd.7) 139 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:52.463767+0000 osd.7 (osd.7) 140 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:51.425339+0000 mgr.y (mgr.14535) 304 : cluster [DBG] pgmap v342: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.1 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:52.129704+0000 mon.a (mon.0) 1021 : cluster [DBG] osdmap e181: 8 total, 8 up, 7 in 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:52.186012+0000 mon.a (mon.0) 1022 : audit [DBG] from='client.? 172.21.15.53:0/2893639511' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:52.460795+0000 osd.7 (osd.7) 139 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:36:53.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:52.463767+0000 osd.7 (osd.7) 140 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:36:53.035 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:36:53.036210+0000 min lat: 0.00121627 max lat: 0.0881957 avg lat: 0.012541 2024-02-22T19:36:53.035 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:36:53.035 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 80 16 102161 102145 79.7119 57.8125 0.0172235 0.012541 2024-02-22T19:36:53.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:36:53.099 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:53.123 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:36:53.128 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:53.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:36:53.229 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:53.282 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:53.682 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:53.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:53.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:53.685 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:52.260771+0000 osd.6 (osd.6) 82 : cluster [DBG] 8.9 scrub ok 2024-02-22T19:36:53.686 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:53.499632+0000 osd.7 (osd.7) 141 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:36:53.686 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:53.503120+0000 osd.7 (osd.7) 142 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:36:53.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:52.260771+0000 osd.6 (osd.6) 82 : cluster [DBG] 8.9 scrub ok 2024-02-22T19:36:53.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:53.499632+0000 osd.7 (osd.7) 141 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:36:53.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:53.503120+0000 osd.7 (osd.7) 142 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:36:53.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:36:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:36:53.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:52.260771+0000 osd.6 (osd.6) 82 : cluster [DBG] 8.9 scrub ok 2024-02-22T19:36:53.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:53.499632+0000 osd.7 (osd.7) 141 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:36:53.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:53.503120+0000 osd.7 (osd.7) 142 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:36:53.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:53.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:36:53.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:36:54.010 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:54.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:36:54.040 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 81 16 103079 103063 79.4316 57.375 0.0138765 0.0125855 2024-02-22T19:36:54.056 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:36:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:36:54.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:54.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:36:54.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:54.319 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:54.552 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:54.555 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:54.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:54.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:54.716 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:54.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:36:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:53.425769+0000 mgr.y (mgr.14535) 305 : cluster [DBG] pgmap v344: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 639 B/s rd, 37 MiB/s wr, 592 op/s 2024-02-22T19:36:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:53.426814+0000 osd.6 (osd.6) 83 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:36:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:53.427942+0000 osd.6 (osd.6) 84 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:36:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:54.485243+0000 osd.7 (osd.7) 143 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:36:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:54.489503+0000 osd.7 (osd.7) 144 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:36:54.853 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:36:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:36:54.881 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:54.981 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:53.425769+0000 mgr.y (mgr.14535) 305 : cluster [DBG] pgmap v344: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 639 B/s rd, 37 MiB/s wr, 592 op/s 2024-02-22T19:36:54.981 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:53.426814+0000 osd.6 (osd.6) 83 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:36:54.981 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:53.427942+0000 osd.6 (osd.6) 84 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:54.485243+0000 osd.7 (osd.7) 143 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:54.489503+0000 osd.7 (osd.7) 144 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:53.425769+0000 mgr.y (mgr.14535) 305 : cluster [DBG] pgmap v344: 158 pgs: 158 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 639 B/s rd, 37 MiB/s wr, 592 op/s 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:53.426814+0000 osd.6 (osd.6) 83 : cluster [DBG] 5.1f scrub starts 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:53.427942+0000 osd.6 (osd.6) 84 : cluster [DBG] 5.1f scrub ok 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:54.485243+0000 osd.7 (osd.7) 143 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:36:54.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:54.489503+0000 osd.7 (osd.7) 144 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:36:55.040 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 82 16 104069 104053 79.2176 61.875 0.0162424 0.0126199 2024-02-22T19:36:55.430 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:36:55.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:55.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:36:55.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:36:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:36:55.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:55.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:36:55.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:55.763 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:54.015173+0000 mgr.y (mgr.14535) 306 : audit [DBG] from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:54.437586+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:54.441641+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:36:54.662645+0000 mgr.y (mgr.14535) 307 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:54.728748+0000 osd.0 (osd.0) 139 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:54.732852+0000 osd.0 (osd.0) 140 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:54.015173+0000 mgr.y (mgr.14535) 306 : audit [DBG] from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:54.437586+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:54.441641+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:36:54.662645+0000 mgr.y (mgr.14535) 307 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:54.728748+0000 osd.0 (osd.0) 139 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:36:55.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:54.732852+0000 osd.0 (osd.0) 140 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:36:55.779 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:54.015173+0000 mgr.y (mgr.14535) 306 : audit [DBG] from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:36:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:54.437586+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:36:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:54.441641+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:36:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:36:54.662645+0000 mgr.y (mgr.14535) 307 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:36:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:54.728748+0000 osd.0 (osd.0) 139 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:36:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:54.732852+0000 osd.0 (osd.0) 140 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:36:55.929 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:55.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:55.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:56.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:36:56.041 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 83 16 105070 105054 79.0164 62.5625 0.00730921 0.0126515 2024-02-22T19:36:56.074 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:56.097 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:36:56.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:56.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:36:56.166 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:36:56.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:36:56.272 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:56.277 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:36:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:36:56.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:689 5.3 5:c609908c:::notify.5:head [watch ping cookie 94129597101696] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 307.120588532, 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86272942500000005, 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 689 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154336+0000", 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154333+0000", 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4004e-05 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154347+0000", 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3070000000000001e-05 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154370+0000", 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2280000000000004e-06 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154379+0000", 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108694 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154488+0000", 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3376e-05 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154501+0000", 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85773138400000004 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012232+0000", 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055974900000000001 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012792+0000", 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040998229999999998 2024-02-22T19:36:56.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.016892+0000", 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6339000000000002e-05 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.016968+0000", 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6751999999999998e-05 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017065+0000", 2024-02-22T19:36:56.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86272942500000005 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:698 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94129597101696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 307.12019284500002, 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87369409200000003, 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 698 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:36:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154731+0000", 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4479999999999996e-06 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154735+0000", 2024-02-22T19:36:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0008e-05 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154745+0000", 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6600000000000003e-06 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154750+0000", 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7904000000000001e-05 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154798+0000", 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1430000000000001e-05 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154809+0000", 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85813308200000005 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.012942+0000", 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0219000000000001e-05 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013002+0000", 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004174891 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017177+0000", 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7285000000000004e-05 2024-02-22T19:36:56.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017245+0000", 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011180936000000001 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.028426+0000", 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87369409200000003 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:114121 7.6 7:6539d17f:::benchmark_data_smithi053_107660_object24914:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150187+0000", 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 192.12473698599999, 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064960559000000001, 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114121 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150187+0000", 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150187+0000", 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150185+0000", 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3499999999999999e-06 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150193+0000", 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150201+0000", 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1109999999999996e-06 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150208+0000", 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0375000000000003e-05 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150278+0000", 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109256 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150387+0000", 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012612100000000001 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150513+0000", 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064495902999999993 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215009+0000", 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9539e-05 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215039+0000", 2024-02-22T19:36:56.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4793999999999999e-05 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215053+0000", 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7370000000000001e-05 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215111+0000", 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6962000000000003e-05 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215148+0000", 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064960559000000001 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:114127 7.4 7:28c1c1b6:::benchmark_data_smithi053_107660_object24920:head [delete] snapc 0=[] ondisk+write+known_if_redirected e135)", 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150220+0000", 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 192.12470443000001, 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065748500000000001, 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114127 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150220+0000", 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150220+0000", 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150219+0000", 2024-02-22T19:36:56.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9629999999999998e-06 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150222+0000", 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9739999999999999e-06 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150228+0000", 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0260000000000001e-06 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150232+0000", 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8846e-05 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150281+0000", 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112027 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150393+0000", 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124556 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:36:56.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150518+0000", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051992050000000001 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155717+0000", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059667896999999998 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215385+0000", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050588300000000005 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215891+0000", 2024-02-22T19:36:56.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2429999999999999e-05 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215933+0000", 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5162e-05 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215968+0000", 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065748500000000001 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114120 7.e e135/132 7:74debd88:::benchmark_data_smithi053_107660_object24913:head v 135'6933, mlcod=135'6931)", 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.150481+0000", 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 192.12444310800001, 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064697999000000006, 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150481+0000", 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150481+0000", 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150480+0000", 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9550000000000003e-06 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150487+0000", 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5440000000000004e-06 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150497+0000", 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.150503+0000", 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002534798 2024-02-22T19:36:56.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.153038+0000", 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8559999999999998e-05 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.153056+0000", 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062094587999999999 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215151+0000", 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8353000000000001e-05 2024-02-22T19:36:56.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215179+0000", 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064697999000000006 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114130 7.d e135/93 7:be62d22d:::benchmark_data_smithi053_107660_object24923:head v 135'7356, mlcod=135'7354)", 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.155746+0000", 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 192.11917843000001, 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064770929000000005, 2024-02-22T19:36:56.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155746+0000", 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155746+0000", 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155745+0000", 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9790000000000005e-06 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155755+0000", 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3139999999999993e-06 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155764+0000", 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155770+0000", 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1980999999999998e-05 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155802+0000", 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9924000000000001e-05 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155822+0000", 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064662440000000002 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220484+0000", 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2431000000000002e-05 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220517+0000", 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064770929000000005 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114124 7.7 e135/93 7:e882f8ed:::benchmark_data_smithi053_107660_object24917:head v 135'7086, mlcod=135'7084)", 2024-02-22T19:36:56.312 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.155781+0000", 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 192.11914335700001, 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064791314000000003, 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155781+0000", 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155781+0000", 2024-02-22T19:36:56.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155780+0000", 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3529999999999997e-06 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155785+0000", 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1499999999999998e-06 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155790+0000", 2024-02-22T19:36:56.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3279999999999999e-06 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155794+0000", 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847999999999998e-05 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155833+0000", 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9712999999999999e-05 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155853+0000", 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064696482999999999 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220549+0000", 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2838999999999999e-05 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220572+0000", 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064791314000000003 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.24670.0:114129 7.5 e135/93 7:a0f0a8d6:::benchmark_data_smithi053_107660_object24922:head v 135'7146, mlcod=135'7144)", 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:44.155812+0000", 2024-02-22T19:36:56.317 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 192.11911263499999, 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064726803999999999, 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155812+0000", 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155812+0000", 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155811+0000", 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1060000000000002e-06 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155815+0000", 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9440000000000005e-06 2024-02-22T19:36:56.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155823+0000", 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.794e-06 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.155827+0000", 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059616852999999997 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215444+0000", 2024-02-22T19:36:56.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9570999999999999e-05 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.215464+0000", 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050555859999999999 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220519+0000", 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9337000000000001e-05 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:44.220538+0000", 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064726803999999999 2024-02-22T19:36:56.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:35124 8.e 8:71cb1430:::benchmark_data_smithi053_130774_object35123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e160)", 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:35:50.338035+0000", 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.936889612000002, 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.060991628999999999, 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 35124 2024-02-22T19:36:56.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338035+0000", 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338035+0000", 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338034+0000", 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2285000000000001e-05 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338056+0000", 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2829999999999998e-06 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338063+0000", 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7859999999999996e-06 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338067+0000", 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9007999999999999e-05 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338156+0000", 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012462199999999999 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338281+0000", 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015745700000000001 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:36:56.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338438+0000", 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059552850000000003 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.344394+0000", 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010070960000000001 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.345401+0000", 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053525760999999998 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.398927+0000", 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5111000000000001e-05 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.398982+0000", 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4592999999999998e-05 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.399026+0000", 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.060991628999999999 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:35130 8.e 8:73005d48:::benchmark_data_smithi053_130774_object35129:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e160)", 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:35:50.338075+0000", 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.936848775000001, 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062432869000000002, 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 35130 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.328 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338075+0000", 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338075+0000", 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338075+0000", 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5617999999999997e-05 2024-02-22T19:36:56.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338111+0000", 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.392e-06 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338117+0000", 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0669999999999999e-06 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.338122+0000", 2024-02-22T19:36:56.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001603089 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.339725+0000", 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012692199999999999 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.339852+0000", 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014577899999999999 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.339998+0000", 2024-02-22T19:36:56.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054089480000000002 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.345407+0000", 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078007700000000001 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.346187+0000", 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054233805000000003 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.400421+0000", 2024-02-22T19:36:56.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9870000000000002e-05 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.400470+0000", 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7880000000000003e-05 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:35:50.400508+0000", 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062432869000000002 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:66650 8.e 8:74ab633a:::benchmark_data_smithi053_130774_object66649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e169)", 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:16.551176+0000", 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.723748395000001, 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.061708437999999997, 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 66650 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551176+0000", 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551176+0000", 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551175+0000", 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1529999999999998e-05 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551207+0000", 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4660000000000002e-06 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551214+0000", 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1240000000000004e-06 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551219+0000", 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-05 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551272+0000", 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013294199999999999 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551405+0000", 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158126 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.551563+0000", 2024-02-22T19:36:56.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077618390000000004 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.559325+0000", 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050573325000000002 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.609898+0000", 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028898259999999999 2024-02-22T19:36:56.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.612788+0000", 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2222999999999998e-05 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.612840+0000", 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4404999999999999e-05 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.612884+0000", 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.061708437999999997 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:66647 8.9 e169/154 8:9b1ef0aa:::benchmark_data_smithi053_130774_object66646:head v 169'4138, mlcod=169'4136)", 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:16.554184+0000", 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.720740708000001, 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059208945999999998, 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554184+0000", 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554184+0000", 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554183+0000", 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9882e-05 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554213+0000", 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1721e-05 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554224+0000", 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8409999999999999e-06 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554229+0000", 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7849999999999998e-05 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554267+0000", 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0057000000000001e-05 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.554287+0000", 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059054991000000001 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.613342+0000", 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0478e-05 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.613392+0000", 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059208945999999998 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.344 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:66828 8.c 8:3142d8c7:::benchmark_data_smithi053_130774_object66827:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e169)", 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:16.783789+0000", 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.491135563999997, 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064225617999999998, 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 66828 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783789+0000", 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783789+0000", 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783787+0000", 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1641000000000002e-05 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783809+0000", 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0990000000000005e-06 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783818+0000", 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7709999999999997e-06 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783822+0000", 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7691000000000002e-05 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.783860+0000", 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014224100000000001 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.784002+0000", 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.784184+0000", 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050183729999999996 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.789202+0000", 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011701839999999999 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.790372+0000", 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.057533277000000001 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.847906+0000", 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1728000000000003e-05 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.847967+0000", 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6848999999999998e-05 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:16.848014+0000", 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064225617999999998 2024-02-22T19:36:56.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:71285 8.1 e171/154 8:80960965:::benchmark_data_smithi053_130774_object71284:head v 171'4488, mlcod=171'4486)", 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:21.019695+0000", 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.255229026999999, 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.061776339, 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019695+0000", 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019695+0000", 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019694+0000", 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0560999999999998e-05 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019715+0000", 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6009999999999997e-06 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019723+0000", 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.301e-06 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019727+0000", 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5873e-05 2024-02-22T19:36:56.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019763+0000", 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0568999999999999e-05 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.019783+0000", 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078013409999999998 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.027585+0000", 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053886932999999998 2024-02-22T19:36:56.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.081472+0000", 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.061776339 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:71284 8.2 e171/154 8:482c922c:::benchmark_data_smithi053_130774_object71283:head v 171'4387, mlcod=171'4382)", 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:21.022138+0000", 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.252786280999999, 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059211892000000002, 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.022138+0000", 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.022138+0000", 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.022137+0000", 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8745000000000002e-05 2024-02-22T19:36:56.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.022186+0000", 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5559999999999996e-06 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.022196+0000", 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.609e-06 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.022201+0000", 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025305330000000002 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.024732+0000", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9434999999999999e-05 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.024751+0000", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056562007999999997 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.081313+0000", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6730000000000002e-05 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.081350+0000", 2024-02-22T19:36:56.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.059211892000000002 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73402 8.e 8:7caa917d:::benchmark_data_smithi053_130774_object73401:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.188290+0000", 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.086634594000003, 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084095189000000001, 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73402 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.359 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188290+0000", 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188290+0000", 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188288+0000", 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265999999999999e-05 2024-02-22T19:36:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188308+0000", 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5920000000000001e-06 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188317+0000", 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4830000000000001e-06 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188321+0000", 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9437999999999997e-05 2024-02-22T19:36:56.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188361+0000", 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012591499999999999 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188487+0000", 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015208600000000001 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188639+0000", 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.072688672999999995 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.261328+0000", 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035452719999999999 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.264873+0000", 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074139499999999999 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.272287+0000", 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2052999999999997e-05 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.272349+0000", 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6072999999999998e-05 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.272385+0000", 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084095189000000001 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:73403 8.1 e172/154 8:897776c2:::benchmark_data_smithi053_130774_object73402:head v 172'4615, mlcod=172'4612)", 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.188737+0000", 2024-02-22T19:36:56.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.086187131999999, 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076422575000000006, 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188737+0000", 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188737+0000", 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188736+0000", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2169000000000003e-05 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188769+0000", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.070243013000000007 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259012+0000", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0359999999999995e-06 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259021+0000", 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015806699999999999 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259179+0000", 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2172999999999999e-05 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259201+0000", 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059261239999999996 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.265127+0000", 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2703000000000002e-05 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.265160+0000", 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076422575000000006 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:73406 8.2 e172/154 8:445e69a0:::benchmark_data_smithi053_130774_object73405:head v 172'4515, mlcod=172'4513)", 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.189761+0000", 2024-02-22T19:36:56.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.085163532999999, 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.074816993999999998, 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189761+0000", 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189761+0000", 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189760+0000", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5698000000000001e-05 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189816+0000", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068142923999999994 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.257958+0000", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1396000000000001e-05 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.257970+0000", 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020552499999999999 2024-02-22T19:36:56.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.258175+0000", 2024-02-22T19:36:56.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3790000000000001e-05 2024-02-22T19:36:56.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.258199+0000", 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063425790000000001 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.264542+0000", 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5939999999999998e-05 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.264578+0000", 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.074816993999999998 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73409 8.6 8:6faa7536:::benchmark_data_smithi053_130774_object73408:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.191289+0000", 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.083635520999998, 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084104599000000002, 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73409 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.191289+0000", 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.191289+0000", 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:36:56.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.191286+0000", 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5418999999999998e-05 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.191312+0000", 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067925464000000005 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259237+0000", 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3489999999999992e-06 2024-02-22T19:36:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259246+0000", 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021018819999999998 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.261347+0000", 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110015 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.261457+0000", 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013690299999999999 2024-02-22T19:36:56.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6,7", 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.261594+0000", 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010800861 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.272395+0000", 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028782590000000002 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.275274+0000", 2024-02-22T19:36:56.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5519000000000003e-05 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.275309+0000", 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.812e-05 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.275357+0000", 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6188e-05 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.275393+0000", 2024-02-22T19:36:56.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084104599000000002 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73771 8.e 8:71a33bda:::benchmark_data_smithi053_130774_object73770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.657332+0000", 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.617592610999999, 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063529285000000005, 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:36:56.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73771 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.657332+0000", 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.657332+0000", 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:36:56.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.657331+0000", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2249e-05 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.657353+0000", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7049999999999999e-06 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.657361+0000", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2739999999999998e-06 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.657366+0000", 2024-02-22T19:36:56.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028157690000000001 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.660182+0000", 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107602 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.660290+0000", 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014548299999999999 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.660435+0000", 2024-02-22T19:36:56.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033990240000000001 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.663834+0000", 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853900000000002 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.664673+0000", 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.056065915000000001 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:36:56.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.720738+0000", 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3200000000000003e-05 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.720822+0000", 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9249000000000003e-05 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.720861+0000", 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063529285000000005 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.387 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:36:56.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:36:56.388 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:36:57.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:55.289344+0000 osd.3 (osd.3) 131 : cluster [DBG] 3.c scrub starts 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:55.297612+0000 osd.3 (osd.3) 132 : cluster [DBG] 3.c scrub ok 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:55.428805+0000 mgr.y (mgr.14535) 308 : cluster [DBG] pgmap v345: 158 pgs: 158 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:55.680800+0000 osd.0 (osd.0) 141 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:55.686987+0000 osd.0 (osd.0) 142 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:56.455639+0000 osd.7 (osd.7) 145 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:56.463498+0000 osd.7 (osd.7) 146 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:55.289344+0000 osd.3 (osd.3) 131 : cluster [DBG] 3.c scrub starts 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:55.297612+0000 osd.3 (osd.3) 132 : cluster [DBG] 3.c scrub ok 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:55.428805+0000 mgr.y (mgr.14535) 308 : cluster [DBG] pgmap v345: 158 pgs: 158 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:55.680800+0000 osd.0 (osd.0) 141 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:55.686987+0000 osd.0 (osd.0) 142 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:36:57.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:56.455639+0000 osd.7 (osd.7) 145 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:36:57.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:56.463498+0000 osd.7 (osd.7) 146 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:36:57.041 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 84 16 106030 106014 78.7901 60 0.00876355 0.0126885 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:55.289344+0000 osd.3 (osd.3) 131 : cluster [DBG] 3.c scrub starts 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:55.297612+0000 osd.3 (osd.3) 132 : cluster [DBG] 3.c scrub ok 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:55.428805+0000 mgr.y (mgr.14535) 308 : cluster [DBG] pgmap v345: 158 pgs: 158 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:55.680800+0000 osd.0 (osd.0) 141 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:55.686987+0000 osd.0 (osd.0) 142 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:56.455639+0000 osd.7 (osd.7) 145 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:36:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:56.463498+0000 osd.7 (osd.7) 146 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:36:57.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:57.432649+0000 osd.7 (osd.7) 147 : cluster [DBG] 3.d scrub starts 2024-02-22T19:36:57.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:57.438109+0000 osd.7 (osd.7) 148 : cluster [DBG] 3.d scrub ok 2024-02-22T19:36:57.822 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-02-22T19:36:53.425511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6466639524,"num_objects":98865,"num_object_clones":0,"num_object_copies":296595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98865,"num_whiteouts":0,"num_read":1175,"num_read_kb":927,"num_write":99101,"num_write_kb":6315113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70783,"ondisk_log_size":70783,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":20973028,"kb_used_data":18867884,"kb_used_omap":0,"kb_used_meta":2105088,"kb_avail":635243036,"statfs":{"total":671965249536,"available":650488868864,"internally_reserved":0,"allocated":19320713216,"data_stored":19317116521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2155610112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":310771712,"num_objects":4742,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4742,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4742,"num_write_kb":303488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.008134"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121263+0000","last_change":"2024-02-22T19:36:14.806137+0000","last_active":"2024-02-22T19:36:42.121263+0000","last_peered":"2024-02-22T19:36:42.121263+0000","last_clean":"2024-02-22T19:36:42.121263+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:36:42.121263+0000","last_undegraded":"2024-02-22T19:36:42.121263+0000","last_fullsized":"2024-02-22T19:36:42.121263+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.806069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.806069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":152,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131042+0000","last_change":"2024-02-22T19:36:13.837113+0000","last_active":"2024-02-22T19:36:42.131042+0000","last_peered":"2024-02-22T19:36:42.131042+0000","last_clean":"2024-02-22T19:36:42.131042+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:36:42.131042+0000","last_undegraded":"2024-02-22T19:36:42.131042+0000","last_fullsized":"2024-02-22T19:36:42.131042+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:13.836981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:13.836981+0000","last_clean_scrub_stamp":"2024-02-22T19:36:13.836981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":154,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.125593+0000","last_change":"2024-02-22T19:34:44.286165+0000","last_active":"2024-02-22T19:36:42.125593+0000","last_peered":"2024-02-22T19:36:42.125593+0000","last_clean":"2024-02-22T19:36:42.125593+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:36:42.125593+0000","last_undegraded":"2024-02-22T19:36:42.125593+0000","last_fullsized":"2024-02-22T19:36:42.125593+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.286059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.286059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":161,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.120759+0000","last_change":"2024-02-22T19:36:10.678582+0000","last_active":"2024-02-22T19:36:42.120759+0000","last_peered":"2024-02-22T19:36:42.120759+0000","last_clean":"2024-02-22T19:36:42.120759+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:36:42.120759+0000","last_undegraded":"2024-02-22T19:36:42.120759+0000","last_fullsized":"2024-02-22T19:36:42.120759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:10.678491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:10.678491+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":133,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127212+0000","last_change":"2024-02-22T19:34:30.923797+0000","last_active":"2024-02-22T19:36:42.127212+0000","last_peered":"2024-02-22T19:36:42.127212+0000","last_clean":"2024-02-22T19:36:42.127212+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:36:42.127212+0000","last_undegraded":"2024-02-22T19:36:42.127212+0000","last_fullsized":"2024-02-22T19:36:42.127212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.923707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.923707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":138,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.120930+0000","last_change":"2024-02-22T19:35:26.000779+0000","last_active":"2024-02-22T19:36:42.120930+0000","last_peered":"2024-02-22T19:36:42.120930+0000","last_clean":"2024-02-22T19:36:42.120930+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:36:42.120930+0000","last_undegraded":"2024-02-22T19:36:42.120930+0000","last_fullsized":"2024-02-22T19:36:42.120930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.000689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.000689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128259+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:36:42.128259+0000","last_peered":"2024-02-22T19:36:42.128259+0000","last_clean":"2024-02-22T19:36:42.128259+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:36:42.128259+0000","last_undegraded":"2024-02-22T19:36:42.128259+0000","last_fullsized":"2024-02-22T19:36:42.128259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129301+0000","last_change":"2024-02-22T19:34:45.948451+0000","last_active":"2024-02-22T19:36:42.129301+0000","last_peered":"2024-02-22T19:36:42.129301+0000","last_clean":"2024-02-22T19:36:42.129301+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:36:42.129301+0000","last_undegraded":"2024-02-22T19:36:42.129301+0000","last_fullsized":"2024-02-22T19:36:42.129301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:45.948358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:45.948358+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124430+0000","last_change":"2024-02-22T19:35:23.244026+0000","last_active":"2024-02-22T19:36:42.124430+0000","last_peered":"2024-02-22T19:36:42.124430+0000","last_clean":"2024-02-22T19:36:42.124430+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:36:42.124430+0000","last_undegraded":"2024-02-22T19:36:42.124430+0000","last_fullsized":"2024-02-22T19:36:42.124430+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:23.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:23.243945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":102,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124511+0000","last_change":"2024-02-22T19:35:26.232511+0000","last_active":"2024-02-22T19:36:42.124511+0000","last_peered":"2024-02-22T19:36:42.124511+0000","last_clean":"2024-02-22T19:36:42.124511+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:36:42.124511+0000","last_undegraded":"2024-02-22T19:36:42.124511+0000","last_fullsized":"2024-02-22T19:36:42.124511+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.232444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.232444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":87,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134234+0000","last_change":"2024-02-22T19:35:22.835205+0000","last_active":"2024-02-22T19:36:42.134234+0000","last_peered":"2024-02-22T19:36:42.134234+0000","last_clean":"2024-02-22T19:36:42.134234+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:36:42.134234+0000","last_undegraded":"2024-02-22T19:36:42.134234+0000","last_fullsized":"2024-02-22T19:36:42.134234+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.835123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.835123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":161,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121492+0000","last_change":"2024-02-22T19:36:21.735619+0000","last_active":"2024-02-22T19:36:42.121492+0000","last_peered":"2024-02-22T19:36:42.121492+0000","last_clean":"2024-02-22T19:36:42.121492+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:36:42.121492+0000","last_undegraded":"2024-02-22T19:36:42.121492+0000","last_fullsized":"2024-02-22T19:36:42.121492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:21.735513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:21.735513+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":102,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.125752+0000","last_change":"2024-02-22T19:36:08.908709+0000","last_active":"2024-02-22T19:36:42.125752+0000","last_peered":"2024-02-22T19:36:42.125752+0000","last_clean":"2024-02-22T19:36:42.125752+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:36:42.125752+0000","last_undegraded":"2024-02-22T19:36:42.125752+0000","last_fullsized":"2024-02-22T19:36:42.125752+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.908571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.908571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128917+0000","last_change":"2024-02-22T19:36:19.511120+0000","last_active":"2024-02-22T19:36:42.128917+0000","last_peered":"2024-02-22T19:36:42.128917+0000","last_clean":"2024-02-22T19:36:42.128917+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:36:42.128917+0000","last_undegraded":"2024-02-22T19:36:42.128917+0000","last_fullsized":"2024-02-22T19:36:42.128917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.511049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.511049+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":109,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121008+0000","last_change":"2024-02-22T19:35:35.930757+0000","last_active":"2024-02-22T19:36:42.121008+0000","last_peered":"2024-02-22T19:36:42.121008+0000","last_clean":"2024-02-22T19:36:42.121008+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:36:42.121008+0000","last_undegraded":"2024-02-22T19:36:42.121008+0000","last_fullsized":"2024-02-22T19:36:42.121008+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:35:35.930654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.930654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.122237+0000","last_change":"2024-02-22T19:34:46.118383+0000","last_active":"2024-02-22T19:36:42.122237+0000","last_peered":"2024-02-22T19:36:42.122237+0000","last_clean":"2024-02-22T19:36:42.122237+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:36:42.122237+0000","last_undegraded":"2024-02-22T19:36:42.122237+0000","last_fullsized":"2024-02-22T19:36:42.122237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:46.118282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.118282+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121976+0000","last_change":"2024-02-22T19:34:56.107642+0000","last_active":"2024-02-22T19:36:42.121976+0000","last_peered":"2024-02-22T19:36:42.121976+0000","last_clean":"2024-02-22T19:36:42.121976+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:36:42.121976+0000","last_undegraded":"2024-02-22T19:36:42.121976+0000","last_fullsized":"2024-02-22T19:36:42.121976+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:56.107558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:56.107558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131729+0000","last_change":"2024-02-22T19:36:18.889514+0000","last_active":"2024-02-22T19:36:42.131729+0000","last_peered":"2024-02-22T19:36:42.131729+0000","last_clean":"2024-02-22T19:36:42.131729+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:36:42.131729+0000","last_undegraded":"2024-02-22T19:36:42.131729+0000","last_fullsized":"2024-02-22T19:36:42.131729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:18.889425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.889425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124822+0000","last_change":"2024-02-22T19:34:53.279758+0000","last_active":"2024-02-22T19:36:42.124822+0000","last_peered":"2024-02-22T19:36:42.124822+0000","last_clean":"2024-02-22T19:36:42.124822+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:36:42.124822+0000","last_undegraded":"2024-02-22T19:36:42.124822+0000","last_fullsized":"2024-02-22T19:36:42.124822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:53.279662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:34:53.279662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"172'22","reported_seq":179,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134301+0000","last_change":"2024-02-22T19:36:24.021213+0000","last_active":"2024-02-22T19:36:42.134301+0000","last_peered":"2024-02-22T19:36:42.134301+0000","last_clean":"2024-02-22T19:36:42.134301+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:36:42.134301+0000","last_undegraded":"2024-02-22T19:36:42.134301+0000","last_fullsized":"2024-02-22T19:36:42.134301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'22","last_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.021111+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":147,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129594+0000","last_change":"2024-02-22T19:36:28.335837+0000","last_active":"2024-02-22T19:36:42.129594+0000","last_peered":"2024-02-22T19:36:42.129594+0000","last_clean":"2024-02-22T19:36:42.129594+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:36:42.129594+0000","last_undegraded":"2024-02-22T19:36:42.129594+0000","last_fullsized":"2024-02-22T19:36:42.129594+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":138,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121211+0000","last_change":"2024-02-22T19:35:24.048780+0000","last_active":"2024-02-22T19:36:42.121211+0000","last_peered":"2024-02-22T19:36:42.121211+0000","last_clean":"2024-02-22T19:36:42.121211+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:36:42.121211+0000","last_undegraded":"2024-02-22T19:36:42.121211+0000","last_fullsized":"2024-02-22T19:36:42.121211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:24.048696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.048696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":142,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121576+0000","last_change":"2024-02-22T19:34:30.007888+0000","last_active":"2024-02-22T19:36:42.121576+0000","last_peered":"2024-02-22T19:36:42.121576+0000","last_clean":"2024-02-22T19:36:42.121576+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:36:42.121576+0000","last_undegraded":"2024-02-22T19:36:42.121576+0000","last_fullsized":"2024-02-22T19:36:42.121576+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:30.007814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:30.007814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":165,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127912+0000","last_change":"2024-02-22T19:35:36.338814+0000","last_active":"2024-02-22T19:36:42.127912+0000","last_peered":"2024-02-22T19:36:42.127912+0000","last_clean":"2024-02-22T19:36:42.127912+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:36:42.127912+0000","last_undegraded":"2024-02-22T19:36:42.127912+0000","last_fullsized":"2024-02-22T19:36:42.127912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":154,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129452+0000","last_change":"2024-02-22T19:36:06.575957+0000","last_active":"2024-02-22T19:36:42.129452+0000","last_peered":"2024-02-22T19:36:42.129452+0000","last_clean":"2024-02-22T19:36:42.129452+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:36:42.129452+0000","last_undegraded":"2024-02-22T19:36:42.129452+0000","last_fullsized":"2024-02-22T19:36:42.129452+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:36:06.575860+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:36:06.575860+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"137'9","reported_seq":251,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.557242+0000","last_change":"2024-02-22T19:35:28.262175+0000","last_active":"2024-02-22T19:36:47.557242+0000","last_peered":"2024-02-22T19:36:47.557242+0000","last_clean":"2024-02-22T19:36:47.557242+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:36:47.557242+0000","last_undegraded":"2024-02-22T19:36:47.557242+0000","last_fullsized":"2024-02-22T19:36:47.557242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:35:28.262089+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:35:28.262089+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":147,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131094+0000","last_change":"2024-02-22T19:35:30.790879+0000","last_active":"2024-02-22T19:36:42.131094+0000","last_peered":"2024-02-22T19:36:42.131094+0000","last_clean":"2024-02-22T19:36:42.131094+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:36:42.131094+0000","last_undegraded":"2024-02-22T19:36:42.131094+0000","last_fullsized":"2024-02-22T19:36:42.131094+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.790805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.790805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":158,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129744+0000","last_change":"2024-02-22T19:36:04.546799+0000","last_active":"2024-02-22T19:36:42.129744+0000","last_peered":"2024-02-22T19:36:42.129744+0000","last_clean":"2024-02-22T19:36:42.129744+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:36:42.129744+0000","last_undegraded":"2024-02-22T19:36:42.129744+0000","last_fullsized":"2024-02-22T19:36:42.129744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:36:04.546703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.546703+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":152,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:48.995124+0000","last_change":"2024-02-22T19:36:48.995124+0000","last_active":"2024-02-22T19:36:48.995124+0000","last_peered":"2024-02-22T19:36:48.995124+0000","last_clean":"2024-02-22T19:36:48.995124+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:36:48.995124+0000","last_undegraded":"2024-02-22T19:36:48.995124+0000","last_fullsized":"2024-02-22T19:36:48.995124+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.995002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.995002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129939+0000","last_change":"2024-02-22T19:36:22.679579+0000","last_active":"2024-02-22T19:36:42.129939+0000","last_peered":"2024-02-22T19:36:42.129939+0000","last_clean":"2024-02-22T19:36:42.129939+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:36:42.129939+0000","last_undegraded":"2024-02-22T19:36:42.129939+0000","last_fullsized":"2024-02-22T19:36:42.129939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:22.679488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:22.679488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126500+0000","last_change":"2024-02-22T19:34:58.165060+0000","last_active":"2024-02-22T19:36:42.126500+0000","last_peered":"2024-02-22T19:36:42.126500+0000","last_clean":"2024-02-22T19:36:42.126500+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:36:42.126500+0000","last_undegraded":"2024-02-22T19:36:42.126500+0000","last_fullsized":"2024-02-22T19:36:42.126500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:34:58.164960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":214,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134195+0000","last_change":"2024-02-22T19:35:29.242870+0000","last_active":"2024-02-22T19:36:42.134195+0000","last_peered":"2024-02-22T19:36:42.134195+0000","last_clean":"2024-02-22T19:36:42.134195+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:36:42.134195+0000","last_undegraded":"2024-02-22T19:36:42.134195+0000","last_fullsized":"2024-02-22T19:36:42.134195+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:35:29.242798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:29.242798+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129074+0000","last_change":"2024-02-22T19:36:07.626299+0000","last_active":"2024-02-22T19:36:42.129074+0000","last_peered":"2024-02-22T19:36:42.129074+0000","last_clean":"2024-02-22T19:36:42.129074+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:36:42.129074+0000","last_undegraded":"2024-02-22T19:36:42.129074+0000","last_fullsized":"2024-02-22T19:36:42.129074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.626201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.626201+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":140,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133481+0000","last_change":"2024-02-22T19:36:05.177403+0000","last_active":"2024-02-22T19:36:42.133481+0000","last_peered":"2024-02-22T19:36:42.133481+0000","last_clean":"2024-02-22T19:36:42.133481+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:36:42.133481+0000","last_undegraded":"2024-02-22T19:36:42.133481+0000","last_fullsized":"2024-02-22T19:36:42.133481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:05.177297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:05.177297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":154,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.125785+0000","last_change":"2024-02-22T19:35:21.292096+0000","last_active":"2024-02-22T19:36:42.125785+0000","last_peered":"2024-02-22T19:36:42.125785+0000","last_clean":"2024-02-22T19:36:42.125785+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:36:42.125785+0000","last_undegraded":"2024-02-22T19:36:42.125785+0000","last_fullsized":"2024-02-22T19:36:42.125785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:21.292020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:21.292020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126585+0000","last_change":"2024-02-22T19:34:39.037075+0000","last_active":"2024-02-22T19:36:42.126585+0000","last_peered":"2024-02-22T19:36:42.126585+0000","last_clean":"2024-02-22T19:36:42.126585+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:36:42.126585+0000","last_undegraded":"2024-02-22T19:36:42.126585+0000","last_fullsized":"2024-02-22T19:36:42.126585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:34:39.036999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.036999+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":133,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127365+0000","last_change":"2024-02-22T19:34:36.044624+0000","last_active":"2024-02-22T19:36:42.127365+0000","last_peered":"2024-02-22T19:36:42.127365+0000","last_clean":"2024-02-22T19:36:42.127365+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:36:42.127365+0000","last_undegraded":"2024-02-22T19:36:42.127365+0000","last_fullsized":"2024-02-22T19:36:42.127365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.044519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.044519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129293+0000","last_change":"2024-02-22T19:35:33.660316+0000","last_active":"2024-02-22T19:36:42.129293+0000","last_peered":"2024-02-22T19:36:42.129293+0000","last_clean":"2024-02-22T19:36:42.129293+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:36:42.129293+0000","last_undegraded":"2024-02-22T19:36:42.129293+0000","last_fullsized":"2024-02-22T19:36:42.129293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.660254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.660254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":146,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129636+0000","last_change":"2024-02-22T19:35:32.621589+0000","last_active":"2024-02-22T19:36:42.129636+0000","last_peered":"2024-02-22T19:36:42.129636+0000","last_clean":"2024-02-22T19:36:42.129636+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:36:42.129636+0000","last_undegraded":"2024-02-22T19:36:42.129636+0000","last_fullsized":"2024-02-22T19:36:42.129636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:32.621508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:32.621508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":156,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129221+0000","last_change":"2024-02-22T19:34:43.915568+0000","last_active":"2024-02-22T19:36:42.129221+0000","last_peered":"2024-02-22T19:36:42.129221+0000","last_clean":"2024-02-22T19:36:42.129221+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:36:42.129221+0000","last_undegraded":"2024-02-22T19:36:42.129221+0000","last_fullsized":"2024-02-22T19:36:42.129221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:43.915463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.915463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124021+0000","last_change":"2024-02-22T19:34:48.292233+0000","last_active":"2024-02-22T19:36:42.124021+0000","last_peered":"2024-02-22T19:36:42.124021+0000","last_clean":"2024-02-22T19:36:42.124021+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:36:42.124021+0000","last_undegraded":"2024-02-22T19:36:42.124021+0000","last_fullsized":"2024-02-22T19:36:42.124021+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:48.292131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:48.292131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":135,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127130+0000","last_change":"2024-02-22T19:34:41.961882+0000","last_active":"2024-02-22T19:36:42.127130+0000","last_peered":"2024-02-22T19:36:42.127130+0000","last_clean":"2024-02-22T19:36:42.127130+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:36:42.127130+0000","last_undegraded":"2024-02-22T19:36:42.127130+0000","last_fullsized":"2024-02-22T19:36:42.127130+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.961788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.961788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129403+0000","last_change":"2024-02-22T19:34:49.595355+0000","last_active":"2024-02-22T19:36:42.129403+0000","last_peered":"2024-02-22T19:36:42.129403+0000","last_clean":"2024-02-22T19:36:42.129403+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:36:42.129403+0000","last_undegraded":"2024-02-22T19:36:42.129403+0000","last_fullsized":"2024-02-22T19:36:42.129403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:49.595253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:34:49.595253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":161,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133495+0000","last_change":"2024-02-22T19:35:35.855558+0000","last_active":"2024-02-22T19:36:42.133495+0000","last_peered":"2024-02-22T19:36:42.133495+0000","last_clean":"2024-02-22T19:36:42.133495+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:36:42.133495+0000","last_undegraded":"2024-02-22T19:36:42.133495+0000","last_fullsized":"2024-02-22T19:36:42.133495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:35.855449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.855449+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":147,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128995+0000","last_change":"2024-02-22T19:36:19.229925+0000","last_active":"2024-02-22T19:36:42.128995+0000","last_peered":"2024-02-22T19:36:42.128995+0000","last_clean":"2024-02-22T19:36:42.128995+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:36:42.128995+0000","last_undegraded":"2024-02-22T19:36:42.128995+0000","last_fullsized":"2024-02-22T19:36:42.128995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.229816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.229816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":152,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.993417+0000","last_change":"2024-02-22T19:36:47.993417+0000","last_active":"2024-02-22T19:36:47.993417+0000","last_peered":"2024-02-22T19:36:47.993417+0000","last_clean":"2024-02-22T19:36:47.993417+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:36:47.993417+0000","last_undegraded":"2024-02-22T19:36:47.993417+0000","last_fullsized":"2024-02-22T19:36:47.993417+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:47.993328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:47.993328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":147,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121092+0000","last_change":"2024-02-22T19:36:27.685508+0000","last_active":"2024-02-22T19:36:42.121092+0000","last_peered":"2024-02-22T19:36:42.121092+0000","last_clean":"2024-02-22T19:36:42.121092+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:36:42.121092+0000","last_undegraded":"2024-02-22T19:36:42.121092+0000","last_fullsized":"2024-02-22T19:36:42.121092+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.685418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.685418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":159,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.122167+0000","last_change":"2024-02-22T19:34:59.142089+0000","last_active":"2024-02-22T19:36:42.122167+0000","last_peered":"2024-02-22T19:36:42.122167+0000","last_clean":"2024-02-22T19:36:42.122167+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:36:42.122167+0000","last_undegraded":"2024-02-22T19:36:42.122167+0000","last_fullsized":"2024-02-22T19:36:42.122167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:59.141999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:59.141999+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129529+0000","last_change":"2024-02-22T19:36:15.699409+0000","last_active":"2024-02-22T19:36:42.129529+0000","last_peered":"2024-02-22T19:36:42.129529+0000","last_clean":"2024-02-22T19:36:42.129529+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:36:42.129529+0000","last_undegraded":"2024-02-22T19:36:42.129529+0000","last_fullsized":"2024-02-22T19:36:42.129529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.699319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.699319+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"100'9","reported_seq":181,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.564799+0000","last_change":"2024-02-22T19:34:57.925328+0000","last_active":"2024-02-22T19:36:47.564799+0000","last_peered":"2024-02-22T19:36:47.564799+0000","last_clean":"2024-02-22T19:36:47.564799+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:36:47.564799+0000","last_undegraded":"2024-02-22T19:36:47.564799+0000","last_fullsized":"2024-02-22T19:36:47.564799+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:57.925253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.925253+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":140,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129086+0000","last_change":"2024-02-22T19:34:42.438663+0000","last_active":"2024-02-22T19:36:42.129086+0000","last_peered":"2024-02-22T19:36:42.129086+0000","last_clean":"2024-02-22T19:36:42.129086+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:36:42.129086+0000","last_undegraded":"2024-02-22T19:36:42.129086+0000","last_fullsized":"2024-02-22T19:36:42.129086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:42.438557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:34:42.438557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":157,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127428+0000","last_change":"2024-02-22T19:34:32.908841+0000","last_active":"2024-02-22T19:36:42.127428+0000","last_peered":"2024-02-22T19:36:42.127428+0000","last_clean":"2024-02-22T19:36:42.127428+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:36:42.127428+0000","last_undegraded":"2024-02-22T19:36:42.127428+0000","last_fullsized":"2024-02-22T19:36:42.127428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:34:32.908721+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.908721+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121666+0000","last_change":"2024-02-22T19:36:26.761319+0000","last_active":"2024-02-22T19:36:42.121666+0000","last_peered":"2024-02-22T19:36:42.121666+0000","last_clean":"2024-02-22T19:36:42.121666+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:36:42.121666+0000","last_undegraded":"2024-02-22T19:36:42.121666+0000","last_fullsized":"2024-02-22T19:36:42.121666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.761226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":237,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.575504+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:36:47.575504+0000","last_peered":"2024-02-22T19:36:47.575504+0000","last_clean":"2024-02-22T19:36:47.575504+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:36:47.575504+0000","last_undegraded":"2024-02-22T19:36:47.575504+0000","last_fullsized":"2024-02-22T19:36:47.575504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.125817+0000","last_change":"2024-02-22T19:34:41.254550+0000","last_active":"2024-02-22T19:36:42.125817+0000","last_peered":"2024-02-22T19:36:42.125817+0000","last_clean":"2024-02-22T19:36:42.125817+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:36:42.125817+0000","last_undegraded":"2024-02-22T19:36:42.125817+0000","last_fullsized":"2024-02-22T19:36:42.125817+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.254449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.254449+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":170,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:48.456721+0000","last_change":"2024-02-22T19:36:48.456721+0000","last_active":"2024-02-22T19:36:48.456721+0000","last_peered":"2024-02-22T19:36:48.456721+0000","last_clean":"2024-02-22T19:36:48.456721+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:36:48.456721+0000","last_undegraded":"2024-02-22T19:36:48.456721+0000","last_fullsized":"2024-02-22T19:36:48.456721+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134527+0000","last_change":"2024-02-22T19:35:31.810419+0000","last_active":"2024-02-22T19:36:42.134527+0000","last_peered":"2024-02-22T19:36:42.134527+0000","last_clean":"2024-02-22T19:36:42.134527+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:36:42.134527+0000","last_undegraded":"2024-02-22T19:36:42.134527+0000","last_fullsized":"2024-02-22T19:36:42.134527+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.810353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":135,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129549+0000","last_change":"2024-02-22T19:36:09.204798+0000","last_active":"2024-02-22T19:36:42.129549+0000","last_peered":"2024-02-22T19:36:42.129549+0000","last_clean":"2024-02-22T19:36:42.129549+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:36:42.129549+0000","last_undegraded":"2024-02-22T19:36:42.129549+0000","last_fullsized":"2024-02-22T19:36:42.129549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.204726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.204726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128863+0000","last_change":"2024-02-22T19:35:30.620049+0000","last_active":"2024-02-22T19:36:42.128863+0000","last_peered":"2024-02-22T19:36:42.128863+0000","last_clean":"2024-02-22T19:36:42.128863+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:36:42.128863+0000","last_undegraded":"2024-02-22T19:36:42.128863+0000","last_fullsized":"2024-02-22T19:36:42.128863+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.619969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":159,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129818+0000","last_change":"2024-02-22T19:35:27.255099+0000","last_active":"2024-02-22T19:36:42.129818+0000","last_peered":"2024-02-22T19:36:42.129818+0000","last_clean":"2024-02-22T19:36:42.129818+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:36:42.129818+0000","last_undegraded":"2024-02-22T19:36:42.129818+0000","last_fullsized":"2024-02-22T19:36:42.129818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129309+0000","last_change":"2024-02-22T19:35:31.614031+0000","last_active":"2024-02-22T19:36:42.129309+0000","last_peered":"2024-02-22T19:36:42.129309+0000","last_clean":"2024-02-22T19:36:42.129309+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:36:42.129309+0000","last_undegraded":"2024-02-22T19:36:42.129309+0000","last_fullsized":"2024-02-22T19:36:42.129309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.613909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.613909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128353+0000","last_change":"2024-02-22T19:34:31.909139+0000","last_active":"2024-02-22T19:36:42.128353+0000","last_peered":"2024-02-22T19:36:42.128353+0000","last_clean":"2024-02-22T19:36:42.128353+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:36:42.128353+0000","last_undegraded":"2024-02-22T19:36:42.128353+0000","last_fullsized":"2024-02-22T19:36:42.128353+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.909036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.909036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":155,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133292+0000","last_change":"2024-02-22T19:35:25.227250+0000","last_active":"2024-02-22T19:36:42.133292+0000","last_peered":"2024-02-22T19:36:42.133292+0000","last_clean":"2024-02-22T19:36:42.133292+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:36:42.133292+0000","last_undegraded":"2024-02-22T19:36:42.133292+0000","last_fullsized":"2024-02-22T19:36:42.133292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:25.227185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.227185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":168,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129639+0000","last_change":"2024-02-22T19:36:26.647888+0000","last_active":"2024-02-22T19:36:42.129639+0000","last_peered":"2024-02-22T19:36:42.129639+0000","last_clean":"2024-02-22T19:36:42.129639+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:36:42.129639+0000","last_undegraded":"2024-02-22T19:36:42.129639+0000","last_fullsized":"2024-02-22T19:36:42.129639+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:36:25.296484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.296484+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"180'6063","reported_seq":6097,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.184609+0000","last_change":"2024-02-22T19:35:19.745999+0000","last_active":"2024-02-22T19:36:49.184609+0000","last_peered":"2024-02-22T19:36:49.184609+0000","last_clean":"2024-02-22T19:36:49.184609+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:36:49.184609+0000","last_undegraded":"2024-02-22T19:36:49.184609+0000","last_fullsized":"2024-02-22T19:36:49.184609+0000","mapping_epoch":154,"log_start":"160'1900","ondisk_log_start":"160'1900","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4163,"ondisk_log_size":4163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":397344768,"num_objects":6063,"num_object_clones":0,"num_object_copies":18189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6063,"num_write_kb":388032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":107,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133539+0000","last_change":"2024-02-22T19:36:17.151389+0000","last_active":"2024-02-22T19:36:42.133539+0000","last_peered":"2024-02-22T19:36:42.133539+0000","last_clean":"2024-02-22T19:36:42.133539+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:36:42.133539+0000","last_undegraded":"2024-02-22T19:36:42.133539+0000","last_fullsized":"2024-02-22T19:36:42.133539+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.151294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.151294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":160,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129885+0000","last_change":"2024-02-22T19:36:04.642477+0000","last_active":"2024-02-22T19:36:42.129885+0000","last_peered":"2024-02-22T19:36:42.129885+0000","last_clean":"2024-02-22T19:36:42.129885+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:36:42.129885+0000","last_undegraded":"2024-02-22T19:36:42.129885+0000","last_fullsized":"2024-02-22T19:36:42.129885+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:36:04.642361+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.642361+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":155,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129160+0000","last_change":"2024-02-22T19:36:09.623594+0000","last_active":"2024-02-22T19:36:42.129160+0000","last_peered":"2024-02-22T19:36:42.129160+0000","last_clean":"2024-02-22T19:36:42.129160+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:36:42.129160+0000","last_undegraded":"2024-02-22T19:36:42.129160+0000","last_fullsized":"2024-02-22T19:36:42.129160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:36:09.623521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.623521+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":238,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.571278+0000","last_change":"2024-02-22T19:34:39.483818+0000","last_active":"2024-02-22T19:36:47.571278+0000","last_peered":"2024-02-22T19:36:47.571278+0000","last_clean":"2024-02-22T19:36:47.571278+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:36:47.571278+0000","last_undegraded":"2024-02-22T19:36:47.571278+0000","last_fullsized":"2024-02-22T19:36:47.571278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:34:39.483740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.483740+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"180'6294","reported_seq":6328,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.211727+0000","last_change":"2024-02-22T19:35:19.226102+0000","last_active":"2024-02-22T19:36:49.211727+0000","last_peered":"2024-02-22T19:36:49.211727+0000","last_clean":"2024-02-22T19:36:49.211727+0000","last_became_active":"2024-02-22T19:35:19.225793+0000","last_became_peered":"2024-02-22T19:35:19.225793+0000","last_unstale":"2024-02-22T19:36:49.211727+0000","last_undegraded":"2024-02-22T19:36:49.211727+0000","last_fullsized":"2024-02-22T19:36:49.211727+0000","mapping_epoch":154,"log_start":"160'1600","ondisk_log_start":"160'1600","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4694,"ondisk_log_size":4694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":412483584,"num_objects":6294,"num_object_clones":0,"num_object_copies":18882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6294,"num_write_kb":402816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":133,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127287+0000","last_change":"2024-02-22T19:36:11.224751+0000","last_active":"2024-02-22T19:36:42.127287+0000","last_peered":"2024-02-22T19:36:42.127287+0000","last_clean":"2024-02-22T19:36:42.127287+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:36:42.127287+0000","last_undegraded":"2024-02-22T19:36:42.127287+0000","last_fullsized":"2024-02-22T19:36:42.127287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.224646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.224646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129716+0000","last_change":"2024-02-22T19:35:22.515913+0000","last_active":"2024-02-22T19:36:42.129716+0000","last_peered":"2024-02-22T19:36:42.129716+0000","last_clean":"2024-02-22T19:36:42.129716+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:36:42.129716+0000","last_undegraded":"2024-02-22T19:36:42.129716+0000","last_fullsized":"2024-02-22T19:36:42.129716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.515824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.515824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":163,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121316+0000","last_change":"2024-02-22T19:34:33.123549+0000","last_active":"2024-02-22T19:36:42.121316+0000","last_peered":"2024-02-22T19:36:42.121316+0000","last_clean":"2024-02-22T19:36:42.121316+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:36:42.121316+0000","last_undegraded":"2024-02-22T19:36:42.121316+0000","last_fullsized":"2024-02-22T19:36:42.121316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:33.123452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.123452+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133155+0000","last_change":"2024-02-22T19:36:19.039536+0000","last_active":"2024-02-22T19:36:42.133155+0000","last_peered":"2024-02-22T19:36:42.133155+0000","last_clean":"2024-02-22T19:36:42.133155+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:36:42.133155+0000","last_undegraded":"2024-02-22T19:36:42.133155+0000","last_fullsized":"2024-02-22T19:36:42.133155+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.039457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.039457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"180'6263","reported_seq":6297,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.335376+0000","last_change":"2024-02-22T19:35:19.225144+0000","last_active":"2024-02-22T19:36:49.335376+0000","last_peered":"2024-02-22T19:36:49.335376+0000","last_clean":"2024-02-22T19:36:49.335376+0000","last_became_active":"2024-02-22T19:35:19.224823+0000","last_became_peered":"2024-02-22T19:35:19.224823+0000","last_unstale":"2024-02-22T19:36:49.335376+0000","last_undegraded":"2024-02-22T19:36:49.335376+0000","last_fullsized":"2024-02-22T19:36:49.335376+0000","mapping_epoch":154,"log_start":"160'1500","ondisk_log_start":"160'1500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4763,"ondisk_log_size":4763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":410451968,"num_objects":6263,"num_object_clones":0,"num_object_copies":18789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6263,"num_write_kb":400832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":134,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128812+0000","last_change":"2024-02-22T19:34:43.434105+0000","last_active":"2024-02-22T19:36:42.128812+0000","last_peered":"2024-02-22T19:36:42.128812+0000","last_clean":"2024-02-22T19:36:42.128812+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:36:42.128812+0000","last_undegraded":"2024-02-22T19:36:42.128812+0000","last_fullsized":"2024-02-22T19:36:42.128812+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:43.434030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.434030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":152,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133502+0000","last_change":"2024-02-22T19:36:14.029499+0000","last_active":"2024-02-22T19:36:42.133502+0000","last_peered":"2024-02-22T19:36:42.133502+0000","last_clean":"2024-02-22T19:36:42.133502+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:36:42.133502+0000","last_undegraded":"2024-02-22T19:36:42.133502+0000","last_fullsized":"2024-02-22T19:36:42.133502+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.029389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":154,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129645+0000","last_change":"2024-02-22T19:34:46.476131+0000","last_active":"2024-02-22T19:36:42.129645+0000","last_peered":"2024-02-22T19:36:42.129645+0000","last_clean":"2024-02-22T19:36:42.129645+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:36:42.129645+0000","last_undegraded":"2024-02-22T19:36:42.129645+0000","last_fullsized":"2024-02-22T19:36:42.129645+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.476034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.476034+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":173,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133771+0000","last_change":"2024-02-22T19:35:22.253473+0000","last_active":"2024-02-22T19:36:42.133771+0000","last_peered":"2024-02-22T19:36:42.133771+0000","last_clean":"2024-02-22T19:36:42.133771+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:36:42.133771+0000","last_undegraded":"2024-02-22T19:36:42.133771+0000","last_fullsized":"2024-02-22T19:36:42.133771+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:22.253381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.253381+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.7","version":"180'6276","reported_seq":6310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.324818+0000","last_change":"2024-02-22T19:35:19.227534+0000","last_active":"2024-02-22T19:36:49.324818+0000","last_peered":"2024-02-22T19:36:49.324818+0000","last_clean":"2024-02-22T19:36:49.324818+0000","last_became_active":"2024-02-22T19:35:19.227265+0000","last_became_peered":"2024-02-22T19:35:19.227265+0000","last_unstale":"2024-02-22T19:36:49.324818+0000","last_undegraded":"2024-02-22T19:36:49.324818+0000","last_fullsized":"2024-02-22T19:36:49.324818+0000","mapping_epoch":154,"log_start":"160'1700","ondisk_log_start":"160'1700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4576,"ondisk_log_size":4576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":411303936,"num_objects":6276,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6276,"num_write_kb":401664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126481+0000","last_change":"2024-02-22T19:36:17.690399+0000","last_active":"2024-02-22T19:36:42.126481+0000","last_peered":"2024-02-22T19:36:42.126481+0000","last_clean":"2024-02-22T19:36:42.126481+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:36:42.126481+0000","last_undegraded":"2024-02-22T19:36:42.126481+0000","last_fullsized":"2024-02-22T19:36:42.126481+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.690298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.690298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133911+0000","last_change":"2024-02-22T19:35:37.831080+0000","last_active":"2024-02-22T19:36:42.133911+0000","last_peered":"2024-02-22T19:36:42.133911+0000","last_clean":"2024-02-22T19:36:42.133911+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:36:42.133911+0000","last_undegraded":"2024-02-22T19:36:42.133911+0000","last_fullsized":"2024-02-22T19:36:42.133911+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.831003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.831003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127506+0000","last_change":"2024-02-22T19:34:04.892928+0000","last_active":"2024-02-22T19:36:42.127506+0000","last_peered":"2024-02-22T19:36:42.127506+0000","last_clean":"2024-02-22T19:36:42.127506+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:36:42.127506+0000","last_undegraded":"2024-02-22T19:36:42.127506+0000","last_fullsized":"2024-02-22T19:36:42.127506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:04.892837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:04.892837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":178,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124970+0000","last_change":"2024-02-22T19:36:18.621857+0000","last_active":"2024-02-22T19:36:42.124970+0000","last_peered":"2024-02-22T19:36:42.124970+0000","last_clean":"2024-02-22T19:36:42.124970+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:36:42.124970+0000","last_undegraded":"2024-02-22T19:36:42.124970+0000","last_fullsized":"2024-02-22T19:36:42.124970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:18.621747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.621747+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.125967+0000","last_change":"2024-02-22T19:36:25.576957+0000","last_active":"2024-02-22T19:36:42.125967+0000","last_peered":"2024-02-22T19:36:42.125967+0000","last_clean":"2024-02-22T19:36:42.125967+0000","last_became_active":"2024-02-22T19:35:58.432364+0000","last_became_peered":"2024-02-22T19:35:58.432364+0000","last_unstale":"2024-02-22T19:36:42.125967+0000","last_undegraded":"2024-02-22T19:36:42.125967+0000","last_fullsized":"2024-02-22T19:36:42.125967+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"180'6301","reported_seq":6335,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.358024+0000","last_change":"2024-02-22T19:35:19.226141+0000","last_active":"2024-02-22T19:36:49.358024+0000","last_peered":"2024-02-22T19:36:49.358024+0000","last_clean":"2024-02-22T19:36:49.358024+0000","last_became_active":"2024-02-22T19:35:19.225860+0000","last_became_peered":"2024-02-22T19:35:19.225860+0000","last_unstale":"2024-02-22T19:36:49.358024+0000","last_undegraded":"2024-02-22T19:36:49.358024+0000","last_fullsized":"2024-02-22T19:36:49.358024+0000","mapping_epoch":154,"log_start":"160'2400","ondisk_log_start":"160'2400","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3901,"ondisk_log_size":3901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":412942336,"num_objects":6301,"num_object_clones":0,"num_object_copies":18903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6301,"num_write_kb":403264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":133,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128493+0000","last_change":"2024-02-22T19:34:46.954016+0000","last_active":"2024-02-22T19:36:42.128493+0000","last_peered":"2024-02-22T19:36:42.128493+0000","last_clean":"2024-02-22T19:36:42.128493+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:36:42.128493+0000","last_undegraded":"2024-02-22T19:36:42.128493+0000","last_fullsized":"2024-02-22T19:36:42.128493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.953951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.953951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133146+0000","last_change":"2024-02-22T19:34:47.963364+0000","last_active":"2024-02-22T19:36:42.133146+0000","last_peered":"2024-02-22T19:36:42.133146+0000","last_clean":"2024-02-22T19:36:42.133146+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:36:42.133146+0000","last_undegraded":"2024-02-22T19:36:42.133146+0000","last_fullsized":"2024-02-22T19:36:42.133146+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:47.963267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:34:47.963267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":142,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.122051+0000","last_change":"2024-02-22T19:34:32.095049+0000","last_active":"2024-02-22T19:36:42.122051+0000","last_peered":"2024-02-22T19:36:42.122051+0000","last_clean":"2024-02-22T19:36:42.122051+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:36:42.122051+0000","last_undegraded":"2024-02-22T19:36:42.122051+0000","last_fullsized":"2024-02-22T19:36:42.122051+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:32.094985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:32.094985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":287,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.564354+0000","last_change":"2024-02-22T19:35:20.843884+0000","last_active":"2024-02-22T19:36:47.564354+0000","last_peered":"2024-02-22T19:36:47.564354+0000","last_clean":"2024-02-22T19:36:47.564354+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:36:47.564354+0000","last_undegraded":"2024-02-22T19:36:47.564354+0000","last_fullsized":"2024-02-22T19:36:47.564354+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:35:20.843809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:20.843809+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.125265+0000","last_change":"2024-02-22T19:36:07.590613+0000","last_active":"2024-02-22T19:36:42.125265+0000","last_peered":"2024-02-22T19:36:42.125265+0000","last_clean":"2024-02-22T19:36:42.125265+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:36:42.125265+0000","last_undegraded":"2024-02-22T19:36:42.125265+0000","last_fullsized":"2024-02-22T19:36:42.125265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.590509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.590509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128963+0000","last_change":"2024-02-22T19:36:24.596597+0000","last_active":"2024-02-22T19:36:42.128963+0000","last_peered":"2024-02-22T19:36:42.128963+0000","last_clean":"2024-02-22T19:36:42.128963+0000","last_became_active":"2024-02-22T19:35:58.429347+0000","last_became_peered":"2024-02-22T19:35:58.429347+0000","last_unstale":"2024-02-22T19:36:42.128963+0000","last_undegraded":"2024-02-22T19:36:42.128963+0000","last_fullsized":"2024-02-22T19:36:42.128963+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:24.596531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.596531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"180'5968","reported_seq":6002,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.354933+0000","last_change":"2024-02-22T19:35:19.225962+0000","last_active":"2024-02-22T19:36:49.354933+0000","last_peered":"2024-02-22T19:36:49.354933+0000","last_clean":"2024-02-22T19:36:49.354933+0000","last_became_active":"2024-02-22T19:35:19.225604+0000","last_became_peered":"2024-02-22T19:35:19.225604+0000","last_unstale":"2024-02-22T19:36:49.354933+0000","last_undegraded":"2024-02-22T19:36:49.354933+0000","last_fullsized":"2024-02-22T19:36:49.354933+0000","mapping_epoch":154,"log_start":"160'2000","ondisk_log_start":"160'2000","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3968,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":391118848,"num_objects":5968,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5968,"num_write_kb":381952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129829+0000","last_change":"2024-02-22T19:36:08.683152+0000","last_active":"2024-02-22T19:36:42.129829+0000","last_peered":"2024-02-22T19:36:42.129829+0000","last_clean":"2024-02-22T19:36:42.129829+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:36:42.129829+0000","last_undegraded":"2024-02-22T19:36:42.129829+0000","last_fullsized":"2024-02-22T19:36:42.129829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.683006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.683006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":147,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126413+0000","last_change":"2024-02-22T19:35:38.917238+0000","last_active":"2024-02-22T19:36:42.126413+0000","last_peered":"2024-02-22T19:36:42.126413+0000","last_clean":"2024-02-22T19:36:42.126413+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:36:42.126413+0000","last_undegraded":"2024-02-22T19:36:42.126413+0000","last_fullsized":"2024-02-22T19:36:42.126413+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:38.917177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.917177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134814+0000","last_change":"2024-02-22T19:35:28.850161+0000","last_active":"2024-02-22T19:36:42.134814+0000","last_peered":"2024-02-22T19:36:42.134814+0000","last_clean":"2024-02-22T19:36:42.134814+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:36:42.134814+0000","last_undegraded":"2024-02-22T19:36:42.134814+0000","last_fullsized":"2024-02-22T19:36:42.134814+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:28.850036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:35:28.850036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":154,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134727+0000","last_change":"2024-02-22T19:36:15.052075+0000","last_active":"2024-02-22T19:36:42.134727+0000","last_peered":"2024-02-22T19:36:42.134727+0000","last_clean":"2024-02-22T19:36:42.134727+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:36:42.134727+0000","last_undegraded":"2024-02-22T19:36:42.134727+0000","last_fullsized":"2024-02-22T19:36:42.134727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:36:15.051953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.051953+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":160,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.141950+0000","last_change":"2024-02-22T19:36:49.141950+0000","last_active":"2024-02-22T19:36:49.141950+0000","last_peered":"2024-02-22T19:36:49.141950+0000","last_clean":"2024-02-22T19:36:49.141950+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:36:49.141950+0000","last_undegraded":"2024-02-22T19:36:49.141950+0000","last_fullsized":"2024-02-22T19:36:49.141950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:49.141855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:49.141855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"180'6202","reported_seq":6236,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.189188+0000","last_change":"2024-02-22T19:35:19.226967+0000","last_active":"2024-02-22T19:36:49.189188+0000","last_peered":"2024-02-22T19:36:49.189188+0000","last_clean":"2024-02-22T19:36:49.189188+0000","last_became_active":"2024-02-22T19:35:19.226638+0000","last_became_peered":"2024-02-22T19:35:19.226638+0000","last_unstale":"2024-02-22T19:36:49.189188+0000","last_undegraded":"2024-02-22T19:36:49.189188+0000","last_fullsized":"2024-02-22T19:36:49.189188+0000","mapping_epoch":154,"log_start":"160'1500","ondisk_log_start":"160'1500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4702,"ondisk_log_size":4702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406454272,"num_objects":6202,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6202,"num_write_kb":396928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.132268+0000","last_change":"2024-02-22T19:35:22.280512+0000","last_active":"2024-02-22T19:36:42.132268+0000","last_peered":"2024-02-22T19:36:42.132268+0000","last_clean":"2024-02-22T19:36:42.132268+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:36:42.132268+0000","last_undegraded":"2024-02-22T19:36:42.132268+0000","last_fullsized":"2024-02-22T19:36:42.132268+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.280418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.280418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129010+0000","last_change":"2024-02-22T19:36:23.598862+0000","last_active":"2024-02-22T19:36:42.129010+0000","last_peered":"2024-02-22T19:36:42.129010+0000","last_clean":"2024-02-22T19:36:42.129010+0000","last_became_active":"2024-02-22T19:35:57.424387+0000","last_became_peered":"2024-02-22T19:35:57.424387+0000","last_unstale":"2024-02-22T19:36:42.129010+0000","last_undegraded":"2024-02-22T19:36:42.129010+0000","last_fullsized":"2024-02-22T19:36:42.129010+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.598758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.598758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":148,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127635+0000","last_change":"2024-02-22T19:34:50.967533+0000","last_active":"2024-02-22T19:36:42.127635+0000","last_peered":"2024-02-22T19:36:42.127635+0000","last_clean":"2024-02-22T19:36:42.127635+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:36:42.127635+0000","last_undegraded":"2024-02-22T19:36:42.127635+0000","last_fullsized":"2024-02-22T19:36:42.127635+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:50.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:50.967435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":171,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134630+0000","last_change":"2024-02-22T19:35:35.065441+0000","last_active":"2024-02-22T19:36:42.134630+0000","last_peered":"2024-02-22T19:36:42.134630+0000","last_clean":"2024-02-22T19:36:42.134630+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:36:42.134630+0000","last_undegraded":"2024-02-22T19:36:42.134630+0000","last_fullsized":"2024-02-22T19:36:42.134630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:35.065332+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.065332+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129558+0000","last_change":"2024-02-22T19:36:12.513753+0000","last_active":"2024-02-22T19:36:42.129558+0000","last_peered":"2024-02-22T19:36:42.129558+0000","last_clean":"2024-02-22T19:36:42.129558+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:36:42.129558+0000","last_undegraded":"2024-02-22T19:36:42.129558+0000","last_fullsized":"2024-02-22T19:36:42.129558+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:12.513648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:12.513648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":25,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131300+0000","last_change":"2024-02-22T19:35:58.418691+0000","last_active":"2024-02-22T19:36:42.131300+0000","last_peered":"2024-02-22T19:36:42.131300+0000","last_clean":"2024-02-22T19:36:42.131300+0000","last_became_active":"2024-02-22T19:35:58.418408+0000","last_became_peered":"2024-02-22T19:35:58.418408+0000","last_unstale":"2024-02-22T19:36:42.131300+0000","last_undegraded":"2024-02-22T19:36:42.131300+0000","last_fullsized":"2024-02-22T19:36:42.131300+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"180'6109","reported_seq":6143,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.308438+0000","last_change":"2024-02-22T19:35:19.748247+0000","last_active":"2024-02-22T19:36:49.308438+0000","last_peered":"2024-02-22T19:36:49.308438+0000","last_clean":"2024-02-22T19:36:49.308438+0000","last_became_active":"2024-02-22T19:35:19.747933+0000","last_became_peered":"2024-02-22T19:35:19.747933+0000","last_unstale":"2024-02-22T19:36:49.308438+0000","last_undegraded":"2024-02-22T19:36:49.308438+0000","last_fullsized":"2024-02-22T19:36:49.308438+0000","mapping_epoch":154,"log_start":"160'1600","ondisk_log_start":"160'1600","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4509,"ondisk_log_size":4509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":400359424,"num_objects":6109,"num_object_clones":0,"num_object_copies":18327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6109,"num_write_kb":390976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":150,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131248+0000","last_change":"2024-02-22T19:36:23.901614+0000","last_active":"2024-02-22T19:36:42.131248+0000","last_peered":"2024-02-22T19:36:42.131248+0000","last_clean":"2024-02-22T19:36:42.131248+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:36:42.131248+0000","last_undegraded":"2024-02-22T19:36:42.131248+0000","last_fullsized":"2024-02-22T19:36:42.131248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.901512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.901512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":145,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126276+0000","last_change":"2024-02-22T19:35:30.342964+0000","last_active":"2024-02-22T19:36:42.126276+0000","last_peered":"2024-02-22T19:36:42.126276+0000","last_clean":"2024-02-22T19:36:42.126276+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:36:42.126276+0000","last_undegraded":"2024-02-22T19:36:42.126276+0000","last_fullsized":"2024-02-22T19:36:42.126276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.342864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.342864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":615,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.790183+0000","last_change":"2024-02-22T19:36:20.553666+0000","last_active":"2024-02-22T19:36:47.790183+0000","last_peered":"2024-02-22T19:36:47.790183+0000","last_clean":"2024-02-22T19:36:47.790183+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:36:47.790183+0000","last_undegraded":"2024-02-22T19:36:47.790183+0000","last_fullsized":"2024-02-22T19:36:47.790183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:36:20.553554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.553554+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":462,"num_read_kb":457,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":156,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129604+0000","last_change":"2024-02-22T19:34:38.184599+0000","last_active":"2024-02-22T19:36:42.129604+0000","last_peered":"2024-02-22T19:36:42.129604+0000","last_clean":"2024-02-22T19:36:42.129604+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:36:42.129604+0000","last_undegraded":"2024-02-22T19:36:42.129604+0000","last_fullsized":"2024-02-22T19:36:42.129604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:34:38.184521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:38.184521+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":102,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.132618+0000","last_change":"2024-02-22T19:36:15.908805+0000","last_active":"2024-02-22T19:36:42.132618+0000","last_peered":"2024-02-22T19:36:42.132618+0000","last_clean":"2024-02-22T19:36:42.132618+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:36:42.132618+0000","last_undegraded":"2024-02-22T19:36:42.132618+0000","last_fullsized":"2024-02-22T19:36:42.132618+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.908714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.908714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133644+0000","last_change":"2024-02-22T19:36:14.230430+0000","last_active":"2024-02-22T19:36:42.133644+0000","last_peered":"2024-02-22T19:36:42.133644+0000","last_clean":"2024-02-22T19:36:42.133644+0000","last_became_active":"2024-02-22T19:35:58.429215+0000","last_became_peered":"2024-02-22T19:35:58.429215+0000","last_unstale":"2024-02-22T19:36:42.133644+0000","last_undegraded":"2024-02-22T19:36:42.133644+0000","last_fullsized":"2024-02-22T19:36:42.133644+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.230365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"180'6134","reported_seq":6168,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.404559+0000","last_change":"2024-02-22T19:35:19.745745+0000","last_active":"2024-02-22T19:36:49.404559+0000","last_peered":"2024-02-22T19:36:49.404559+0000","last_clean":"2024-02-22T19:36:49.404559+0000","last_became_active":"2024-02-22T19:35:19.745488+0000","last_became_peered":"2024-02-22T19:35:19.745488+0000","last_unstale":"2024-02-22T19:36:49.404559+0000","last_undegraded":"2024-02-22T19:36:49.404559+0000","last_fullsized":"2024-02-22T19:36:49.404559+0000","mapping_epoch":154,"log_start":"160'2200","ondisk_log_start":"160'2200","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3934,"ondisk_log_size":3934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":401997824,"num_objects":6134,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6134,"num_write_kb":392576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":136,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128247+0000","last_change":"2024-02-22T19:34:36.003355+0000","last_active":"2024-02-22T19:36:42.128247+0000","last_peered":"2024-02-22T19:36:42.128247+0000","last_clean":"2024-02-22T19:36:42.128247+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:36:42.128247+0000","last_undegraded":"2024-02-22T19:36:42.128247+0000","last_fullsized":"2024-02-22T19:36:42.128247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.003297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.003297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":156,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131364+0000","last_change":"2024-02-22T19:36:19.862015+0000","last_active":"2024-02-22T19:36:42.131364+0000","last_peered":"2024-02-22T19:36:42.131364+0000","last_clean":"2024-02-22T19:36:42.131364+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:36:42.131364+0000","last_undegraded":"2024-02-22T19:36:42.131364+0000","last_fullsized":"2024-02-22T19:36:42.131364+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.861921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.861921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":206,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124746+0000","last_change":"2024-02-22T19:35:35.426393+0000","last_active":"2024-02-22T19:36:42.124746+0000","last_peered":"2024-02-22T19:36:42.124746+0000","last_clean":"2024-02-22T19:36:42.124746+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:36:42.124746+0000","last_undegraded":"2024-02-22T19:36:42.124746+0000","last_fullsized":"2024-02-22T19:36:42.124746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:35.426296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.426296+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":151,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133855+0000","last_change":"2024-02-22T19:35:31.179969+0000","last_active":"2024-02-22T19:36:42.133855+0000","last_peered":"2024-02-22T19:36:42.133855+0000","last_clean":"2024-02-22T19:36:42.133855+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:36:42.133855+0000","last_undegraded":"2024-02-22T19:36:42.133855+0000","last_fullsized":"2024-02-22T19:36:42.133855+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.179891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.179891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133834+0000","last_change":"2024-02-22T19:36:07.161771+0000","last_active":"2024-02-22T19:36:42.133834+0000","last_peered":"2024-02-22T19:36:42.133834+0000","last_clean":"2024-02-22T19:36:42.133834+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:36:42.133834+0000","last_undegraded":"2024-02-22T19:36:42.133834+0000","last_fullsized":"2024-02-22T19:36:42.133834+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.161538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.161538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"180'6200","reported_seq":6234,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.324519+0000","last_change":"2024-02-22T19:35:19.745825+0000","last_active":"2024-02-22T19:36:49.324519+0000","last_peered":"2024-02-22T19:36:49.324519+0000","last_clean":"2024-02-22T19:36:49.324519+0000","last_became_active":"2024-02-22T19:35:19.745522+0000","last_became_peered":"2024-02-22T19:35:19.745522+0000","last_unstale":"2024-02-22T19:36:49.324519+0000","last_undegraded":"2024-02-22T19:36:49.324519+0000","last_fullsized":"2024-02-22T19:36:49.324519+0000","mapping_epoch":154,"log_start":"160'1500","ondisk_log_start":"160'1500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4700,"ondisk_log_size":4700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406323200,"num_objects":6200,"num_object_clones":0,"num_object_copies":18600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":396800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.124139+0000","last_change":"2024-02-22T19:35:33.417797+0000","last_active":"2024-02-22T19:36:42.124139+0000","last_peered":"2024-02-22T19:36:42.124139+0000","last_clean":"2024-02-22T19:36:42.124139+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:36:42.124139+0000","last_undegraded":"2024-02-22T19:36:42.124139+0000","last_fullsized":"2024-02-22T19:36:42.124139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127843+0000","last_change":"2024-02-22T19:34:33.941965+0000","last_active":"2024-02-22T19:36:42.127843+0000","last_peered":"2024-02-22T19:36:42.127843+0000","last_clean":"2024-02-22T19:36:42.127843+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:36:42.127843+0000","last_undegraded":"2024-02-22T19:36:42.127843+0000","last_fullsized":"2024-02-22T19:36:42.127843+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:33.941905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:33.941905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":181,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133411+0000","last_change":"2024-02-22T19:35:37.886857+0000","last_active":"2024-02-22T19:36:42.133411+0000","last_peered":"2024-02-22T19:36:42.133411+0000","last_clean":"2024-02-22T19:36:42.133411+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:36:42.133411+0000","last_undegraded":"2024-02-22T19:36:42.133411+0000","last_fullsized":"2024-02-22T19:36:42.133411+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:37.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.886787+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":256,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.547313+0000","last_change":"2024-02-22T19:36:23.659056+0000","last_active":"2024-02-22T19:36:47.547313+0000","last_peered":"2024-02-22T19:36:47.547313+0000","last_clean":"2024-02-22T19:36:47.547313+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:36:47.547313+0000","last_undegraded":"2024-02-22T19:36:47.547313+0000","last_fullsized":"2024-02-22T19:36:47.547313+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:36:23.658957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.658957+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127042+0000","last_change":"2024-02-22T19:35:58.422975+0000","last_active":"2024-02-22T19:36:42.127042+0000","last_peered":"2024-02-22T19:36:42.127042+0000","last_clean":"2024-02-22T19:36:42.127042+0000","last_became_active":"2024-02-22T19:35:58.422662+0000","last_became_peered":"2024-02-22T19:35:58.422662+0000","last_unstale":"2024-02-22T19:36:42.127042+0000","last_undegraded":"2024-02-22T19:36:42.127042+0000","last_fullsized":"2024-02-22T19:36:42.127042+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"180'6206","reported_seq":6240,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.131164+0000","last_change":"2024-02-22T19:35:19.746052+0000","last_active":"2024-02-22T19:36:49.131164+0000","last_peered":"2024-02-22T19:36:49.131164+0000","last_clean":"2024-02-22T19:36:49.131164+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:36:49.131164+0000","last_undegraded":"2024-02-22T19:36:49.131164+0000","last_fullsized":"2024-02-22T19:36:49.131164+0000","mapping_epoch":154,"log_start":"160'1700","ondisk_log_start":"160'1700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4506,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406716416,"num_objects":6206,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6206,"num_write_kb":397184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129258+0000","last_change":"2024-02-22T19:36:11.554490+0000","last_active":"2024-02-22T19:36:42.129258+0000","last_peered":"2024-02-22T19:36:42.129258+0000","last_clean":"2024-02-22T19:36:42.129258+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:36:42.129258+0000","last_undegraded":"2024-02-22T19:36:42.129258+0000","last_fullsized":"2024-02-22T19:36:42.129258+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.554382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.554382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":145,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:48.680783+0000","last_change":"2024-02-22T19:36:48.680783+0000","last_active":"2024-02-22T19:36:48.680783+0000","last_peered":"2024-02-22T19:36:48.680783+0000","last_clean":"2024-02-22T19:36:48.680783+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:36:48.680783+0000","last_undegraded":"2024-02-22T19:36:48.680783+0000","last_fullsized":"2024-02-22T19:36:48.680783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.680723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.680723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":193,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121144+0000","last_change":"2024-02-22T19:35:26.969937+0000","last_active":"2024-02-22T19:36:42.121144+0000","last_peered":"2024-02-22T19:36:42.121144+0000","last_clean":"2024-02-22T19:36:42.121144+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:36:42.121144+0000","last_undegraded":"2024-02-22T19:36:42.121144+0000","last_fullsized":"2024-02-22T19:36:42.121144+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:26.969827+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.969827+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":237,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.574348+0000","last_change":"2024-02-22T19:35:25.316903+0000","last_active":"2024-02-22T19:36:47.574348+0000","last_peered":"2024-02-22T19:36:47.574348+0000","last_clean":"2024-02-22T19:36:47.574348+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:36:47.574348+0000","last_undegraded":"2024-02-22T19:36:47.574348+0000","last_fullsized":"2024-02-22T19:36:47.574348+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121031+0000","last_change":"2024-02-22T19:35:58.417513+0000","last_active":"2024-02-22T19:36:42.121031+0000","last_peered":"2024-02-22T19:36:42.121031+0000","last_clean":"2024-02-22T19:36:42.121031+0000","last_became_active":"2024-02-22T19:35:58.417154+0000","last_became_peered":"2024-02-22T19:35:58.417154+0000","last_unstale":"2024-02-22T19:36:42.121031+0000","last_undegraded":"2024-02-22T19:36:42.121031+0000","last_fullsized":"2024-02-22T19:36:42.121031+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"180'6191","reported_seq":6225,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.354376+0000","last_change":"2024-02-22T19:35:19.227135+0000","last_active":"2024-02-22T19:36:49.354376+0000","last_peered":"2024-02-22T19:36:49.354376+0000","last_clean":"2024-02-22T19:36:49.354376+0000","last_became_active":"2024-02-22T19:35:19.226763+0000","last_became_peered":"2024-02-22T19:35:19.226763+0000","last_unstale":"2024-02-22T19:36:49.354376+0000","last_undegraded":"2024-02-22T19:36:49.354376+0000","last_fullsized":"2024-02-22T19:36:49.354376+0000","mapping_epoch":154,"log_start":"160'1500","ondisk_log_start":"160'1500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4691,"ondisk_log_size":4691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":405733376,"num_objects":6191,"num_object_clones":0,"num_object_copies":18573,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6191,"num_write_kb":396224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121349+0000","last_change":"2024-02-22T19:36:20.739814+0000","last_active":"2024-02-22T19:36:42.121349+0000","last_peered":"2024-02-22T19:36:42.121349+0000","last_clean":"2024-02-22T19:36:42.121349+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:36:42.121349+0000","last_undegraded":"2024-02-22T19:36:42.121349+0000","last_fullsized":"2024-02-22T19:36:42.121349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":139,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.129470+0000","last_change":"2024-02-22T19:34:40.000683+0000","last_active":"2024-02-22T19:36:42.129470+0000","last_peered":"2024-02-22T19:36:42.129470+0000","last_clean":"2024-02-22T19:36:42.129470+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:36:42.129470+0000","last_undegraded":"2024-02-22T19:36:42.129470+0000","last_fullsized":"2024-02-22T19:36:42.129470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:40.000583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:40.000583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":176,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126753+0000","last_change":"2024-02-22T19:36:26.284565+0000","last_active":"2024-02-22T19:36:42.126753+0000","last_peered":"2024-02-22T19:36:42.126753+0000","last_clean":"2024-02-22T19:36:42.126753+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:36:42.126753+0000","last_undegraded":"2024-02-22T19:36:42.126753+0000","last_fullsized":"2024-02-22T19:36:42.126753+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.120937+0000","last_change":"2024-02-22T19:35:36.900633+0000","last_active":"2024-02-22T19:36:42.120937+0000","last_peered":"2024-02-22T19:36:42.120937+0000","last_clean":"2024-02-22T19:36:42.120937+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:36:42.120937+0000","last_undegraded":"2024-02-22T19:36:42.120937+0000","last_fullsized":"2024-02-22T19:36:42.120937+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":30,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133594+0000","last_change":"2024-02-22T19:36:20.071572+0000","last_active":"2024-02-22T19:36:42.133594+0000","last_peered":"2024-02-22T19:36:42.133594+0000","last_clean":"2024-02-22T19:36:42.133594+0000","last_became_active":"2024-02-22T19:35:58.418647+0000","last_became_peered":"2024-02-22T19:35:58.418647+0000","last_unstale":"2024-02-22T19:36:42.133594+0000","last_undegraded":"2024-02-22T19:36:42.133594+0000","last_fullsized":"2024-02-22T19:36:42.133594+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.071500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.071500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"180'6047","reported_seq":6081,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.111249+0000","last_change":"2024-02-22T19:35:19.745819+0000","last_active":"2024-02-22T19:36:49.111249+0000","last_peered":"2024-02-22T19:36:49.111249+0000","last_clean":"2024-02-22T19:36:49.111249+0000","last_became_active":"2024-02-22T19:35:19.745509+0000","last_became_peered":"2024-02-22T19:35:19.745509+0000","last_unstale":"2024-02-22T19:36:49.111249+0000","last_undegraded":"2024-02-22T19:36:49.111249+0000","last_fullsized":"2024-02-22T19:36:49.111249+0000","mapping_epoch":154,"log_start":"160'1500","ondisk_log_start":"160'1500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4547,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":396296192,"num_objects":6047,"num_object_clones":0,"num_object_copies":18141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6047,"num_write_kb":387008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131575+0000","last_change":"2024-02-22T19:34:51.981539+0000","last_active":"2024-02-22T19:36:42.131575+0000","last_peered":"2024-02-22T19:36:42.131575+0000","last_clean":"2024-02-22T19:36:42.131575+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:36:42.131575+0000","last_undegraded":"2024-02-22T19:36:42.131575+0000","last_fullsized":"2024-02-22T19:36:42.131575+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:51.981444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:51.981444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":253,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.553191+0000","last_change":"2024-02-22T19:34:43.126256+0000","last_active":"2024-02-22T19:36:47.553191+0000","last_peered":"2024-02-22T19:36:47.553191+0000","last_clean":"2024-02-22T19:36:47.553191+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:36:47.553191+0000","last_undegraded":"2024-02-22T19:36:47.553191+0000","last_fullsized":"2024-02-22T19:36:47.553191+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":144,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.126664+0000","last_change":"2024-02-22T19:35:27.352610+0000","last_active":"2024-02-22T19:36:42.126664+0000","last_peered":"2024-02-22T19:36:42.126664+0000","last_clean":"2024-02-22T19:36:42.126664+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:36:42.126664+0000","last_undegraded":"2024-02-22T19:36:42.126664+0000","last_fullsized":"2024-02-22T19:36:42.126664+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:27.352545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.352545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":37,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.132262+0000","last_change":"2024-02-22T19:36:16.249180+0000","last_active":"2024-02-22T19:36:42.132262+0000","last_peered":"2024-02-22T19:36:42.132262+0000","last_clean":"2024-02-22T19:36:42.132262+0000","last_became_active":"2024-02-22T19:36:16.154859+0000","last_became_peered":"2024-02-22T19:36:16.154859+0000","last_unstale":"2024-02-22T19:36:42.132262+0000","last_undegraded":"2024-02-22T19:36:42.132262+0000","last_fullsized":"2024-02-22T19:36:42.132262+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:35:24.795027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.795027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"180'6096","reported_seq":6130,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.150340+0000","last_change":"2024-02-22T19:35:19.225232+0000","last_active":"2024-02-22T19:36:49.150340+0000","last_peered":"2024-02-22T19:36:49.150340+0000","last_clean":"2024-02-22T19:36:49.150340+0000","last_became_active":"2024-02-22T19:35:19.224941+0000","last_became_peered":"2024-02-22T19:35:19.224941+0000","last_unstale":"2024-02-22T19:36:49.150340+0000","last_undegraded":"2024-02-22T19:36:49.150340+0000","last_fullsized":"2024-02-22T19:36:49.150340+0000","mapping_epoch":154,"log_start":"160'1900","ondisk_log_start":"160'1900","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4196,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":399507456,"num_objects":6096,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6096,"num_write_kb":390144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":142,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127345+0000","last_change":"2024-02-22T19:35:37.337349+0000","last_active":"2024-02-22T19:36:42.127345+0000","last_peered":"2024-02-22T19:36:42.127345+0000","last_clean":"2024-02-22T19:36:42.127345+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:36:42.127345+0000","last_undegraded":"2024-02-22T19:36:42.127345+0000","last_fullsized":"2024-02-22T19:36:42.127345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":97,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121586+0000","last_change":"2024-02-22T19:34:39.082294+0000","last_active":"2024-02-22T19:36:42.121586+0000","last_peered":"2024-02-22T19:36:42.121586+0000","last_clean":"2024-02-22T19:36:42.121586+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:36:42.121586+0000","last_undegraded":"2024-02-22T19:36:42.121586+0000","last_fullsized":"2024-02-22T19:36:42.121586+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:39.082190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.082190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":159,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121010+0000","last_change":"2024-02-22T19:36:08.709226+0000","last_active":"2024-02-22T19:36:42.121010+0000","last_peered":"2024-02-22T19:36:42.121010+0000","last_clean":"2024-02-22T19:36:42.121010+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:36:42.121010+0000","last_undegraded":"2024-02-22T19:36:42.121010+0000","last_fullsized":"2024-02-22T19:36:42.121010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.709163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.709163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":164,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121981+0000","last_change":"2024-02-22T19:36:11.687594+0000","last_active":"2024-02-22T19:36:42.121981+0000","last_peered":"2024-02-22T19:36:42.121981+0000","last_clean":"2024-02-22T19:36:42.121981+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:36:42.121981+0000","last_undegraded":"2024-02-22T19:36:42.121981+0000","last_fullsized":"2024-02-22T19:36:42.121981+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:36:11.687522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.687522+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":30,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.132890+0000","last_change":"2024-02-22T19:36:23.022761+0000","last_active":"2024-02-22T19:36:42.132890+0000","last_peered":"2024-02-22T19:36:42.132890+0000","last_clean":"2024-02-22T19:36:42.132890+0000","last_became_active":"2024-02-22T19:35:58.422620+0000","last_became_peered":"2024-02-22T19:35:58.422620+0000","last_unstale":"2024-02-22T19:36:42.132890+0000","last_undegraded":"2024-02-22T19:36:42.132890+0000","last_fullsized":"2024-02-22T19:36:42.132890+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.022678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.022678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"180'6193","reported_seq":6227,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.206818+0000","last_change":"2024-02-22T19:35:19.745821+0000","last_active":"2024-02-22T19:36:49.206818+0000","last_peered":"2024-02-22T19:36:49.206818+0000","last_clean":"2024-02-22T19:36:49.206818+0000","last_became_active":"2024-02-22T19:35:19.745498+0000","last_became_peered":"2024-02-22T19:35:19.745498+0000","last_unstale":"2024-02-22T19:36:49.206818+0000","last_undegraded":"2024-02-22T19:36:49.206818+0000","last_fullsized":"2024-02-22T19:36:49.206818+0000","mapping_epoch":154,"log_start":"160'1700","ondisk_log_start":"160'1700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4493,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":405864448,"num_objects":6193,"num_object_clones":0,"num_object_copies":18579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6193,"num_write_kb":396352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":141,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.134581+0000","last_change":"2024-02-22T19:36:11.024673+0000","last_active":"2024-02-22T19:36:42.134581+0000","last_peered":"2024-02-22T19:36:42.134581+0000","last_clean":"2024-02-22T19:36:42.134581+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:36:42.134581+0000","last_undegraded":"2024-02-22T19:36:42.134581+0000","last_fullsized":"2024-02-22T19:36:42.134581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.024573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.024573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":143,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.131484+0000","last_change":"2024-02-22T19:36:09.886117+0000","last_active":"2024-02-22T19:36:42.131484+0000","last_peered":"2024-02-22T19:36:42.131484+0000","last_clean":"2024-02-22T19:36:42.131484+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:36:42.131484+0000","last_undegraded":"2024-02-22T19:36:42.131484+0000","last_fullsized":"2024-02-22T19:36:42.131484+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.885998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.885998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":149,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.133627+0000","last_change":"2024-02-22T19:36:28.180370+0000","last_active":"2024-02-22T19:36:42.133627+0000","last_peered":"2024-02-22T19:36:42.133627+0000","last_clean":"2024-02-22T19:36:42.133627+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:36:42.133627+0000","last_undegraded":"2024-02-22T19:36:42.133627+0000","last_fullsized":"2024-02-22T19:36:42.133627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.180297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.180297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":168,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121536+0000","last_change":"2024-02-22T19:34:57.145803+0000","last_active":"2024-02-22T19:36:42.121536+0000","last_peered":"2024-02-22T19:36:42.121536+0000","last_clean":"2024-02-22T19:36:42.121536+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:36:42.121536+0000","last_undegraded":"2024-02-22T19:36:42.121536+0000","last_fullsized":"2024-02-22T19:36:42.121536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:34:57.145716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.145716+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.128721+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:36:42.128721+0000","last_peered":"2024-02-22T19:36:42.128721+0000","last_clean":"2024-02-22T19:36:42.128721+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:36:42.128721+0000","last_undegraded":"2024-02-22T19:36:42.128721+0000","last_fullsized":"2024-02-22T19:36:42.128721+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"180'6130","reported_seq":6164,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:49.391550+0000","last_change":"2024-02-22T19:35:19.229848+0000","last_active":"2024-02-22T19:36:49.391550+0000","last_peered":"2024-02-22T19:36:49.391550+0000","last_clean":"2024-02-22T19:36:49.391550+0000","last_became_active":"2024-02-22T19:35:19.229525+0000","last_became_peered":"2024-02-22T19:35:19.229525+0000","last_unstale":"2024-02-22T19:36:49.391550+0000","last_undegraded":"2024-02-22T19:36:49.391550+0000","last_fullsized":"2024-02-22T19:36:49.391550+0000","mapping_epoch":154,"log_start":"160'2200","ondisk_log_start":"160'2200","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3930,"ondisk_log_size":3930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":401735680,"num_objects":6130,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":392320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":140,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.122076+0000","last_change":"2024-02-22T19:34:54.019363+0000","last_active":"2024-02-22T19:36:42.122076+0000","last_peered":"2024-02-22T19:36:42.122076+0000","last_clean":"2024-02-22T19:36:42.122076+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:36:42.122076+0000","last_undegraded":"2024-02-22T19:36:42.122076+0000","last_fullsized":"2024-02-22T19:36:42.122076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:34:54.019270+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:34:54.019270+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":258,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-02-22T19:36:47.554965+0000","last_change":"2024-02-22T19:36:04.927145+0000","last_active":"2024-02-22T19:36:47.554965+0000","last_peered":"2024-02-22T19:36:47.554965+0000","last_clean":"2024-02-22T19:36:47.554965+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:36:47.554965+0000","last_undegraded":"2024-02-22T19:36:47.554965+0000","last_fullsized":"2024-02-22T19:36:47.554965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:36:04.927065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.927065+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":148,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.127037+0000","last_change":"2024-02-22T19:34:28.943926+0000","last_active":"2024-02-22T19:36:42.127037+0000","last_peered":"2024-02-22T19:36:42.127037+0000","last_clean":"2024-02-22T19:36:42.127037+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:36:42.127037+0000","last_undegraded":"2024-02-22T19:36:42.127037+0000","last_fullsized":"2024-02-22T19:36:42.127037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:34:28.943835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:28.943835+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":155,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-02-22T19:36:42.121183+0000","last_change":"2024-02-22T19:35:33.971551+0000","last_active":"2024-02-22T19:36:42.121183+0000","last_peered":"2024-02-22T19:36:42.121183+0000","last_clean":"2024-02-22T19:36:42.121183+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:36:42.121183+0000","last_undegraded":"2024-02-22T19:36:42.121183+0000","last_fullsized":"2024-02-22T19:36:42.121183+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:33.971446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.971446+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6466633728,"num_objects":98673,"num_object_clones":0,"num_object_copies":296019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98673,"num_write_kb":6315072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19400818688,"data_stored":19400818688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70273,"ondisk_log_size":70273,"up":48,"acting":48,"num_store_stats":7},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":462,"num_read_kb":457,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":7},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1650,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266785,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2294032,"kb_used_data":1984520,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91451120,"statfs":{"total":95995035648,"available":93645946880,"internally_reserved":0,"allocated":2032148480,"data_stored":2031632861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":1.9419999999999999,"15min":1.0229999999999999},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.0739999999999998,"5min":42.597999999999999,"15min":42.597999999999999},"last":5.4009999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.8400000000000001,"15min":0.96299999999999997},"min":{"1min":0.255,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.8659999999999997,"5min":42.155999999999999,"15min":42.155999999999999},"last":0.252}]},{"osd":1,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.796,"15min":0.97699999999999998},"min":{"1min":0.28199999999999997,"5min":0.248,"15min":0.248},"max":{"1min":5.4260000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":2.3029999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.9710000000000001,"15min":1.016},"min":{"1min":0.28699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":4.2190000000000003,"5min":37.938000000000002,"15min":37.938000000000002},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.4870000000000001,"15min":0.86099999999999999},"min":{"1min":0.32100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.165,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.764,"15min":0.98099999999999998},"min":{"1min":0.26400000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.3650000000000002,"5min":41.722000000000001,"15min":41.722000000000001},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.802,"15min":0.99399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.9009999999999998,"5min":41.457000000000001,"15min":41.457000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.651,"15min":0.96099999999999997},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6699999999999999,"5min":41.311999999999998,"15min":41.311999999999998},"last":0.308}]},{"osd":4,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54700000000000004,"15min":0.48699999999999999},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.393,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.499,"15min":0.47299999999999998},"min":{"1min":0.33400000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.8819999999999999,"5min":3.4780000000000002,"15min":3.4780000000000002},"last":0.35899999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.091,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60799999999999998,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0900000000000001,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.36799999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77100000000000002,"15min":0.60599999999999998},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1259999999999999,"5min":12.199999999999999,"15min":12.199999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.825,"5min":12.067,"15min":12.067},"last":0.495}]}]},{"osd":6,"up_from":45,"seq":193273528422,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3049844,"kb_used_data":2752620,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":90695308,"statfs":{"total":95995035648,"available":92871995392,"internally_reserved":0,"allocated":2818682880,"data_stored":2818195001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.548,"15min":0.85899999999999999},"min":{"1min":0.23100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":6.0510000000000002,"5min":17.885000000000002,"15min":17.885000000000002},"last":0.23100000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.389,"15min":0.80800000000000005},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.5939999999999999,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":2.931}]},{"osd":1,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2050000000000001,"15min":0.76900000000000002},"min":{"1min":0.30499999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.931,"5min":11.301,"15min":11.301},"last":2.4769999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.1679999999999999,"15min":0.73899999999999999},"min":{"1min":0.27800000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.516,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.6100000000000001,"15min":0.89300000000000002},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.249},"max":{"1min":7.2910000000000004,"5min":12.859999999999999,"15min":12.859999999999999},"last":2.77},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.411,"15min":0.83399999999999996},"min":{"1min":0.308,"5min":0.25,"15min":0.25},"max":{"1min":6.3890000000000002,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":2.71}]},{"osd":3,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.9490000000000001,"15min":1.0089999999999999},"min":{"1min":0.33100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.9870000000000001,"5min":38.762,"15min":38.762},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":2.2360000000000002,"15min":1.1160000000000001},"min":{"1min":0.33600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.3849999999999998,"5min":35.612000000000002,"15min":35.612000000000002},"last":2.3849999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.31},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.48999999999999999,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.54700000000000004,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.60999999999999999,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":4.0380000000000003,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.59199999999999997,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.0129999999999999,"5min":3.488,"15min":3.488},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.67800000000000005,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5600000000000001,"5min":3.593,"15min":3.593},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3032424,"kb_used_data":2735200,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":90712728,"statfs":{"total":95995035648,"available":92889833472,"internally_reserved":0,"allocated":2800844800,"data_stored":2800305159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":2.0590000000000002,"15min":0.91700000000000004},"min":{"1min":0.20899999999999999,"5min":0.20699999999999999,"15min":0.20499999999999999},"max":{"1min":5.3550000000000004,"5min":15.613,"15min":15.613},"last":4.0389999999999997},{"interface":"front","average":{"1min":1.2330000000000001,"5min":2.1880000000000002,"15min":0.97499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23899999999999999},"max":{"1min":7.3620000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":8.1820000000000004}]},{"osd":2,"last update":"Thu Feb 22 19:36:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":2.4729999999999999,"15min":1.091},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":5.29,"5min":16.503,"15min":16.503},"last":9.0540000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":2.2320000000000002,"15min":1.01},"min":{"1min":0.20200000000000001,"5min":0.192,"15min":0.192},"max":{"1min":2.859,"5min":13.461,"15min":13.461},"last":14.994}]},{"osd":3,"last update":"Thu Feb 22 19:36:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0640000000000001,"5min":2.899,"15min":1.2609999999999999},"min":{"1min":0.26600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":10.734999999999999,"5min":25.145,"15min":25.145},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":2.7599999999999998,"15min":1.2529999999999999},"min":{"1min":0.371,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":4.6349999999999998,"5min":22.25,"15min":22.25},"last":1.6000000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":1.591,"15min":0.93200000000000005},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.669,"5min":15.542999999999999,"15min":15.542999999999999},"last":0.503},{"interface":"front","average":{"1min":1.365,"5min":1.4159999999999999,"15min":0.84599999999999997},"min":{"1min":0.32500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":8.5500000000000007,"5min":8.5500000000000007,"15min":8.5500000000000007},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.6930000000000001,"15min":0.95999999999999996},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.5389999999999997,"5min":15.587999999999999,"15min":15.587999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.5600000000000001,"15min":0.90100000000000002},"min":{"1min":0.34799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.2930000000000001,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":3.8719999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":2.2519999999999998,"5min":1.696,"15min":0.92600000000000005},"min":{"1min":0.35199999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":10.631,"5min":10.631,"15min":10.631},"last":0.625},{"interface":"front","average":{"1min":1.657,"5min":1.8169999999999999,"15min":1},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.5810000000000004,"5min":15.651999999999999,"15min":15.651999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:36:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.538,"15min":0.86499999999999999},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.1299999999999999,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":13.169},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.5389999999999999,"15min":0.877},"min":{"1min":0.45900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":8.5869999999999997,"5min":8.5869999999999997,"15min":8.5869999999999997},"last":0.44600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3064780,"kb_used_data":2763460,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":90680372,"statfs":{"total":95995035648,"available":92856700928,"internally_reserved":0,"allocated":2829783040,"data_stored":2829271867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":1.946,"15min":0.88300000000000001},"min":{"1min":0.245,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":3.5590000000000002,"5min":12.755000000000001,"15min":12.755000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.774,"15min":0.83299999999999996},"min":{"1min":0.255,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":5.9560000000000004,"5min":13.013999999999999,"15min":13.013999999999999},"last":1.49}]},{"osd":2,"last update":"Thu Feb 22 19:36:17 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":2.5590000000000002,"15min":1.0880000000000001},"min":{"1min":0.25800000000000001,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":4.3700000000000001,"5min":19.927,"15min":19.927},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":2.3719999999999999,"15min":1.0409999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.8889999999999998,"5min":17.907,"15min":17.907},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:36:32 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.1880000000000002,"15min":1.006},"min":{"1min":0.27900000000000003,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":6.6109999999999998,"5min":13.06,"15min":13.06},"last":0.55200000000000005},{"interface":"front","average":{"1min":2.4180000000000001,"5min":2.903,"15min":1.256},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":8.8659999999999997,"5min":22.936,"15min":22.936},"last":0.27000000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":1.2989999999999999,"15min":0.80700000000000005},"min":{"1min":0.45900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.419,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.992,"15min":1.044},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89200000000000002,"5min":13.874000000000001,"15min":13.874000000000001},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:36:26 2024","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":2.274,"15min":1.129},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33100000000000002},"max":{"1min":8.8369999999999997,"5min":13.903,"15min":13.903},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.3640000000000001,"15min":0.83199999999999996},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.5230000000000001,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:36:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.4390000000000001,"15min":0.872},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.4609999999999999,"5min":7.0830000000000002,"15min":7.0830000000000002},"last":2.0539999999999998},{"interface":"front","average":{"1min":1.839,"5min":1.5669999999999999,"15min":0.91800000000000004},"min":{"1min":0.40899999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.4550000000000001,"5min":7.5410000000000004,"15min":7.5410000000000004},"last":3.96}]},{"osd":7,"last update":"Thu Feb 22 19:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.381,"15min":0.84099999999999997},"min":{"1min":0.41999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":3.492,"5min":7.0620000000000003,"15min":7.0620000000000003},"last":4.7140000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.45,"15min":0.873},"min":{"1min":0.42899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.4139999999999997,"5min":7.423,"15min":7.423},"last":0.58599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411449,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3051904,"kb_used_data":2746488,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":90693248,"statfs":{"total":95995035648,"available":92869885952,"internally_reserved":0,"allocated":2812403712,"data_stored":2811903663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":1.756,"15min":0.83199999999999996},"min":{"1min":0.249,"5min":0.183,"15min":0.183},"max":{"1min":2.2989999999999999,"5min":16.356999999999999,"15min":16.356999999999999},"last":1.016},{"interface":"front","average":{"1min":1.357,"5min":2.3100000000000001,"15min":1.0169999999999999},"min":{"1min":0.30299999999999999,"5min":0.191,"15min":0.191},"max":{"1min":9.5530000000000008,"5min":18.059999999999999,"15min":18.059999999999999},"last":1.399}]},{"osd":1,"last update":"Thu Feb 22 19:36:07 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":2.024,"15min":0.92700000000000005},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":3.4830000000000001,"5min":10.183,"15min":10.183},"last":1.226},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.577,"15min":0.80000000000000004},"min":{"1min":0.25600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":6.4329999999999998,"5min":12.393000000000001,"15min":12.393000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:36:41 2024","interfaces":[{"interface":"back","average":{"1min":2.1110000000000002,"5min":2.4860000000000002,"15min":1.133},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.6639999999999997,"5min":30.071000000000002,"15min":30.071000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":2.343,"5min":2.895,"15min":1.2549999999999999},"min":{"1min":0.222,"5min":0.192,"15min":0.192},"max":{"1min":7.2160000000000002,"5min":32.119,"15min":32.119},"last":0.94799999999999995}]},{"osd":4,"last update":"Thu Feb 22 19:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.575,"15min":0.90500000000000003},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.2080000000000002,"5min":6.3890000000000002,"15min":6.3890000000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.94699999999999995,"15min":0.70599999999999996},"min":{"1min":0.28100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":2.1219999999999999,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.0049999999999999,"15min":0.749},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.895,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.6639999999999999,"15min":0.95099999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.2410000000000001,"5min":6.4320000000000004,"15min":6.4320000000000004},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.905,"15min":1.038},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.0659999999999998,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.1639999999999999,"15min":0.80600000000000005},"min":{"1min":0.318,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.79,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:36:07 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.73,"15min":0.96699999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.3530000000000002,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":1.117,"15min":0.77100000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.173,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.54200000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149811,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3845912,"kb_used_data":3548688,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":89899240,"statfs":{"total":95995035648,"available":92056821760,"internally_reserved":0,"allocated":3633856512,"data_stored":3633333407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":2.4409999999999998,"15min":1.093},"min":{"1min":0.22,"5min":0.19400000000000001,"15min":0.193},"max":{"1min":3.9020000000000001,"5min":12.929,"15min":12.929},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.954,"5min":2.8959999999999999,"15min":1.272},"min":{"1min":0.29099999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.8710000000000004,"5min":14.831,"15min":14.831},"last":6.4710000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:36:31 2024","interfaces":[{"interface":"back","average":{"1min":2.2250000000000001,"5min":2.927,"15min":1.2749999999999999},"min":{"1min":0.29599999999999999,"5min":0.22,"15min":0.22},"max":{"1min":6.2229999999999999,"5min":18.259,"15min":18.259},"last":2.4769999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":2.1419999999999999,"15min":1.0069999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":6.4539999999999997,"5min":13.154,"15min":13.154},"last":16.943999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.867,"5min":2.7170000000000001,"15min":1.2210000000000001},"min":{"1min":0.35499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":8.2460000000000004,"5min":16.271000000000001,"15min":16.271000000000001},"last":2.1520000000000001},{"interface":"front","average":{"1min":1.335,"5min":2.5019999999999998,"15min":1.139},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":3.6760000000000002,"5min":15.143000000000001,"15min":15.143000000000001},"last":6.2519999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:35:54 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.853,"15min":1.026},"min":{"1min":0.432,"5min":0.249,"15min":0.249},"max":{"1min":4.0780000000000003,"5min":13.680999999999999,"15min":13.680999999999999},"last":6.3869999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":1.55,"15min":0.91600000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3440000000000001,"5min":11.366,"15min":11.366},"last":6.1909999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:36:21 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.752,"15min":0.97099999999999997},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":8.0259999999999998,"5min":12.879,"15min":12.879},"last":2.2389999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.9650000000000001,"15min":1.0389999999999999},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.1070000000000002,"5min":13.707000000000001,"15min":13.707000000000001},"last":2.4369999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.867,"15min":1.0089999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.9640000000000004,"5min":12.898999999999999,"15min":12.898999999999999},"last":2.2749999999999999},{"interface":"front","average":{"1min":1.9530000000000001,"5min":2.2320000000000002,"15min":1.165},"min":{"1min":0.42699999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":9.048,"5min":13.803000000000001,"15min":13.803000000000001},"last":2.504}]},{"osd":7,"last update":"Thu Feb 22 19:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.6779999999999999,"15min":0.94199999999999995},"min":{"1min":0.443,"5min":0.317,"15min":0.317},"max":{"1min":3.8250000000000002,"5min":11.348000000000001,"15min":11.348000000000001},"last":6.2329999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.6879999999999999,"15min":0.96699999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.8359999999999999,"5min":12.846,"15min":12.846},"last":6.2869999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953585,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2634132,"kb_used_data":2336908,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":91111020,"statfs":{"total":95995035648,"available":93297684480,"internally_reserved":0,"allocated":2392993792,"data_stored":2392474563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.5229999999999999,"15min":0.84999999999999998},"min":{"1min":0.246,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.94699999999999995,"5min":24.009,"15min":24.009},"last":6.0810000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":1.5249999999999999,"15min":0.86599999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.8480000000000001,"5min":24.263000000000002,"15min":24.263000000000002},"last":3.5030000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":1.2789999999999999,"15min":0.78600000000000003},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":2.9500000000000002,"5min":6.3029999999999999,"15min":6.3029999999999999},"last":8.3209999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0649999999999999,"15min":0.69399999999999995},"min":{"1min":0.29499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.5939999999999999,"5min":12.445,"15min":12.445},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.3009999999999999,"15min":0.80600000000000005},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.4219999999999999,"5min":23.082000000000001,"15min":23.082000000000001},"last":8.3369999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.627,"15min":0.90000000000000002},"min":{"1min":0.28199999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.2149999999999999,"5min":23.138999999999999,"15min":23.138999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.7,"15min":0.93100000000000005},"min":{"1min":0.29599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.4450000000000001,"5min":13.476000000000001,"15min":13.476000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":1.3280000000000001,"15min":0.79600000000000004},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2470000000000001,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.55300000000000005,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.34399999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.54100000000000004,"15min":0.49299999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.53200000000000003,"15min":0.5},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.2559999999999998,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96599999999999997,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.313,"5min":0.307,"15min":0.307},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.437}]}]},{"osd":5,"up_from":0,"seq":167503724651,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2848784384,"data_stored":2848784384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2808283136,"data_stored":2808283136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2816475136,"data_stored":2816475136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3633315840,"data_stored":3633315840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2406809600,"data_stored":2406809600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2840461312,"data_stored":2840461312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2046689280,"data_stored":2046689280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:36:57.907 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:36:58.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:57.432649+0000 osd.7 (osd.7) 147 : cluster [DBG] 3.d scrub starts 2024-02-22T19:36:58.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:57.438109+0000 osd.7 (osd.7) 148 : cluster [DBG] 3.d scrub ok 2024-02-22T19:36:58.041 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 85 16 107063 107047 78.6228 64.5625 0.00764145 0.0127146 2024-02-22T19:36:58.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:57.432649+0000 osd.7 (osd.7) 147 : cluster [DBG] 3.d scrub starts 2024-02-22T19:36:58.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:57.438109+0000 osd.7 (osd.7) 148 : cluster [DBG] 3.d scrub ok 2024-02-22T19:36:58.380 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:36:58.634 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:57.180421+0000 osd.2 (osd.2) 98 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:36:58.634 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:57.185714+0000 osd.2 (osd.2) 99 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:36:58.635 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:57.429420+0000 mgr.y (mgr.14535) 309 : cluster [DBG] pgmap v346: 158 pgs: 158 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-02-22T19:36:58.635 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:57.180421+0000 osd.2 (osd.2) 98 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:36:58.635 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:57.185714+0000 osd.2 (osd.2) 99 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:36:58.635 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:57.429420+0000 mgr.y (mgr.14535) 309 : cluster [DBG] pgmap v346: 158 pgs: 158 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-02-22T19:36:58.791 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:36:58.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:58.797 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:36:58.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:36:58.961 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:36:59.043 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 86 16 108048 108032 78.4236 61.5625 0.0118468 0.0127468 2024-02-22T19:36:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:57.180421+0000 osd.2 (osd.2) 98 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:36:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:57.185714+0000 osd.2 (osd.2) 99 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:36:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:57.429420+0000 mgr.y (mgr.14535) 309 : cluster [DBG] pgmap v346: 158 pgs: 158 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-02-22T19:36:59.655 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:36:59] "GET /metrics HTTP/1.1" 200 175800 "" "Prometheus/2.33.4" 2024-02-22T19:36:59.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:58.212771+0000 osd.2 (osd.2) 100 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:36:59.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:58.219812+0000 osd.2 (osd.2) 101 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:36:59.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:58.251349+0000 osd.3 (osd.3) 133 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:36:59.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:58.263830+0000 osd.3 (osd.3) 134 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:36:59.656 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:59.442267+0000 osd.7 (osd.7) 149 : cluster [DBG] 8.d scrub starts 2024-02-22T19:36:59.830 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:36:59.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:36:59.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:36:59.937 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:36:59.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:58.212771+0000 osd.2 (osd.2) 100 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:36:59.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:58.219812+0000 osd.2 (osd.2) 101 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:36:59.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:58.251349+0000 osd.3 (osd.3) 133 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:36:59.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:58.263830+0000 osd.3 (osd.3) 134 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:36:59.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:36:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:59.442267+0000 osd.7 (osd.7) 149 : cluster [DBG] 8.d scrub starts 2024-02-22T19:36:59.998 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:00.043 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 87 16 109024 109008 78.2234 61 0.0164188 0.0127793 2024-02-22T19:37:00.070 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:00.072 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:00.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:58.212771+0000 osd.2 (osd.2) 100 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:37:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:58.219812+0000 osd.2 (osd.2) 101 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:37:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:58.251349+0000 osd.3 (osd.3) 133 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:37:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:58.263830+0000 osd.3 (osd.3) 134 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:37:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:36:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:59.442267+0000 osd.7 (osd.7) 149 : cluster [DBG] 8.d scrub starts 2024-02-22T19:37:00.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:00.225 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:00.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:00.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:00.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:00.378 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:00.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:00.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:00.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:00.546 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:00.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:00.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:00.679 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:00.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:58.947436+0000 osd.1 (osd.1) 115 : cluster [DBG] 6.d scrub starts 2024-02-22T19:37:00.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:58.979462+0000 osd.1 (osd.1) 116 : cluster [DBG] 6.d scrub ok 2024-02-22T19:37:00.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:59.253308+0000 osd.3 (osd.3) 135 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:37:00.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:59.298015+0000 osd.3 (osd.3) 136 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:37:00.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:36:59.430259+0000 mgr.y (mgr.14535) 310 : cluster [DBG] pgmap v347: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 60 MiB/s wr, 964 op/s 2024-02-22T19:37:00.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:00.547152+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.53:0/4017316520' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:00.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:00.547873+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:37:00.711 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:00.742 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:00.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:00.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:00.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:58.947436+0000 osd.1 (osd.1) 115 : cluster [DBG] 6.d scrub starts 2024-02-22T19:37:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:58.979462+0000 osd.1 (osd.1) 116 : cluster [DBG] 6.d scrub ok 2024-02-22T19:37:00.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:59.253308+0000 osd.3 (osd.3) 135 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:37:00.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:59.298015+0000 osd.3 (osd.3) 136 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:37:00.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:36:59.430259+0000 mgr.y (mgr.14535) 310 : cluster [DBG] pgmap v347: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 60 MiB/s wr, 964 op/s 2024-02-22T19:37:00.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:00.547152+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.53:0/4017316520' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:00.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:00.547873+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:00.885 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:00.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:00.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:58.947436+0000 osd.1 (osd.1) 115 : cluster [DBG] 6.d scrub starts 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:58.979462+0000 osd.1 (osd.1) 116 : cluster [DBG] 6.d scrub ok 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:59.253308+0000 osd.3 (osd.3) 135 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:59.298015+0000 osd.3 (osd.3) 136 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:36:59.430259+0000 mgr.y (mgr.14535) 310 : cluster [DBG] pgmap v347: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 60 MiB/s wr, 964 op/s 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:00.547152+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.53:0/4017316520' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:00.547873+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:01.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:01.045 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 88 16 109948 109932 77.9888 57.75 0.0195224 0.0128175 2024-02-22T19:37:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:01.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:01.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:01.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:01.227 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:01.252 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:01.256 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:01.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:01.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:01.396 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:01.421 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:01.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:01.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:01.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:01.594 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:01.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:01.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:00.652020+0000 mon.a (mon.0) 1024 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:00.652060+0000 mon.a (mon.0) 1025 : cluster [INF] Cluster is now healthy 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:00.661222+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:00.661290+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e182: 8 total, 8 up, 7 in 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:00.652020+0000 mon.a (mon.0) 1024 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:00.652060+0000 mon.a (mon.0) 1025 : cluster [INF] Cluster is now healthy 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:00.661222+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:37:01.761 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:00.661290+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e182: 8 total, 8 up, 7 in 2024-02-22T19:37:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:00.652020+0000 mon.a (mon.0) 1024 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:37:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:00.652060+0000 mon.a (mon.0) 1025 : cluster [INF] Cluster is now healthy 2024-02-22T19:37:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:00.661222+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:37:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:00.661290+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e182: 8 total, 8 up, 7 in 2024-02-22T19:37:01.879 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:37:01.925 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:37:01.953 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:37:01.996 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:37:02.028 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:37:02.045 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 89 16 110861 110845 77.7539 57.0625 0.0119167 0.0128569 2024-02-22T19:37:02.054 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:37:02.088 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:37:02.134 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.151 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:37:02.831 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:37:02.831 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:37:02.831 INFO:tasks.thrashosds.thrasher:Shrinking pool default.rgw.control 2024-02-22T19:37:02.831 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool default.rgw.control by 10 min 8 2024-02-22T19:37:02.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:37:03.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:01.430669+0000 mgr.y (mgr.14535) 311 : cluster [DBG] pgmap v349: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:37:03.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:01.648421+0000 osd.7 (osd.7) 150 : cluster [DBG] 8.d scrub ok 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:01.784488+0000 osd.1 (osd.1) 117 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:01.784554+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:01.834421+0000 mon.b (mon.1) 56 : audit [DBG] from='client.? 172.21.15.53:0/1850625959' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:02.137264+0000 mon.a (mon.0) 1028 : cluster [DBG] osdmap e183: 8 total, 8 up, 7 in 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:01.430669+0000 mgr.y (mgr.14535) 311 : cluster [DBG] pgmap v349: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:01.648421+0000 osd.7 (osd.7) 150 : cluster [DBG] 8.d scrub ok 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:01.784488+0000 osd.1 (osd.1) 117 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:01.784554+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:01.834421+0000 mon.b (mon.1) 56 : audit [DBG] from='client.? 172.21.15.53:0/1850625959' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:03.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:02.137264+0000 mon.a (mon.0) 1028 : cluster [DBG] osdmap e183: 8 total, 8 up, 7 in 2024-02-22T19:37:03.045 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total time run: 90.0079 2024-02-22T19:37:03.045 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total writes made: 111731 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Write size: 65536 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object size: 65536 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Bandwidth (MB/sec): 77.5842 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Bandwidth: 27.6042 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max bandwidth (MB/sec): 177.25 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min bandwidth (MB/sec): 49.3125 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average IOPS: 1241 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev IOPS: 441.667 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max IOPS: 2836 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min IOPS: 789 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average Latency(s): 0.0128858 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Latency(s): 0.00718789 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max latency(s): 0.0881957 2024-02-22T19:37:03.046 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min latency(s): 0.00121627 2024-02-22T19:37:03.050 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Cleaning up (deleting benchmark objects) 2024-02-22T19:37:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:01.430669+0000 mgr.y (mgr.14535) 311 : cluster [DBG] pgmap v349: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:37:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:01.648421+0000 osd.7 (osd.7) 150 : cluster [DBG] 8.d scrub ok 2024-02-22T19:37:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:01.784488+0000 osd.1 (osd.1) 117 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:01.784554+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:01.834421+0000 mon.b (mon.1) 56 : audit [DBG] from='client.? 172.21.15.53:0/1850625959' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:02.137264+0000 mon.a (mon.0) 1028 : cluster [DBG] osdmap e183: 8 total, 8 up, 7 in 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:01.956753+0000 osd.1 (osd.1) 119 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:01.957847+0000 osd.1 (osd.1) 120 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:02.223912+0000 osd.2 (osd.2) 102 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:02.235952+0000 osd.2 (osd.2) 103 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:02.251823+0000 osd.3 (osd.3) 137 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:02.261533+0000 osd.3 (osd.3) 138 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:03.572413+0000 osd.4 (osd.4) 127 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:37:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:03.578312+0000 osd.4 (osd.4) 128 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:01.956753+0000 osd.1 (osd.1) 119 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:01.957847+0000 osd.1 (osd.1) 120 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:02.223912+0000 osd.2 (osd.2) 102 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:02.235952+0000 osd.2 (osd.2) 103 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:02.251823+0000 osd.3 (osd.3) 137 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:02.261533+0000 osd.3 (osd.3) 138 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:03.572413+0000 osd.4 (osd.4) 127 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:37:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:03.578312+0000 osd.4 (osd.4) 128 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:37:04.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:37:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:01.956753+0000 osd.1 (osd.1) 119 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:37:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:01.957847+0000 osd.1 (osd.1) 120 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:37:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:02.223912+0000 osd.2 (osd.2) 102 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:37:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:02.235952+0000 osd.2 (osd.2) 103 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:37:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:02.251823+0000 osd.3 (osd.3) 137 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:37:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:02.261533+0000 osd.3 (osd.3) 138 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:37:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:03.572413+0000 osd.4 (osd.4) 127 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:37:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:03.578312+0000 osd.4 (osd.4) 128 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:37:05.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:03.431079+0000 mgr.y (mgr.14535) 312 : cluster [DBG] pgmap v351: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 39 MiB/s wr, 623 op/s 2024-02-22T19:37:05.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:03.431079+0000 mgr.y (mgr.14535) 312 : cluster [DBG] pgmap v351: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 39 MiB/s wr, 623 op/s 2024-02-22T19:37:05.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:37:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:37:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:03.431079+0000 mgr.y (mgr.14535) 312 : cluster [DBG] pgmap v351: 158 pgs: 158 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 39 MiB/s wr, 623 op/s 2024-02-22T19:37:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:37:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:37:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:03.962275+0000 osd.1 (osd.1) 121 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:37:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:03.977842+0000 osd.1 (osd.1) 122 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:37:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:04.429482+0000 osd.6 (osd.6) 87 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:37:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:04.431651+0000 osd.6 (osd.6) 88 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:37:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:04.673127+0000 mgr.y (mgr.14535) 313 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:04.696198+0000 osd.0 (osd.0) 143 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:37:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:04.719699+0000 osd.0 (osd.0) 144 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:37:06.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:03.962275+0000 osd.1 (osd.1) 121 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:37:06.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:03.977842+0000 osd.1 (osd.1) 122 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:37:06.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:04.429482+0000 osd.6 (osd.6) 87 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:37:06.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:04.431651+0000 osd.6 (osd.6) 88 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:37:06.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:04.673127+0000 mgr.y (mgr.14535) 313 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:06.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:04.696198+0000 osd.0 (osd.0) 143 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:37:06.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:04.719699+0000 osd.0 (osd.0) 144 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:03.962275+0000 osd.1 (osd.1) 121 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:03.977842+0000 osd.1 (osd.1) 122 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:04.429482+0000 osd.6 (osd.6) 87 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:04.431651+0000 osd.6 (osd.6) 88 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:04.673127+0000 mgr.y (mgr.14535) 313 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:04.696198+0000 osd.0 (osd.0) 143 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:37:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:04.719699+0000 osd.0 (osd.0) 144 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:04.953331+0000 osd.1 (osd.1) 123 : cluster [DBG] 5.15 deep-scrub starts 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:04.973723+0000 osd.1 (osd.1) 124 : cluster [DBG] 5.15 deep-scrub ok 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:05.348021+0000 osd.3 (osd.3) 139 : cluster [DBG] 3.b scrub starts 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:05.357873+0000 osd.3 (osd.3) 140 : cluster [DBG] 3.b scrub ok 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:05.433607+0000 mgr.y (mgr.14535) 314 : cluster [DBG] pgmap v352: 158 pgs: 158 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.4 KiB/s rd, 66 MiB/s wr, 1.22k op/s 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:05.638650+0000 osd.4 (osd.4) 129 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:37:07.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:05.645696+0000 osd.4 (osd.4) 130 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:37:07.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:04.953331+0000 osd.1 (osd.1) 123 : cluster [DBG] 5.15 deep-scrub starts 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:04.973723+0000 osd.1 (osd.1) 124 : cluster [DBG] 5.15 deep-scrub ok 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:05.348021+0000 osd.3 (osd.3) 139 : cluster [DBG] 3.b scrub starts 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:05.357873+0000 osd.3 (osd.3) 140 : cluster [DBG] 3.b scrub ok 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:05.433607+0000 mgr.y (mgr.14535) 314 : cluster [DBG] pgmap v352: 158 pgs: 158 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.4 KiB/s rd, 66 MiB/s wr, 1.22k op/s 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:05.638650+0000 osd.4 (osd.4) 129 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:05.645696+0000 osd.4 (osd.4) 130 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:04.953331+0000 osd.1 (osd.1) 123 : cluster [DBG] 5.15 deep-scrub starts 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:04.973723+0000 osd.1 (osd.1) 124 : cluster [DBG] 5.15 deep-scrub ok 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:05.348021+0000 osd.3 (osd.3) 139 : cluster [DBG] 3.b scrub starts 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:05.357873+0000 osd.3 (osd.3) 140 : cluster [DBG] 3.b scrub ok 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:05.433607+0000 mgr.y (mgr.14535) 314 : cluster [DBG] pgmap v352: 158 pgs: 158 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.4 KiB/s rd, 66 MiB/s wr, 1.22k op/s 2024-02-22T19:37:07.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:05.638650+0000 osd.4 (osd.4) 129 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:37:07.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:05.645696+0000 osd.4 (osd.4) 130 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:37:07.399 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:07.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:07.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:07.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:07.554 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:07.583 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:07.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:07.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:07.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:07.725 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:07.745 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:07.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:07.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:07.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:07.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:06.311703+0000 osd.3 (osd.3) 141 : cluster [DBG] 3.7 deep-scrub starts 2024-02-22T19:37:07.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:06.317119+0000 osd.3 (osd.3) 142 : cluster [DBG] 3.7 deep-scrub ok 2024-02-22T19:37:07.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:06.486312+0000 osd.6 (osd.6) 89 : cluster [DBG] 8.5 scrub starts 2024-02-22T19:37:07.886 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:07.914 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:07.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:07.920 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:08.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:08.076 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:08.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:08.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:08.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:08.236 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:08.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:06.311703+0000 osd.3 (osd.3) 141 : cluster [DBG] 3.7 deep-scrub starts 2024-02-22T19:37:08.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:06.317119+0000 osd.3 (osd.3) 142 : cluster [DBG] 3.7 deep-scrub ok 2024-02-22T19:37:08.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:06.486312+0000 osd.6 (osd.6) 89 : cluster [DBG] 8.5 scrub starts 2024-02-22T19:37:08.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:06.311703+0000 osd.3 (osd.3) 141 : cluster [DBG] 3.7 deep-scrub starts 2024-02-22T19:37:08.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:06.317119+0000 osd.3 (osd.3) 142 : cluster [DBG] 3.7 deep-scrub ok 2024-02-22T19:37:08.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:06.486312+0000 osd.6 (osd.6) 89 : cluster [DBG] 8.5 scrub starts 2024-02-22T19:37:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:07.433984+0000 mgr.y (mgr.14535) 315 : cluster [DBG] pgmap v353: 158 pgs: 158 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 767 B/s rd, 27 MiB/s wr, 598 op/s 2024-02-22T19:37:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:07.737800+0000 osd.6 (osd.6) 90 : cluster [DBG] 8.5 scrub ok 2024-02-22T19:37:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:08.353983+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.451520+0000 osd.7 (osd.7) 151 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:37:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.454838+0000 osd.7 (osd.7) 152 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:37:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.578970+0000 osd.4 (osd.4) 131 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:37:09.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.583918+0000 osd.4 (osd.4) 132 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:07.433984+0000 mgr.y (mgr.14535) 315 : cluster [DBG] pgmap v353: 158 pgs: 158 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 767 B/s rd, 27 MiB/s wr, 598 op/s 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:07.737800+0000 osd.6 (osd.6) 90 : cluster [DBG] 8.5 scrub ok 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:08.353983+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.451520+0000 osd.7 (osd.7) 151 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.454838+0000 osd.7 (osd.7) 152 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.578970+0000 osd.4 (osd.4) 131 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.583918+0000 osd.4 (osd.4) 132 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:37:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:07.433984+0000 mgr.y (mgr.14535) 315 : cluster [DBG] pgmap v353: 158 pgs: 158 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 767 B/s rd, 27 MiB/s wr, 598 op/s 2024-02-22T19:37:09.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:07.737800+0000 osd.6 (osd.6) 90 : cluster [DBG] 8.5 scrub ok 2024-02-22T19:37:09.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:08.353983+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:09.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.451520+0000 osd.7 (osd.7) 151 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:37:09.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.454838+0000 osd.7 (osd.7) 152 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:37:09.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.578970+0000 osd.4 (osd.4) 131 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:37:09.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.583918+0000 osd.4 (osd.4) 132 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:37:09.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:09] "GET /metrics HTTP/1.1" 200 175804 "" "Prometheus/2.33.4" 2024-02-22T19:37:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.030926+0000 osd.1 (osd.1) 125 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:37:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.036665+0000 osd.1 (osd.1) 126 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:37:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.790052+0000 osd.0 (osd.0) 145 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:37:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.467271+0000 osd.7 (osd.7) 153 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:37:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.470143+0000 osd.7 (osd.7) 154 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:37:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.030926+0000 osd.1 (osd.1) 125 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:37:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.036665+0000 osd.1 (osd.1) 126 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:37:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.790052+0000 osd.0 (osd.0) 145 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:37:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.467271+0000 osd.7 (osd.7) 153 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:37:10.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.470143+0000 osd.7 (osd.7) 154 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:37:10.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.030926+0000 osd.1 (osd.1) 125 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:37:10.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.036665+0000 osd.1 (osd.1) 126 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:37:10.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.790052+0000 osd.0 (osd.0) 145 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:37:10.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.467271+0000 osd.7 (osd.7) 153 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:37:10.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.470143+0000 osd.7 (osd.7) 154 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:37:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:08.878444+0000 osd.0 (osd.0) 146 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:37:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.049577+0000 osd.2 (osd.2) 104 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.049629+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.197973+0000 osd.2 (osd.2) 106 : cluster [DBG] 5.a scrub starts 2024-02-22T19:37:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.207746+0000 osd.2 (osd.2) 107 : cluster [DBG] 5.a scrub ok 2024-02-22T19:37:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.340606+0000 osd.3 (osd.3) 143 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:37:11.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.345312+0000 osd.3 (osd.3) 144 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:37:11.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.434804+0000 mgr.y (mgr.14535) 316 : cluster [DBG] pgmap v354: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.3 KiB/s rd, 24 MiB/s wr, 1.15k op/s 2024-02-22T19:37:11.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.783728+0000 mon.a (mon.0) 1030 : cluster [DBG] osdmap e184: 8 total, 8 up, 7 in 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:08.878444+0000 osd.0 (osd.0) 146 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.049577+0000 osd.2 (osd.2) 104 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.049629+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.197973+0000 osd.2 (osd.2) 106 : cluster [DBG] 5.a scrub starts 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.207746+0000 osd.2 (osd.2) 107 : cluster [DBG] 5.a scrub ok 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.340606+0000 osd.3 (osd.3) 143 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:37:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.345312+0000 osd.3 (osd.3) 144 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:37:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.434804+0000 mgr.y (mgr.14535) 316 : cluster [DBG] pgmap v354: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.3 KiB/s rd, 24 MiB/s wr, 1.15k op/s 2024-02-22T19:37:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.783728+0000 mon.a (mon.0) 1030 : cluster [DBG] osdmap e184: 8 total, 8 up, 7 in 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:08.878444+0000 osd.0 (osd.0) 146 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.049577+0000 osd.2 (osd.2) 104 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.049629+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.197973+0000 osd.2 (osd.2) 106 : cluster [DBG] 5.a scrub starts 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.207746+0000 osd.2 (osd.2) 107 : cluster [DBG] 5.a scrub ok 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.340606+0000 osd.3 (osd.3) 143 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.345312+0000 osd.3 (osd.3) 144 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:37:11.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.434804+0000 mgr.y (mgr.14535) 316 : cluster [DBG] pgmap v354: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.3 KiB/s rd, 24 MiB/s wr, 1.15k op/s 2024-02-22T19:37:11.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.783728+0000 mon.a (mon.0) 1030 : cluster [DBG] osdmap e184: 8 total, 8 up, 7 in 2024-02-22T19:37:12.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.867133+0000 osd.0 (osd.0) 147 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:37:12.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:09.873134+0000 osd.0 (osd.0) 148 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:37:12.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:10.027495+0000 osd.1 (osd.1) 127 : cluster [DBG] 8.a scrub starts 2024-02-22T19:37:12.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.867133+0000 osd.0 (osd.0) 147 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:37:12.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:09.873134+0000 osd.0 (osd.0) 148 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:37:12.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:10.027495+0000 osd.1 (osd.1) 127 : cluster [DBG] 8.a scrub starts 2024-02-22T19:37:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.867133+0000 osd.0 (osd.0) 147 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:37:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:09.873134+0000 osd.0 (osd.0) 148 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:37:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:10.027495+0000 osd.1 (osd.1) 127 : cluster [DBG] 8.a scrub starts 2024-02-22T19:37:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:11.437232+0000 mgr.y (mgr.14535) 317 : cluster [DBG] pgmap v356: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 1.08k op/s 2024-02-22T19:37:13.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:12.135638+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e185: 8 total, 8 up, 7 in 2024-02-22T19:37:13.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:11.437232+0000 mgr.y (mgr.14535) 317 : cluster [DBG] pgmap v356: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 1.08k op/s 2024-02-22T19:37:13.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:12.135638+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e185: 8 total, 8 up, 7 in 2024-02-22T19:37:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:11.437232+0000 mgr.y (mgr.14535) 317 : cluster [DBG] pgmap v356: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 1.08k op/s 2024-02-22T19:37:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:12.135638+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e185: 8 total, 8 up, 7 in 2024-02-22T19:37:13.387 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:13.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:13.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:13.443 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:13.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:13.565 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:13.741 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:13.763 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:37:13.763 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:37:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:13.229255+0000 osd.2 (osd.2) 108 : cluster [DBG] 5.d scrub starts 2024-02-22T19:37:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:13.243495+0000 osd.2 (osd.2) 109 : cluster [DBG] 5.d scrub ok 2024-02-22T19:37:14.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:13.229255+0000 osd.2 (osd.2) 108 : cluster [DBG] 5.d scrub starts 2024-02-22T19:37:14.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:13.243495+0000 osd.2 (osd.2) 109 : cluster [DBG] 5.d scrub ok 2024-02-22T19:37:14.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:13.229255+0000 osd.2 (osd.2) 108 : cluster [DBG] 5.d scrub starts 2024-02-22T19:37:14.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:13.243495+0000 osd.2 (osd.2) 109 : cluster [DBG] 5.d scrub ok 2024-02-22T19:37:14.272 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:14.279 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:14.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:14.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:14.441 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:14.707 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:14.713 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:14.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:14.715 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:37:14.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:14.916 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:15.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:12.962209+0000 osd.1 (osd.1) 128 : cluster [DBG] 8.a scrub ok 2024-02-22T19:37:15.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:13.437640+0000 mgr.y (mgr.14535) 318 : cluster [DBG] pgmap v358: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 657 op/s 2024-02-22T19:37:15.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:12.962209+0000 osd.1 (osd.1) 128 : cluster [DBG] 8.a scrub ok 2024-02-22T19:37:15.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:13.437640+0000 mgr.y (mgr.14535) 318 : cluster [DBG] pgmap v358: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 657 op/s 2024-02-22T19:37:15.030 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:15.036 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:15.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:15.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:37:15.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:37:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:37:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:12.962209+0000 osd.1 (osd.1) 128 : cluster [DBG] 8.a scrub ok 2024-02-22T19:37:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:13.437640+0000 mgr.y (mgr.14535) 318 : cluster [DBG] pgmap v358: 158 pgs: 158 active+clean; 6.4 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 657 op/s 2024-02-22T19:37:15.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:37:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:37:15.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:15.199 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:15.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:15.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:15.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:15.333 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:15.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:15.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:15.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:15.524 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:15.634 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:15.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:15.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:15.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:15.951 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:15.957 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:15.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:15.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:14.049475+0000 osd.1 (osd.1) 129 : cluster [DBG] 8.7 scrub starts 2024-02-22T19:37:15.960 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:14.683423+0000 mgr.y (mgr.14535) 319 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:15.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:14.049475+0000 osd.1 (osd.1) 129 : cluster [DBG] 8.7 scrub starts 2024-02-22T19:37:15.964 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:14.683423+0000 mgr.y (mgr.14535) 319 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:16.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:14.049475+0000 osd.1 (osd.1) 129 : cluster [DBG] 8.7 scrub starts 2024-02-22T19:37:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:14.683423+0000 mgr.y (mgr.14535) 319 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:16.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:15.440280+0000 mgr.y (mgr.14535) 320 : cluster [DBG] pgmap v359: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-02-22T19:37:16.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:15.900214+0000 osd.0 (osd.0) 149 : cluster [DBG] 8.e scrub starts 2024-02-22T19:37:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:17.116 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:17.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:17.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:15.440280+0000 mgr.y (mgr.14535) 320 : cluster [DBG] pgmap v359: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-02-22T19:37:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:15.900214+0000 osd.0 (osd.0) 149 : cluster [DBG] 8.e scrub starts 2024-02-22T19:37:17.285 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:15.440280+0000 mgr.y (mgr.14535) 320 : cluster [DBG] pgmap v359: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-02-22T19:37:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:15.900214+0000 osd.0 (osd.0) 149 : cluster [DBG] 8.e scrub starts 2024-02-22T19:37:17.425 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:17.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:17.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:17.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:17.655 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:17.665 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:17.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:17.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:17.795 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:17.820 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:17.820 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:17.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:17.997 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:18.151 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:18.160 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:18.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:18.165 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:18.170 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:18.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:17.093415+0000 osd.1 (osd.1) 130 : cluster [DBG] 8.7 scrub ok 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:17.277151+0000 osd.3 (osd.3) 145 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:17.299252+0000 osd.3 (osd.3) 146 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:17.440841+0000 mgr.y (mgr.14535) 321 : cluster [DBG] pgmap v360: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 616 op/s 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:17.093415+0000 osd.1 (osd.1) 130 : cluster [DBG] 8.7 scrub ok 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:17.277151+0000 osd.3 (osd.3) 145 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:17.299252+0000 osd.3 (osd.3) 146 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:37:18.263 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:17.440841+0000 mgr.y (mgr.14535) 321 : cluster [DBG] pgmap v360: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 616 op/s 2024-02-22T19:37:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:18.318 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:18.319 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:18.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:18.450 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:17.093415+0000 osd.1 (osd.1) 130 : cluster [DBG] 8.7 scrub ok 2024-02-22T19:37:18.450 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:17.277151+0000 osd.3 (osd.3) 145 : cluster [DBG] 5.16 scrub starts 2024-02-22T19:37:18.450 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:17.299252+0000 osd.3 (osd.3) 146 : cluster [DBG] 5.16 scrub ok 2024-02-22T19:37:18.450 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:17.440841+0000 mgr.y (mgr.14535) 321 : cluster [DBG] pgmap v360: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 616 op/s 2024-02-22T19:37:18.452 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:18.477 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:37:18.483 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:18.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:18.485 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:37:18.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:18.642 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:18.645 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:18.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:18.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:18.825 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:18.888 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:18.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:18.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:18.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:19.038 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:19.172 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:19.179 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:19.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:19.230 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:37:19.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:19.317 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-02-22T19:37:17.440599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6565467836,"num_objects":100374,"num_object_clones":0,"num_object_copies":301122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100374,"num_whiteouts":0,"num_read":1201,"num_read_kb":953,"num_write":123710,"num_write_kb":7150826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70793,"ondisk_log_size":70793,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":21452436,"kb_used_data":19330652,"kb_used_omap":0,"kb_used_meta":2121472,"kb_avail":634763628,"statfs":{"total":671965249536,"available":649997955072,"internally_reserved":0,"allocated":19794587648,"data_stored":19790786027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2172387328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-322961408,"num_objects":-4928,"num_object_clones":0,"num_object_copies":-14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4928,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4928,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.006460"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798551+0000","last_change":"2024-02-22T19:36:14.806137+0000","last_active":"2024-02-22T19:37:09.798551+0000","last_peered":"2024-02-22T19:37:09.798551+0000","last_clean":"2024-02-22T19:37:09.798551+0000","last_became_active":"2024-02-22T19:31:50.004112+0000","last_became_peered":"2024-02-22T19:31:50.004112+0000","last_unstale":"2024-02-22T19:37:09.798551+0000","last_undegraded":"2024-02-22T19:37:09.798551+0000","last_fullsized":"2024-02-22T19:37:09.798551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.806069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.806069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":157,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789474+0000","last_change":"2024-02-22T19:36:13.837113+0000","last_active":"2024-02-22T19:37:09.789474+0000","last_peered":"2024-02-22T19:37:09.789474+0000","last_clean":"2024-02-22T19:37:09.789474+0000","last_became_active":"2024-02-22T19:31:50.002878+0000","last_became_peered":"2024-02-22T19:31:50.002878+0000","last_unstale":"2024-02-22T19:37:09.789474+0000","last_undegraded":"2024-02-22T19:37:09.789474+0000","last_fullsized":"2024-02-22T19:37:09.789474+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:13.836981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:13.836981+0000","last_clean_scrub_stamp":"2024-02-22T19:36:13.836981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.859354+0000","last_change":"2024-02-22T19:36:55.687119+0000","last_active":"2024-02-22T19:37:09.859354+0000","last_peered":"2024-02-22T19:37:09.859354+0000","last_clean":"2024-02-22T19:37:09.859354+0000","last_became_active":"2024-02-22T19:31:50.012648+0000","last_became_peered":"2024-02-22T19:31:50.012648+0000","last_unstale":"2024-02-22T19:37:09.859354+0000","last_undegraded":"2024-02-22T19:37:09.859354+0000","last_fullsized":"2024-02-22T19:37:09.859354+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":166,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798196+0000","last_change":"2024-02-22T19:36:10.678582+0000","last_active":"2024-02-22T19:37:09.798196+0000","last_peered":"2024-02-22T19:37:09.798196+0000","last_clean":"2024-02-22T19:37:09.798196+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:37:09.798196+0000","last_undegraded":"2024-02-22T19:37:09.798196+0000","last_fullsized":"2024-02-22T19:37:09.798196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:10.678491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:10.678491+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861592+0000","last_change":"2024-02-22T19:36:59.298132+0000","last_active":"2024-02-22T19:37:09.861592+0000","last_peered":"2024-02-22T19:37:09.861592+0000","last_clean":"2024-02-22T19:37:09.861592+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:37:09.861592+0000","last_undegraded":"2024-02-22T19:37:09.861592+0000","last_fullsized":"2024-02-22T19:37:09.861592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:59.298021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:36:59.298021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798292+0000","last_change":"2024-02-22T19:35:26.000779+0000","last_active":"2024-02-22T19:37:09.798292+0000","last_peered":"2024-02-22T19:37:09.798292+0000","last_clean":"2024-02-22T19:37:09.798292+0000","last_became_active":"2024-02-22T19:29:05.098457+0000","last_became_peered":"2024-02-22T19:29:05.098457+0000","last_unstale":"2024-02-22T19:37:09.798292+0000","last_undegraded":"2024-02-22T19:37:09.798292+0000","last_fullsized":"2024-02-22T19:37:09.798292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.000689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.000689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.862446+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:37:09.862446+0000","last_peered":"2024-02-22T19:37:09.862446+0000","last_clean":"2024-02-22T19:37:09.862446+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:37:09.862446+0000","last_undegraded":"2024-02-22T19:37:09.862446+0000","last_fullsized":"2024-02-22T19:37:09.862446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797210+0000","last_change":"2024-02-22T19:34:45.948451+0000","last_active":"2024-02-22T19:37:09.797210+0000","last_peered":"2024-02-22T19:37:09.797210+0000","last_clean":"2024-02-22T19:37:09.797210+0000","last_became_active":"2024-02-22T19:29:03.078661+0000","last_became_peered":"2024-02-22T19:29:03.078661+0000","last_unstale":"2024-02-22T19:37:09.797210+0000","last_undegraded":"2024-02-22T19:37:09.797210+0000","last_fullsized":"2024-02-22T19:37:09.797210+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:45.948358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:45.948358+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798332+0000","last_change":"2024-02-22T19:35:23.244026+0000","last_active":"2024-02-22T19:37:09.798332+0000","last_peered":"2024-02-22T19:37:09.798332+0000","last_clean":"2024-02-22T19:37:09.798332+0000","last_became_active":"2024-02-22T19:29:07.100857+0000","last_became_peered":"2024-02-22T19:29:07.100857+0000","last_unstale":"2024-02-22T19:37:09.798332+0000","last_undegraded":"2024-02-22T19:37:09.798332+0000","last_fullsized":"2024-02-22T19:37:09.798332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:23.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:23.243945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":107,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798354+0000","last_change":"2024-02-22T19:35:26.232511+0000","last_active":"2024-02-22T19:37:09.798354+0000","last_peered":"2024-02-22T19:37:09.798354+0000","last_clean":"2024-02-22T19:37:09.798354+0000","last_became_active":"2024-02-22T19:31:49.998682+0000","last_became_peered":"2024-02-22T19:31:49.998682+0000","last_unstale":"2024-02-22T19:37:09.798354+0000","last_undegraded":"2024-02-22T19:37:09.798354+0000","last_fullsized":"2024-02-22T19:37:09.798354+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:26.232444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.232444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":97,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.794693+0000","last_change":"2024-02-22T19:36:58.219910+0000","last_active":"2024-02-22T19:37:09.794693+0000","last_peered":"2024-02-22T19:37:09.794693+0000","last_clean":"2024-02-22T19:37:09.794693+0000","last_became_active":"2024-02-22T19:32:25.384977+0000","last_became_peered":"2024-02-22T19:32:25.384977+0000","last_unstale":"2024-02-22T19:37:09.794693+0000","last_undegraded":"2024-02-22T19:37:09.794693+0000","last_fullsized":"2024-02-22T19:37:09.794693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":166,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798783+0000","last_change":"2024-02-22T19:36:21.735619+0000","last_active":"2024-02-22T19:37:09.798783+0000","last_peered":"2024-02-22T19:37:09.798783+0000","last_clean":"2024-02-22T19:37:09.798783+0000","last_became_active":"2024-02-22T19:29:03.077292+0000","last_became_peered":"2024-02-22T19:29:03.077292+0000","last_unstale":"2024-02-22T19:37:09.798783+0000","last_undegraded":"2024-02-22T19:37:09.798783+0000","last_fullsized":"2024-02-22T19:37:09.798783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:21.735513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:21.735513+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":107,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789379+0000","last_change":"2024-02-22T19:36:08.908709+0000","last_active":"2024-02-22T19:37:09.789379+0000","last_peered":"2024-02-22T19:37:09.789379+0000","last_clean":"2024-02-22T19:37:09.789379+0000","last_became_active":"2024-02-22T19:31:50.004705+0000","last_became_peered":"2024-02-22T19:31:50.004705+0000","last_unstale":"2024-02-22T19:37:09.789379+0000","last_undegraded":"2024-02-22T19:37:09.789379+0000","last_fullsized":"2024-02-22T19:37:09.789379+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.908571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.908571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801886+0000","last_change":"2024-02-22T19:36:19.511120+0000","last_active":"2024-02-22T19:37:09.801886+0000","last_peered":"2024-02-22T19:37:09.801886+0000","last_clean":"2024-02-22T19:37:09.801886+0000","last_became_active":"2024-02-22T19:29:05.093874+0000","last_became_peered":"2024-02-22T19:29:05.093874+0000","last_unstale":"2024-02-22T19:37:09.801886+0000","last_undegraded":"2024-02-22T19:37:09.801886+0000","last_fullsized":"2024-02-22T19:37:09.801886+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.511049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.619448+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.511049+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":114,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802728+0000","last_change":"2024-02-22T19:35:35.930757+0000","last_active":"2024-02-22T19:37:09.802728+0000","last_peered":"2024-02-22T19:37:09.802728+0000","last_clean":"2024-02-22T19:37:09.802728+0000","last_became_active":"2024-02-22T19:31:50.005979+0000","last_became_peered":"2024-02-22T19:31:50.005979+0000","last_unstale":"2024-02-22T19:37:09.802728+0000","last_undegraded":"2024-02-22T19:37:09.802728+0000","last_fullsized":"2024-02-22T19:37:09.802728+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:35:35.930654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.930654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798849+0000","last_change":"2024-02-22T19:34:46.118383+0000","last_active":"2024-02-22T19:37:09.798849+0000","last_peered":"2024-02-22T19:37:09.798849+0000","last_clean":"2024-02-22T19:37:09.798849+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:37:09.798849+0000","last_undegraded":"2024-02-22T19:37:09.798849+0000","last_fullsized":"2024-02-22T19:37:09.798849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:46.118282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.118282+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":150,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803510+0000","last_change":"2024-02-22T19:34:56.107642+0000","last_active":"2024-02-22T19:37:09.803510+0000","last_peered":"2024-02-22T19:37:09.803510+0000","last_clean":"2024-02-22T19:37:09.803510+0000","last_became_active":"2024-02-22T19:31:50.008987+0000","last_became_peered":"2024-02-22T19:31:50.008987+0000","last_unstale":"2024-02-22T19:37:09.803510+0000","last_undegraded":"2024-02-22T19:37:09.803510+0000","last_fullsized":"2024-02-22T19:37:09.803510+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:56.107558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:56.107558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":148,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.790513+0000","last_change":"2024-02-22T19:36:18.889514+0000","last_active":"2024-02-22T19:37:09.790513+0000","last_peered":"2024-02-22T19:37:09.790513+0000","last_clean":"2024-02-22T19:37:09.790513+0000","last_became_active":"2024-02-22T19:29:05.088299+0000","last_became_peered":"2024-02-22T19:29:05.088299+0000","last_unstale":"2024-02-22T19:37:09.790513+0000","last_undegraded":"2024-02-22T19:37:09.790513+0000","last_fullsized":"2024-02-22T19:37:09.790513+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:18.889425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:41.981349+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.889425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798586+0000","last_change":"2024-02-22T19:36:54.732926+0000","last_active":"2024-02-22T19:37:09.798586+0000","last_peered":"2024-02-22T19:37:09.798586+0000","last_clean":"2024-02-22T19:37:09.798586+0000","last_became_active":"2024-02-22T19:29:01.013366+0000","last_became_peered":"2024-02-22T19:29:01.013366+0000","last_unstale":"2024-02-22T19:37:09.798586+0000","last_undegraded":"2024-02-22T19:37:09.798586+0000","last_fullsized":"2024-02-22T19:37:09.798586+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.732856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.732856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"172'22","reported_seq":184,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.794285+0000","last_change":"2024-02-22T19:36:24.021213+0000","last_active":"2024-02-22T19:37:09.794285+0000","last_peered":"2024-02-22T19:37:09.794285+0000","last_clean":"2024-02-22T19:37:09.794285+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:37:09.794285+0000","last_undegraded":"2024-02-22T19:37:09.794285+0000","last_fullsized":"2024-02-22T19:37:09.794285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'22","last_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.021111+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861967+0000","last_change":"2024-02-22T19:36:28.335837+0000","last_active":"2024-02-22T19:37:09.861967+0000","last_peered":"2024-02-22T19:37:09.861967+0000","last_clean":"2024-02-22T19:37:09.861967+0000","last_became_active":"2024-02-22T19:31:50.014222+0000","last_became_peered":"2024-02-22T19:31:50.014222+0000","last_unstale":"2024-02-22T19:37:09.861967+0000","last_undegraded":"2024-02-22T19:37:09.861967+0000","last_fullsized":"2024-02-22T19:37:09.861967+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803005+0000","last_change":"2024-02-22T19:35:24.048780+0000","last_active":"2024-02-22T19:37:09.803005+0000","last_peered":"2024-02-22T19:37:09.803005+0000","last_clean":"2024-02-22T19:37:09.803005+0000","last_became_active":"2024-02-22T19:29:05.096515+0000","last_became_peered":"2024-02-22T19:29:05.096515+0000","last_unstale":"2024-02-22T19:37:09.803005+0000","last_undegraded":"2024-02-22T19:37:09.803005+0000","last_fullsized":"2024-02-22T19:37:09.803005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:24.048696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.048696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798846+0000","last_change":"2024-02-22T19:37:08.584019+0000","last_active":"2024-02-22T19:37:09.798846+0000","last_peered":"2024-02-22T19:37:09.798846+0000","last_clean":"2024-02-22T19:37:09.798846+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:37:09.798846+0000","last_undegraded":"2024-02-22T19:37:09.798846+0000","last_fullsized":"2024-02-22T19:37:09.798846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.583926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.583926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797460+0000","last_change":"2024-02-22T19:35:36.338814+0000","last_active":"2024-02-22T19:37:09.797460+0000","last_peered":"2024-02-22T19:37:09.797460+0000","last_clean":"2024-02-22T19:37:09.797460+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:37:09.797460+0000","last_undegraded":"2024-02-22T19:37:09.797460+0000","last_fullsized":"2024-02-22T19:37:09.797460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":159,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802599+0000","last_change":"2024-02-22T19:36:06.575957+0000","last_active":"2024-02-22T19:37:09.802599+0000","last_peered":"2024-02-22T19:37:09.802599+0000","last_clean":"2024-02-22T19:37:09.802599+0000","last_became_active":"2024-02-22T19:31:49.999321+0000","last_became_peered":"2024-02-22T19:31:49.999321+0000","last_unstale":"2024-02-22T19:37:09.802599+0000","last_undegraded":"2024-02-22T19:37:09.802599+0000","last_fullsized":"2024-02-22T19:37:09.802599+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:36:06.575860+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:36:06.575860+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"137'9","reported_seq":266,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.135978+0000","last_change":"2024-02-22T19:36:53.428054+0000","last_active":"2024-02-22T19:37:12.135978+0000","last_peered":"2024-02-22T19:37:12.135978+0000","last_clean":"2024-02-22T19:37:12.135978+0000","last_became_active":"2024-02-22T19:29:05.092539+0000","last_became_peered":"2024-02-22T19:29:05.092539+0000","last_unstale":"2024-02-22T19:37:12.135978+0000","last_undegraded":"2024-02-22T19:37:12.135978+0000","last_fullsized":"2024-02-22T19:37:12.135978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:36:53.427951+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.427951+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789529+0000","last_change":"2024-02-22T19:35:30.790879+0000","last_active":"2024-02-22T19:37:09.789529+0000","last_peered":"2024-02-22T19:37:09.789529+0000","last_clean":"2024-02-22T19:37:09.789529+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:37:09.789529+0000","last_undegraded":"2024-02-22T19:37:09.789529+0000","last_fullsized":"2024-02-22T19:37:09.789529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.790805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.790805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":163,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.860144+0000","last_change":"2024-02-22T19:36:04.546799+0000","last_active":"2024-02-22T19:37:09.860144+0000","last_peered":"2024-02-22T19:37:09.860144+0000","last_clean":"2024-02-22T19:37:09.860144+0000","last_became_active":"2024-02-22T19:29:03.446854+0000","last_became_peered":"2024-02-22T19:29:03.446854+0000","last_unstale":"2024-02-22T19:37:09.860144+0000","last_undegraded":"2024-02-22T19:37:09.860144+0000","last_fullsized":"2024-02-22T19:37:09.860144+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:36:04.546703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.546703+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":157,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.790646+0000","last_change":"2024-02-22T19:36:48.995124+0000","last_active":"2024-02-22T19:37:09.790646+0000","last_peered":"2024-02-22T19:37:09.790646+0000","last_clean":"2024-02-22T19:37:09.790646+0000","last_became_active":"2024-02-22T19:31:50.007907+0000","last_became_peered":"2024-02-22T19:31:50.007907+0000","last_unstale":"2024-02-22T19:37:09.790646+0000","last_undegraded":"2024-02-22T19:37:09.790646+0000","last_fullsized":"2024-02-22T19:37:09.790646+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.995002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.995002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.860360+0000","last_change":"2024-02-22T19:36:22.679579+0000","last_active":"2024-02-22T19:37:09.860360+0000","last_peered":"2024-02-22T19:37:09.860360+0000","last_clean":"2024-02-22T19:37:09.860360+0000","last_became_active":"2024-02-22T19:29:05.092155+0000","last_became_peered":"2024-02-22T19:29:05.092155+0000","last_unstale":"2024-02-22T19:37:09.860360+0000","last_undegraded":"2024-02-22T19:37:09.860360+0000","last_fullsized":"2024-02-22T19:37:09.860360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:22.679488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:22.679488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797189+0000","last_change":"2024-02-22T19:34:58.165060+0000","last_active":"2024-02-22T19:37:09.797189+0000","last_peered":"2024-02-22T19:37:09.797189+0000","last_clean":"2024-02-22T19:37:09.797189+0000","last_became_active":"2024-02-22T19:29:01.012816+0000","last_became_peered":"2024-02-22T19:29:01.012816+0000","last_unstale":"2024-02-22T19:37:09.797189+0000","last_undegraded":"2024-02-22T19:37:09.797189+0000","last_fullsized":"2024-02-22T19:37:09.797189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:34:58.164960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":219,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797254+0000","last_change":"2024-02-22T19:35:29.242870+0000","last_active":"2024-02-22T19:37:09.797254+0000","last_peered":"2024-02-22T19:37:09.797254+0000","last_clean":"2024-02-22T19:37:09.797254+0000","last_became_active":"2024-02-22T19:31:50.009209+0000","last_became_peered":"2024-02-22T19:31:50.009209+0000","last_unstale":"2024-02-22T19:37:09.797254+0000","last_undegraded":"2024-02-22T19:37:09.797254+0000","last_fullsized":"2024-02-22T19:37:09.797254+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:35:29.242798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:29.242798+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802036+0000","last_change":"2024-02-22T19:36:07.626299+0000","last_active":"2024-02-22T19:37:09.802036+0000","last_peered":"2024-02-22T19:37:09.802036+0000","last_clean":"2024-02-22T19:37:09.802036+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:37:09.802036+0000","last_undegraded":"2024-02-22T19:37:09.802036+0000","last_fullsized":"2024-02-22T19:37:09.802036+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.626201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.626201+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":145,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796570+0000","last_change":"2024-02-22T19:36:05.177403+0000","last_active":"2024-02-22T19:37:09.796570+0000","last_peered":"2024-02-22T19:37:09.796570+0000","last_clean":"2024-02-22T19:37:09.796570+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:37:09.796570+0000","last_undegraded":"2024-02-22T19:37:09.796570+0000","last_fullsized":"2024-02-22T19:37:09.796570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:05.177297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:05.177297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":159,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.859534+0000","last_change":"2024-02-22T19:35:21.292096+0000","last_active":"2024-02-22T19:37:09.859534+0000","last_peered":"2024-02-22T19:37:09.859534+0000","last_clean":"2024-02-22T19:37:09.859534+0000","last_became_active":"2024-02-22T19:31:50.010352+0000","last_became_peered":"2024-02-22T19:31:50.010352+0000","last_unstale":"2024-02-22T19:37:09.859534+0000","last_undegraded":"2024-02-22T19:37:09.859534+0000","last_fullsized":"2024-02-22T19:37:09.859534+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:21.292020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:21.292020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797291+0000","last_change":"2024-02-22T19:36:58.263929+0000","last_active":"2024-02-22T19:37:09.797291+0000","last_peered":"2024-02-22T19:37:09.797291+0000","last_clean":"2024-02-22T19:37:09.797291+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:37:09.797291+0000","last_undegraded":"2024-02-22T19:37:09.797291+0000","last_fullsized":"2024-02-22T19:37:09.797291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:36:58.263834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.263834+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":138,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861737+0000","last_change":"2024-02-22T19:34:36.044624+0000","last_active":"2024-02-22T19:37:09.861737+0000","last_peered":"2024-02-22T19:37:09.861737+0000","last_clean":"2024-02-22T19:37:09.861737+0000","last_became_active":"2024-02-22T19:29:07.199597+0000","last_became_peered":"2024-02-22T19:29:07.199597+0000","last_unstale":"2024-02-22T19:37:09.861737+0000","last_undegraded":"2024-02-22T19:37:09.861737+0000","last_fullsized":"2024-02-22T19:37:09.861737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:36.044519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:36.044519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802112+0000","last_change":"2024-02-22T19:35:33.660316+0000","last_active":"2024-02-22T19:37:09.802112+0000","last_peered":"2024-02-22T19:37:09.802112+0000","last_clean":"2024-02-22T19:37:09.802112+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:37:09.802112+0000","last_undegraded":"2024-02-22T19:37:09.802112+0000","last_fullsized":"2024-02-22T19:37:09.802112+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.660254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.660254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":156,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802989+0000","last_change":"2024-02-22T19:37:08.454952+0000","last_active":"2024-02-22T19:37:09.802989+0000","last_peered":"2024-02-22T19:37:09.802989+0000","last_clean":"2024-02-22T19:37:09.802989+0000","last_became_active":"2024-02-22T19:31:50.000731+0000","last_became_peered":"2024-02-22T19:31:50.000731+0000","last_unstale":"2024-02-22T19:37:09.802989+0000","last_undegraded":"2024-02-22T19:37:09.802989+0000","last_fullsized":"2024-02-22T19:37:09.802989+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797165+0000","last_change":"2024-02-22T19:34:43.915568+0000","last_active":"2024-02-22T19:37:09.797165+0000","last_peered":"2024-02-22T19:37:09.797165+0000","last_clean":"2024-02-22T19:37:09.797165+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:37:09.797165+0000","last_undegraded":"2024-02-22T19:37:09.797165+0000","last_fullsized":"2024-02-22T19:37:09.797165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:43.915463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.915463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798108+0000","last_change":"2024-02-22T19:37:04.719801+0000","last_active":"2024-02-22T19:37:09.798108+0000","last_peered":"2024-02-22T19:37:09.798108+0000","last_clean":"2024-02-22T19:37:09.798108+0000","last_became_active":"2024-02-22T19:31:50.013881+0000","last_became_peered":"2024-02-22T19:31:50.013881+0000","last_unstale":"2024-02-22T19:37:09.798108+0000","last_undegraded":"2024-02-22T19:37:09.798108+0000","last_fullsized":"2024-02-22T19:37:09.798108+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":140,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861508+0000","last_change":"2024-02-22T19:34:41.961882+0000","last_active":"2024-02-22T19:37:09.861508+0000","last_peered":"2024-02-22T19:37:09.861508+0000","last_clean":"2024-02-22T19:37:09.861508+0000","last_became_active":"2024-02-22T19:29:05.092172+0000","last_became_peered":"2024-02-22T19:29:05.092172+0000","last_unstale":"2024-02-22T19:37:09.861508+0000","last_undegraded":"2024-02-22T19:37:09.861508+0000","last_fullsized":"2024-02-22T19:37:09.861508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.961788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.961788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":159,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803340+0000","last_change":"2024-02-22T19:36:52.463891+0000","last_active":"2024-02-22T19:37:09.803340+0000","last_peered":"2024-02-22T19:37:09.803340+0000","last_clean":"2024-02-22T19:37:09.803340+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:37:09.803340+0000","last_undegraded":"2024-02-22T19:37:09.803340+0000","last_fullsized":"2024-02-22T19:37:09.803340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:52.463774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.463774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":166,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.791780+0000","last_change":"2024-02-22T19:35:35.855558+0000","last_active":"2024-02-22T19:37:09.791780+0000","last_peered":"2024-02-22T19:37:09.791780+0000","last_clean":"2024-02-22T19:37:09.791780+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:37:09.791780+0000","last_undegraded":"2024-02-22T19:37:09.791780+0000","last_fullsized":"2024-02-22T19:37:09.791780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:35.855449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.855449+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796947+0000","last_change":"2024-02-22T19:36:19.229925+0000","last_active":"2024-02-22T19:37:09.796947+0000","last_peered":"2024-02-22T19:37:09.796947+0000","last_clean":"2024-02-22T19:37:09.796947+0000","last_became_active":"2024-02-22T19:31:50.007388+0000","last_became_peered":"2024-02-22T19:31:50.007388+0000","last_unstale":"2024-02-22T19:37:09.796947+0000","last_undegraded":"2024-02-22T19:37:09.796947+0000","last_fullsized":"2024-02-22T19:37:09.796947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.229816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.229816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":157,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.791590+0000","last_change":"2024-02-22T19:36:47.993417+0000","last_active":"2024-02-22T19:37:09.791590+0000","last_peered":"2024-02-22T19:37:09.791590+0000","last_clean":"2024-02-22T19:37:09.791590+0000","last_became_active":"2024-02-22T19:31:50.012026+0000","last_became_peered":"2024-02-22T19:31:50.012026+0000","last_unstale":"2024-02-22T19:37:09.791590+0000","last_undegraded":"2024-02-22T19:37:09.791590+0000","last_fullsized":"2024-02-22T19:37:09.791590+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:47.993328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:47.993328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802807+0000","last_change":"2024-02-22T19:36:27.685508+0000","last_active":"2024-02-22T19:37:09.802807+0000","last_peered":"2024-02-22T19:37:09.802807+0000","last_clean":"2024-02-22T19:37:09.802807+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:37:09.802807+0000","last_undegraded":"2024-02-22T19:37:09.802807+0000","last_fullsized":"2024-02-22T19:37:09.802807+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.685418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.685418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798664+0000","last_change":"2024-02-22T19:34:59.142089+0000","last_active":"2024-02-22T19:37:09.798664+0000","last_peered":"2024-02-22T19:37:09.798664+0000","last_clean":"2024-02-22T19:37:09.798664+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:37:09.798664+0000","last_undegraded":"2024-02-22T19:37:09.798664+0000","last_fullsized":"2024-02-22T19:37:09.798664+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:59.141999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:59.141999+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.859898+0000","last_change":"2024-02-22T19:36:15.699409+0000","last_active":"2024-02-22T19:37:09.859898+0000","last_peered":"2024-02-22T19:37:09.859898+0000","last_clean":"2024-02-22T19:37:09.859898+0000","last_became_active":"2024-02-22T19:29:07.106840+0000","last_became_peered":"2024-02-22T19:29:07.106840+0000","last_unstale":"2024-02-22T19:37:09.859898+0000","last_undegraded":"2024-02-22T19:37:09.859898+0000","last_fullsized":"2024-02-22T19:37:09.859898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.699319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.699319+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"183'10","reported_seq":197,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.135230+0000","last_change":"2024-02-22T19:37:04.973800+0000","last_active":"2024-02-22T19:37:12.135230+0000","last_peered":"2024-02-22T19:37:12.135230+0000","last_clean":"2024-02-22T19:37:12.135230+0000","last_became_active":"2024-02-22T19:31:50.012730+0000","last_became_peered":"2024-02-22T19:31:50.012730+0000","last_unstale":"2024-02-22T19:37:12.135230+0000","last_undegraded":"2024-02-22T19:37:12.135230+0000","last_fullsized":"2024-02-22T19:37:12.135230+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":150,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802286+0000","last_change":"2024-02-22T19:36:56.463572+0000","last_active":"2024-02-22T19:37:09.802286+0000","last_peered":"2024-02-22T19:37:09.802286+0000","last_clean":"2024-02-22T19:37:09.802286+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:37:09.802286+0000","last_undegraded":"2024-02-22T19:37:09.802286+0000","last_fullsized":"2024-02-22T19:37:09.802286+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:56.463502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:36:56.463502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.858494+0000","last_change":"2024-02-22T19:37:09.345416+0000","last_active":"2024-02-22T19:37:09.858494+0000","last_peered":"2024-02-22T19:37:09.858494+0000","last_clean":"2024-02-22T19:37:09.858494+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:37:09.858494+0000","last_undegraded":"2024-02-22T19:37:09.858494+0000","last_fullsized":"2024-02-22T19:37:09.858494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:37:09.345319+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.345319+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803652+0000","last_change":"2024-02-22T19:36:26.761319+0000","last_active":"2024-02-22T19:37:09.803652+0000","last_peered":"2024-02-22T19:37:09.803652+0000","last_clean":"2024-02-22T19:37:09.803652+0000","last_became_active":"2024-02-22T19:31:50.015421+0000","last_became_peered":"2024-02-22T19:31:50.015421+0000","last_unstale":"2024-02-22T19:37:09.803652+0000","last_undegraded":"2024-02-22T19:37:09.803652+0000","last_fullsized":"2024-02-22T19:37:09.803652+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.761226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":247,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.148911+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:37:12.148911+0000","last_peered":"2024-02-22T19:37:12.148911+0000","last_clean":"2024-02-22T19:37:12.148911+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:37:12.148911+0000","last_undegraded":"2024-02-22T19:37:12.148911+0000","last_fullsized":"2024-02-22T19:37:12.148911+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.873277+0000","last_change":"2024-02-22T19:37:09.873235+0000","last_active":"2024-02-22T19:37:09.873277+0000","last_peered":"2024-02-22T19:37:09.873277+0000","last_clean":"2024-02-22T19:37:09.873277+0000","last_became_active":"2024-02-22T19:29:01.010717+0000","last_became_peered":"2024-02-22T19:29:01.010717+0000","last_unstale":"2024-02-22T19:37:09.873277+0000","last_undegraded":"2024-02-22T19:37:09.873277+0000","last_fullsized":"2024-02-22T19:37:09.873277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":175,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803580+0000","last_change":"2024-02-22T19:36:48.456721+0000","last_active":"2024-02-22T19:37:09.803580+0000","last_peered":"2024-02-22T19:37:09.803580+0000","last_clean":"2024-02-22T19:37:09.803580+0000","last_became_active":"2024-02-22T19:31:49.999950+0000","last_became_peered":"2024-02-22T19:31:49.999950+0000","last_unstale":"2024-02-22T19:37:09.803580+0000","last_undegraded":"2024-02-22T19:37:09.803580+0000","last_fullsized":"2024-02-22T19:37:09.803580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":151,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.794448+0000","last_change":"2024-02-22T19:37:02.236048+0000","last_active":"2024-02-22T19:37:09.794448+0000","last_peered":"2024-02-22T19:37:09.794448+0000","last_clean":"2024-02-22T19:37:09.794448+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:37:09.794448+0000","last_undegraded":"2024-02-22T19:37:09.794448+0000","last_fullsized":"2024-02-22T19:37:09.794448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.235958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.235958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":140,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861883+0000","last_change":"2024-02-22T19:36:09.204798+0000","last_active":"2024-02-22T19:37:09.861883+0000","last_peered":"2024-02-22T19:37:09.861883+0000","last_clean":"2024-02-22T19:37:09.861883+0000","last_became_active":"2024-02-22T19:29:05.096267+0000","last_became_peered":"2024-02-22T19:29:05.096267+0000","last_unstale":"2024-02-22T19:37:09.861883+0000","last_undegraded":"2024-02-22T19:37:09.861883+0000","last_fullsized":"2024-02-22T19:37:09.861883+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.204726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.204726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801968+0000","last_change":"2024-02-22T19:37:09.470209+0000","last_active":"2024-02-22T19:37:09.801968+0000","last_peered":"2024-02-22T19:37:09.801968+0000","last_clean":"2024-02-22T19:37:09.801968+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:37:09.801968+0000","last_undegraded":"2024-02-22T19:37:09.801968+0000","last_fullsized":"2024-02-22T19:37:09.801968+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.470149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.470149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.860227+0000","last_change":"2024-02-22T19:35:27.255099+0000","last_active":"2024-02-22T19:37:09.860227+0000","last_peered":"2024-02-22T19:37:09.860227+0000","last_clean":"2024-02-22T19:37:09.860227+0000","last_became_active":"2024-02-22T19:29:03.447708+0000","last_became_peered":"2024-02-22T19:29:03.447708+0000","last_unstale":"2024-02-22T19:37:09.860227+0000","last_undegraded":"2024-02-22T19:37:09.860227+0000","last_fullsized":"2024-02-22T19:37:09.860227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802298+0000","last_change":"2024-02-22T19:35:31.614031+0000","last_active":"2024-02-22T19:37:09.802298+0000","last_peered":"2024-02-22T19:37:09.802298+0000","last_clean":"2024-02-22T19:37:09.802298+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:37:09.802298+0000","last_undegraded":"2024-02-22T19:37:09.802298+0000","last_fullsized":"2024-02-22T19:37:09.802298+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:31.613909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:31.613909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":102,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.862586+0000","last_change":"2024-02-22T19:34:31.909139+0000","last_active":"2024-02-22T19:37:09.862586+0000","last_peered":"2024-02-22T19:37:09.862586+0000","last_clean":"2024-02-22T19:37:09.862586+0000","last_became_active":"2024-02-22T19:31:50.013532+0000","last_became_peered":"2024-02-22T19:31:50.013532+0000","last_unstale":"2024-02-22T19:37:09.862586+0000","last_undegraded":"2024-02-22T19:37:09.862586+0000","last_fullsized":"2024-02-22T19:37:09.862586+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:31.909036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:31.909036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796166+0000","last_change":"2024-02-22T19:37:04.431720+0000","last_active":"2024-02-22T19:37:09.796166+0000","last_peered":"2024-02-22T19:37:09.796166+0000","last_clean":"2024-02-22T19:37:09.796166+0000","last_became_active":"2024-02-22T19:31:50.006038+0000","last_became_peered":"2024-02-22T19:31:50.006038+0000","last_unstale":"2024-02-22T19:37:09.796166+0000","last_undegraded":"2024-02-22T19:37:09.796166+0000","last_fullsized":"2024-02-22T19:37:09.796166+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.431655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.431655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.862054+0000","last_change":"2024-02-22T19:36:26.647888+0000","last_active":"2024-02-22T19:37:09.862054+0000","last_peered":"2024-02-22T19:37:09.862054+0000","last_clean":"2024-02-22T19:37:09.862054+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:37:09.862054+0000","last_undegraded":"2024-02-22T19:37:09.862054+0000","last_fullsized":"2024-02-22T19:37:09.862054+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:36:25.296484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.296484+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"185'7594","reported_seq":7633,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.200698+0000","last_change":"2024-02-22T19:35:19.745999+0000","last_active":"2024-02-22T19:37:14.200698+0000","last_peered":"2024-02-22T19:37:14.200698+0000","last_clean":"2024-02-22T19:37:14.200698+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:37:14.200698+0000","last_undegraded":"2024-02-22T19:37:14.200698+0000","last_fullsized":"2024-02-22T19:37:14.200698+0000","mapping_epoch":154,"log_start":"166'3400","ondisk_log_start":"166'3400","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4194,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406585344,"num_objects":6204,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7594,"num_write_kb":441536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":112,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796294+0000","last_change":"2024-02-22T19:36:17.151389+0000","last_active":"2024-02-22T19:37:09.796294+0000","last_peered":"2024-02-22T19:37:09.796294+0000","last_clean":"2024-02-22T19:37:09.796294+0000","last_became_active":"2024-02-22T19:31:50.006079+0000","last_became_peered":"2024-02-22T19:31:50.006079+0000","last_unstale":"2024-02-22T19:37:09.796294+0000","last_undegraded":"2024-02-22T19:37:09.796294+0000","last_fullsized":"2024-02-22T19:37:09.796294+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.151294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.151294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802601+0000","last_change":"2024-02-22T19:36:04.642477+0000","last_active":"2024-02-22T19:37:09.802601+0000","last_peered":"2024-02-22T19:37:09.802601+0000","last_clean":"2024-02-22T19:37:09.802601+0000","last_became_active":"2024-02-22T19:29:01.005596+0000","last_became_peered":"2024-02-22T19:29:01.005596+0000","last_unstale":"2024-02-22T19:37:09.802601+0000","last_undegraded":"2024-02-22T19:37:09.802601+0000","last_fullsized":"2024-02-22T19:37:09.802601+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:36:04.642361+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.642361+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":160,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801966+0000","last_change":"2024-02-22T19:36:09.623594+0000","last_active":"2024-02-22T19:37:09.801966+0000","last_peered":"2024-02-22T19:37:09.801966+0000","last_clean":"2024-02-22T19:37:09.801966+0000","last_became_active":"2024-02-22T19:31:50.004229+0000","last_became_peered":"2024-02-22T19:31:50.004229+0000","last_unstale":"2024-02-22T19:37:09.801966+0000","last_undegraded":"2024-02-22T19:37:09.801966+0000","last_fullsized":"2024-02-22T19:37:09.801966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:36:09.623521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.623521+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":253,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.139892+0000","last_change":"2024-02-22T19:36:53.503236+0000","last_active":"2024-02-22T19:37:12.139892+0000","last_peered":"2024-02-22T19:37:12.139892+0000","last_clean":"2024-02-22T19:37:12.139892+0000","last_became_active":"2024-02-22T19:29:05.094805+0000","last_became_peered":"2024-02-22T19:29:05.094805+0000","last_unstale":"2024-02-22T19:37:12.139892+0000","last_undegraded":"2024-02-22T19:37:12.139892+0000","last_fullsized":"2024-02-22T19:37:12.139892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:36:53.503128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.503128+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"185'7815","reported_seq":7859,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.201379+0000","last_change":"2024-02-22T19:37:07.737898+0000","last_active":"2024-02-22T19:37:14.201379+0000","last_peered":"2024-02-22T19:37:14.201379+0000","last_clean":"2024-02-22T19:37:14.201379+0000","last_became_active":"2024-02-22T19:35:19.225793+0000","last_became_peered":"2024-02-22T19:35:19.225793+0000","last_unstale":"2024-02-22T19:37:14.201379+0000","last_undegraded":"2024-02-22T19:37:14.201379+0000","last_fullsized":"2024-02-22T19:37:14.201379+0000","mapping_epoch":154,"log_start":"165'3100","ondisk_log_start":"165'3100","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'7399","last_scrub_stamp":"2024-02-22T19:37:07.737806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:07.737806+0000","log_size":4715,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":414646272,"num_objects":6327,"num_object_clones":0,"num_object_copies":18981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7815,"num_write_kb":452544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":138,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861666+0000","last_change":"2024-02-22T19:36:11.224751+0000","last_active":"2024-02-22T19:37:09.861666+0000","last_peered":"2024-02-22T19:37:09.861666+0000","last_clean":"2024-02-22T19:37:09.861666+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:37:09.861666+0000","last_undegraded":"2024-02-22T19:37:09.861666+0000","last_fullsized":"2024-02-22T19:37:09.861666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.224646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.224646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":152,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.517307+0000","last_change":"2024-02-22T19:37:12.518070+0000","last_active":"2024-02-22T19:37:12.517307+0000","last_peered":"2024-02-22T19:37:12.517307+0000","last_clean":"2024-02-22T19:37:12.517307+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:37:12.517307+0000","last_undegraded":"2024-02-22T19:37:12.517307+0000","last_fullsized":"2024-02-22T19:37:12.517307+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.515824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.515824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798569+0000","last_change":"2024-02-22T19:37:05.645806+0000","last_active":"2024-02-22T19:37:09.798569+0000","last_peered":"2024-02-22T19:37:09.798569+0000","last_clean":"2024-02-22T19:37:09.798569+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:37:09.798569+0000","last_undegraded":"2024-02-22T19:37:09.798569+0000","last_fullsized":"2024-02-22T19:37:09.798569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:05.645703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.645703+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":148,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793759+0000","last_change":"2024-02-22T19:36:19.039536+0000","last_active":"2024-02-22T19:37:09.793759+0000","last_peered":"2024-02-22T19:37:09.793759+0000","last_clean":"2024-02-22T19:37:09.793759+0000","last_became_active":"2024-02-22T19:29:05.092147+0000","last_became_peered":"2024-02-22T19:29:05.092147+0000","last_unstale":"2024-02-22T19:37:09.793759+0000","last_undegraded":"2024-02-22T19:37:09.793759+0000","last_fullsized":"2024-02-22T19:37:09.793759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.039457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.039457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"185'7801","reported_seq":7840,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.312300+0000","last_change":"2024-02-22T19:35:19.225144+0000","last_active":"2024-02-22T19:37:14.312300+0000","last_peered":"2024-02-22T19:37:14.312300+0000","last_clean":"2024-02-22T19:37:14.312300+0000","last_became_active":"2024-02-22T19:35:19.224823+0000","last_became_peered":"2024-02-22T19:35:19.224823+0000","last_unstale":"2024-02-22T19:37:14.312300+0000","last_undegraded":"2024-02-22T19:37:14.312300+0000","last_fullsized":"2024-02-22T19:37:14.312300+0000","mapping_epoch":154,"log_start":"165'3100","ondisk_log_start":"165'3100","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4701,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":418185216,"num_objects":6381,"num_object_clones":0,"num_object_copies":19143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7801,"num_write_kb":453824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801638+0000","last_change":"2024-02-22T19:36:54.489612+0000","last_active":"2024-02-22T19:37:09.801638+0000","last_peered":"2024-02-22T19:37:09.801638+0000","last_clean":"2024-02-22T19:37:09.801638+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:37:09.801638+0000","last_undegraded":"2024-02-22T19:37:09.801638+0000","last_fullsized":"2024-02-22T19:37:09.801638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.489509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.489509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":157,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793950+0000","last_change":"2024-02-22T19:36:14.029499+0000","last_active":"2024-02-22T19:37:09.793950+0000","last_peered":"2024-02-22T19:37:09.793950+0000","last_clean":"2024-02-22T19:37:09.793950+0000","last_became_active":"2024-02-22T19:31:50.008459+0000","last_became_peered":"2024-02-22T19:31:50.008459+0000","last_unstale":"2024-02-22T19:37:09.793950+0000","last_undegraded":"2024-02-22T19:37:09.793950+0000","last_fullsized":"2024-02-22T19:37:09.793950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.029389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803506+0000","last_change":"2024-02-22T19:36:57.438182+0000","last_active":"2024-02-22T19:37:09.803506+0000","last_peered":"2024-02-22T19:37:09.803506+0000","last_clean":"2024-02-22T19:37:09.803506+0000","last_became_active":"2024-02-22T19:31:49.998818+0000","last_became_peered":"2024-02-22T19:31:49.998818+0000","last_unstale":"2024-02-22T19:37:09.803506+0000","last_undegraded":"2024-02-22T19:37:09.803506+0000","last_fullsized":"2024-02-22T19:37:09.803506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.438112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.438112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796535+0000","last_change":"2024-02-22T19:35:22.253473+0000","last_active":"2024-02-22T19:37:09.796535+0000","last_peered":"2024-02-22T19:37:09.796535+0000","last_clean":"2024-02-22T19:37:09.796535+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:37:09.796535+0000","last_undegraded":"2024-02-22T19:37:09.796535+0000","last_fullsized":"2024-02-22T19:37:09.796535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:22.253381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.253381+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.7","version":"185'7838","reported_seq":7879,"reported_epoch":185,"state":"active+clean+scrubbing","last_fresh":"2024-02-22T19:37:14.348186+0000","last_change":"2024-02-22T19:37:14.044770+0000","last_active":"2024-02-22T19:37:14.348186+0000","last_peered":"2024-02-22T19:37:14.348186+0000","last_clean":"2024-02-22T19:37:14.348186+0000","last_became_active":"2024-02-22T19:35:19.227265+0000","last_became_peered":"2024-02-22T19:35:19.227265+0000","last_unstale":"2024-02-22T19:37:14.348186+0000","last_undegraded":"2024-02-22T19:37:14.348186+0000","last_fullsized":"2024-02-22T19:37:14.348186+0000","mapping_epoch":154,"log_start":"165'3300","ondisk_log_start":"165'3300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4538,"ondisk_log_size":4538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":414973952,"num_objects":6332,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":453440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798617+0000","last_change":"2024-02-22T19:36:17.690399+0000","last_active":"2024-02-22T19:37:09.798617+0000","last_peered":"2024-02-22T19:37:09.798617+0000","last_clean":"2024-02-22T19:37:09.798617+0000","last_became_active":"2024-02-22T19:29:07.101588+0000","last_became_peered":"2024-02-22T19:29:07.101588+0000","last_unstale":"2024-02-22T19:37:09.798617+0000","last_undegraded":"2024-02-22T19:37:09.798617+0000","last_fullsized":"2024-02-22T19:37:09.798617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.690298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.690298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793985+0000","last_change":"2024-02-22T19:37:09.207841+0000","last_active":"2024-02-22T19:37:09.793985+0000","last_peered":"2024-02-22T19:37:09.793985+0000","last_clean":"2024-02-22T19:37:09.793985+0000","last_became_active":"2024-02-22T19:29:05.096059+0000","last_became_peered":"2024-02-22T19:29:05.096059+0000","last_unstale":"2024-02-22T19:37:09.793985+0000","last_undegraded":"2024-02-22T19:37:09.793985+0000","last_fullsized":"2024-02-22T19:37:09.793985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.207753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.207753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":107,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.858582+0000","last_change":"2024-02-22T19:36:55.297715+0000","last_active":"2024-02-22T19:37:09.858582+0000","last_peered":"2024-02-22T19:37:09.858582+0000","last_clean":"2024-02-22T19:37:09.858582+0000","last_became_active":"2024-02-22T19:31:50.013728+0000","last_became_peered":"2024-02-22T19:31:50.013728+0000","last_unstale":"2024-02-22T19:37:09.858582+0000","last_undegraded":"2024-02-22T19:37:09.858582+0000","last_fullsized":"2024-02-22T19:37:09.858582+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.297621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.297621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":183,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798667+0000","last_change":"2024-02-22T19:36:18.621857+0000","last_active":"2024-02-22T19:37:09.798667+0000","last_peered":"2024-02-22T19:37:09.798667+0000","last_clean":"2024-02-22T19:37:09.798667+0000","last_became_active":"2024-02-22T19:30:24.409676+0000","last_became_peered":"2024-02-22T19:30:24.409676+0000","last_unstale":"2024-02-22T19:37:09.798667+0000","last_undegraded":"2024-02-22T19:37:09.798667+0000","last_fullsized":"2024-02-22T19:37:09.798667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:18.621747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.621747+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798100+0000","last_change":"2024-02-22T19:36:25.576957+0000","last_active":"2024-02-22T19:37:09.798100+0000","last_peered":"2024-02-22T19:37:09.798100+0000","last_clean":"2024-02-22T19:37:09.798100+0000","last_became_active":"2024-02-22T19:35:58.432364+0000","last_became_peered":"2024-02-22T19:35:58.432364+0000","last_unstale":"2024-02-22T19:37:09.798100+0000","last_undegraded":"2024-02-22T19:37:09.798100+0000","last_fullsized":"2024-02-22T19:37:09.798100+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"185'7855","reported_seq":7894,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.334095+0000","last_change":"2024-02-22T19:35:19.226141+0000","last_active":"2024-02-22T19:37:14.334095+0000","last_peered":"2024-02-22T19:37:14.334095+0000","last_clean":"2024-02-22T19:37:14.334095+0000","last_became_active":"2024-02-22T19:35:19.225860+0000","last_became_peered":"2024-02-22T19:35:19.225860+0000","last_unstale":"2024-02-22T19:37:14.334095+0000","last_undegraded":"2024-02-22T19:37:14.334095+0000","last_fullsized":"2024-02-22T19:37:14.334095+0000","mapping_epoch":154,"log_start":"166'3900","ondisk_log_start":"166'3900","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3955,"ondisk_log_size":3955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":416350208,"num_objects":6353,"num_object_clones":0,"num_object_copies":19059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7855,"num_write_kb":454656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":138,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.862781+0000","last_change":"2024-02-22T19:34:46.954016+0000","last_active":"2024-02-22T19:37:09.862781+0000","last_peered":"2024-02-22T19:37:09.862781+0000","last_clean":"2024-02-22T19:37:09.862781+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:37:09.862781+0000","last_undegraded":"2024-02-22T19:37:09.862781+0000","last_fullsized":"2024-02-22T19:37:09.862781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:46.953951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.953951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":109,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.791379+0000","last_change":"2024-02-22T19:37:08.036758+0000","last_active":"2024-02-22T19:37:09.791379+0000","last_peered":"2024-02-22T19:37:09.791379+0000","last_clean":"2024-02-22T19:37:09.791379+0000","last_became_active":"2024-02-22T19:31:50.009079+0000","last_became_peered":"2024-02-22T19:31:50.009079+0000","last_unstale":"2024-02-22T19:37:09.791379+0000","last_undegraded":"2024-02-22T19:37:09.791379+0000","last_fullsized":"2024-02-22T19:37:09.791379+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":154,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798522+0000","last_change":"2024-02-22T19:37:03.578378+0000","last_active":"2024-02-22T19:37:09.798522+0000","last_peered":"2024-02-22T19:37:09.798522+0000","last_clean":"2024-02-22T19:37:09.798522+0000","last_became_active":"2024-02-22T19:29:01.001360+0000","last_became_peered":"2024-02-22T19:29:01.001360+0000","last_unstale":"2024-02-22T19:37:09.798522+0000","last_undegraded":"2024-02-22T19:37:09.798522+0000","last_fullsized":"2024-02-22T19:37:09.798522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.578317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.578317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":297,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.137999+0000","last_change":"2024-02-22T19:35:20.843884+0000","last_active":"2024-02-22T19:37:12.137999+0000","last_peered":"2024-02-22T19:37:12.137999+0000","last_clean":"2024-02-22T19:37:12.137999+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:37:12.137999+0000","last_undegraded":"2024-02-22T19:37:12.137999+0000","last_fullsized":"2024-02-22T19:37:12.137999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:35:20.843809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:20.843809+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.858645+0000","last_change":"2024-02-22T19:36:07.590613+0000","last_active":"2024-02-22T19:37:09.858645+0000","last_peered":"2024-02-22T19:37:09.858645+0000","last_clean":"2024-02-22T19:37:09.858645+0000","last_became_active":"2024-02-22T19:29:05.090070+0000","last_became_peered":"2024-02-22T19:29:05.090070+0000","last_unstale":"2024-02-22T19:37:09.858645+0000","last_undegraded":"2024-02-22T19:37:09.858645+0000","last_fullsized":"2024-02-22T19:37:09.858645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.590509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.590509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801900+0000","last_change":"2024-02-22T19:36:24.596597+0000","last_active":"2024-02-22T19:37:09.801900+0000","last_peered":"2024-02-22T19:37:09.801900+0000","last_clean":"2024-02-22T19:37:09.801900+0000","last_became_active":"2024-02-22T19:35:58.429347+0000","last_became_peered":"2024-02-22T19:35:58.429347+0000","last_unstale":"2024-02-22T19:37:09.801900+0000","last_undegraded":"2024-02-22T19:37:09.801900+0000","last_fullsized":"2024-02-22T19:37:09.801900+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:24.596531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.596531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"185'7515","reported_seq":7553,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.373243+0000","last_change":"2024-02-22T19:35:19.225962+0000","last_active":"2024-02-22T19:37:14.373243+0000","last_peered":"2024-02-22T19:37:14.373243+0000","last_clean":"2024-02-22T19:37:14.373243+0000","last_became_active":"2024-02-22T19:35:19.225604+0000","last_became_peered":"2024-02-22T19:35:19.225604+0000","last_unstale":"2024-02-22T19:37:14.373243+0000","last_undegraded":"2024-02-22T19:37:14.373243+0000","last_fullsized":"2024-02-22T19:37:14.373243+0000","mapping_epoch":154,"log_start":"166'3600","ondisk_log_start":"166'3600","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3915,"ondisk_log_size":3915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":401408000,"num_objects":6125,"num_object_clones":0,"num_object_copies":18375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":436480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803652+0000","last_change":"2024-02-22T19:36:08.683152+0000","last_active":"2024-02-22T19:37:09.803652+0000","last_peered":"2024-02-22T19:37:09.803652+0000","last_clean":"2024-02-22T19:37:09.803652+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:37:09.803652+0000","last_undegraded":"2024-02-22T19:37:09.803652+0000","last_fullsized":"2024-02-22T19:37:09.803652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.683006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.683006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789786+0000","last_change":"2024-02-22T19:35:38.917238+0000","last_active":"2024-02-22T19:37:09.789786+0000","last_peered":"2024-02-22T19:37:09.789786+0000","last_clean":"2024-02-22T19:37:09.789786+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:37:09.789786+0000","last_undegraded":"2024-02-22T19:37:09.789786+0000","last_fullsized":"2024-02-22T19:37:09.789786+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:38.917177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.917177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":159,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.794523+0000","last_change":"2024-02-22T19:36:57.185812+0000","last_active":"2024-02-22T19:37:09.794523+0000","last_peered":"2024-02-22T19:37:09.794523+0000","last_clean":"2024-02-22T19:37:09.794523+0000","last_became_active":"2024-02-22T19:29:01.010074+0000","last_became_peered":"2024-02-22T19:29:01.010074+0000","last_unstale":"2024-02-22T19:37:09.794523+0000","last_undegraded":"2024-02-22T19:37:09.794523+0000","last_fullsized":"2024-02-22T19:37:09.794523+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.185719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.185719+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":159,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.795874+0000","last_change":"2024-02-22T19:36:15.052075+0000","last_active":"2024-02-22T19:37:09.795874+0000","last_peered":"2024-02-22T19:37:09.795874+0000","last_clean":"2024-02-22T19:37:09.795874+0000","last_became_active":"2024-02-22T19:31:50.010808+0000","last_became_peered":"2024-02-22T19:31:50.010808+0000","last_unstale":"2024-02-22T19:37:09.795874+0000","last_undegraded":"2024-02-22T19:37:09.795874+0000","last_fullsized":"2024-02-22T19:37:09.795874+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:36:15.051953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.051953+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.795675+0000","last_change":"2024-02-22T19:36:49.141950+0000","last_active":"2024-02-22T19:37:09.795675+0000","last_peered":"2024-02-22T19:37:09.795675+0000","last_clean":"2024-02-22T19:37:09.795675+0000","last_became_active":"2024-02-22T19:31:50.005828+0000","last_became_peered":"2024-02-22T19:31:50.005828+0000","last_unstale":"2024-02-22T19:37:09.795675+0000","last_undegraded":"2024-02-22T19:37:09.795675+0000","last_fullsized":"2024-02-22T19:37:09.795675+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:49.141855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:49.141855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"185'7728","reported_seq":7772,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.197449+0000","last_change":"2024-02-22T19:36:52.260887+0000","last_active":"2024-02-22T19:37:14.197449+0000","last_peered":"2024-02-22T19:37:14.197449+0000","last_clean":"2024-02-22T19:37:14.197449+0000","last_became_active":"2024-02-22T19:35:19.226638+0000","last_became_peered":"2024-02-22T19:35:19.226638+0000","last_unstale":"2024-02-22T19:37:14.197449+0000","last_undegraded":"2024-02-22T19:37:14.197449+0000","last_fullsized":"2024-02-22T19:37:14.197449+0000","mapping_epoch":154,"log_start":"165'3100","ondisk_log_start":"165'3100","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'6381","last_scrub_stamp":"2024-02-22T19:36:52.260780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.260780+0000","log_size":4628,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":417595392,"num_objects":6372,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7728,"num_write_kb":451200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":102,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861807+0000","last_change":"2024-02-22T19:35:22.280512+0000","last_active":"2024-02-22T19:37:09.861807+0000","last_peered":"2024-02-22T19:37:09.861807+0000","last_clean":"2024-02-22T19:37:09.861807+0000","last_became_active":"2024-02-22T19:31:50.014430+0000","last_became_peered":"2024-02-22T19:31:50.014430+0000","last_unstale":"2024-02-22T19:37:09.861807+0000","last_undegraded":"2024-02-22T19:37:09.861807+0000","last_fullsized":"2024-02-22T19:37:09.861807+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.280418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.280418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801806+0000","last_change":"2024-02-22T19:36:23.598862+0000","last_active":"2024-02-22T19:37:09.801806+0000","last_peered":"2024-02-22T19:37:09.801806+0000","last_clean":"2024-02-22T19:37:09.801806+0000","last_became_active":"2024-02-22T19:35:57.424387+0000","last_became_peered":"2024-02-22T19:35:57.424387+0000","last_unstale":"2024-02-22T19:37:09.801806+0000","last_undegraded":"2024-02-22T19:37:09.801806+0000","last_fullsized":"2024-02-22T19:37:09.801806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.598758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.598758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.859162+0000","last_change":"2024-02-22T19:34:50.967533+0000","last_active":"2024-02-22T19:37:09.859162+0000","last_peered":"2024-02-22T19:37:09.859162+0000","last_clean":"2024-02-22T19:37:09.859162+0000","last_became_active":"2024-02-22T19:31:50.009285+0000","last_became_peered":"2024-02-22T19:31:50.009285+0000","last_unstale":"2024-02-22T19:37:09.859162+0000","last_undegraded":"2024-02-22T19:37:09.859162+0000","last_fullsized":"2024-02-22T19:37:09.859162+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:50.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:50.967435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":176,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796992+0000","last_change":"2024-02-22T19:35:35.065441+0000","last_active":"2024-02-22T19:37:09.796992+0000","last_peered":"2024-02-22T19:37:09.796992+0000","last_clean":"2024-02-22T19:37:09.796992+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:37:09.796992+0000","last_undegraded":"2024-02-22T19:37:09.796992+0000","last_fullsized":"2024-02-22T19:37:09.796992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:35.065332+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.065332+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802684+0000","last_change":"2024-02-22T19:36:12.513753+0000","last_active":"2024-02-22T19:37:09.802684+0000","last_peered":"2024-02-22T19:37:09.802684+0000","last_clean":"2024-02-22T19:37:09.802684+0000","last_became_active":"2024-02-22T19:31:49.995147+0000","last_became_peered":"2024-02-22T19:31:49.995147+0000","last_unstale":"2024-02-22T19:37:09.802684+0000","last_undegraded":"2024-02-22T19:37:09.802684+0000","last_fullsized":"2024-02-22T19:37:09.802684+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:12.513648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:12.513648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789630+0000","last_change":"2024-02-22T19:37:01.957962+0000","last_active":"2024-02-22T19:37:09.789630+0000","last_peered":"2024-02-22T19:37:09.789630+0000","last_clean":"2024-02-22T19:37:09.789630+0000","last_became_active":"2024-02-22T19:35:58.418408+0000","last_became_peered":"2024-02-22T19:35:58.418408+0000","last_unstale":"2024-02-22T19:37:09.789630+0000","last_undegraded":"2024-02-22T19:37:09.789630+0000","last_fullsized":"2024-02-22T19:37:09.789630+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:01.957854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:01.957854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"185'7628","reported_seq":7672,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.343620+0000","last_change":"2024-02-22T19:37:12.962333+0000","last_active":"2024-02-22T19:37:14.343620+0000","last_peered":"2024-02-22T19:37:14.343620+0000","last_clean":"2024-02-22T19:37:14.343620+0000","last_became_active":"2024-02-22T19:35:19.747933+0000","last_became_peered":"2024-02-22T19:35:19.747933+0000","last_unstale":"2024-02-22T19:37:14.343620+0000","last_undegraded":"2024-02-22T19:37:14.343620+0000","last_fullsized":"2024-02-22T19:37:14.343620+0000","mapping_epoch":154,"log_start":"165'3100","ondisk_log_start":"165'3100","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'7550","last_scrub_stamp":"2024-02-22T19:37:12.962216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:12.962216+0000","log_size":4528,"ondisk_log_size":4528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":404619264,"num_objects":6174,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":441664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":155,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789630+0000","last_change":"2024-02-22T19:36:23.901614+0000","last_active":"2024-02-22T19:37:09.789630+0000","last_peered":"2024-02-22T19:37:09.789630+0000","last_clean":"2024-02-22T19:37:09.789630+0000","last_became_active":"2024-02-22T19:31:50.012010+0000","last_became_peered":"2024-02-22T19:31:50.012010+0000","last_unstale":"2024-02-22T19:37:09.789630+0000","last_undegraded":"2024-02-22T19:37:09.789630+0000","last_fullsized":"2024-02-22T19:37:09.789630+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.901512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.901512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":150,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798394+0000","last_change":"2024-02-22T19:35:30.342964+0000","last_active":"2024-02-22T19:37:09.798394+0000","last_peered":"2024-02-22T19:37:09.798394+0000","last_clean":"2024-02-22T19:37:09.798394+0000","last_became_active":"2024-02-22T19:29:01.013575+0000","last_became_peered":"2024-02-22T19:29:01.013575+0000","last_unstale":"2024-02-22T19:37:09.798394+0000","last_undegraded":"2024-02-22T19:37:09.798394+0000","last_fullsized":"2024-02-22T19:37:09.798394+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.342864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.342864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":645,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.818321+0000","last_change":"2024-02-22T19:36:20.553666+0000","last_active":"2024-02-22T19:37:12.818321+0000","last_peered":"2024-02-22T19:37:12.818321+0000","last_clean":"2024-02-22T19:37:12.818321+0000","last_became_active":"2024-02-22T19:29:01.008261+0000","last_became_peered":"2024-02-22T19:29:01.008261+0000","last_unstale":"2024-02-22T19:37:12.818321+0000","last_undegraded":"2024-02-22T19:37:12.818321+0000","last_fullsized":"2024-02-22T19:37:12.818321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:36:20.553554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.553554+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":487,"num_read_kb":482,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":166,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.859979+0000","last_change":"2024-02-22T19:37:08.878576+0000","last_active":"2024-02-22T19:37:09.859979+0000","last_peered":"2024-02-22T19:37:09.859979+0000","last_clean":"2024-02-22T19:37:09.859979+0000","last_became_active":"2024-02-22T19:29:03.083017+0000","last_became_peered":"2024-02-22T19:29:03.083017+0000","last_unstale":"2024-02-22T19:37:09.859979+0000","last_undegraded":"2024-02-22T19:37:09.859979+0000","last_fullsized":"2024-02-22T19:37:09.859979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:08.878452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.878452+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":107,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793530+0000","last_change":"2024-02-22T19:36:15.908805+0000","last_active":"2024-02-22T19:37:09.793530+0000","last_peered":"2024-02-22T19:37:09.793530+0000","last_clean":"2024-02-22T19:37:09.793530+0000","last_became_active":"2024-02-22T19:31:50.009784+0000","last_became_peered":"2024-02-22T19:31:50.009784+0000","last_unstale":"2024-02-22T19:37:09.793530+0000","last_undegraded":"2024-02-22T19:37:09.793530+0000","last_fullsized":"2024-02-22T19:37:09.793530+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.908714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.908714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796505+0000","last_change":"2024-02-22T19:36:14.230430+0000","last_active":"2024-02-22T19:37:09.796505+0000","last_peered":"2024-02-22T19:37:09.796505+0000","last_clean":"2024-02-22T19:37:09.796505+0000","last_became_active":"2024-02-22T19:35:58.429215+0000","last_became_peered":"2024-02-22T19:35:58.429215+0000","last_unstale":"2024-02-22T19:37:09.796505+0000","last_undegraded":"2024-02-22T19:37:09.796505+0000","last_fullsized":"2024-02-22T19:37:09.796505+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.230365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"185'7641","reported_seq":7680,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.404367+0000","last_change":"2024-02-22T19:35:19.745745+0000","last_active":"2024-02-22T19:37:14.404367+0000","last_peered":"2024-02-22T19:37:14.404367+0000","last_clean":"2024-02-22T19:37:14.404367+0000","last_became_active":"2024-02-22T19:35:19.745488+0000","last_became_peered":"2024-02-22T19:35:19.745488+0000","last_unstale":"2024-02-22T19:37:14.404367+0000","last_undegraded":"2024-02-22T19:37:14.404367+0000","last_fullsized":"2024-02-22T19:37:14.404367+0000","mapping_epoch":154,"log_start":"166'3700","ondisk_log_start":"166'3700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3941,"ondisk_log_size":3941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408616960,"num_objects":6235,"num_object_clones":0,"num_object_copies":18705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7641,"num_write_kb":444032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789444+0000","last_change":"2024-02-22T19:37:03.977945+0000","last_active":"2024-02-22T19:37:09.789444+0000","last_peered":"2024-02-22T19:37:09.789444+0000","last_clean":"2024-02-22T19:37:09.789444+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:37:09.789444+0000","last_undegraded":"2024-02-22T19:37:09.789444+0000","last_fullsized":"2024-02-22T19:37:09.789444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.977850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.977850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.789847+0000","last_change":"2024-02-22T19:36:19.862015+0000","last_active":"2024-02-22T19:37:09.789847+0000","last_peered":"2024-02-22T19:37:09.789847+0000","last_clean":"2024-02-22T19:37:09.789847+0000","last_became_active":"2024-02-22T19:31:50.009645+0000","last_became_peered":"2024-02-22T19:31:50.009645+0000","last_unstale":"2024-02-22T19:37:09.789847+0000","last_undegraded":"2024-02-22T19:37:09.789847+0000","last_fullsized":"2024-02-22T19:37:09.789847+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.861921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.861921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":211,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798530+0000","last_change":"2024-02-22T19:35:35.426393+0000","last_active":"2024-02-22T19:37:09.798530+0000","last_peered":"2024-02-22T19:37:09.798530+0000","last_clean":"2024-02-22T19:37:09.798530+0000","last_became_active":"2024-02-22T19:31:50.011631+0000","last_became_peered":"2024-02-22T19:31:50.011631+0000","last_unstale":"2024-02-22T19:37:09.798530+0000","last_undegraded":"2024-02-22T19:37:09.798530+0000","last_fullsized":"2024-02-22T19:37:09.798530+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:35.426296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.426296+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796620+0000","last_change":"2024-02-22T19:36:54.441714+0000","last_active":"2024-02-22T19:37:09.796620+0000","last_peered":"2024-02-22T19:37:09.796620+0000","last_clean":"2024-02-22T19:37:09.796620+0000","last_became_active":"2024-02-22T19:31:50.006441+0000","last_became_peered":"2024-02-22T19:31:50.006441+0000","last_unstale":"2024-02-22T19:37:09.796620+0000","last_undegraded":"2024-02-22T19:37:09.796620+0000","last_fullsized":"2024-02-22T19:37:09.796620+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.441645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.441645+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797029+0000","last_change":"2024-02-22T19:36:07.161771+0000","last_active":"2024-02-22T19:37:09.797029+0000","last_peered":"2024-02-22T19:37:09.797029+0000","last_clean":"2024-02-22T19:37:09.797029+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:37:09.797029+0000","last_undegraded":"2024-02-22T19:37:09.797029+0000","last_fullsized":"2024-02-22T19:37:09.797029+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.161538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.161538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"185'7766","reported_seq":7807,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.372024+0000","last_change":"2024-02-22T19:37:12.866374+0000","last_active":"2024-02-22T19:37:14.372024+0000","last_peered":"2024-02-22T19:37:14.372024+0000","last_clean":"2024-02-22T19:37:14.372024+0000","last_became_active":"2024-02-22T19:35:19.745522+0000","last_became_peered":"2024-02-22T19:35:19.745522+0000","last_unstale":"2024-02-22T19:37:14.372024+0000","last_undegraded":"2024-02-22T19:37:14.372024+0000","last_fullsized":"2024-02-22T19:37:14.372024+0000","mapping_epoch":154,"log_start":"165'3000","ondisk_log_start":"165'3000","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4766,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":412745728,"num_objects":6298,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7766,"num_write_kb":450048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798150+0000","last_change":"2024-02-22T19:35:33.417797+0000","last_active":"2024-02-22T19:37:09.798150+0000","last_peered":"2024-02-22T19:37:09.798150+0000","last_clean":"2024-02-22T19:37:09.798150+0000","last_became_active":"2024-02-22T19:29:07.102776+0000","last_became_peered":"2024-02-22T19:29:07.102776+0000","last_unstale":"2024-02-22T19:37:09.798150+0000","last_undegraded":"2024-02-22T19:37:09.798150+0000","last_fullsized":"2024-02-22T19:37:09.798150+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":107,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797301+0000","last_change":"2024-02-22T19:37:02.261620+0000","last_active":"2024-02-22T19:37:09.797301+0000","last_peered":"2024-02-22T19:37:09.797301+0000","last_clean":"2024-02-22T19:37:09.797301+0000","last_became_active":"2024-02-22T19:31:50.006281+0000","last_became_peered":"2024-02-22T19:31:50.006281+0000","last_unstale":"2024-02-22T19:37:09.797301+0000","last_undegraded":"2024-02-22T19:37:09.797301+0000","last_fullsized":"2024-02-22T19:37:09.797301+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.261537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.261537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":186,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.791684+0000","last_change":"2024-02-22T19:35:37.886857+0000","last_active":"2024-02-22T19:37:09.791684+0000","last_peered":"2024-02-22T19:37:09.791684+0000","last_clean":"2024-02-22T19:37:09.791684+0000","last_became_active":"2024-02-22T19:31:50.009721+0000","last_became_peered":"2024-02-22T19:31:50.009721+0000","last_unstale":"2024-02-22T19:37:09.791684+0000","last_undegraded":"2024-02-22T19:37:09.791684+0000","last_fullsized":"2024-02-22T19:37:09.791684+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:37.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.886787+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"100'9","reported_seq":266,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.144286+0000","last_change":"2024-02-22T19:36:23.659056+0000","last_active":"2024-02-22T19:37:12.144286+0000","last_peered":"2024-02-22T19:37:12.144286+0000","last_clean":"2024-02-22T19:37:12.144286+0000","last_became_active":"2024-02-22T19:31:50.010861+0000","last_became_peered":"2024-02-22T19:31:50.010861+0000","last_unstale":"2024-02-22T19:37:12.144286+0000","last_undegraded":"2024-02-22T19:37:12.144286+0000","last_fullsized":"2024-02-22T19:37:12.144286+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:36:23.658957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.658957+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797383+0000","last_change":"2024-02-22T19:35:58.422975+0000","last_active":"2024-02-22T19:37:09.797383+0000","last_peered":"2024-02-22T19:37:09.797383+0000","last_clean":"2024-02-22T19:37:09.797383+0000","last_became_active":"2024-02-22T19:35:58.422662+0000","last_became_peered":"2024-02-22T19:35:58.422662+0000","last_unstale":"2024-02-22T19:37:09.797383+0000","last_undegraded":"2024-02-22T19:37:09.797383+0000","last_fullsized":"2024-02-22T19:37:09.797383+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"185'7805","reported_seq":7850,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.221036+0000","last_change":"2024-02-22T19:37:01.648510+0000","last_active":"2024-02-22T19:37:14.221036+0000","last_peered":"2024-02-22T19:37:14.221036+0000","last_clean":"2024-02-22T19:37:14.221036+0000","last_became_active":"2024-02-22T19:35:19.745627+0000","last_became_peered":"2024-02-22T19:35:19.745627+0000","last_unstale":"2024-02-22T19:37:14.221036+0000","last_undegraded":"2024-02-22T19:37:14.221036+0000","last_fullsized":"2024-02-22T19:37:14.221036+0000","mapping_epoch":154,"log_start":"165'3300","ondisk_log_start":"165'3300","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'6967","last_scrub_stamp":"2024-02-22T19:37:01.648425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:01.648425+0000","log_size":4505,"ondisk_log_size":4505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":411959320,"num_objects":6287,"num_object_clones":0,"num_object_copies":18861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7805,"num_write_kb":450881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":144,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802566+0000","last_change":"2024-02-22T19:36:11.554490+0000","last_active":"2024-02-22T19:37:09.802566+0000","last_peered":"2024-02-22T19:37:09.802566+0000","last_clean":"2024-02-22T19:37:09.802566+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:37:09.802566+0000","last_undegraded":"2024-02-22T19:37:09.802566+0000","last_fullsized":"2024-02-22T19:37:09.802566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.554382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.554382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":150,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.860055+0000","last_change":"2024-02-22T19:36:48.680783+0000","last_active":"2024-02-22T19:37:09.860055+0000","last_peered":"2024-02-22T19:37:09.860055+0000","last_clean":"2024-02-22T19:37:09.860055+0000","last_became_active":"2024-02-22T19:29:01.006022+0000","last_became_peered":"2024-02-22T19:29:01.006022+0000","last_unstale":"2024-02-22T19:37:09.860055+0000","last_undegraded":"2024-02-22T19:37:09.860055+0000","last_fullsized":"2024-02-22T19:37:09.860055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.680723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.680723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":198,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798488+0000","last_change":"2024-02-22T19:35:26.969937+0000","last_active":"2024-02-22T19:37:09.798488+0000","last_peered":"2024-02-22T19:37:09.798488+0000","last_clean":"2024-02-22T19:37:09.798488+0000","last_became_active":"2024-02-22T19:31:50.007334+0000","last_became_peered":"2024-02-22T19:31:50.007334+0000","last_unstale":"2024-02-22T19:37:09.798488+0000","last_undegraded":"2024-02-22T19:37:09.798488+0000","last_fullsized":"2024-02-22T19:37:09.798488+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:26.969827+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.969827+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":247,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:12.151305+0000","last_change":"2024-02-22T19:35:25.316903+0000","last_active":"2024-02-22T19:37:12.151305+0000","last_peered":"2024-02-22T19:37:12.151305+0000","last_clean":"2024-02-22T19:37:12.151305+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:37:12.151305+0000","last_undegraded":"2024-02-22T19:37:12.151305+0000","last_fullsized":"2024-02-22T19:37:12.151305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:11.560498+0000","last_change":"2024-02-22T19:37:11.560498+0000","last_active":"2024-02-22T19:37:11.560498+0000","last_peered":"2024-02-22T19:37:11.560498+0000","last_clean":"2024-02-22T19:37:11.560498+0000","last_became_active":"2024-02-22T19:35:58.417154+0000","last_became_peered":"2024-02-22T19:35:58.417154+0000","last_unstale":"2024-02-22T19:37:11.560498+0000","last_undegraded":"2024-02-22T19:37:11.560498+0000","last_fullsized":"2024-02-22T19:37:11.560498+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:44.432446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:34:44.432446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"185'7686","reported_seq":7725,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.355630+0000","last_change":"2024-02-22T19:35:19.227135+0000","last_active":"2024-02-22T19:37:14.355630+0000","last_peered":"2024-02-22T19:37:14.355630+0000","last_clean":"2024-02-22T19:37:14.355630+0000","last_became_active":"2024-02-22T19:35:19.226763+0000","last_became_peered":"2024-02-22T19:35:19.226763+0000","last_unstale":"2024-02-22T19:37:14.355630+0000","last_undegraded":"2024-02-22T19:37:14.355630+0000","last_fullsized":"2024-02-22T19:37:14.355630+0000","mapping_epoch":154,"log_start":"164'2900","ondisk_log_start":"164'2900","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4786,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408944640,"num_objects":6240,"num_object_clones":0,"num_object_copies":18720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7686,"num_write_kb":445632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798630+0000","last_change":"2024-02-22T19:36:20.739814+0000","last_active":"2024-02-22T19:37:09.798630+0000","last_peered":"2024-02-22T19:37:09.798630+0000","last_clean":"2024-02-22T19:37:09.798630+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:37:09.798630+0000","last_undegraded":"2024-02-22T19:37:09.798630+0000","last_fullsized":"2024-02-22T19:37:09.798630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":149,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.858977+0000","last_change":"2024-02-22T19:37:06.317218+0000","last_active":"2024-02-22T19:37:09.858977+0000","last_peered":"2024-02-22T19:37:09.858977+0000","last_clean":"2024-02-22T19:37:09.858977+0000","last_became_active":"2024-02-22T19:29:01.008970+0000","last_became_peered":"2024-02-22T19:29:01.008970+0000","last_unstale":"2024-02-22T19:37:09.858977+0000","last_undegraded":"2024-02-22T19:37:09.858977+0000","last_fullsized":"2024-02-22T19:37:09.858977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:37:06.317125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":181,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797463+0000","last_change":"2024-02-22T19:36:26.284565+0000","last_active":"2024-02-22T19:37:09.797463+0000","last_peered":"2024-02-22T19:37:09.797463+0000","last_clean":"2024-02-22T19:37:09.797463+0000","last_became_active":"2024-02-22T19:29:03.447564+0000","last_became_peered":"2024-02-22T19:29:03.447564+0000","last_unstale":"2024-02-22T19:37:09.797463+0000","last_undegraded":"2024-02-22T19:37:09.797463+0000","last_fullsized":"2024-02-22T19:37:09.797463+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":148,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798301+0000","last_change":"2024-02-22T19:35:36.900633+0000","last_active":"2024-02-22T19:37:09.798301+0000","last_peered":"2024-02-22T19:37:09.798301+0000","last_clean":"2024-02-22T19:37:09.798301+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:37:09.798301+0000","last_undegraded":"2024-02-22T19:37:09.798301+0000","last_fullsized":"2024-02-22T19:37:09.798301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":35,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793819+0000","last_change":"2024-02-22T19:36:20.071572+0000","last_active":"2024-02-22T19:37:09.793819+0000","last_peered":"2024-02-22T19:37:09.793819+0000","last_clean":"2024-02-22T19:37:09.793819+0000","last_became_active":"2024-02-22T19:35:58.418647+0000","last_became_peered":"2024-02-22T19:35:58.418647+0000","last_unstale":"2024-02-22T19:37:09.793819+0000","last_undegraded":"2024-02-22T19:37:09.793819+0000","last_fullsized":"2024-02-22T19:37:09.793819+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.071500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.071500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"185'7605","reported_seq":7644,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.349324+0000","last_change":"2024-02-22T19:35:19.745819+0000","last_active":"2024-02-22T19:37:14.349324+0000","last_peered":"2024-02-22T19:37:14.349324+0000","last_clean":"2024-02-22T19:37:14.349324+0000","last_became_active":"2024-02-22T19:35:19.745509+0000","last_became_peered":"2024-02-22T19:35:19.745509+0000","last_unstale":"2024-02-22T19:37:14.349324+0000","last_undegraded":"2024-02-22T19:37:14.349324+0000","last_fullsized":"2024-02-22T19:37:14.349324+0000","mapping_epoch":154,"log_start":"165'3100","ondisk_log_start":"165'3100","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4505,"ondisk_log_size":4505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":402980864,"num_objects":6149,"num_object_clones":0,"num_object_copies":18447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7605,"num_write_kb":440128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":111,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.790233+0000","last_change":"2024-02-22T19:36:58.979544+0000","last_active":"2024-02-22T19:37:09.790233+0000","last_peered":"2024-02-22T19:37:09.790233+0000","last_clean":"2024-02-22T19:37:09.790233+0000","last_became_active":"2024-02-22T19:31:50.008757+0000","last_became_peered":"2024-02-22T19:31:50.008757+0000","last_unstale":"2024-02-22T19:37:09.790233+0000","last_undegraded":"2024-02-22T19:37:09.790233+0000","last_fullsized":"2024-02-22T19:37:09.790233+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.979469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.979469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"100'9","reported_seq":262,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803592+0000","last_change":"2024-02-22T19:34:43.126256+0000","last_active":"2024-02-22T19:37:09.803592+0000","last_peered":"2024-02-22T19:37:09.803592+0000","last_clean":"2024-02-22T19:37:09.803592+0000","last_became_active":"2024-02-22T19:31:50.006085+0000","last_became_peered":"2024-02-22T19:31:50.006085+0000","last_unstale":"2024-02-22T19:37:09.803592+0000","last_undegraded":"2024-02-22T19:37:09.803592+0000","last_fullsized":"2024-02-22T19:37:09.803592+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":149,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.797382+0000","last_change":"2024-02-22T19:35:27.352610+0000","last_active":"2024-02-22T19:37:09.797382+0000","last_peered":"2024-02-22T19:37:09.797382+0000","last_clean":"2024-02-22T19:37:09.797382+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:37:09.797382+0000","last_undegraded":"2024-02-22T19:37:09.797382+0000","last_fullsized":"2024-02-22T19:37:09.797382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:27.352545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.352545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":42,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793163+0000","last_change":"2024-02-22T19:36:16.249180+0000","last_active":"2024-02-22T19:37:09.793163+0000","last_peered":"2024-02-22T19:37:09.793163+0000","last_clean":"2024-02-22T19:37:09.793163+0000","last_became_active":"2024-02-22T19:36:16.154859+0000","last_became_peered":"2024-02-22T19:36:16.154859+0000","last_unstale":"2024-02-22T19:37:09.793163+0000","last_undegraded":"2024-02-22T19:37:09.793163+0000","last_fullsized":"2024-02-22T19:37:09.793163+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:35:24.795027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.795027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"185'7644","reported_seq":7683,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.230567+0000","last_change":"2024-02-22T19:35:19.225232+0000","last_active":"2024-02-22T19:37:14.230567+0000","last_peered":"2024-02-22T19:37:14.230567+0000","last_clean":"2024-02-22T19:37:14.230567+0000","last_became_active":"2024-02-22T19:35:19.224941+0000","last_became_peered":"2024-02-22T19:35:19.224941+0000","last_unstale":"2024-02-22T19:37:14.230567+0000","last_undegraded":"2024-02-22T19:37:14.230567+0000","last_fullsized":"2024-02-22T19:37:14.230567+0000","mapping_epoch":154,"log_start":"166'3500","ondisk_log_start":"166'3500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4144,"ondisk_log_size":4144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406716416,"num_objects":6206,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7644,"num_write_kb":443200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":147,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.858272+0000","last_change":"2024-02-22T19:35:37.337349+0000","last_active":"2024-02-22T19:37:09.858272+0000","last_peered":"2024-02-22T19:37:09.858272+0000","last_clean":"2024-02-22T19:37:09.858272+0000","last_became_active":"2024-02-22T19:31:50.006191+0000","last_became_peered":"2024-02-22T19:31:50.006191+0000","last_unstale":"2024-02-22T19:37:09.858272+0000","last_undegraded":"2024-02-22T19:37:09.858272+0000","last_fullsized":"2024-02-22T19:37:09.858272+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":102,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803540+0000","last_change":"2024-02-22T19:34:39.082294+0000","last_active":"2024-02-22T19:37:09.803540+0000","last_peered":"2024-02-22T19:37:09.803540+0000","last_clean":"2024-02-22T19:37:09.803540+0000","last_became_active":"2024-02-22T19:31:50.015908+0000","last_became_peered":"2024-02-22T19:31:50.015908+0000","last_unstale":"2024-02-22T19:37:09.803540+0000","last_undegraded":"2024-02-22T19:37:09.803540+0000","last_fullsized":"2024-02-22T19:37:09.803540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:39.082190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:39.082190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.798357+0000","last_change":"2024-02-22T19:36:08.709226+0000","last_active":"2024-02-22T19:37:09.798357+0000","last_peered":"2024-02-22T19:37:09.798357+0000","last_clean":"2024-02-22T19:37:09.798357+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:37:09.798357+0000","last_undegraded":"2024-02-22T19:37:09.798357+0000","last_fullsized":"2024-02-22T19:37:09.798357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.709163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.709163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":169,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803428+0000","last_change":"2024-02-22T19:36:11.687594+0000","last_active":"2024-02-22T19:37:09.803428+0000","last_peered":"2024-02-22T19:37:09.803428+0000","last_clean":"2024-02-22T19:37:09.803428+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:37:09.803428+0000","last_undegraded":"2024-02-22T19:37:09.803428+0000","last_fullsized":"2024-02-22T19:37:09.803428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:36:11.687522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.687522+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":35,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.793551+0000","last_change":"2024-02-22T19:36:23.022761+0000","last_active":"2024-02-22T19:37:09.793551+0000","last_peered":"2024-02-22T19:37:09.793551+0000","last_clean":"2024-02-22T19:37:09.793551+0000","last_became_active":"2024-02-22T19:35:58.422620+0000","last_became_peered":"2024-02-22T19:35:58.422620+0000","last_unstale":"2024-02-22T19:37:09.793551+0000","last_undegraded":"2024-02-22T19:37:09.793551+0000","last_fullsized":"2024-02-22T19:37:09.793551+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.022678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.022678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"185'7750","reported_seq":7789,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.293434+0000","last_change":"2024-02-22T19:35:19.745821+0000","last_active":"2024-02-22T19:37:14.293434+0000","last_peered":"2024-02-22T19:37:14.293434+0000","last_clean":"2024-02-22T19:37:14.293434+0000","last_became_active":"2024-02-22T19:35:19.745498+0000","last_became_peered":"2024-02-22T19:35:19.745498+0000","last_unstale":"2024-02-22T19:37:14.293434+0000","last_undegraded":"2024-02-22T19:37:14.293434+0000","last_fullsized":"2024-02-22T19:37:14.293434+0000","mapping_epoch":154,"log_start":"165'3200","ondisk_log_start":"165'3200","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4550,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":410779648,"num_objects":6268,"num_object_clones":0,"num_object_copies":18804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7750,"num_write_kb":448576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":146,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.795730+0000","last_change":"2024-02-22T19:36:11.024673+0000","last_active":"2024-02-22T19:37:09.795730+0000","last_peered":"2024-02-22T19:37:09.795730+0000","last_clean":"2024-02-22T19:37:09.795730+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:37:09.795730+0000","last_undegraded":"2024-02-22T19:37:09.795730+0000","last_fullsized":"2024-02-22T19:37:09.795730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.024573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.024573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":148,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.790073+0000","last_change":"2024-02-22T19:36:09.886117+0000","last_active":"2024-02-22T19:37:09.790073+0000","last_peered":"2024-02-22T19:37:09.790073+0000","last_clean":"2024-02-22T19:37:09.790073+0000","last_became_active":"2024-02-22T19:29:05.088493+0000","last_became_peered":"2024-02-22T19:29:05.088493+0000","last_unstale":"2024-02-22T19:37:09.790073+0000","last_undegraded":"2024-02-22T19:37:09.790073+0000","last_fullsized":"2024-02-22T19:37:09.790073+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.885998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.885998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":154,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.796384+0000","last_change":"2024-02-22T19:36:28.180370+0000","last_active":"2024-02-22T19:37:09.796384+0000","last_peered":"2024-02-22T19:37:09.796384+0000","last_clean":"2024-02-22T19:37:09.796384+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:37:09.796384+0000","last_undegraded":"2024-02-22T19:37:09.796384+0000","last_fullsized":"2024-02-22T19:37:09.796384+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.180297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.180297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801334+0000","last_change":"2024-02-22T19:34:57.145803+0000","last_active":"2024-02-22T19:37:09.801334+0000","last_peered":"2024-02-22T19:37:09.801334+0000","last_clean":"2024-02-22T19:37:09.801334+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:37:09.801334+0000","last_undegraded":"2024-02-22T19:37:09.801334+0000","last_fullsized":"2024-02-22T19:37:09.801334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:34:57.145716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:57.145716+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.801649+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:37:09.801649+0000","last_peered":"2024-02-22T19:37:09.801649+0000","last_clean":"2024-02-22T19:37:09.801649+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:37:09.801649+0000","last_undegraded":"2024-02-22T19:37:09.801649+0000","last_fullsized":"2024-02-22T19:37:09.801649+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"185'7611","reported_seq":7650,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:14.372192+0000","last_change":"2024-02-22T19:35:19.229848+0000","last_active":"2024-02-22T19:37:14.372192+0000","last_peered":"2024-02-22T19:37:14.372192+0000","last_clean":"2024-02-22T19:37:14.372192+0000","last_became_active":"2024-02-22T19:35:19.229525+0000","last_became_peered":"2024-02-22T19:35:19.229525+0000","last_unstale":"2024-02-22T19:37:14.372192+0000","last_undegraded":"2024-02-22T19:37:14.372192+0000","last_fullsized":"2024-02-22T19:37:14.372192+0000","mapping_epoch":154,"log_start":"166'3700","ondisk_log_start":"166'3700","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":3911,"ondisk_log_size":3911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408354816,"num_objects":6231,"num_object_clones":0,"num_object_copies":18693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7611,"num_write_kb":442944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":145,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.803679+0000","last_change":"2024-02-22T19:34:54.019363+0000","last_active":"2024-02-22T19:37:09.803679+0000","last_peered":"2024-02-22T19:37:09.803679+0000","last_clean":"2024-02-22T19:37:09.803679+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:37:09.803679+0000","last_undegraded":"2024-02-22T19:37:09.803679+0000","last_fullsized":"2024-02-22T19:37:09.803679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:34:54.019270+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:34:54.019270+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"100'9","reported_seq":273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-02-22T19:37:13.243600+0000","last_change":"2024-02-22T19:37:13.243565+0000","last_active":"2024-02-22T19:37:13.243600+0000","last_peered":"2024-02-22T19:37:13.243600+0000","last_clean":"2024-02-22T19:37:13.243600+0000","last_became_active":"2024-02-22T19:31:50.012418+0000","last_became_peered":"2024-02-22T19:31:50.012418+0000","last_unstale":"2024-02-22T19:37:13.243600+0000","last_undegraded":"2024-02-22T19:37:13.243600+0000","last_fullsized":"2024-02-22T19:37:13.243600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:37:13.243499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:13.243499+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":158,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.861437+0000","last_change":"2024-02-22T19:37:05.357980+0000","last_active":"2024-02-22T19:37:09.861437+0000","last_peered":"2024-02-22T19:37:09.861437+0000","last_clean":"2024-02-22T19:37:09.861437+0000","last_became_active":"2024-02-22T19:29:01.014044+0000","last_became_peered":"2024-02-22T19:29:01.014044+0000","last_unstale":"2024-02-22T19:37:09.861437+0000","last_undegraded":"2024-02-22T19:37:09.861437+0000","last_fullsized":"2024-02-22T19:37:09.861437+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:37:05.357880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.357880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":160,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-02-22T19:37:09.802936+0000","last_change":"2024-02-22T19:35:33.971551+0000","last_active":"2024-02-22T19:37:09.802936+0000","last_peered":"2024-02-22T19:37:09.802936+0000","last_clean":"2024-02-22T19:37:09.802936+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:37:09.802936+0000","last_undegraded":"2024-02-22T19:37:09.802936+0000","last_fullsized":"2024-02-22T19:37:09.802936+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:33.971446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.971446+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6565462040,"num_objects":100182,"num_object_clones":0,"num_object_copies":300546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123282,"num_write_kb":7150785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19696660480,"data_stored":19696648264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70282,"ondisk_log_size":70282,"up":48,"acting":48,"num_store_stats":7},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":487,"num_read_kb":482,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":7},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":696,"num_read_kb":453,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1650,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266790,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2348240,"kb_used_data":2038728,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91396912,"statfs":{"total":95995035648,"available":93590437888,"internally_reserved":0,"allocated":2087657472,"data_stored":2087110251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":2.1680000000000001,"15min":1.117},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":6.8040000000000003,"5min":42.597999999999999,"15min":42.597999999999999},"last":0.23200000000000001},{"interface":"front","average":{"1min":1.867,"5min":2.085,"15min":1.052},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.7729999999999997,"5min":42.155999999999999,"15min":42.155999999999999},"last":0.318}]},{"osd":1,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.9830000000000001,"15min":1.042},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":7.1840000000000002,"5min":38.188000000000002,"15min":38.188000000000002},"last":7.1840000000000002},{"interface":"front","average":{"1min":2.2189999999999999,"5min":2.2829999999999999,"15min":1.1279999999999999},"min":{"1min":0.28299999999999997,"5min":0.219,"15min":0.219},"max":{"1min":7.9020000000000001,"5min":37.938000000000002,"15min":37.938000000000002},"last":7.9020000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.6859999999999999,"15min":0.93100000000000005},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":6.5419999999999998,"5min":41.640999999999998,"15min":41.640999999999998},"last":0.31},{"interface":"front","average":{"1min":1.391,"5min":1.875,"15min":1.034},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":8.2929999999999993,"5min":41.722000000000001,"15min":41.722000000000001},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.0049999999999999,"5min":2.0390000000000001,"15min":1.087},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":7.2510000000000003,"5min":41.457000000000001,"15min":41.457000000000001},"last":7.2510000000000003},{"interface":"front","average":{"1min":2.2370000000000001,"5min":1.98,"15min":1.069},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":7.3940000000000001,"5min":41.311999999999998,"15min":41.311999999999998},"last":7.3940000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59899999999999998,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.1800000000000002,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53300000000000003,"15min":0.48299999999999998},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.253},"max":{"1min":3.028,"5min":3.4780000000000002,"15min":3.4780000000000002},"last":0.33200000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.59699999999999998,"15min":0.52100000000000002},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.66100000000000003,"15min":0.55400000000000005},"min":{"1min":0.30399999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.2029999999999998,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.71999999999999997,"15min":0.625},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.2189999999999999,"5min":4.7160000000000002,"15min":12.199999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5509999999999999,"5min":3.4430000000000001,"15min":12.067},"last":0.35599999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528427,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3149108,"kb_used_data":2843692,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":90596044,"statfs":{"total":95995035648,"available":92770349056,"internally_reserved":0,"allocated":2911940608,"data_stored":2911421127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.548,"15min":0.85899999999999999},"min":{"1min":0.23100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":6.0510000000000002,"5min":17.885000000000002,"15min":17.885000000000002},"last":1.53},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.389,"15min":0.80800000000000005},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.5939999999999999,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":1.1479999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2050000000000001,"15min":0.76900000000000002},"min":{"1min":0.30499999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.931,"5min":11.301,"15min":11.301},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1679999999999999,"15min":0.73899999999999999},"min":{"1min":0.27800000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.516,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":1.6850000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.6100000000000001,"15min":0.89300000000000002},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.249},"max":{"1min":7.2910000000000004,"5min":12.859999999999999,"15min":12.859999999999999},"last":2.024},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.411,"15min":0.83399999999999996},"min":{"1min":0.308,"5min":0.25,"15min":0.25},"max":{"1min":6.3890000000000002,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":2.1389999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.9490000000000001,"15min":1.0089999999999999},"min":{"1min":0.33100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.9870000000000001,"5min":38.762,"15min":38.762},"last":0.374},{"interface":"front","average":{"1min":0.99199999999999999,"5min":2.2360000000000002,"15min":1.1160000000000001},"min":{"1min":0.33600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.3849999999999998,"5min":35.612000000000002,"15min":35.612000000000002},"last":1.615}]},{"osd":4,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.48999999999999999,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.318}]},{"osd":5,"last update":"Thu Feb 22 19:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.54700000000000004,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.60999999999999999,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":4.0380000000000003,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.38}]},{"osd":7,"last update":"Thu Feb 22 19:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.54600000000000004},"min":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.899,"5min":3.488,"15min":3.488},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":0.60799999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9550000000000001,"5min":3.593,"15min":3.593},"last":0.35799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3092964,"kb_used_data":2795740,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":90652188,"statfs":{"total":95995035648,"available":92827840512,"internally_reserved":0,"allocated":2862837760,"data_stored":2862270589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4689999999999999,"5min":2.48,"15min":1.0600000000000001},"min":{"1min":0.308,"5min":0.20699999999999999,"15min":0.20499999999999999},"max":{"1min":10.6,"5min":15.613,"15min":15.613},"last":0.60899999999999999},{"interface":"front","average":{"1min":3.25,"5min":2.7490000000000001,"15min":1.1679999999999999},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.23899999999999999},"max":{"1min":8.1820000000000004,"5min":13.515000000000001,"15min":13.515000000000001},"last":0.20000000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:37:06 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":2.6709999999999998,"15min":1.1850000000000001},"min":{"1min":0.35399999999999998,"5min":0.22,"15min":0.22},"max":{"1min":9.0540000000000003,"5min":16.503,"15min":16.503},"last":0.29799999999999999},{"interface":"front","average":{"1min":4.3310000000000004,"5min":2.988,"15min":1.272},"min":{"1min":0.24399999999999999,"5min":0.192,"15min":0.192},"max":{"1min":14.994,"5min":14.994,"15min":14.994},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:36:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0640000000000001,"5min":2.899,"15min":1.2609999999999999},"min":{"1min":0.26600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":10.734999999999999,"5min":25.145,"15min":25.145},"last":1.8879999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":2.7599999999999998,"15min":1.2529999999999999},"min":{"1min":0.371,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":4.6349999999999998,"5min":22.25,"15min":22.25},"last":0.251}]},{"osd":4,"last update":"Thu Feb 22 19:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.7829999999999999,"15min":0.99399999999999999},"min":{"1min":0.34599999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.5129999999999999,"5min":15.542999999999999,"15min":15.542999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6279999999999999,"15min":0.91700000000000004},"min":{"1min":0.30499999999999999,"5min":0.245,"15min":0.245},"max":{"1min":10.706,"5min":10.706,"15min":10.706},"last":1.335}]},{"osd":5,"last update":"Thu Feb 22 19:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.6930000000000001,"15min":0.95999999999999996},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":4.5389999999999997,"5min":15.587999999999999,"15min":15.587999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.5600000000000001,"15min":0.90100000000000002},"min":{"1min":0.34799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.2930000000000001,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.37}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":2.2519999999999998,"5min":1.696,"15min":0.92600000000000005},"min":{"1min":0.35199999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":10.631,"5min":10.631,"15min":10.631},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.8169999999999999,"15min":1},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.5810000000000004,"5min":15.651999999999999,"15min":15.651999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:37:06 2024","interfaces":[{"interface":"back","average":{"1min":2.96,"5min":1.9910000000000001,"15min":1.0269999999999999},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":13.169,"5min":13.169,"15min":13.169},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.746,"5min":1.7330000000000001,"15min":0.95699999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":10.673,"5min":10.673,"15min":10.673},"last":1.3660000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738505,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3136456,"kb_used_data":2839168,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":90608696,"statfs":{"total":95995035648,"available":92783304704,"internally_reserved":0,"allocated":2907308032,"data_stored":2906769329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:36:58 2024","interfaces":[{"interface":"back","average":{"1min":1.966,"5min":2.262,"15min":0.98999999999999999},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":14.352,"5min":14.352,"15min":14.352},"last":3.726},{"interface":"front","average":{"1min":4.2530000000000001,"5min":2.54,"15min":1.093},"min":{"1min":0.248,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":10.189,"5min":13.013999999999999,"15min":13.013999999999999},"last":3.0939999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:36:17 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":2.5590000000000002,"15min":1.0880000000000001},"min":{"1min":0.25800000000000001,"5min":0.23200000000000001,"15min":0.22600000000000001},"max":{"1min":4.3700000000000001,"5min":19.927,"15min":19.927},"last":5.508},{"interface":"front","average":{"1min":0.89200000000000002,"5min":2.3719999999999999,"15min":1.0409999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.8889999999999998,"5min":17.907,"15min":17.907},"last":3.5369999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:36:32 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.1880000000000002,"15min":1.006},"min":{"1min":0.27900000000000003,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":6.6109999999999998,"5min":13.06,"15min":13.06},"last":12.754},{"interface":"front","average":{"1min":2.4180000000000001,"5min":2.903,"15min":1.256},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":8.8659999999999997,"5min":22.936,"15min":22.936},"last":0.433}]},{"osd":4,"last update":"Thu Feb 22 19:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.3340000000000001,"15min":0.82699999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":3.0939999999999999,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":0.38},{"interface":"front","average":{"1min":2.3959999999999999,"5min":2.3330000000000002,"15min":1.165},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.8040000000000003,"5min":13.874000000000001,"15min":13.874000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:36:26 2024","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":2.274,"15min":1.129},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33100000000000002},"max":{"1min":8.8369999999999997,"5min":13.903,"15min":13.903},"last":0.498},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.3640000000000001,"15min":0.83199999999999996},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.5230000000000001,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:36:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.4390000000000001,"15min":0.872},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.4609999999999999,"5min":7.0830000000000002,"15min":7.0830000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.839,"5min":1.5669999999999999,"15min":0.91800000000000004},"min":{"1min":0.40899999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.4550000000000001,"5min":7.5410000000000004,"15min":7.5410000000000004},"last":3.0640000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:37:10 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5369999999999999,"15min":0.90000000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":4.7140000000000004,"5min":7.0620000000000003,"15min":7.0620000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":2.4089999999999998,"5min":1.8069999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":8.4450000000000003,"5min":8.4450000000000003,"15min":8.4450000000000003},"last":3.024}]}]},{"osd":2,"up_from":18,"seq":77309411454,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3100864,"kb_used_data":2795384,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":90644288,"statfs":{"total":95995035648,"available":92819750912,"internally_reserved":0,"allocated":2862473216,"data_stored":2861941565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:09 2024","interfaces":[{"interface":"back","average":{"1min":2.919,"5min":2.0289999999999999,"15min":1.002},"min":{"1min":0.245,"5min":0.183,"15min":0.183},"max":{"1min":22.338999999999999,"5min":22.338999999999999,"15min":22.338999999999999},"last":0.219},{"interface":"front","average":{"1min":3.4870000000000001,"5min":2.5539999999999998,"15min":1.2250000000000001},"min":{"1min":0.21099999999999999,"5min":0.191,"15min":0.191},"max":{"1min":22.170000000000002,"5min":22.170000000000002,"15min":22.170000000000002},"last":2.9289999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:37:09 2024","interfaces":[{"interface":"back","average":{"1min":2.4990000000000001,"5min":2.3260000000000001,"15min":1.069},"min":{"1min":0.21299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":9.5370000000000008,"5min":10.183,"15min":10.183},"last":0.42399999999999999},{"interface":"front","average":{"1min":2.363,"5min":1.9550000000000001,"15min":0.93100000000000005},"min":{"1min":0.22900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":9.1039999999999992,"5min":12.393000000000001,"15min":12.393000000000001},"last":2.1600000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:36:41 2024","interfaces":[{"interface":"back","average":{"1min":2.1110000000000002,"5min":2.4860000000000002,"15min":1.133},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.6639999999999997,"5min":30.071000000000002,"15min":30.071000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":2.343,"5min":2.895,"15min":1.2549999999999999},"min":{"1min":0.222,"5min":0.192,"15min":0.192},"max":{"1min":7.2160000000000002,"5min":32.119,"15min":32.119},"last":2.839}]},{"osd":4,"last update":"Thu Feb 22 19:36:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.756,"15min":0.97099999999999997},"min":{"1min":0.28100000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.9690000000000003,"5min":6.3890000000000002,"15min":6.3890000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.55,"5min":1.131,"15min":0.77000000000000002},"min":{"1min":0.34399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":4.7619999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.0049999999999999,"15min":0.749},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.895,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.6639999999999999,"15min":0.95099999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.2410000000000001,"5min":6.4320000000000004,"15min":6.4320000000000004},"last":2.7519999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.905,"15min":1.038},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.0659999999999998,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":2.718},{"interface":"front","average":{"1min":1.319,"5min":1.1639999999999999,"15min":0.80600000000000005},"min":{"1min":0.318,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.79,"5min":9.5990000000000002,"15min":9.5990000000000002},"last":0.504}]},{"osd":7,"last update":"Thu Feb 22 19:37:09 2024","interfaces":[{"interface":"back","average":{"1min":2.1190000000000002,"5min":1.99,"15min":1.0700000000000001},"min":{"1min":0.34899999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.0369999999999999,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.9019999999999999,"5min":1.3640000000000001,"15min":0.85799999999999998},"min":{"1min":0.36099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.835,"5min":9.5660000000000007,"15min":9.5660000000000007},"last":0.54600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149816,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3911444,"kb_used_data":3605900,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":89833708,"statfs":{"total":95995035648,"available":91989716992,"internally_reserved":0,"allocated":3692441600,"data_stored":3691890965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":2.4409999999999998,"15min":1.093},"min":{"1min":0.22,"5min":0.19400000000000001,"15min":0.193},"max":{"1min":3.9020000000000001,"5min":12.929,"15min":12.929},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.954,"5min":2.8959999999999999,"15min":1.272},"min":{"1min":0.29099999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.8710000000000004,"5min":14.831,"15min":14.831},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:36:31 2024","interfaces":[{"interface":"back","average":{"1min":2.2250000000000001,"5min":2.927,"15min":1.2749999999999999},"min":{"1min":0.29599999999999999,"5min":0.22,"15min":0.22},"max":{"1min":6.2229999999999999,"5min":18.259,"15min":18.259},"last":6.9429999999999996},{"interface":"front","average":{"1min":1.3700000000000001,"5min":2.1419999999999999,"15min":1.0069999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":6.4539999999999997,"5min":13.154,"15min":13.154},"last":2.6680000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.867,"5min":2.7170000000000001,"15min":1.2210000000000001},"min":{"1min":0.35499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":8.2460000000000004,"5min":16.271000000000001,"15min":16.271000000000001},"last":8.1389999999999993},{"interface":"front","average":{"1min":1.335,"5min":2.5019999999999998,"15min":1.139},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":3.6760000000000002,"5min":15.143000000000001,"15min":15.143000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":2.0680000000000001,"15min":1.1140000000000001},"min":{"1min":0.27300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":9.4130000000000003,"5min":13.680999999999999,"15min":13.680999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.649,"5min":1.706,"15min":0.98499999999999999},"min":{"1min":0.27800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":8.1470000000000002,"5min":11.366,"15min":11.366},"last":2.637}]},{"osd":5,"last update":"Thu Feb 22 19:36:21 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.752,"15min":0.97099999999999997},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":8.0259999999999998,"5min":12.879,"15min":12.879},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.395,"5min":1.9650000000000001,"15min":1.0389999999999999},"min":{"1min":0.36699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.1070000000000002,"5min":13.707000000000001,"15min":13.707000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.867,"15min":1.0089999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.9640000000000004,"5min":12.898999999999999,"15min":12.898999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.9530000000000001,"5min":2.2320000000000002,"15min":1.165},"min":{"1min":0.42699999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":9.048,"5min":13.803000000000001,"15min":13.803000000000001},"last":1.79}]},{"osd":7,"last update":"Thu Feb 22 19:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.6990000000000001,"15min":0.996},"min":{"1min":0.32300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":6.2329999999999997,"5min":11.348000000000001,"15min":11.348000000000001},"last":2.6840000000000002},{"interface":"front","average":{"1min":1.974,"5min":1.8899999999999999,"15min":1.0580000000000001},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":7.9909999999999997,"5min":12.846,"15min":12.846},"last":0.629}]}]},{"osd":4,"up_from":32,"seq":137438953590,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2713360,"kb_used_data":2412040,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91031792,"statfs":{"total":95995035648,"available":93216555008,"internally_reserved":0,"allocated":2469928960,"data_stored":2469382201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:36:58 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":1.796,"15min":0.94599999999999995},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":6.4859999999999998,"5min":24.009,"15min":24.009},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.6870000000000001,"15min":0.92500000000000004},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.5030000000000001,"5min":24.263000000000002,"15min":24.263000000000002},"last":0.98699999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:36:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.5620000000000001,"15min":0.88200000000000001},"min":{"1min":0.23899999999999999,"5min":0.215,"15min":0.215},"max":{"1min":8.3209999999999997,"5min":8.3209999999999997,"15min":8.3209999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.2310000000000001,"15min":0.755},"min":{"1min":0.27100000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.25,"5min":12.445,"15min":12.445},"last":0.251}]},{"osd":2,"last update":"Thu Feb 22 19:36:58 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5209999999999999,"15min":0.879},"min":{"1min":0.32500000000000001,"5min":0.247,"15min":0.247},"max":{"1min":8.3369999999999997,"5min":23.082000000000001,"15min":23.082000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.8180000000000001,"15min":0.96799999999999997},"min":{"1min":0.254,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":5.556,"5min":23.138999999999999,"15min":23.138999999999999},"last":1.1890000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:36:58 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":1.95,"15min":1.0229999999999999},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":9.9139999999999997,"5min":13.476000000000001,"15min":13.476000000000001},"last":0.23100000000000001},{"interface":"front","average":{"1min":1.9790000000000001,"5min":1.609,"15min":0.89300000000000002},"min":{"1min":0.314,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.7930000000000001,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":0.315}]},{"osd":5,"last update":"Thu Feb 22 19:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.55300000000000005,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.54100000000000004,"15min":0.49299999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.53200000000000003,"15min":0.5},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.2559999999999998,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.417,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.437}]}]},{"osd":5,"up_from":0,"seq":167503724656,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2890268672,"data_stored":2890268672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2852388864,"data_stored":2852388864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2856128512,"data_stored":2856124440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3685548032,"data_stored":3685548032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2452619264,"data_stored":2452619264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2887979008,"data_stored":2887974936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2071728128,"data_stored":2071724056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:37:19.323 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-02-22T19:37:19.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set default.rgw.control pg_num 22 2024-02-22T19:37:19.335 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:19.433 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:19.438 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:19.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:19.438 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.264233+0000 osd.0 (osd.0) 150 : cluster [DBG] 8.e scrub ok 2024-02-22T19:37:19.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:18.275956+0000 mon.a (mon.0) 1032 : audit [INF] from='client.? 172.21.15.53:0/3064061054' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:37:19.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.362676+0000 osd.6 (osd.6) 91 : cluster [DBG] 4.a scrub starts 2024-02-22T19:37:19.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.374791+0000 osd.6 (osd.6) 92 : cluster [DBG] 4.a scrub ok 2024-02-22T19:37:19.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:18.457123+0000 mgr.y (mgr.14535) 322 : audit [DBG] from='client.25006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:37:19.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.577781+0000 osd.4 (osd.4) 133 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:37:19.439 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.583032+0000 osd.4 (osd.4) 134 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.264233+0000 osd.0 (osd.0) 150 : cluster [DBG] 8.e scrub ok 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:18.275956+0000 mon.a (mon.0) 1032 : audit [INF] from='client.? 172.21.15.53:0/3064061054' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.362676+0000 osd.6 (osd.6) 91 : cluster [DBG] 4.a scrub starts 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.374791+0000 osd.6 (osd.6) 92 : cluster [DBG] 4.a scrub ok 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:18.457123+0000 mgr.y (mgr.14535) 322 : audit [DBG] from='client.25006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.577781+0000 osd.4 (osd.4) 133 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.583032+0000 osd.4 (osd.4) 134 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:37:19.440 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:37:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:19] "GET /metrics HTTP/1.1" 200 175804 "" "Prometheus/2.33.4" 2024-02-22T19:37:19.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.264233+0000 osd.0 (osd.0) 150 : cluster [DBG] 8.e scrub ok 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:18.275956+0000 mon.a (mon.0) 1032 : audit [INF] from='client.? 172.21.15.53:0/3064061054' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.362676+0000 osd.6 (osd.6) 91 : cluster [DBG] 4.a scrub starts 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.374791+0000 osd.6 (osd.6) 92 : cluster [DBG] 4.a scrub ok 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:18.457123+0000 mgr.y (mgr.14535) 322 : audit [DBG] from='client.25006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.577781+0000 osd.4 (osd.4) 133 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:37:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.583032+0000 osd.4 (osd.4) 134 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:37:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:19.721 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:19.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:19.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:19.836 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:19.869 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:19.895 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:19.898 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:19.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:19.999 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:20.070 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:20.137 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:20.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:20.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:20.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:20.302 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:20.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.920808+0000 osd.0 (osd.0) 151 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:37:20.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:18.925501+0000 osd.0 (osd.0) 152 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:37:20.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:19.177157+0000 osd.1 (osd.1) 131 : cluster [DBG] 8.3 scrub starts 2024-02-22T19:37:20.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:19.186532+0000 mon.a (mon.0) 1033 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:20.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:19.207927+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? 172.21.15.53:0/3064061054' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:19.207981+0000 mon.a (mon.0) 1035 : cluster [DBG] osdmap e186: 8 total, 8 up, 7 in 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:19.441574+0000 mgr.y (mgr.14535) 323 : cluster [DBG] pgmap v362: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.920808+0000 osd.0 (osd.0) 151 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:18.925501+0000 osd.0 (osd.0) 152 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:19.177157+0000 osd.1 (osd.1) 131 : cluster [DBG] 8.3 scrub starts 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:19.186532+0000 mon.a (mon.0) 1033 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:19.207927+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? 172.21.15.53:0/3064061054' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:19.207981+0000 mon.a (mon.0) 1035 : cluster [DBG] osdmap e186: 8 total, 8 up, 7 in 2024-02-22T19:37:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:19.441574+0000 mgr.y (mgr.14535) 323 : cluster [DBG] pgmap v362: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-02-22T19:37:20.579 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:20.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:20.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.920808+0000 osd.0 (osd.0) 151 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:18.925501+0000 osd.0 (osd.0) 152 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:19.177157+0000 osd.1 (osd.1) 131 : cluster [DBG] 8.3 scrub starts 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:19.186532+0000 mon.a (mon.0) 1033 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:19.207927+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? 172.21.15.53:0/3064061054' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:19.207981+0000 mon.a (mon.0) 1035 : cluster [DBG] osdmap e186: 8 total, 8 up, 7 in 2024-02-22T19:37:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:19.441574+0000 mgr.y (mgr.14535) 323 : cluster [DBG] pgmap v362: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-02-22T19:37:20.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:20.722 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:20.744 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:20.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:20.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:20.848 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:20.909 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:21.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:21.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:21.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:21.283 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:21.670 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:21.734 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:21.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:21.835 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:21.837 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:37:21.838 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:37:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:21.904 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:21.904 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:22.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:22.039 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:22.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:22.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:22.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:22.651 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:37:22.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:21.441971+0000 mgr.y (mgr.14535) 324 : cluster [DBG] pgmap v363: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-02-22T19:37:22.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:21.441971+0000 mgr.y (mgr.14535) 324 : cluster [DBG] pgmap v363: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-02-22T19:37:22.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:21.441971+0000 mgr.y (mgr.14535) 324 : cluster [DBG] pgmap v363: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-02-22T19:37:22.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:37:23.032 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:23.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:23.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:23.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:23.272 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:23.301 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:23.549 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:23.554 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:23.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:23.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:23.803 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:23.815 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:23.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:23.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:22.570887+0000 mon.a (mon.0) 1036 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:37:23.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:23.359215+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:23.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:23.389787+0000 mon.a (mon.0) 1038 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:37:23.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:23.413657+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:37:23.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:23.426898+0000 mon.a (mon.0) 1040 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:23.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:22.570887+0000 mon.a (mon.0) 1036 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:37:23.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:23.359215+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:23.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:23.389787+0000 mon.a (mon.0) 1038 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:37:23.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:23.413657+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:37:23.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:23.426898+0000 mon.a (mon.0) 1040 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:22.570887+0000 mon.a (mon.0) 1036 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:37:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:23.359215+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:23.389787+0000 mon.a (mon.0) 1038 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:37:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:23.413657+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:37:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:23.426898+0000 mon.a (mon.0) 1040 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:37:23.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:24.009 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:24.155 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:37:24.193 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:37:24.225 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:37:24.275 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:37:24.310 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:37:24.349 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:37:24.381 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:37:24.415 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:37:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:37:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:23.442343+0000 mgr.y (mgr.14535) 325 : cluster [DBG] pgmap v364: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 961 op/s 2024-02-22T19:37:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:23.929126+0000 osd.6 (osd.6) 93 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:23.929197+0000 osd.6 (osd.6) 94 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:24.114065+0000 mon.b (mon.1) 57 : audit [DBG] from='client.? 172.21.15.53:0/9479746' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:23.442343+0000 mgr.y (mgr.14535) 325 : cluster [DBG] pgmap v364: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 961 op/s 2024-02-22T19:37:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:23.929126+0000 osd.6 (osd.6) 93 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:23.929197+0000 osd.6 (osd.6) 94 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:24.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:24.114065+0000 mon.b (mon.1) 57 : audit [DBG] from='client.? 172.21.15.53:0/9479746' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:23.442343+0000 mgr.y (mgr.14535) 325 : cluster [DBG] pgmap v364: 158 pgs: 158 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 961 op/s 2024-02-22T19:37:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:23.929126+0000 osd.6 (osd.6) 93 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:23.929197+0000 osd.6 (osd.6) 94 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:24.114065+0000 mon.b (mon.1) 57 : audit [DBG] from='client.? 172.21.15.53:0/9479746' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:24.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:37:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:37:24.853 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:37:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:37:25.322 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:25.331 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:25.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:25.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:25.553 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:25.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:25.571 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:24.538408+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e187: 8 total, 8 up, 7 in 2024-02-22T19:37:25.572 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:24.695721+0000 mgr.y (mgr.14535) 326 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:25.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:24.538408+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e187: 8 total, 8 up, 7 in 2024-02-22T19:37:25.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:24.695721+0000 mgr.y (mgr.14535) 326 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:25.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:25.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:25.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:25.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:24.538408+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e187: 8 total, 8 up, 7 in 2024-02-22T19:37:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:24.695721+0000 mgr.y (mgr.14535) 326 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:25.906 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:26.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:26.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:26.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:26.432 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:26.435 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:26.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:26.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:26.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:25.456521+0000 mgr.y (mgr.14535) 327 : cluster [DBG] pgmap v366: 158 pgs: 158 active+clean; 5.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-02-22T19:37:26.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:26.224269+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.53:0/3352689217' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:37:26.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:26.224893+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:37:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 5 pg_num to 22 2024-02-22T19:37:26.602 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:26.606 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:26.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:26.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:26.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:25.456521+0000 mgr.y (mgr.14535) 327 : cluster [DBG] pgmap v366: 158 pgs: 158 active+clean; 5.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-02-22T19:37:26.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:26.224269+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.53:0/3352689217' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:37:26.738 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:26.224893+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:37:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:26.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:26.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:26.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:26.906 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:26.930 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:26.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:26.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:26.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:25.456521+0000 mgr.y (mgr.14535) 327 : cluster [DBG] pgmap v366: 158 pgs: 158 active+clean; 5.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-02-22T19:37:26.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:26.224269+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.53:0/3352689217' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:37:26.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:26.224893+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:37:27.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:27.114 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:27.661 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:26.553961+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]': finished 2024-02-22T19:37:27.661 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:26.554029+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e188: 8 total, 8 up, 7 in 2024-02-22T19:37:27.661 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:27.461011+0000 mon.a (mon.0) 1045 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-22T19:37:27.661 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:26.553961+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]': finished 2024-02-22T19:37:27.661 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:26.554029+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e188: 8 total, 8 up, 7 in 2024-02-22T19:37:27.662 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:27.461011+0000 mon.a (mon.0) 1045 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-22T19:37:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:27.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:27.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:27.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:27.842 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:27.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:26.553961+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "22"}]': finished 2024-02-22T19:37:27.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:26.554029+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e188: 8 total, 8 up, 7 in 2024-02-22T19:37:27.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:27.461011+0000 mon.a (mon.0) 1045 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-22T19:37:28.435 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:27.456933+0000 mgr.y (mgr.14535) 328 : cluster [DBG] pgmap v368: 158 pgs: 158 active+clean; 5.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 638 B/s rd, 0 B/s wr, 618 op/s 2024-02-22T19:37:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:28.188745+0000 mon.a (mon.0) 1046 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-22T19:37:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:28.188807+0000 mon.a (mon.0) 1047 : cluster [DBG] osdmap e189: 8 total, 8 up, 7 in 2024-02-22T19:37:28.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:27.456933+0000 mgr.y (mgr.14535) 328 : cluster [DBG] pgmap v368: 158 pgs: 158 active+clean; 5.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 638 B/s rd, 0 B/s wr, 618 op/s 2024-02-22T19:37:28.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:28.188745+0000 mon.a (mon.0) 1046 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-22T19:37:28.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:28.188807+0000 mon.a (mon.0) 1047 : cluster [DBG] osdmap e189: 8 total, 8 up, 7 in 2024-02-22T19:37:28.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:27.456933+0000 mgr.y (mgr.14535) 328 : cluster [DBG] pgmap v368: 158 pgs: 158 active+clean; 5.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 638 B/s rd, 0 B/s wr, 618 op/s 2024-02-22T19:37:28.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:28.188745+0000 mon.a (mon.0) 1046 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-22T19:37:28.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:28.188807+0000 mon.a (mon.0) 1047 : cluster [DBG] osdmap e189: 8 total, 8 up, 7 in 2024-02-22T19:37:29.128 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:29.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:29.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:29.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:29.274 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:29.299 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:29.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:29.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:29.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:37:29.404 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:29.429 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:37:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:29] "GET /metrics HTTP/1.1" 200 175821 "" "Prometheus/2.33.4" 2024-02-22T19:37:29.512 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:29.832 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20843 7.0 7:0b664233:::benchmark_data_smithi053_107660_object20842:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.46517853400002, 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472188600000001, 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20843 2024-02-22T19:37:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005336+0000", 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005334+0000", 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4664999999999998e-05 2024-02-22T19:37:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005359+0000", 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1853e-05 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005371+0000", 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005378+0000", 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016187800000000001 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005539+0000", 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1446e-05 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.005551+0000", 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98239731299999999 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987948+0000", 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4382000000000001e-05 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987963+0000", 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.054996391 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.042959+0000", 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041492100000000001 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.043374+0000", 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016716699999999999 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.043541+0000", 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003835649 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047377+0000", 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011540490000000001 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048531+0000", 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033136839999999999 2024-02-22T19:37:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051844+0000", 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1493999999999999e-05 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051896+0000", 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065935400000000002 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052555+0000", 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472188600000001 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20848 7.0 7:00506f5c:::benchmark_data_smithi053_107660_object20847:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.45286152699998, 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0355339840000002, 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20848 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017653+0000", 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017651+0000", 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6801999999999999e-05 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017678+0000", 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1832e-05 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017690+0000", 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9899999999999997e-06 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017698+0000", 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0209e-05 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017748+0000", 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.017755+0000", 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97023272699999996 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987988+0000", 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0849999999999997e-06 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.987996+0000", 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0570709490000001 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045067+0000", 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016401100000000001 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045231+0000", 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158806 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045390+0000", 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031485200000000001 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048538+0000", 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040701499999999998 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052608+0000", 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2032e-05 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052650+0000", 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2859000000000003e-05 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052703+0000", 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000484162 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053187+0000", 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0355339840000002 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20867 7.b 7:d5ccad63:::benchmark_data_smithi053_107660_object20866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.44576369100002, 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0358723360000002, 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20867 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024751+0000", 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024749+0000", 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8314e-05 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024777+0000", 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953000000000001e-05 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024792+0000", 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8079999999999994e-06 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024801+0000", 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9221000000000001e-05 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024860+0000", 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0496000000000001e-05 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024871+0000", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.964611467 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989482+0000", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0557000000000001e-05 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989493+0000", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06140933 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050902+0000", 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015795699999999999 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051060+0000", 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000234997 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051295+0000", 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029240099999999999 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054219+0000", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001188099 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055407+0000", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042592480000000002 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059666+0000", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0272000000000002e-05 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059727+0000", 2024-02-22T19:37:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089686099999999999 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060624+0000", 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0358723360000002 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20879 7.b 7:d83f7aa7:::benchmark_data_smithi053_107660_object20878:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.43959817400003, 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0357336730000002, 2024-02-22T19:37:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20879 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030917+0000", 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030915+0000", 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4876e-05 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030940+0000", 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7564e-05 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030957+0000", 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5020000000000003e-06 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030965+0000", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2354999999999997e-05 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.031007+0000", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5809999999999999e-06 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.031015+0000", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.958500719 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989516+0000", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5569999999999998e-06 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989523+0000", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635326220000001 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053056+0000", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014994699999999999 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053206+0000", 2024-02-22T19:37:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169927 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053376+0000", 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049563699999999999 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058332+0000", 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079005799999999995 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059122+0000", 2024-02-22T19:37:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066417830000000001 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.065764+0000", 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0451999999999997e-05 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.065814+0000", 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083617100000000003 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066650+0000", 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0357336730000002 2024-02-22T19:37:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20885 7.0 7:074624f9:::benchmark_data_smithi053_107660_object20884:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.43459821300002, 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0216042769999998, 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20885 2024-02-22T19:37:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035917+0000", 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035915+0000", 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7532999999999999e-05 2024-02-22T19:37:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035942+0000", 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6866000000000001e-05 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035959+0000", 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3299999999999999e-06 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035968+0000", 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0296000000000006e-05 2024-02-22T19:37:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036038+0000", 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2227e-05 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036050+0000", 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95196719699999999 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988017+0000", 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5079999999999999e-06 2024-02-22T19:37:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988025+0000", 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0581795979999999 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046204+0000", 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137909 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046342+0000", 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017299200000000001 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046515+0000", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046217719999999997 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051137+0000", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003247869 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054385+0000", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022844919999999999 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.056669+0000", 2024-02-22T19:37:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0172e-05 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.056720+0000", 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080141399999999997 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057521+0000", 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0216042769999998 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20888 7.0 7:0390592f:::benchmark_data_smithi053_107660_object20887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.429874493, 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.017606019, 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20888 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040640+0000", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040639+0000", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7110999999999998e-05 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040666+0000", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4090999999999999e-05 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040680+0000", 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2790000000000004e-06 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040686+0000", 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4023999999999996e-05 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040750+0000", 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2262000000000001e-05 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040763+0000", 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.9472855 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988048+0000", 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8350000000000004e-06 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.988056+0000", 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.059385043 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047441+0000", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012125600000000001 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047562+0000", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017584799999999999 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047738+0000", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054622200000000003 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.053200+0000", 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001243353 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054444+0000", 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031954219999999998 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057639+0000", 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4193000000000001e-05 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057693+0000", 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055305899999999995 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058246+0000", 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.017606019 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20889 7.b 7:deffbca2:::benchmark_data_smithi053_107660_object20888:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.42981639999999, 2024-02-22T19:37:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0267241290000002, 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20889 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040699+0000", 2024-02-22T19:37:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040698+0000", 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7917000000000001e-05 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040716+0000", 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6410000000000001e-06 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040721+0000", 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0340000000000001e-06 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040724+0000", 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3059000000000001e-05 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040777+0000", 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3380000000000001e-05 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.040790+0000", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94875468299999999 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989545+0000", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.374e-06 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.989552+0000", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0647073950000001 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054259+0000", 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142843 2024-02-22T19:37:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054402+0000", 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018314000000000001 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054585+0000", 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037587060000000001 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058344+0000", 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002519851 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060864+0000", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058517530000000003 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066716+0000", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0093999999999998e-05 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.066776+0000", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064683000000000004 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.067423+0000", 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0267241290000002 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:701 5.15 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 340.31570499700001, 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86739943600000002, 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:37:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 701 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154810+0000", 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154809+0000", 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6779999999999999e-05 2024-02-22T19:37:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154825+0000", 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3336000000000002e-05 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154849+0000", 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6969999999999998e-06 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154854+0000", 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3915999999999998e-05 2024-02-22T19:37:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154898+0000", 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.413e-06 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.154906+0000", 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85827278699999998 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013179+0000", 2024-02-22T19:37:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030207999999999998 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for degraded object", 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.013481+0000", 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003549445 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017030+0000", 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3808999999999997e-05 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.017114+0000", 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005095502 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.022209+0000", 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86739943600000002 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:71551 8.7 8:ea6f489d:::benchmark_data_smithi053_130774_object71550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:21.306287+0000", 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 68.164228387999998, 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079092576999999997, 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 71551 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.306287+0000", 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.306287+0000", 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.306286+0000", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0214e-05 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.306306+0000", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2060000000000001e-06 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.306313+0000", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7319999999999998e-06 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.306318+0000", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019939599999999999 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.308312+0000", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9474000000000005e-05 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.308391+0000", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2348000000000004e-05 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.308484+0000", 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023929419999999999 2024-02-22T19:37:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.310877+0000", 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021209520000000002 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.312998+0000", 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.072286885999999995 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.385285+0000", 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1613000000000002e-05 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.385336+0000", 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2925e-05 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:21.385379+0000", 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079092576999999997 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73403 8.1 8:897776c2:::benchmark_data_smithi053_130774_object73402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.188391+0000", 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.282124365000001, 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077094157999999996, 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73403 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188391+0000", 2024-02-22T19:37:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188391+0000", 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188389+0000", 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5871999999999999e-05 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188415+0000", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1999e-05 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188427+0000", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.849e-06 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188434+0000", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6563999999999997e-05 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188480+0000", 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9517000000000005e-05 2024-02-22T19:37:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188560+0000", 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5194999999999999e-05 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.188655+0000", 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.071025950000000004 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.259681+0000", 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045496300000000001 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.260136+0000", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052419930000000003 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.265378+0000", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3919999999999998e-05 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.265442+0000", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3040000000000001e-05 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.265485+0000", 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077094157999999996 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73411 8.7 8:e4726e41:::benchmark_data_smithi053_130774_object73410:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.189564+0000", 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.280950908999998, 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.080288261999999999, 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73411 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189564+0000", 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189564+0000", 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189563+0000", 2024-02-22T19:37:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3802e-05 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.189587+0000", 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068087745000000005 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.257674+0000", 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2734e-05 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.257687+0000", 2024-02-22T19:37:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4862000000000003e-05 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.257752+0000", 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142826 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.257895+0000", 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019328100000000001 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.258088+0000", 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073727300000000001 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.265461+0000", 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039218839999999996 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.269383+0000", 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037741199999999998 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.269760+0000", 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6866999999999997e-05 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.269807+0000", 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5188000000000002e-05 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.269852+0000", 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.080288261999999999 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73621 8.7 8:e29c69a5:::benchmark_data_smithi053_130774_object73620:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.456520+0000", 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.013994991000004, 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076119779999999998, 2024-02-22T19:37:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73621 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456520+0000", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456520+0000", 2024-02-22T19:37:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456518+0000", 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5279999999999999e-05 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456544+0000", 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1051e-05 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456555+0000", 2024-02-22T19:37:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4930000000000003e-06 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456561+0000", 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3640000000000003e-05 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456595+0000", 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1546999999999996e-05 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456686+0000", 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100115 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456786+0000", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.070816085000000001 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.527602+0000", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028384980000000001 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.530441+0000", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020864820000000002 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532527+0000", 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7567e-05 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532585+0000", 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4697000000000002e-05 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532640+0000", 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076119779999999998 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73622 8.7 8:eb0fc023:::benchmark_data_smithi053_130774_object73621:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.456570+0000", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.013944524999999, 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076158668999999998, 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73622 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456570+0000", 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456570+0000", 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456570+0000", 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1586999999999998e-05 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456591+0000", 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1330000000000004e-06 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456598+0000", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3769999999999998e-06 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456602+0000", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026725809999999998 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.459274+0000", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8679999999999998e-05 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.459363+0000", 2024-02-22T19:37:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012292600000000001 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.459486+0000", 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.070911605000000003 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.530398+0000", 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018024250000000001 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532200+0000", 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044920300000000002 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532649+0000", 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1850000000000001e-05 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532691+0000", 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7880999999999999e-05 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532729+0000", 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076158668999999998 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:73620 8.f e172/154 8:ff4d5cdb:::benchmark_data_smithi053_130774_object73619:head v 172'4525, mlcod=172'4521)", 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.456669+0000", 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.013846295999997, 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.075844497999999996, 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456669+0000", 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456669+0000", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456668+0000", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1515999999999999e-05 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456690+0000", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7389999999999996e-06 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456696+0000", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8399999999999997e-06 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456699+0000", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015541400000000001 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456855+0000", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9712e-05 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.456875+0000", 2024-02-22T19:37:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.075597673000000004 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532472+0000", 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0927999999999998e-05 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532513+0000", 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.075844497999999996 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73629 8.7 8:e1fa6050:::benchmark_data_smithi053_130774_object73628:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.460768+0000", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.009746899999996, 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077445384000000006, 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73629 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460768+0000", 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460768+0000", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460766+0000", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3876e-05 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460790+0000", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6199999999999994e-06 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460800+0000", 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9989999999999997e-06 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460806+0000", 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066825352000000005 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.527631+0000", 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664699999999999 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.527768+0000", 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016927199999999999 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.527937+0000", 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048309549999999996 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532768+0000", 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532775+0000", 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053466420000000004 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538122+0000", 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8216000000000001e-05 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538170+0000", 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3152000000000003e-05 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538213+0000", 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077445384000000006 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73631 8.3 8:c43d6a0c:::benchmark_data_smithi053_130774_object73630:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.460963+0000", 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.009551883, 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076894573999999993, 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73631 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460963+0000", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460963+0000", 2024-02-22T19:37:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460962+0000", 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8045999999999999e-05 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.460990+0000", 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0061000000000001e-05 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461000+0000", 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1489999999999999e-06 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461006+0000", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9350000000000001e-05 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461045+0000", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066503172999999999 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.527548+0000", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018168399999999999 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:37:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.527730+0000", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058834109999999998 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.533613+0000", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027244130000000002 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.536338+0000", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014397710000000001 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.537778+0000", 2024-02-22T19:37:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7122e-05 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.537825+0000", 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2864000000000002e-05 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.537858+0000", 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.076894573999999993 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73632 8.7 8:e10dff87:::benchmark_data_smithi053_130774_object73631:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.461017+0000", 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.009497558000007, 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077315808, 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73632 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461017+0000", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461017+0000", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461017+0000", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2731e-05 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461039+0000", 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5009999999999998e-06 2024-02-22T19:37:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461047+0000", 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5790000000000003e-06 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461053+0000", 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.071747776999999999 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532800+0000", 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014224600000000001 2024-02-22T19:37:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.532943+0000", 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021121399999999999 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.533154+0000", 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049205669999999998 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538074+0000", 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2884000000000001e-05 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538087+0000", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167947 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538255+0000", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1130000000000001e-05 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538296+0000", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6862e-05 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538333+0000", 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077315808 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:73634 8.3 8:c6fe26e3:::benchmark_data_smithi053_130774_object73633:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:23.461058+0000", 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.009456592999996, 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077045884999999995, 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 73634 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461058+0000", 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461058+0000", 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461058+0000", 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0570000000000001e-05 2024-02-22T19:37:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461078+0000", 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9015e-05 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461097+0000", 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.461102+0000", 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069734286000000006 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.530836+0000", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155129 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.530992+0000", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018451700000000001 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,6", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.531176+0000", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051466849999999998 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.536323+0000", 2024-02-22T19:37:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5317e-05 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.536368+0000", 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016351670000000001 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538003+0000", 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5794000000000001e-05 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538059+0000", 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5225999999999997e-05 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:23.538104+0000", 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.077045884999999995 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:74285 8.7 8:ec0c760c:::benchmark_data_smithi053_130774_object74284:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:24.170753+0000", 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.299761489999995, 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.082141344000000005, 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74285 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170753+0000", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170753+0000", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170753+0000", 2024-02-22T19:37:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4799000000000001e-05 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170778+0000", 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3319999999999997e-06 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170784+0000", 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.25e-06 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170788+0000", 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054646210000000002 2024-02-22T19:37:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.176253+0000", 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118206 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.176371+0000", 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158488 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,6", 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.176530+0000", 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024109650000000002 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.178941+0000", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002311872 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.181252+0000", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.071550164999999999 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.252803+0000", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1654999999999999e-05 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.252854+0000", 2024-02-22T19:37:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.252895+0000", 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.082141344000000005 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:74291 8.1 8:80ed6d8d:::benchmark_data_smithi053_130774_object74290:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:36:24.170958+0000", 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.299557265000004, 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079007685999999994, 2024-02-22T19:37:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74291 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170958+0000", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170958+0000", 2024-02-22T19:37:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170957+0000", 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1168999999999998e-05 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.170998+0000", 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2509999999999994e-06 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.171007+0000", 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0260000000000001e-06 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.171011+0000", 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077169279999999996 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.178728+0000", 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133409 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.178861+0000", 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015805600000000001 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,2", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.179019+0000", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.060871298999999997 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.239890+0000", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034197999999999997 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.240232+0000", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096314769999999994 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.249864+0000", 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5182000000000001e-05 2024-02-22T19:37:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.249919+0000", 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6385999999999999e-05 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:36:24.249965+0000", 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079007685999999994 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:29.948 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:37:30.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:28.645577+0000 osd.5 (osd.5) 59 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:30.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:28.645639+0000 osd.5 (osd.5) 60 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:30.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:29.192667+0000 mon.a (mon.0) 1048 : cluster [DBG] osdmap e190: 8 total, 8 up, 7 in 2024-02-22T19:37:30.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:29.457673+0000 mgr.y (mgr.14535) 329 : cluster [DBG] pgmap v371: 158 pgs: 158 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:30.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:29.459108+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-22T19:37:30.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:28.645577+0000 osd.5 (osd.5) 59 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:30.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:28.645639+0000 osd.5 (osd.5) 60 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:30.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:29.192667+0000 mon.a (mon.0) 1048 : cluster [DBG] osdmap e190: 8 total, 8 up, 7 in 2024-02-22T19:37:30.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:29.457673+0000 mgr.y (mgr.14535) 329 : cluster [DBG] pgmap v371: 158 pgs: 158 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:30.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:29.459108+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-22T19:37:30.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:37:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:28.645577+0000 osd.5 (osd.5) 59 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:28.645639+0000 osd.5 (osd.5) 60 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:29.192667+0000 mon.a (mon.0) 1048 : cluster [DBG] osdmap e190: 8 total, 8 up, 7 in 2024-02-22T19:37:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:29.457673+0000 mgr.y (mgr.14535) 329 : cluster [DBG] pgmap v371: 158 pgs: 158 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:30.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:29.459108+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-22T19:37:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:30.611 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:30.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:30.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:30.744 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:30.767 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:30.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:30.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:30.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:30.937 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:31.051 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:31.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:31.060 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:31.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:31.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:30.199869+0000 mon.a (mon.0) 1050 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-22T19:37:31.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:30.199927+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e191: 8 total, 8 up, 7 in 2024-02-22T19:37:31.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:30.199869+0000 mon.a (mon.0) 1050 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-22T19:37:31.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:30.199927+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e191: 8 total, 8 up, 7 in 2024-02-22T19:37:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:31.398 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:31.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:31.405 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:31.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:31.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:30.199869+0000 mon.a (mon.0) 1050 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-22T19:37:31.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:30.199927+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e191: 8 total, 8 up, 7 in 2024-02-22T19:37:31.779 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:31.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:31.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:31.883 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:31.953 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:32.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:31.211370+0000 mon.a (mon.0) 1052 : cluster [DBG] osdmap e192: 8 total, 8 up, 7 in 2024-02-22T19:37:32.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:31.458132+0000 mgr.y (mgr.14535) 330 : cluster [DBG] pgmap v374: 158 pgs: 158 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:32.376 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:32.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:31.211370+0000 mon.a (mon.0) 1052 : cluster [DBG] osdmap e192: 8 total, 8 up, 7 in 2024-02-22T19:37:32.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:31.458132+0000 mgr.y (mgr.14535) 330 : cluster [DBG] pgmap v374: 158 pgs: 158 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:31.211370+0000 mon.a (mon.0) 1052 : cluster [DBG] osdmap e192: 8 total, 8 up, 7 in 2024-02-22T19:37:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:31.458132+0000 mgr.y (mgr.14535) 330 : cluster [DBG] pgmap v374: 158 pgs: 158 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:32.918 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:32.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:32.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:33.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:33.059 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:33.083 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:33.086 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:33.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:33.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:33.219 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:33.247 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:33.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:33.272 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:37:33.348 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:32.254818+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e193: 8 total, 8 up, 7 in 2024-02-22T19:37:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:32.280415+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? 172.21.15.53:0/2968409066' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:33.420 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:32.254818+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e193: 8 total, 8 up, 7 in 2024-02-22T19:37:33.420 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:32.280415+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? 172.21.15.53:0/2968409066' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:33.420 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:32.254818+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e193: 8 total, 8 up, 7 in 2024-02-22T19:37:33.421 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:32.280415+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? 172.21.15.53:0/2968409066' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:37:33.422 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:33.517 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:33.521 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:33.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:33.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:33.682 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:33.889 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:37:33.889 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:37:33.889 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-02-22T19:37:33.890 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:37:33.890 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:37:33.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-02-22T19:37:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:37:34.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:33.248638+0000 mon.a (mon.0) 1055 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:34.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:33.254428+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.53:0/2968409066' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:37:34.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:33.248638+0000 mon.a (mon.0) 1055 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:34.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:33.254428+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.53:0/2968409066' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:37:34.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:33.254484+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e194: 8 total, 8 up, 7 in 2024-02-22T19:37:34.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:33.458537+0000 mgr.y (mgr.14535) 331 : cluster [DBG] pgmap v377: 157 pgs: 157 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail 2024-02-22T19:37:34.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:33.459854+0000 mon.a (mon.0) 1058 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-22T19:37:34.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:33.248638+0000 mon.a (mon.0) 1055 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:34.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:33.254428+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.53:0/2968409066' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:37:34.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:33.254484+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e194: 8 total, 8 up, 7 in 2024-02-22T19:37:34.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:33.458537+0000 mgr.y (mgr.14535) 331 : cluster [DBG] pgmap v377: 157 pgs: 157 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail 2024-02-22T19:37:34.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:33.459854+0000 mon.a (mon.0) 1058 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-22T19:37:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:33.254484+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e194: 8 total, 8 up, 7 in 2024-02-22T19:37:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:33.458537+0000 mgr.y (mgr.14535) 331 : cluster [DBG] pgmap v377: 157 pgs: 157 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail 2024-02-22T19:37:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:33.459854+0000 mon.a (mon.0) 1058 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-22T19:37:35.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:37:35.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:37:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:37:35.199 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:35.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:35.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:35.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:35.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:34.274350+0000 mon.a (mon.0) 1059 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-22T19:37:35.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:34.274425+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e195: 8 total, 8 up, 7 in 2024-02-22T19:37:35.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:34.556851+0000 osd.0 (osd.0) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:35.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:34.556889+0000 osd.0 (osd.0) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:35.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:34.706589+0000 mgr.y (mgr.14535) 332 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:35.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:35.271038+0000 mon.a (mon.0) 1061 : cluster [DBG] osdmap e196: 8 total, 8 up, 7 in 2024-02-22T19:37:35.358 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:35.392 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:37:35.437 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:37:35.475 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:37:35.508 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:37:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:35.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:35.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:35.545 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:37:35.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:34.274350+0000 mon.a (mon.0) 1059 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-22T19:37:35.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:34.274425+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e195: 8 total, 8 up, 7 in 2024-02-22T19:37:35.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:34.556851+0000 osd.0 (osd.0) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:35.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:34.556889+0000 osd.0 (osd.0) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:35.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:34.706589+0000 mgr.y (mgr.14535) 332 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:35.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:35.271038+0000 mon.a (mon.0) 1061 : cluster [DBG] osdmap e196: 8 total, 8 up, 7 in 2024-02-22T19:37:35.580 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:37:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:34.274350+0000 mon.a (mon.0) 1059 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-22T19:37:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:34.274425+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e195: 8 total, 8 up, 7 in 2024-02-22T19:37:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:34.556851+0000 osd.0 (osd.0) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:34.556889+0000 osd.0 (osd.0) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:34.706589+0000 mgr.y (mgr.14535) 332 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:35.271038+0000 mon.a (mon.0) 1061 : cluster [DBG] osdmap e196: 8 total, 8 up, 7 in 2024-02-22T19:37:35.605 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:37:35.631 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:37:35.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:37:35.727 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:36.437 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:37:36.437 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:37:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:35.350615+0000 mon.a (mon.0) 1062 : audit [DBG] from='client.? 172.21.15.53:0/3574227703' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:35.459406+0000 mgr.y (mgr.14535) 333 : cluster [DBG] pgmap v380: 157 pgs: 157 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-02-22T19:37:36.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:35.350615+0000 mon.a (mon.0) 1062 : audit [DBG] from='client.? 172.21.15.53:0/3574227703' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:36.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:35.459406+0000 mgr.y (mgr.14535) 333 : cluster [DBG] pgmap v380: 157 pgs: 157 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-02-22T19:37:36.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:35.350615+0000 mon.a (mon.0) 1062 : audit [DBG] from='client.? 172.21.15.53:0/3574227703' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:36.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:35.459406+0000 mgr.y (mgr.14535) 333 : cluster [DBG] pgmap v380: 157 pgs: 157 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-02-22T19:37:37.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:37:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:36.313164+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e197: 8 total, 8 up, 7 in 2024-02-22T19:37:37.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:36.313164+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e197: 8 total, 8 up, 7 in 2024-02-22T19:37:37.581 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:36.313164+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e197: 8 total, 8 up, 7 in 2024-02-22T19:37:37.606 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:37.614 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:37.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:37.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:37.753 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:37.763 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:37.842 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:37.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:37.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:37.882 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:37:37.882 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "" 2024-02-22T19:37:37.882 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:37:37.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:37.984 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:38.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:38.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:38.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:37:38.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:38.284 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:38.358 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:37.315877+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e198: 8 total, 8 up, 7 in 2024-02-22T19:37:38.359 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:37.459779+0000 mgr.y (mgr.14535) 334 : cluster [DBG] pgmap v383: 156 pgs: 156 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:38.359 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:37.461007+0000 mon.a (mon.0) 1065 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-22T19:37:38.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:37.315877+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e198: 8 total, 8 up, 7 in 2024-02-22T19:37:38.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:37.459779+0000 mgr.y (mgr.14535) 334 : cluster [DBG] pgmap v383: 156 pgs: 156 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:38.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:37.461007+0000 mon.a (mon.0) 1065 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-22T19:37:38.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:37.510922+0000 osd.7 (osd.7) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:38.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:37.510961+0000 osd.7 (osd.7) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:37.315877+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e198: 8 total, 8 up, 7 in 2024-02-22T19:37:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:37.459779+0000 mgr.y (mgr.14535) 334 : cluster [DBG] pgmap v383: 156 pgs: 156 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:37:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:37.461007+0000 mon.a (mon.0) 1065 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-22T19:37:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:37.510922+0000 osd.7 (osd.7) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:37.510961+0000 osd.7 (osd.7) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:38.640 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:38.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:38.643 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:38.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:37.510922+0000 osd.7 (osd.7) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:38.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:37.510961+0000 osd.7 (osd.7) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:38.795 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:38.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:38.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:38.995 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:39.023 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:39.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:39.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:39.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:39.190 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:39.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:38.336825+0000 mon.a (mon.0) 1066 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-22T19:37:39.498 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:38.336887+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e199: 8 total, 8 up, 7 in 2024-02-22T19:37:39.499 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:38.336825+0000 mon.a (mon.0) 1066 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-22T19:37:39.499 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:38.336887+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e199: 8 total, 8 up, 7 in 2024-02-22T19:37:39.500 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:37:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:39] "GET /metrics HTTP/1.1" 200 175842 "" "Prometheus/2.33.4" 2024-02-22T19:37:39.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:37:39.576 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:39.587 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:39.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:39.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:39.716 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:38.336825+0000 mon.a (mon.0) 1066 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-22T19:37:39.716 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:38.336887+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e199: 8 total, 8 up, 7 in 2024-02-22T19:37:39.719 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:39.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:39.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:39.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:39.946 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:40.074 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:40.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:40.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:40.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:40.206 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:40.230 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:37:40.230 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:37:40.234 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:40.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:40.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:40.291 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:40.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:40.411 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:39.364219+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e200: 8 total, 8 up, 7 in 2024-02-22T19:37:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:39.461465+0000 mgr.y (mgr.14535) 335 : cluster [DBG] pgmap v386: 156 pgs: 2 clean+premerge+peered, 154 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 978 op/s 2024-02-22T19:37:40.725 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:39.364219+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e200: 8 total, 8 up, 7 in 2024-02-22T19:37:40.726 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:39.461465+0000 mgr.y (mgr.14535) 335 : cluster [DBG] pgmap v386: 156 pgs: 2 clean+premerge+peered, 154 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 978 op/s 2024-02-22T19:37:40.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:39.364219+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e200: 8 total, 8 up, 7 in 2024-02-22T19:37:40.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:39.461465+0000 mgr.y (mgr.14535) 335 : cluster [DBG] pgmap v386: 156 pgs: 2 clean+premerge+peered, 154 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 978 op/s 2024-02-22T19:37:40.791 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:40.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:40.799 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:40.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:40.933 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:40.960 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:40.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:40.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:41.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:41.205 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:41.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:41.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:41.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:41.381 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:41.539 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:41.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:41.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:41.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:41.692 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:40.386335+0000 mon.a (mon.0) 1069 : cluster [DBG] osdmap e201: 8 total, 8 up, 7 in 2024-02-22T19:37:41.692 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:40.386335+0000 mon.a (mon.0) 1069 : cluster [DBG] osdmap e201: 8 total, 8 up, 7 in 2024-02-22T19:37:41.693 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:41.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:40.386335+0000 mon.a (mon.0) 1069 : cluster [DBG] osdmap e201: 8 total, 8 up, 7 in 2024-02-22T19:37:42.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:37:42.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:41.408375+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e202: 8 total, 8 up, 7 in 2024-02-22T19:37:42.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:41.461899+0000 mgr.y (mgr.14535) 336 : cluster [DBG] pgmap v389: 155 pgs: 1 clean+premerge+peered, 154 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-02-22T19:37:42.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:41.462760+0000 mon.a (mon.0) 1071 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-22T19:37:42.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:41.408375+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e202: 8 total, 8 up, 7 in 2024-02-22T19:37:42.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:41.461899+0000 mgr.y (mgr.14535) 336 : cluster [DBG] pgmap v389: 155 pgs: 1 clean+premerge+peered, 154 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-02-22T19:37:42.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:41.462760+0000 mon.a (mon.0) 1071 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-22T19:37:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:41.408375+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e202: 8 total, 8 up, 7 in 2024-02-22T19:37:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:41.461899+0000 mgr.y (mgr.14535) 336 : cluster [DBG] pgmap v389: 155 pgs: 1 clean+premerge+peered, 154 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-02-22T19:37:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:41.462760+0000 mon.a (mon.0) 1071 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-22T19:37:42.874 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:42.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:42.880 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:42.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:43.017 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:43.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:43.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:43.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:43.217 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:43.343 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:43.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:43.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:43.443 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:42.431058+0000 mon.a (mon.0) 1072 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-22T19:37:43.443 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:42.431115+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e203: 8 total, 8 up, 7 in 2024-02-22T19:37:43.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:43.534 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:43.610 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:37:43.610 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:37:43.610 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-02-22T19:37:43.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 5 2024-02-22T19:37:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:43.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:43.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:43.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:42.431058+0000 mon.a (mon.0) 1072 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-22T19:37:43.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:42.431115+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e203: 8 total, 8 up, 7 in 2024-02-22T19:37:43.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:43.830 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:42.431058+0000 mon.a (mon.0) 1072 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-22T19:37:43.830 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:42.431115+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e203: 8 total, 8 up, 7 in 2024-02-22T19:37:43.834 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:43.858 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:43.861 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:43.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:43.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:44.021 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:44.186 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:44.191 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:44.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:44.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:44.357 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:44.361 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:44.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:44.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:44.509 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:37:44.521 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:43.452803+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e204: 8 total, 8 up, 7 in 2024-02-22T19:37:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:43.464873+0000 mgr.y (mgr.14535) 337 : cluster [DBG] pgmap v392: 155 pgs: 1 clean+premerge+peered, 154 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 304 op/s 2024-02-22T19:37:44.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:44.280257+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? 172.21.15.53:0/3671188686' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:37:44.611 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:37:44.650 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:37:44.681 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:37:44.751 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:37:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:43.452803+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e204: 8 total, 8 up, 7 in 2024-02-22T19:37:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:43.464873+0000 mgr.y (mgr.14535) 337 : cluster [DBG] pgmap v392: 155 pgs: 1 clean+premerge+peered, 154 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 304 op/s 2024-02-22T19:37:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:44.280257+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? 172.21.15.53:0/3671188686' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:37:44.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:43.452803+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e204: 8 total, 8 up, 7 in 2024-02-22T19:37:44.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:43.464873+0000 mgr.y (mgr.14535) 337 : cluster [DBG] pgmap v392: 155 pgs: 1 clean+premerge+peered, 154 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 304 op/s 2024-02-22T19:37:44.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:44.280257+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? 172.21.15.53:0/3671188686' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:37:44.789 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:37:44.838 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:37:44.872 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:37:44.897 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:37:44.909 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.909 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.909 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.909 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.910 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.910 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.910 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:37:44.910 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:37:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:37:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:37:45.104 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:37:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:43.949622+0000 osd.4 (osd.4) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:43.949661+0000 osd.4 (osd.4) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:44.450127+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:44.480183+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? 172.21.15.53:0/3671188686' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:44.480242+0000 mon.a (mon.0) 1078 : cluster [DBG] osdmap e205: 8 total, 8 up, 7 in 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:44.550968+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.53:0/2701515130' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:45.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:44.718427+0000 mgr.y (mgr.14535) 338 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:43.949622+0000 osd.4 (osd.4) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:43.949661+0000 osd.4 (osd.4) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:44.450127+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:44.480183+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? 172.21.15.53:0/3671188686' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:44.480242+0000 mon.a (mon.0) 1078 : cluster [DBG] osdmap e205: 8 total, 8 up, 7 in 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:44.550968+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.53:0/2701515130' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:45.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:44.718427+0000 mgr.y (mgr.14535) 338 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:43.949622+0000 osd.4 (osd.4) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:43.949661+0000 osd.4 (osd.4) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:44.450127+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:44.480183+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? 172.21.15.53:0/3671188686' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:44.480242+0000 mon.a (mon.0) 1078 : cluster [DBG] osdmap e205: 8 total, 8 up, 7 in 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:44.550968+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.53:0/2701515130' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:37:45.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:44.718427+0000 mgr.y (mgr.14535) 338 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.099219+0000 osd.2 (osd.2) 110 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:37:46.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.104941+0000 osd.2 (osd.2) 111 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.128724+0000 osd.1 (osd.1) 132 : cluster [DBG] 8.3 scrub starts 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.310972+0000 osd.3 (osd.3) 147 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.317433+0000 osd.3 (osd.3) 148 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.465298+0000 mgr.y (mgr.14535) 339 : cluster [DBG] pgmap v394: 154 pgs: 1 clean+premerge+peered, 153 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:45.477000+0000 mon.a (mon.0) 1079 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:45.491582+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:45.491637+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e206: 8 total, 8 up, 7 in 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.099219+0000 osd.2 (osd.2) 110 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.104941+0000 osd.2 (osd.2) 111 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.128724+0000 osd.1 (osd.1) 132 : cluster [DBG] 8.3 scrub starts 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.310972+0000 osd.3 (osd.3) 147 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:37:46.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.317433+0000 osd.3 (osd.3) 148 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:37:46.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.465298+0000 mgr.y (mgr.14535) 339 : cluster [DBG] pgmap v394: 154 pgs: 1 clean+premerge+peered, 153 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-02-22T19:37:46.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:45.477000+0000 mon.a (mon.0) 1079 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-22T19:37:46.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:45.491582+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-22T19:37:46.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:45.491637+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e206: 8 total, 8 up, 7 in 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.099219+0000 osd.2 (osd.2) 110 : cluster [DBG] 5.1b scrub starts 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.104941+0000 osd.2 (osd.2) 111 : cluster [DBG] 5.1b scrub ok 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.128724+0000 osd.1 (osd.1) 132 : cluster [DBG] 8.3 scrub starts 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.310972+0000 osd.3 (osd.3) 147 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.317433+0000 osd.3 (osd.3) 148 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.465298+0000 mgr.y (mgr.14535) 339 : cluster [DBG] pgmap v394: 154 pgs: 1 clean+premerge+peered, 153 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:45.477000+0000 mon.a (mon.0) 1079 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-22T19:37:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:45.491582+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-22T19:37:46.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:45.491637+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e206: 8 total, 8 up, 7 in 2024-02-22T19:37:47.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:46.501906+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e207: 8 total, 8 up, 7 in 2024-02-22T19:37:48.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:46.501906+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e207: 8 total, 8 up, 7 in 2024-02-22T19:37:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:46.501906+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e207: 8 total, 8 up, 7 in 2024-02-22T19:37:48.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:47.213279+0000 osd.1 (osd.1) 133 : cluster [DBG] 8.3 scrub ok 2024-02-22T19:37:48.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:47.419697+0000 osd.6 (osd.6) 95 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:37:48.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:47.421618+0000 osd.6 (osd.6) 96 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:37:48.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:47.465686+0000 mgr.y (mgr.14535) 340 : cluster [DBG] pgmap v397: 154 pgs: 1 clean+premerge+peered, 153 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-02-22T19:37:48.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:47.518820+0000 mon.a (mon.0) 1083 : cluster [DBG] osdmap e208: 8 total, 8 up, 7 in 2024-02-22T19:37:49.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:47.213279+0000 osd.1 (osd.1) 133 : cluster [DBG] 8.3 scrub ok 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:47.419697+0000 osd.6 (osd.6) 95 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:47.421618+0000 osd.6 (osd.6) 96 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:47.465686+0000 mgr.y (mgr.14535) 340 : cluster [DBG] pgmap v397: 154 pgs: 1 clean+premerge+peered, 153 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:47.518820+0000 mon.a (mon.0) 1083 : cluster [DBG] osdmap e208: 8 total, 8 up, 7 in 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:47.213279+0000 osd.1 (osd.1) 133 : cluster [DBG] 8.3 scrub ok 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:47.419697+0000 osd.6 (osd.6) 95 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:47.421618+0000 osd.6 (osd.6) 96 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:47.465686+0000 mgr.y (mgr.14535) 340 : cluster [DBG] pgmap v397: 154 pgs: 1 clean+premerge+peered, 153 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-02-22T19:37:49.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:47.518820+0000 mon.a (mon.0) 1083 : cluster [DBG] osdmap e208: 8 total, 8 up, 7 in 2024-02-22T19:37:49.553 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:49] "GET /metrics HTTP/1.1" 200 175842 "" "Prometheus/2.33.4" 2024-02-22T19:37:49.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:48.161122+0000 osd.0 (osd.0) 155 : cluster [DBG] 8.6 scrub starts 2024-02-22T19:37:49.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:48.374406+0000 osd.4 (osd.4) 137 : cluster [DBG] 4.d scrub starts 2024-02-22T19:37:49.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:48.379785+0000 osd.4 (osd.4) 138 : cluster [DBG] 4.d scrub ok 2024-02-22T19:37:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:48.161122+0000 osd.0 (osd.0) 155 : cluster [DBG] 8.6 scrub starts 2024-02-22T19:37:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:48.374406+0000 osd.4 (osd.4) 137 : cluster [DBG] 4.d scrub starts 2024-02-22T19:37:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:48.379785+0000 osd.4 (osd.4) 138 : cluster [DBG] 4.d scrub ok 2024-02-22T19:37:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:48.525099+0000 mon.a (mon.0) 1084 : cluster [DBG] osdmap e209: 8 total, 8 up, 7 in 2024-02-22T19:37:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:49.468441+0000 mon.a (mon.0) 1085 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:48.161122+0000 osd.0 (osd.0) 155 : cluster [DBG] 8.6 scrub starts 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:48.374406+0000 osd.4 (osd.4) 137 : cluster [DBG] 4.d scrub starts 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:48.379785+0000 osd.4 (osd.4) 138 : cluster [DBG] 4.d scrub ok 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:48.525099+0000 mon.a (mon.0) 1084 : cluster [DBG] osdmap e209: 8 total, 8 up, 7 in 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:49.468441+0000 mon.a (mon.0) 1085 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:48.525099+0000 mon.a (mon.0) 1084 : cluster [DBG] osdmap e209: 8 total, 8 up, 7 in 2024-02-22T19:37:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:49.468441+0000 mon.a (mon.0) 1085 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-22T19:37:50.659 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:50.669 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:50.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:50.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:50.811 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:49.466771+0000 mgr.y (mgr.14535) 341 : cluster [DBG] pgmap v400: 153 pgs: 1 active+clean+scrubbing, 152 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 892 op/s 2024-02-22T19:37:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:49.542638+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-22T19:37:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:49.542697+0000 mon.a (mon.0) 1087 : cluster [DBG] osdmap e210: 8 total, 8 up, 7 in 2024-02-22T19:37:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:49.752280+0000 osd.0 (osd.0) 156 : cluster [DBG] 8.6 scrub ok 2024-02-22T19:37:50.983 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:49.466771+0000 mgr.y (mgr.14535) 341 : cluster [DBG] pgmap v400: 153 pgs: 1 active+clean+scrubbing, 152 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 892 op/s 2024-02-22T19:37:50.983 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:49.542638+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-22T19:37:50.983 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:49.542697+0000 mon.a (mon.0) 1087 : cluster [DBG] osdmap e210: 8 total, 8 up, 7 in 2024-02-22T19:37:50.983 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:49.752280+0000 osd.0 (osd.0) 156 : cluster [DBG] 8.6 scrub ok 2024-02-22T19:37:50.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:49.466771+0000 mgr.y (mgr.14535) 341 : cluster [DBG] pgmap v400: 153 pgs: 1 active+clean+scrubbing, 152 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 892 op/s 2024-02-22T19:37:50.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:49.542638+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-22T19:37:50.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:49.542697+0000 mon.a (mon.0) 1087 : cluster [DBG] osdmap e210: 8 total, 8 up, 7 in 2024-02-22T19:37:50.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:49.752280+0000 osd.0 (osd.0) 156 : cluster [DBG] 8.6 scrub ok 2024-02-22T19:37:51.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:50.139939+0000 osd.1 (osd.1) 134 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:37:51.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:50.144172+0000 osd.1 (osd.1) 135 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:37:51.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:50.139939+0000 osd.1 (osd.1) 134 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:37:51.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:50.144172+0000 osd.1 (osd.1) 135 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:37:51.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:50.561999+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e211: 8 total, 8 up, 7 in 2024-02-22T19:37:51.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:51.425335+0000 osd.7 (osd.7) 157 : cluster [DBG] 3.e scrub starts 2024-02-22T19:37:51.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:51.431523+0000 osd.7 (osd.7) 158 : cluster [DBG] 3.e scrub ok 2024-02-22T19:37:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:50.139939+0000 osd.1 (osd.1) 134 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:37:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:50.144172+0000 osd.1 (osd.1) 135 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:37:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:50.561999+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e211: 8 total, 8 up, 7 in 2024-02-22T19:37:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:51.425335+0000 osd.7 (osd.7) 157 : cluster [DBG] 3.e scrub starts 2024-02-22T19:37:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:51.431523+0000 osd.7 (osd.7) 158 : cluster [DBG] 3.e scrub ok 2024-02-22T19:37:52.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:50.561999+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e211: 8 total, 8 up, 7 in 2024-02-22T19:37:52.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:51.425335+0000 osd.7 (osd.7) 157 : cluster [DBG] 3.e scrub starts 2024-02-22T19:37:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:51.431523+0000 osd.7 (osd.7) 158 : cluster [DBG] 3.e scrub ok 2024-02-22T19:37:52.196 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:52.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:52.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:52.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:52.435 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:52.461 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:52.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:52.562 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:52.626 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:52.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:52.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:52.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:52.764 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:51.415700+0000 osd.3 (osd.3) 149 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:37:52.764 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:51.454672+0000 osd.3 (osd.3) 150 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:37:52.764 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:51.467264+0000 mgr.y (mgr.14535) 342 : cluster [DBG] pgmap v403: 153 pgs: 1 active+clean+scrubbing, 152 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-02-22T19:37:52.765 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:51.596947+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e212: 8 total, 8 up, 7 in 2024-02-22T19:37:52.767 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:52.790 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:52.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:52.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:52.894 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:51.415700+0000 osd.3 (osd.3) 149 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:51.454672+0000 osd.3 (osd.3) 150 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:51.467264+0000 mgr.y (mgr.14535) 342 : cluster [DBG] pgmap v403: 153 pgs: 1 active+clean+scrubbing, 152 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:51.596947+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e212: 8 total, 8 up, 7 in 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:51.415700+0000 osd.3 (osd.3) 149 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:51.454672+0000 osd.3 (osd.3) 150 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:51.467264+0000 mgr.y (mgr.14535) 342 : cluster [DBG] pgmap v403: 153 pgs: 1 active+clean+scrubbing, 152 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-02-22T19:37:52.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:51.596947+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e212: 8 total, 8 up, 7 in 2024-02-22T19:37:52.966 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:52.973 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:53.001 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:53.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:53.006 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:53.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:53.139 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:53.161 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:53.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:53.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:53.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:53.351 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:53.400 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:53.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:53.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:53.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:53.587 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:53.621 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:53.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:53.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:53.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:37:53.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:53.757 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:53.780 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:53.785 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:53.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:53.886 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:53.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:52.186906+0000 osd.2 (osd.2) 112 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:37:53.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:52.198220+0000 osd.2 (osd.2) 113 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:37:53.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:52.429138+0000 osd.6 (osd.6) 97 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:37:53.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:52.432671+0000 osd.6 (osd.6) 98 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:37:53.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:52.604499+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e213: 8 total, 8 up, 7 in 2024-02-22T19:37:53.913 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.381245+0000 osd.7 (osd.7) 159 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:37:53.913 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.392175+0000 osd.7 (osd.7) 160 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:37:53.913 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:53.469057+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-22T19:37:53.914 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:53.936 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:53.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:53.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:52.186906+0000 osd.2 (osd.2) 112 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:52.198220+0000 osd.2 (osd.2) 113 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:52.429138+0000 osd.6 (osd.6) 97 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:52.432671+0000 osd.6 (osd.6) 98 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:52.604499+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e213: 8 total, 8 up, 7 in 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.381245+0000 osd.7 (osd.7) 159 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.392175+0000 osd.7 (osd.7) 160 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:37:54.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:53.469057+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:52.186906+0000 osd.2 (osd.2) 112 : cluster [DBG] 5.18 scrub starts 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:52.198220+0000 osd.2 (osd.2) 113 : cluster [DBG] 5.18 scrub ok 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:52.429138+0000 osd.6 (osd.6) 97 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:52.432671+0000 osd.6 (osd.6) 98 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:52.604499+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e213: 8 total, 8 up, 7 in 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.381245+0000 osd.7 (osd.7) 159 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.392175+0000 osd.7 (osd.7) 160 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:37:54.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:53.469057+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-22T19:37:54.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:54.071 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:54.093 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:54.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:54.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:54.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:37:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:54.263 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:37:54.265 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:54.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:37:54.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:54.434 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:54.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:54.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:54.589 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:54.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:54.592 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:54.693 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:37:54.739 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20820 7.e 7:75479808:::benchmark_data_smithi053_107660_object20819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.98164331100003, 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0656461249999998, 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:37:54.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20820 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980812+0000", 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980810+0000", 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.211e-05 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980833+0000", 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8699999999999992e-06 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980840+0000", 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9620000000000001e-06 2024-02-22T19:37:54.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.980845+0000", 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084919699999999999 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981695+0000", 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000152538 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981847+0000", 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117994 2024-02-22T19:37:54.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5", 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981965+0000", 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001797442 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.983763+0000", 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007165561 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990928+0000", 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6160000000000008e-06 2024-02-22T19:37:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990938+0000", 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004499088 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995437+0000", 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2568000000000001e-05 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995449+0000", 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509275339999999 2024-02-22T19:37:54.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046377+0000", 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1163999999999997e-05 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046458+0000", 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0656461249999998 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20825 7.e 7:74b6997e:::benchmark_data_smithi053_107660_object20824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:37:54.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.98067619599999, 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0648078949999999, 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20825 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981779+0000", 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981778+0000", 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6081000000000006e-05 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981844+0000", 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7260000000000002e-06 2024-02-22T19:37:54.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981852+0000", 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0959999999999997e-06 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.981858+0000", 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088905199999999998 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.982747+0000", 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118694 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.982866+0000", 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014208600000000001 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5", 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.983008+0000", 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00131347 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.984321+0000", 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006647231 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990968+0000", 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.109e-06 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.990976+0000", 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004509871 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995485+0000", 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4160000000000004e-06 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995494+0000", 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0510047179999999 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046499+0000", 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8375999999999995e-05 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046587+0000", 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0648078949999999 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20834 7.e 7:73899197:::benchmark_data_smithi053_107660_object20833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.97467829099998, 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0674701959999999, 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20834 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:37:54.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987777+0000", 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987775+0000", 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0616999999999999e-05 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987796+0000", 2024-02-22T19:37:54.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7879999999999995e-06 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987805+0000", 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9380000000000003e-06 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.987812+0000", 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043811340000000001 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.992193+0000", 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9510000000000001e-06 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:48.992203+0000", 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003316238 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995519+0000", 2024-02-22T19:37:54.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0299999999999994e-06 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995527+0000", 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0510881539999999 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046615+0000", 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012597899999999999 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046741+0000", 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014336500000000001 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.046885+0000", 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073188910000000001 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054204+0000", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1601999999999997e-05 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054265+0000", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010688699999999999 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054372+0000", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2679000000000001e-05 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054435+0000", 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081248500000000005 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055247+0000", 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0674701959999999 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20860 7.e 7:7d18546b:::benchmark_data_smithi053_107660_object20859:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.93820795800002, 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.034585796, 2024-02-22T19:37:54.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20860 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024247+0000", 2024-02-22T19:37:54.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024245+0000", 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7005e-05 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024272+0000", 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3974999999999999e-05 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024286+0000", 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6139999999999997e-06 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024293+0000", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6604000000000002e-05 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024349+0000", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6550000000000004e-06 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.024357+0000", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.971194102 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995551+0000", 2024-02-22T19:37:54.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3649999999999998e-06 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995558+0000", 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521218619999999 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047680+0000", 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114424 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.047795+0000", 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210046 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048005+0000", 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062518670000000004 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054257+0000", 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5546e-05 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054272+0000", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003673805 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057946+0000", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8497000000000005e-05 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058014+0000", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081864399999999999 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058833+0000", 2024-02-22T19:37:54.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.034585796 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20873 7.e 7:70011a90:::benchmark_data_smithi053_107660_object20872:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.93448898999998, 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0315724149999999, 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20873 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027966+0000", 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027965+0000", 2024-02-22T19:37:54.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3580000000000001e-05 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.027988+0000", 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.577e-05 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028004+0000", 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028011+0000", 2024-02-22T19:37:54.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6941999999999999e-05 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028058+0000", 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3840000000000002e-06 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.028064+0000", 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96751664000000004 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995581+0000", 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3509999999999996e-06 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995588+0000", 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0532674879999999 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048856+0000", 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015597000000000001 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049012+0000", 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015027700000000001 2024-02-22T19:37:54.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049162+0000", 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051174610000000002 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054279+0000", 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.541e-05 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054335+0000", 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045385269999999997 2024-02-22T19:37:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058873+0000", 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5123000000000001e-05 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058919+0000", 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062019400000000002 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059539+0000", 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0315724149999999 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20878 7.e 7:7891943c:::benchmark_data_smithi053_107660_object20877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.93177517999999, 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0297337099999999, 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20878 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030678+0000", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2857000000000002e-05 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030700+0000", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3713000000000001e-05 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030714+0000", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1090000000000003e-06 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030720+0000", 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9595000000000002e-05 2024-02-22T19:37:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030770+0000", 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1749999999999994e-06 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030779+0000", 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96483299300000003 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995612+0000", 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5809999999999999e-06 2024-02-22T19:37:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995620+0000", 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0543208209999999 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.049940+0000", 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7372000000000003e-05 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050028+0000", 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013467700000000001 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050162+0000", 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041241170000000001 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054287+0000", 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010961230000000001 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.055383+0000", 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042013659999999998 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059584+0000", 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8198000000000001e-05 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.059642+0000", 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077155300000000002 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060414+0000", 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0297337099999999 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20883 7.e 7:7c7f3c6f:::benchmark_data_smithi053_107660_object20882:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.92782498999998, 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265122409999998, 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20883 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034630+0000", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034628+0000", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6835e-05 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034655+0000", 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8521000000000001e-05 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034674+0000", 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2910000000000002e-06 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034681+0000", 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9109999999999994e-05 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034750+0000", 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0144e-05 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.034760+0000", 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96088248899999995 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995643+0000", 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995650+0000", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.055232344 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.050883+0000", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016314899999999999 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051046+0000", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000208643 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051255+0000", 2024-02-22T19:37:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030399609999999999 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.054295+0000", 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061688639999999996 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060463+0000", 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0683000000000002e-05 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060524+0000", 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1838e-05 2024-02-22T19:37:54.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.060586+0000", 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055658699999999999 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061143+0000", 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0265122409999998 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20886 7.e 7:75e1811d:::benchmark_data_smithi053_107660_object20885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:37:54.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 364.92648147599999, 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0257961170000001, 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20886 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035974+0000", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.035972+0000", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7784e-05 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036000+0000", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8953e-05 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036019+0000", 2024-02-22T19:37:54.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1119999999999996e-06 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036027+0000", 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106254 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036133+0000", 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2412000000000001e-05 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.036146+0000", 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95952591799999998 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995672+0000", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5299999999999999e-06 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.995679+0000", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0562192079999999 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051898+0000", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000153024 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052051+0000", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195556 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.052247+0000", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049309180000000003 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057178+0000", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040298340000000004 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061208+0000", 2024-02-22T19:37:54.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6218999999999995e-05 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061304+0000", 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3900000000000002e-05 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061358+0000", 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041208899999999998 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.061770+0000", 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0257961170000001 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110795 7.e 7:7fb91f79:::benchmark_data_smithi053_107660_object21588:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.084742+0000", 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.877712919, 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025483795, 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110795 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084742+0000", 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084742+0000", 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084741+0000", 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1459999999999996e-06 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084749+0000", 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2155e-05 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084761+0000", 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5529999999999998e-06 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.084768+0000", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024508709999999999 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.087219+0000", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7408000000000001e-05 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.087306+0000", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2807999999999996e-05 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.087399+0000", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061312190000000003 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.093530+0000", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065161530000000002 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.100046+0000", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0083518299999999993 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108398+0000", 2024-02-22T19:37:54.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6590000000000002e-06 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108408+0000", 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.001729877 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110137+0000", 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8690000000000006e-05 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110226+0000", 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025483795 2024-02-22T19:37:54.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110799 7.e 7:726922b8:::benchmark_data_smithi053_107660_object21592:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.088641+0000", 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.873814756, 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021687464, 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110799 2024-02-22T19:37:54.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088641+0000", 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088641+0000", 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088638+0000", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8400000000000001e-06 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088647+0000", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0780000000000002e-06 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088656+0000", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3720000000000001e-06 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.088663+0000", 2024-02-22T19:37:54.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001396865 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.090060+0000", 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014967299999999999 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.090209+0000", 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169808 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.090379+0000", 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031592500000000002 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.093538+0000", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014899720999999999 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108438+0000", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1609999999999997e-06 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108445+0000", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.001810976 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110256+0000", 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1786999999999996e-05 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110328+0000", 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021687464 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110802 7.e 7:74556c42:::benchmark_data_smithi053_107660_object21595:head [delete] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.097107+0000", 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.86534806399999, 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0133390470000001, 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110802 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097107+0000", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097107+0000", 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097106+0000", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097114+0000", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2564e-05 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097127+0000", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4839999999999998e-06 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097134+0000", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5037999999999997e-05 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097189+0000", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013183699999999999 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097321+0000", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013346899999999999 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.097454+0000", 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003854845 2024-02-22T19:37:54.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.101309+0000", 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071630269999999998 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108472+0000", 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0369999999999997e-06 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.108479+0000", 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0018937649999999 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110373+0000", 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3134999999999997e-05 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110446+0000", 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0133390470000001 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110814 7.e 7:740d9b1f:::benchmark_data_smithi053_107660_object21607:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.117657+0000", 2024-02-22T19:37:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.844798088, 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027530931, 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110814 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117657+0000", 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117657+0000", 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117655+0000", 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9629999999999998e-06 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117663+0000", 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2947e-05 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117676+0000", 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1865000000000001e-05 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.117688+0000", 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046482799999999998 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.118153+0000", 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3170000000000008e-06 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.118162+0000", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99231844199999997 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110481+0000", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012363900000000001 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110604+0000", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017102900000000001 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.110775+0000", 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011035908000000001 2024-02-22T19:37:54.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121811+0000", 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023248329000000002 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145060+0000", 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1023999999999999e-05 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145101+0000", 2024-02-22T19:37:54.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7902999999999999e-05 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145149+0000", 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9548000000000002e-05 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145188+0000", 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027530931 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110819 7.e 7:7eaeb1ee:::benchmark_data_smithi053_107660_object21612:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.124807+0000", 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.83764860299999, 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0205201960000001, 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110819 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124807+0000", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124807+0000", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124805+0000", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4290000000000001e-06 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124813+0000", 2024-02-22T19:37:54.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1292e-05 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124825+0000", 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4409999999999998e-06 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124831+0000", 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7509000000000002e-05 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124879+0000", 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-02-22T19:37:54.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124885+0000", 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98871795900000004 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.113603+0000", 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7225999999999994e-05 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.113701+0000", 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014236399999999999 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.113843+0000", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0079748839999999998 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121818+0000", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023407692000000001 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145226+0000", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0882e-05 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145256+0000", 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6516000000000001e-05 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145293+0000", 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3945000000000001e-05 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145327+0000", 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0205201960000001 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110820 7.e 7:754a5adc:::benchmark_data_smithi053_107660_object21613:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.837611821, 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026599126, 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110820 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124843+0000", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.005e-06 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124846+0000", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124851+0000", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6149999999999996e-06 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124857+0000", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7889000000000002e-05 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124905+0000", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9429999999999997e-06 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.124911+0000", 2024-02-22T19:37:54.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99109971699999999 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.116010+0000", 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013614500000000001 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.116147+0000", 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015206200000000001 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.116299+0000", 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016877981 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133177+0000", 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012185431 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145362+0000", 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059805309999999999 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151343+0000", 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4162000000000001e-05 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151397+0000", 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5803000000000002e-05 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151443+0000", 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026599126 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110830 7.e 7:7342ed68:::benchmark_data_smithi053_107660_object21623:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.138800+0000", 2024-02-22T19:37:54.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.823655145, 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012783733, 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110830 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138800+0000", 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138800+0000", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138798+0000", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1530000000000003e-06 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138807+0000", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7183000000000001e-05 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138824+0000", 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7870000000000004e-06 2024-02-22T19:37:54.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138831+0000", 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2577999999999997e-05 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138884+0000", 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5100000000000004e-06 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.138893+0000", 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98002727999999995 2024-02-22T19:37:54.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.118920+0000", 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013081900000000001 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.119051+0000", 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000165923 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.119217+0000", 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013967171 2024-02-22T19:37:54.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133184+0000", 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012208811 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145393+0000", 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061003960000000001 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151494+0000", 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6493999999999999e-05 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151540+0000", 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3776999999999998e-05 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151584+0000", 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012783733 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110847 7.e 7:7e4d7ad4:::benchmark_data_smithi053_107660_object21640:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.156297+0000", 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.80615835399999, 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99545414099999996, 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110847 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156297+0000", 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156297+0000", 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156296+0000", 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7729999999999996e-06 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156301+0000", 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0149999999999999e-05 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156312+0000", 2024-02-22T19:37:54.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156317+0000", 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4360999999999997e-05 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156371+0000", 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.156383+0000", 2024-02-22T19:37:54.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96546441100000002 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121847+0000", 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107951 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.121955+0000", 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144988 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.122100+0000", 2024-02-22T19:37:54.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011091546000000001 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133192+0000", 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012232875000000001 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145425+0000", 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006235127 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151660+0000", 2024-02-22T19:37:54.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7330999999999998e-05 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151707+0000", 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989000000000002e-05 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151751+0000", 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99545414099999996 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110896 7.e 7:73002236:::benchmark_data_smithi053_107660_object21689:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.212855+0000", 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.74960076299999, 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.93903675200000003, 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110896 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212855+0000", 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212855+0000", 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212852+0000", 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4409999999999999e-06 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212862+0000", 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6711e-05 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212878+0000", 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6020000000000002e-06 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212886+0000", 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2688999999999996e-05 2024-02-22T19:37:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212949+0000", 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1256e-05 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.212960+0000", 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.9117461 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.124706+0000", 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000121069 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.124827+0000", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016258900000000001 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.124990+0000", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0082091390000000007 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133199+0000", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012256107 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145455+0000", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063481570000000001 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151803+0000", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6461999999999997e-05 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151850+0000", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1643000000000001e-05 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151891+0000", 2024-02-22T19:37:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.93903675200000003 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110920 7.e 7:76c1d686:::benchmark_data_smithi053_107660_object21713:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.235434+0000", 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.72702174599999, 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91662422200000004, 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110920 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235434+0000", 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235434+0000", 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235431+0000", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-06 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235440+0000", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.366e-05 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235453+0000", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4159999999999998e-06 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235461+0000", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2850999999999998e-05 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235524+0000", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1752e-05 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.235536+0000", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89218530900000004 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.127721+0000", 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012830799999999999 2024-02-22T19:37:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.127849+0000", 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017058500000000001 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.128020+0000", 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017466145999999998 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145486+0000", 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051044749999999998 2024-02-22T19:37:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.150590+0000", 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001387969 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.151978+0000", 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1754e-05 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152020+0000", 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7688999999999998e-05 2024-02-22T19:37:54.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152058+0000", 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91662422200000004 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110928 7.e 7:7b4e694f:::benchmark_data_smithi053_107660_object21721:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.247486+0000", 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.714969766, 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90479611400000004, 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110928 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247486+0000", 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247486+0000", 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247484+0000", 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0610000000000001e-06 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247490+0000", 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4266e-05 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247505+0000", 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8329999999999996e-06 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247511+0000", 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2525999999999998e-05 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247564+0000", 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0601000000000001e-05 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.247575+0000", 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.88291269800000005 2024-02-22T19:37:54.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.130487+0000", 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109404 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.130597+0000", 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036100000000001 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.130737+0000", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014779439 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145516+0000", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050812640000000003 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.150598+0000", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001600186 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152198+0000", 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4004000000000001e-05 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152242+0000", 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9711e-05 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.152282+0000", 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90479611400000004 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:110952 7.e 7:77e3b0a4:::benchmark_data_smithi053_107660_object21745:head [delete] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:39.279954+0000", 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 254.682501035, 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87986089099999998, 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 110952 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:37:54.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279954+0000", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279954+0000", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279952+0000", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2339999999999994e-06 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:37:54.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279961+0000", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3113e-05 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279974+0000", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1180000000000003e-06 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.279981+0000", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2351e-05 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.280043+0000", 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0951000000000001e-05 2024-02-22T19:37:54.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:39.280054+0000", 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85318869900000005 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133243+0000", 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114162 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133357+0000", 2024-02-22T19:37:54.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4803999999999996e-05 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.133452+0000", 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012095134 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.145547+0000", 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050563459999999998 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.150603+0000", 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091255810000000007 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.159729+0000", 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6208999999999998e-05 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.159775+0000", 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9975000000000003e-05 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:40.159815+0000", 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87986089099999998 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:37:54.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:37:54.867 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:37:54.867 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:54.867 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:37:54.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.397614+0000 osd.6 (osd.6) 99 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.400527+0000 osd.6 (osd.6) 100 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.457352+0000 osd.4 (osd.4) 139 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.461280+0000 osd.4 (osd.4) 140 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.467854+0000 mgr.y (mgr.14535) 343 : cluster [DBG] pgmap v406: 152 pgs: 1 active+clean+scrubbing, 151 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 300 op/s 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:53.615981+0000 mon.a (mon.0) 1092 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:53.616033+0000 mon.a (mon.0) 1093 : cluster [DBG] osdmap e214: 8 total, 8 up, 7 in 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:54.414931+0000 osd.7 (osd.7) 161 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:54.418950+0000 osd.7 (osd.7) 162 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:37:54.868 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:37:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.397614+0000 osd.6 (osd.6) 99 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.400527+0000 osd.6 (osd.6) 100 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.457352+0000 osd.4 (osd.4) 139 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.461280+0000 osd.4 (osd.4) 140 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.467854+0000 mgr.y (mgr.14535) 343 : cluster [DBG] pgmap v406: 152 pgs: 1 active+clean+scrubbing, 151 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 300 op/s 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:53.615981+0000 mon.a (mon.0) 1092 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:53.616033+0000 mon.a (mon.0) 1093 : cluster [DBG] osdmap e214: 8 total, 8 up, 7 in 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:54.414931+0000 osd.7 (osd.7) 161 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:37:55.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:54.418950+0000 osd.7 (osd.7) 162 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.397614+0000 osd.6 (osd.6) 99 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.400527+0000 osd.6 (osd.6) 100 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.457352+0000 osd.4 (osd.4) 139 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.461280+0000 osd.4 (osd.4) 140 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.467854+0000 mgr.y (mgr.14535) 343 : cluster [DBG] pgmap v406: 152 pgs: 1 active+clean+scrubbing, 151 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 300 op/s 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:53.615981+0000 mon.a (mon.0) 1092 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:53.616033+0000 mon.a (mon.0) 1093 : cluster [DBG] osdmap e214: 8 total, 8 up, 7 in 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:54.414931+0000 osd.7 (osd.7) 161 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:37:55.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:54.418950+0000 osd.7 (osd.7) 162 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:37:55.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:37:55.451 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:37:55.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:37:55.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:55.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:37:55.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:55.744 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:55.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:55.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:55.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:55.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:55.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:54.234226+0000 osd.1 (osd.1) 136 : cluster [DBG] 8.1 scrub starts 2024-02-22T19:37:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:54.624255+0000 mon.a (mon.0) 1094 : cluster [DBG] osdmap e215: 8 total, 8 up, 7 in 2024-02-22T19:37:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:54.728353+0000 mgr.y (mgr.14535) 344 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:56.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:54.234226+0000 osd.1 (osd.1) 136 : cluster [DBG] 8.1 scrub starts 2024-02-22T19:37:56.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:54.624255+0000 mon.a (mon.0) 1094 : cluster [DBG] osdmap e215: 8 total, 8 up, 7 in 2024-02-22T19:37:56.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:54.728353+0000 mgr.y (mgr.14535) 344 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:56.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:37:56.038 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:56.040 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:54.234226+0000 osd.1 (osd.1) 136 : cluster [DBG] 8.1 scrub starts 2024-02-22T19:37:56.040 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:54.624255+0000 mon.a (mon.0) 1094 : cluster [DBG] osdmap e215: 8 total, 8 up, 7 in 2024-02-22T19:37:56.040 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:54.728353+0000 mgr.y (mgr.14535) 344 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:37:56.065 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:37:56.067 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:56.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:37:56.114 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:56.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:37:56.185 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:37:56.225 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:37:56.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:56.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:37:56.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:37:56.397 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:56.439 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:37:56.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:56.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:37:56.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:37:56.579 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:56.601 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:37:56.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:56.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:37:56.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:37:56.467+0000 7fe2d794e700 -1 mon.a@0(leader).osd e216 definitely_dead 0 2024-02-22T19:37:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.5. 2024-02-22T19:37:56.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:37:56.737 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:55.443402+0000 osd.3 (osd.3) 151 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:37:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:55.451186+0000 osd.3 (osd.3) 152 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:37:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:55.470565+0000 mgr.y (mgr.14535) 345 : cluster [DBG] pgmap v409: 152 pgs: 2 peering, 1 active+clean+scrubbing, 149 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:37:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:55.650894+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e216: 8 total, 8 up, 7 in 2024-02-22T19:37:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:56.401492+0000 osd.7 (osd.7) 163 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:37:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:56.406756+0000 osd.7 (osd.7) 164 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:37:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:56.468373+0000 mon.a (mon.0) 1096 : audit [DBG] from='client.? 172.21.15.53:0/1126052466' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["5"]}]: dispatch 2024-02-22T19:37:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:56.653164+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? 172.21.15.53:0/1126052466' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["5"]}]': finished 2024-02-22T19:37:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:56.653221+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-22T19:37:56.936 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:55.443402+0000 osd.3 (osd.3) 151 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:55.451186+0000 osd.3 (osd.3) 152 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:55.470565+0000 mgr.y (mgr.14535) 345 : cluster [DBG] pgmap v409: 152 pgs: 2 peering, 1 active+clean+scrubbing, 149 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:55.650894+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e216: 8 total, 8 up, 7 in 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:56.401492+0000 osd.7 (osd.7) 163 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:56.406756+0000 osd.7 (osd.7) 164 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:56.468373+0000 mon.a (mon.0) 1096 : audit [DBG] from='client.? 172.21.15.53:0/1126052466' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["5"]}]: dispatch 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:56.653164+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? 172.21.15.53:0/1126052466' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["5"]}]': finished 2024-02-22T19:37:56.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:56.653221+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:55.443402+0000 osd.3 (osd.3) 151 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:55.451186+0000 osd.3 (osd.3) 152 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:55.470565+0000 mgr.y (mgr.14535) 345 : cluster [DBG] pgmap v409: 152 pgs: 2 peering, 1 active+clean+scrubbing, 149 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:55.650894+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e216: 8 total, 8 up, 7 in 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:56.401492+0000 osd.7 (osd.7) 163 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:56.406756+0000 osd.7 (osd.7) 164 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:56.468373+0000 mon.a (mon.0) 1096 : audit [DBG] from='client.? 172.21.15.53:0/1126052466' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["5"]}]: dispatch 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:56.653164+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? 172.21.15.53:0/1126052466' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["5"]}]': finished 2024-02-22T19:37:56.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:56.653221+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-22T19:37:56.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:56.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:37:57.042 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:57.071 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:56.009514+0000 osd.1 (osd.1) 137 : cluster [DBG] 8.1 scrub ok 2024-02-22T19:37:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:56.240202+0000 osd.2 (osd.2) 114 : cluster [DBG] 6.f scrub starts 2024-02-22T19:37:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:56.312684+0000 osd.2 (osd.2) 115 : cluster [DBG] 6.f scrub ok 2024-02-22T19:37:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:57.160592+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-22T19:37:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:57.471656+0000 mon.a (mon.0) 1100 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-22T19:37:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:56.009514+0000 osd.1 (osd.1) 137 : cluster [DBG] 8.1 scrub ok 2024-02-22T19:37:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:56.240202+0000 osd.2 (osd.2) 114 : cluster [DBG] 6.f scrub starts 2024-02-22T19:37:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:56.312684+0000 osd.2 (osd.2) 115 : cluster [DBG] 6.f scrub ok 2024-02-22T19:37:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:57.160592+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-22T19:37:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:57.471656+0000 mon.a (mon.0) 1100 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-22T19:37:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:56.009514+0000 osd.1 (osd.1) 137 : cluster [DBG] 8.1 scrub ok 2024-02-22T19:37:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:56.240202+0000 osd.2 (osd.2) 114 : cluster [DBG] 6.f scrub starts 2024-02-22T19:37:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:56.312684+0000 osd.2 (osd.2) 115 : cluster [DBG] 6.f scrub ok 2024-02-22T19:37:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:57.160592+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-22T19:37:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:57.471656+0000 mon.a (mon.0) 1100 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-22T19:37:58.697 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:57.470932+0000 mgr.y (mgr.14535) 346 : cluster [DBG] pgmap v413: 151 pgs: 1 peering, 1 active+clean+scrubbing, 149 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-02-22T19:37:58.698 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:58.166102+0000 mon.a (mon.0) 1101 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-22T19:37:58.698 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.166175+0000 mon.a (mon.0) 1102 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-22T19:37:58.698 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.167894+0000 osd.4 (osd.4) 141 : cluster [DBG] 8.2 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10260 2024-02-22T19:37:58.698 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.170640+0000 osd.4 (osd.4) 142 : cluster [DBG] 5.e starting backfill to osd.5 from (0'0,0'0] MAX to 199'11 2024-02-22T19:37:58.834 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:37:58.849 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:37:58.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:37:58.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:37:58.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:57.470932+0000 mgr.y (mgr.14535) 346 : cluster [DBG] pgmap v413: 151 pgs: 1 peering, 1 active+clean+scrubbing, 149 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-02-22T19:37:58.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:58.166102+0000 mon.a (mon.0) 1101 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-22T19:37:58.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.166175+0000 mon.a (mon.0) 1102 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-22T19:37:58.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.167894+0000 osd.4 (osd.4) 141 : cluster [DBG] 8.2 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10260 2024-02-22T19:37:58.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.170640+0000 osd.4 (osd.4) 142 : cluster [DBG] 5.e starting backfill to osd.5 from (0'0,0'0] MAX to 199'11 2024-02-22T19:37:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:37:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:57.470932+0000 mgr.y (mgr.14535) 346 : cluster [DBG] pgmap v413: 151 pgs: 1 peering, 1 active+clean+scrubbing, 149 active+clean; 3.7 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-02-22T19:37:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:58.166102+0000 mon.a (mon.0) 1101 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-22T19:37:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.166175+0000 mon.a (mon.0) 1102 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-22T19:37:59.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.167894+0000 osd.4 (osd.4) 141 : cluster [DBG] 8.2 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10260 2024-02-22T19:37:59.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.170640+0000 osd.4 (osd.4) 142 : cluster [DBG] 5.e starting backfill to osd.5 from (0'0,0'0] MAX to 199'11 2024-02-22T19:37:59.530 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-02-22T19:37:59.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:37:59] "GET /metrics HTTP/1.1" 200 175840 "" "Prometheus/2.33.4" 2024-02-22T19:38:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.164886+0000 osd.1 (osd.1) 138 : cluster [DBG] 8.a starting backfill to osd.5 from (0'0,0'0] MAX to 216'10231 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.165607+0000 osd.3 (osd.3) 153 : cluster [DBG] 8.0 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10215 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.165644+0000 osd.1 (osd.1) 139 : cluster [DBG] 8.3 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10152 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.168022+0000 osd.0 (osd.0) 157 : cluster [DBG] 8.8 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10585 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.169156+0000 osd.2 (osd.2) 116 : cluster [DBG] 8.4 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10122 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.170565+0000 osd.6 (osd.6) 101 : cluster [DBG] 8.9 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10381 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.175430+0000 osd.2 (osd.2) 117 : cluster [DBG] 5.f starting backfill to osd.5 from (0'0,0'0] MAX to 194'12 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.189485+0000 osd.2 (osd.2) 118 : cluster [DBG] 8.d starting backfill to osd.5 from (0'0,0'0] MAX to 216'10472 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:58.200709+0000 osd.2 (osd.2) 119 : cluster [DBG] 5.d starting backfill to osd.5 from (0'0,0'0] MAX to 202'11 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:59.169879+0000 mon.a (mon.0) 1103 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:59.568401+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.53:0/3096360178' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:00.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:37:59.571175+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:00.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.164886+0000 osd.1 (osd.1) 138 : cluster [DBG] 8.a starting backfill to osd.5 from (0'0,0'0] MAX to 216'10231 2024-02-22T19:38:00.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.165607+0000 osd.3 (osd.3) 153 : cluster [DBG] 8.0 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10215 2024-02-22T19:38:00.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.165644+0000 osd.1 (osd.1) 139 : cluster [DBG] 8.3 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10152 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.168022+0000 osd.0 (osd.0) 157 : cluster [DBG] 8.8 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10585 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.169156+0000 osd.2 (osd.2) 116 : cluster [DBG] 8.4 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10122 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.170565+0000 osd.6 (osd.6) 101 : cluster [DBG] 8.9 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10381 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.175430+0000 osd.2 (osd.2) 117 : cluster [DBG] 5.f starting backfill to osd.5 from (0'0,0'0] MAX to 194'12 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.189485+0000 osd.2 (osd.2) 118 : cluster [DBG] 8.d starting backfill to osd.5 from (0'0,0'0] MAX to 216'10472 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:58.200709+0000 osd.2 (osd.2) 119 : cluster [DBG] 5.d starting backfill to osd.5 from (0'0,0'0] MAX to 202'11 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:59.169879+0000 mon.a (mon.0) 1103 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:59.568401+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.53:0/3096360178' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:00.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:37:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:37:59.571175+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:00.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.164886+0000 osd.1 (osd.1) 138 : cluster [DBG] 8.a starting backfill to osd.5 from (0'0,0'0] MAX to 216'10231 2024-02-22T19:38:00.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.165607+0000 osd.3 (osd.3) 153 : cluster [DBG] 8.0 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10215 2024-02-22T19:38:00.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.165644+0000 osd.1 (osd.1) 139 : cluster [DBG] 8.3 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10152 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.168022+0000 osd.0 (osd.0) 157 : cluster [DBG] 8.8 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10585 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.169156+0000 osd.2 (osd.2) 116 : cluster [DBG] 8.4 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10122 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.170565+0000 osd.6 (osd.6) 101 : cluster [DBG] 8.9 starting backfill to osd.5 from (0'0,0'0] MAX to 216'10381 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.175430+0000 osd.2 (osd.2) 117 : cluster [DBG] 5.f starting backfill to osd.5 from (0'0,0'0] MAX to 194'12 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.189485+0000 osd.2 (osd.2) 118 : cluster [DBG] 8.d starting backfill to osd.5 from (0'0,0'0] MAX to 216'10472 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:58.200709+0000 osd.2 (osd.2) 119 : cluster [DBG] 5.d starting backfill to osd.5 from (0'0,0'0] MAX to 202'11 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:59.169879+0000 mon.a (mon.0) 1103 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:59.568401+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.53:0/3096360178' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:00.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:37:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:37:59.571175+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:00.228 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:38:00.247 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:00.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:00.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:00.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:00.389 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:00.421 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:00.425 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:00.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:00.526 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:00.713 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:37:59.472862+0000 mgr.y (mgr.14535) 347 : cluster [DBG] pgmap v416: 151 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 2 clean+premerge+peered, 139 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 614 op/s; 25026/174840 objects misplaced (14.314%); 135 KiB/s, 11 objects/s recovering 2024-02-22T19:38:00.714 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:00.168156+0000 mon.a (mon.0) 1105 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:00.714 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:00.171801+0000 mon.a (mon.0) 1106 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:38:00.714 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:00.171826+0000 mon.a (mon.0) 1107 : cluster [INF] Cluster is now healthy 2024-02-22T19:38:00.714 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:00.194458+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:00.714 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:00.194529+0000 mon.a (mon.0) 1109 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-22T19:38:00.915 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:37:59.472862+0000 mgr.y (mgr.14535) 347 : cluster [DBG] pgmap v416: 151 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 2 clean+premerge+peered, 139 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 614 op/s; 25026/174840 objects misplaced (14.314%); 135 KiB/s, 11 objects/s recovering 2024-02-22T19:38:00.915 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:00.168156+0000 mon.a (mon.0) 1105 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:00.915 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:00.171801+0000 mon.a (mon.0) 1106 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:38:00.915 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:00.171826+0000 mon.a (mon.0) 1107 : cluster [INF] Cluster is now healthy 2024-02-22T19:38:00.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:00.194458+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:00.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:00.194529+0000 mon.a (mon.0) 1109 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-22T19:38:00.916 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:37:59.472862+0000 mgr.y (mgr.14535) 347 : cluster [DBG] pgmap v416: 151 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 2 clean+premerge+peered, 139 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 614 op/s; 25026/174840 objects misplaced (14.314%); 135 KiB/s, 11 objects/s recovering 2024-02-22T19:38:00.916 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:00.168156+0000 mon.a (mon.0) 1105 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:00.916 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:00.171801+0000 mon.a (mon.0) 1106 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:38:00.916 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:00.171826+0000 mon.a (mon.0) 1107 : cluster [INF] Cluster is now healthy 2024-02-22T19:38:00.917 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:00.194458+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:00.917 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:00.194529+0000 mon.a (mon.0) 1109 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-22T19:38:01.167 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:01.171 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:01.172 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:01.273 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:01.554 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:01.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:01.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:01.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:01.714 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:01.848 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:01.857 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:01.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:01.898 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:38:01.898 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:38:01.898 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-02-22T19:38:01.898 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:38:01.958 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:02.042 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:02.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:02.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:02.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:02.241 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:02.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:02.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:02.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:02.453 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:02.457 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:02.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:02.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:01.196851+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-22T19:38:02.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:01.473451+0000 mgr.y (mgr.14535) 348 : cluster [DBG] pgmap v419: 150 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 1 clean+premerge+peered, 138 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 719 op/s; 28571/173232 objects misplaced (16.493%); 336 KiB/s, 14 objects/s recovering 2024-02-22T19:38:02.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:01.475012+0000 mon.a (mon.0) 1111 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-22T19:38:02.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:02.165544+0000 mon.a (mon.0) 1112 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-22T19:38:02.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:02.165604+0000 mon.a (mon.0) 1113 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-22T19:38:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:01.196851+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-22T19:38:02.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:01.473451+0000 mgr.y (mgr.14535) 348 : cluster [DBG] pgmap v419: 150 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 1 clean+premerge+peered, 138 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 719 op/s; 28571/173232 objects misplaced (16.493%); 336 KiB/s, 14 objects/s recovering 2024-02-22T19:38:02.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:01.475012+0000 mon.a (mon.0) 1111 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-22T19:38:02.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:02.165544+0000 mon.a (mon.0) 1112 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-22T19:38:02.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:02.165604+0000 mon.a (mon.0) 1113 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-22T19:38:02.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:01.196851+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-22T19:38:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:01.473451+0000 mgr.y (mgr.14535) 348 : cluster [DBG] pgmap v419: 150 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 1 clean+premerge+peered, 138 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 719 op/s; 28571/173232 objects misplaced (16.493%); 336 KiB/s, 14 objects/s recovering 2024-02-22T19:38:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:01.475012+0000 mon.a (mon.0) 1111 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-22T19:38:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:02.165544+0000 mon.a (mon.0) 1112 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-22T19:38:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:02.165604+0000 mon.a (mon.0) 1113 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-22T19:38:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:02.602 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:38:02.621 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:02.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:02.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:02.648 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:38:02.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:38:02.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:02.751 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:38:02.788 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:38:02.790 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:02.829 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:38:02.881 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:38:02.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:38:02.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:38:03.224 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:02.556204+0000 mon.c (mon.2) 50 : audit [DBG] from='client.? 172.21.15.53:0/2378087733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:03.224 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:02.628291+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:03.224 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:02.628330+0000 osd.3 (osd.3) 155 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:03.224 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:03.174556+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-22T19:38:03.224 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:02.556204+0000 mon.c (mon.2) 50 : audit [DBG] from='client.? 172.21.15.53:0/2378087733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:03.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:02.628291+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:03.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:02.628330+0000 osd.3 (osd.3) 155 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:03.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:38:03.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:03.174556+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-22T19:38:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:02.556204+0000 mon.c (mon.2) 50 : audit [DBG] from='client.? 172.21.15.53:0/2378087733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:02.628291+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:02.628330+0000 osd.3 (osd.3) 155 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:03.174556+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-22T19:38:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:03.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:03.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:03.913 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:03.952 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:04.234 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:04.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:04.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:04.339 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:04.404 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:03.479518+0000 mgr.y (mgr.14535) 349 : cluster [DBG] pgmap v422: 150 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 1 clean+premerge+peered, 138 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 133 op/s; 28571/173232 objects misplaced (16.493%); 208 KiB/s, 3 objects/s recovering 2024-02-22T19:38:04.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:04.175760+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-22T19:38:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:03.479518+0000 mgr.y (mgr.14535) 349 : cluster [DBG] pgmap v422: 150 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 1 clean+premerge+peered, 138 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 133 op/s; 28571/173232 objects misplaced (16.493%); 208 KiB/s, 3 objects/s recovering 2024-02-22T19:38:04.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:04.175760+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-22T19:38:04.531 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:38:04.531 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:38:04.531 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 6, 7, 5] 2024-02-22T19:38:04.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 0 2024-02-22T19:38:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:03.479518+0000 mgr.y (mgr.14535) 349 : cluster [DBG] pgmap v422: 150 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 3 peering, 1 clean+premerge+peered, 138 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 133 op/s; 28571/173232 objects misplaced (16.493%); 208 KiB/s, 3 objects/s recovering 2024-02-22T19:38:04.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:04.175760+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-22T19:38:04.993 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:04.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:04.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:05.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:05.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:38:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:38:05.103 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:38:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:38:05.142 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:05.169 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:05.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:05.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:05.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:04.743427+0000 mgr.y (mgr.14535) 350 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:05.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:05.180978+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-22T19:38:05.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:04.743427+0000 mgr.y (mgr.14535) 350 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:05.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:05.180978+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-22T19:38:05.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:05.350 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:05.607 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:04.743427+0000 mgr.y (mgr.14535) 350 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:05.607 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:05.180978+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-22T19:38:05.644 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:05.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:05.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:05.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:05.792 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:05.819 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:05.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:05.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:38:05.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:05.958 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:05.990 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:05.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:06.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:06.174 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:06.455 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:06.466 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:05.482348+0000 mgr.y (mgr.14535) 351 : cluster [DBG] pgmap v425: 149 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 1 peering, 140 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 26410/160722 objects misplaced (16.432%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:38:06.466 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:05.483655+0000 mon.a (mon.0) 1117 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-22T19:38:06.466 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:05.482348+0000 mgr.y (mgr.14535) 351 : cluster [DBG] pgmap v425: 149 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 1 peering, 140 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 26410/160722 objects misplaced (16.432%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:38:06.466 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:05.483655+0000 mon.a (mon.0) 1117 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-22T19:38:06.467 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:06.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:06.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:06.606 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:05.482348+0000 mgr.y (mgr.14535) 351 : cluster [DBG] pgmap v425: 149 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 1 peering, 140 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 26410/160722 objects misplaced (16.432%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:38:06.606 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:05.483655+0000 mon.a (mon.0) 1117 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-22T19:38:06.614 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:06.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:06.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:06.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:06.806 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:06.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:06.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:06.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:06.980 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:07.442 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:07.443 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:07.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:06.271304+0000 mon.a (mon.0) 1118 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-22T19:38:07.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:06.271367+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-22T19:38:07.469 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:07.167491+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-22T19:38:07.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:06.271304+0000 mon.a (mon.0) 1118 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-22T19:38:07.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:06.271367+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-22T19:38:07.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:07.167491+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-22T19:38:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:06.271304+0000 mon.a (mon.0) 1118 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-22T19:38:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:06.271367+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-22T19:38:07.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:07.167491+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-22T19:38:07.889 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:08.223 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:08.230 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:38:08.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:08.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:08.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:38:08.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:08.414 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:08.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:08.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:07.482796+0000 mgr.y (mgr.14535) 352 : cluster [DBG] pgmap v428: 149 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 1 peering, 140 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 26410/160722 objects misplaced (16.432%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:38:08.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:07.665364+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.53:0/2003305884' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:38:08.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:07.665734+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:38:08.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:08.182193+0000 mon.a (mon.0) 1122 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:08.188112+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:08.188178+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:07.482796+0000 mgr.y (mgr.14535) 352 : cluster [DBG] pgmap v428: 149 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 1 peering, 140 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 26410/160722 objects misplaced (16.432%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:07.665364+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.53:0/2003305884' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:07.665734+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:08.182193+0000 mon.a (mon.0) 1122 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:08.188112+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:38:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:08.188178+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-22T19:38:08.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:08.594 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:07.482796+0000 mgr.y (mgr.14535) 352 : cluster [DBG] pgmap v428: 149 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 1 peering, 140 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 26410/160722 objects misplaced (16.432%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:38:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:07.665364+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.53:0/2003305884' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:38:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:07.665734+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:38:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:08.182193+0000 mon.a (mon.0) 1122 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:08.188112+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:38:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:08.188178+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-22T19:38:08.747 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:08.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:08.754 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:08.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:08.894 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:08.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:08.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:09.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:09.092 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:09.344 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:09.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:09.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:09.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:09.514 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:09.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:08.379648+0000 mon.a (mon.0) 1125 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:09.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:08.699764+0000 mon.a (mon.0) 1126 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:09.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:09.199369+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-22T19:38:09.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:09] "GET /metrics HTTP/1.1" 200 175945 "" "Prometheus/2.33.4" 2024-02-22T19:38:09.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:08.379648+0000 mon.a (mon.0) 1125 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:09.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:08.699764+0000 mon.a (mon.0) 1126 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:09.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:09.199369+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-22T19:38:09.827 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:08.379648+0000 mon.a (mon.0) 1125 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:08.699764+0000 mon.a (mon.0) 1126 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:09.199369+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-22T19:38:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:10.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:09.484067+0000 mgr.y (mgr.14535) 353 : cluster [DBG] pgmap v431: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 804 op/s; 24620/151083 objects misplaced (16.296%); 928 KiB/s, 14 objects/s recovering 2024-02-22T19:38:10.588 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:09.484067+0000 mgr.y (mgr.14535) 353 : cluster [DBG] pgmap v431: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 804 op/s; 24620/151083 objects misplaced (16.296%); 928 KiB/s, 14 objects/s recovering 2024-02-22T19:38:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:09.484067+0000 mgr.y (mgr.14535) 353 : cluster [DBG] pgmap v431: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 804 op/s; 24620/151083 objects misplaced (16.296%); 928 KiB/s, 14 objects/s recovering 2024-02-22T19:38:11.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:38:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:11.426 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:11.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:11.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:11.582 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:11.697 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:10.059113+0000 osd.2 (osd.2) 120 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:11.697 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:10.059168+0000 osd.2 (osd.2) 121 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:11.698 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:10.401756+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-22T19:38:11.698 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:38:11.604+0000 7fe2d794e700 -1 mon.a@0(leader).osd e231 definitely_dead 0 2024-02-22T19:38:11.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:10.059113+0000 osd.2 (osd.2) 120 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:11.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:10.059168+0000 osd.2 (osd.2) 121 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:11.698 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:10.401756+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-22T19:38:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:10.059113+0000 osd.2 (osd.2) 120 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:10.059168+0000 osd.2 (osd.2) 121 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:10.401756+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-22T19:38:12.179 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.0. 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:11.485473+0000 mgr.y (mgr.14535) 354 : cluster [DBG] pgmap v433: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 913 op/s; 24327/148908 objects misplaced (16.337%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:11.605759+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? 172.21.15.53:0/2793908192' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:11.605862+0000 mon.a (mon.0) 1130 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:12.173717+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? 172.21.15.53:0/2793908192' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:12.173776+0000 mon.a (mon.0) 1132 : cluster [DBG] osdmap e232: 8 total, 8 up, 7 in 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:11.485473+0000 mgr.y (mgr.14535) 354 : cluster [DBG] pgmap v433: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 913 op/s; 24327/148908 objects misplaced (16.337%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:11.605759+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? 172.21.15.53:0/2793908192' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:38:12.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:11.605862+0000 mon.a (mon.0) 1130 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-22T19:38:12.495 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:12.173717+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? 172.21.15.53:0/2793908192' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-22T19:38:12.495 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:12.173776+0000 mon.a (mon.0) 1132 : cluster [DBG] osdmap e232: 8 total, 8 up, 7 in 2024-02-22T19:38:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:11.485473+0000 mgr.y (mgr.14535) 354 : cluster [DBG] pgmap v433: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 913 op/s; 24327/148908 objects misplaced (16.337%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:38:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:11.605759+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? 172.21.15.53:0/2793908192' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:38:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:11.605862+0000 mon.a (mon.0) 1130 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-22T19:38:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:12.173717+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? 172.21.15.53:0/2793908192' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-22T19:38:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:12.173776+0000 mon.a (mon.0) 1132 : cluster [DBG] osdmap e232: 8 total, 8 up, 7 in 2024-02-22T19:38:13.088 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:38:13.094 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:13.097 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:13.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:13.117 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:38:13.134 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:38:13.151 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:38:13.170 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:38:13.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:13.232 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:38:13.300 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:38:13.307 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:13.343 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:38:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:38:13.360 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:13.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:13.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:13.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:13.501 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:13.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:13.071236+0000 mon.a (mon.0) 1133 : audit [DBG] from='client.? 172.21.15.53:0/2317940903' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:13.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:13.174251+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e233: 8 total, 8 up, 7 in 2024-02-22T19:38:13.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:13.071236+0000 mon.a (mon.0) 1133 : audit [DBG] from='client.? 172.21.15.53:0/2317940903' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:13.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:13.174251+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e233: 8 total, 8 up, 7 in 2024-02-22T19:38:13.654 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:13.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:13.756 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:13.784 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:13.071236+0000 mon.a (mon.0) 1133 : audit [DBG] from='client.? 172.21.15.53:0/2317940903' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:13.174251+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e233: 8 total, 8 up, 7 in 2024-02-22T19:38:13.888 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:13.893 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:13.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:13.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:14.040 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:14.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:14.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:14.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:14.170 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:14.212 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:14.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:14.235 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:14.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:14.375 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:14.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:14.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:14.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:14.560 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:14.594 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:14.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:14.596 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:14.667 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:13.485918+0000 mgr.y (mgr.14535) 355 : cluster [DBG] pgmap v436: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.0 GiB data, 9.8 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 919 op/s; 24327/148908 objects misplaced (16.337%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:38:14.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.175359+0000 mon.a (mon.0) 1135 : cluster [DBG] osdmap e234: 8 total, 8 up, 7 in 2024-02-22T19:38:14.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:38:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:38:14.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:14.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:13.485918+0000 mgr.y (mgr.14535) 355 : cluster [DBG] pgmap v436: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.0 GiB data, 9.8 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 919 op/s; 24327/148908 objects misplaced (16.337%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:38:14.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.175359+0000 mon.a (mon.0) 1135 : cluster [DBG] osdmap e234: 8 total, 8 up, 7 in 2024-02-22T19:38:14.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:13.485918+0000 mgr.y (mgr.14535) 355 : cluster [DBG] pgmap v436: 148 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 140 active+clean; 3.0 GiB data, 9.8 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 919 op/s; 24327/148908 objects misplaced (16.337%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:38:14.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.175359+0000 mon.a (mon.0) 1135 : cluster [DBG] osdmap e234: 8 total, 8 up, 7 in 2024-02-22T19:38:14.771 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:14.830 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:14.844 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:14.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:14.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:15.002 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:15.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:38:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:38:15.443 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:15.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:15.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:15.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:15.596 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:15.629 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:15.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:15.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:15.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.173936+0000 osd.1 (osd.1) 140 : cluster [DBG] 8.1 starting backfill to osd.5 from (0'0,0'0] MAX to 231'11039 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.175276+0000 osd.3 (osd.3) 156 : cluster [DBG] 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 231'11027 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.177017+0000 osd.4 (osd.4) 143 : cluster [INF] 8.2 continuing backfill to osd.5 from (179'5996,231'10926] MIN to 231'10926 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.177787+0000 osd.4 (osd.4) 144 : cluster [DBG] 5.e starting backfill to osd.2 from (0'0,0'0] MAX to 221'14 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.178915+0000 osd.2 (osd.2) 122 : cluster [DBG] 8.e starting backfill to osd.7 from (0'0,0'0] MAX to 231'11039 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.179178+0000 osd.6 (osd.6) 102 : cluster [INF] 8.9 continuing backfill to osd.5 from (177'5500,231'11056] 8:90cb756c:::benchmark_data_smithi053_130774_object96502:head to 231'11056 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:14.181872+0000 osd.6 (osd.6) 103 : cluster [DBG] 8.6 starting backfill to osd.2 from (0'0,0'0] MAX to 231'11199 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:14.754192+0000 mgr.y (mgr.14535) 356 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:15.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:15.182918+0000 mon.a (mon.0) 1136 : cluster [DBG] osdmap e235: 8 total, 8 up, 7 in 2024-02-22T19:38:15.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.173936+0000 osd.1 (osd.1) 140 : cluster [DBG] 8.1 starting backfill to osd.5 from (0'0,0'0] MAX to 231'11039 2024-02-22T19:38:15.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.175276+0000 osd.3 (osd.3) 156 : cluster [DBG] 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 231'11027 2024-02-22T19:38:15.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.177017+0000 osd.4 (osd.4) 143 : cluster [INF] 8.2 continuing backfill to osd.5 from (179'5996,231'10926] MIN to 231'10926 2024-02-22T19:38:15.768 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.177787+0000 osd.4 (osd.4) 144 : cluster [DBG] 5.e starting backfill to osd.2 from (0'0,0'0] MAX to 221'14 2024-02-22T19:38:15.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.178915+0000 osd.2 (osd.2) 122 : cluster [DBG] 8.e starting backfill to osd.7 from (0'0,0'0] MAX to 231'11039 2024-02-22T19:38:15.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.179178+0000 osd.6 (osd.6) 102 : cluster [INF] 8.9 continuing backfill to osd.5 from (177'5500,231'11056] 8:90cb756c:::benchmark_data_smithi053_130774_object96502:head to 231'11056 2024-02-22T19:38:15.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:14.181872+0000 osd.6 (osd.6) 103 : cluster [DBG] 8.6 starting backfill to osd.2 from (0'0,0'0] MAX to 231'11199 2024-02-22T19:38:15.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:14.754192+0000 mgr.y (mgr.14535) 356 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:15.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:15.182918+0000 mon.a (mon.0) 1136 : cluster [DBG] osdmap e235: 8 total, 8 up, 7 in 2024-02-22T19:38:15.811 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:15.811 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:15.852 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.173936+0000 osd.1 (osd.1) 140 : cluster [DBG] 8.1 starting backfill to osd.5 from (0'0,0'0] MAX to 231'11039 2024-02-22T19:38:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.175276+0000 osd.3 (osd.3) 156 : cluster [DBG] 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 231'11027 2024-02-22T19:38:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.177017+0000 osd.4 (osd.4) 143 : cluster [INF] 8.2 continuing backfill to osd.5 from (179'5996,231'10926] MIN to 231'10926 2024-02-22T19:38:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.177787+0000 osd.4 (osd.4) 144 : cluster [DBG] 5.e starting backfill to osd.2 from (0'0,0'0] MAX to 221'14 2024-02-22T19:38:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.178915+0000 osd.2 (osd.2) 122 : cluster [DBG] 8.e starting backfill to osd.7 from (0'0,0'0] MAX to 231'11039 2024-02-22T19:38:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.179178+0000 osd.6 (osd.6) 102 : cluster [INF] 8.9 continuing backfill to osd.5 from (177'5500,231'11056] 8:90cb756c:::benchmark_data_smithi053_130774_object96502:head to 231'11056 2024-02-22T19:38:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:14.181872+0000 osd.6 (osd.6) 103 : cluster [DBG] 8.6 starting backfill to osd.2 from (0'0,0'0] MAX to 231'11199 2024-02-22T19:38:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:14.754192+0000 mgr.y (mgr.14535) 356 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:15.182918+0000 mon.a (mon.0) 1136 : cluster [DBG] osdmap e235: 8 total, 8 up, 7 in 2024-02-22T19:38:15.857 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:15.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20875 7.a 7:5368db82:::benchmark_data_smithi053_107660_object20874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:38:15.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 386.84663498899999, 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001, 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20875 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:38:15.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030654+0000", 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5442000000000001e-05 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1599e-05 2024-02-22T19:38:15.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030691+0000", 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8590000000000001e-06 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030697+0000", 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5996999999999998e-05 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030763+0000", 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1030000000000002e-06 2024-02-22T19:38:15.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030771+0000", 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96976369299999998 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000535+0000", 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3981999999999999e-05 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000549+0000", 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449743229999999 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045523+0000", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015328299999999999 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045676+0000", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167651 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045844+0000", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003101946 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048946+0000", 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029887199999999998 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051935+0000", 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057814199999999998 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057716+0000", 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5281000000000001e-05 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057771+0000", 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092475900000000004 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058696+0000", 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111852 7.a 7:5162ddfb:::benchmark_data_smithi053_107660_object22645:head [delete] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.76847338599998, 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.030160867, 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111852 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:38:15.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108815+0000", 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8550000000000008e-06 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108824+0000", 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3647e-05 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108837+0000", 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9809999999999997e-06 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108844+0000", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9802000000000001e-05 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108904+0000", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013143000000000001 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.109036+0000", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016598399999999999 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.109202+0000", 2024-02-22T19:38:15.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012752025 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121954+0000", 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1360000000000002e-06 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121960+0000", 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016965211 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.138925+0000", 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3065000000000001e-05 2024-02-22T19:38:15.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.138978+0000", 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.030160867 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111854 7.a 7:5a0cbb42:::benchmark_data_smithi053_107660_object22647:head [delete] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.76349776900003, 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531219629999999, 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111854 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113791+0000", 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3090000000000003e-06 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113797+0000", 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8690000000000004e-06 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113805+0000", 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6439999999999997e-06 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113810+0000", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109718 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113920+0000", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125108 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.114045+0000", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016840599999999999 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.114214+0000", 2024-02-22T19:38:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077607329999999997 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121974+0000", 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6069999999999999e-06 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121978+0000", 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170384509999999 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139016+0000", 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100129 2024-02-22T19:38:15.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139116+0000", 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027745477000000001 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166862+0000", 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2849000000000002e-05 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166915+0000", 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531219629999999 2024-02-22T19:38:15.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111874 7.a 7:51c8a40d:::benchmark_data_smithi053_107660_object22667:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.744160206, 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0339878849999999, 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111874 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133128+0000", 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7869999999999996e-06 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133138+0000", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2543e-05 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133150+0000", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8499999999999994e-06 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133158+0000", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019007130000000001 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.135059+0000", 2024-02-22T19:38:15.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2880000000000004e-06 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.135066+0000", 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0040841840000001 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139150+0000", 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011695400000000001 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139267+0000", 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014209699999999999 2024-02-22T19:38:15.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139409+0000", 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011988443999999999 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151398+0000", 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015560497 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166958+0000", 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1764000000000002e-05 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167000+0000", 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4150000000000003e-05 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167034+0000", 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4145000000000002e-05 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167118+0000", 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0339878849999999 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111911 7.a 7:53f6a651:::benchmark_data_smithi053_107660_object22704:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.68832976700003, 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97843876299999999, 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111911 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188959+0000", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188967+0000", 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1484000000000001e-05 2024-02-22T19:38:15.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188978+0000", 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7379999999999997e-06 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188985+0000", 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00214072 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.191126+0000", 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1008e-05 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.191137+0000", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95036631800000004 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141503+0000", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001082 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141611+0000", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015199899999999999 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141763+0000", 2024-02-22T19:38:15.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096414059999999999 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151405+0000", 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015767331999999998 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167172+0000", 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9851000000000003e-05 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167272+0000", 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2466999999999999e-05 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167304+0000", 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5302000000000003e-05 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167400+0000", 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97843876299999999 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111925 7.a 7:55d29d8b:::benchmark_data_smithi053_107660_object22718:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:38:15.912 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.68005287900002, 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97039658799999995, 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111925 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197237+0000", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.563e-06 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197241+0000", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3770000000000004e-06 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197246+0000", 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-02-22T19:38:15.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197250+0000", 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2380999999999997e-05 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197322+0000", 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.268e-06 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197332+0000", 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94598222200000004 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143314+0000", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013365499999999999 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143448+0000", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014185900000000001 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143589+0000", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078197079999999999 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151409+0000", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015815124 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167224+0000", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027372900000000001 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167498+0000", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5688000000000001e-05 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167534+0000", 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010068699999999999 2024-02-22T19:38:15.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167634+0000", 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97039658799999995 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111926 7.a 7:5f8a638d:::benchmark_data_smithi053_107660_object22719:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.68002942200002, 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97061990799999998, 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111926 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5790000000000002e-06 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197263+0000", 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197268+0000", 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6339999999999999e-06 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197272+0000", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0308000000000004e-05 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197352+0000", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.28e-06 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197358+0000", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94796157400000003 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145320+0000", 2024-02-22T19:38:15.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012371600000000001 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145444+0000", 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158498 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145602+0000", 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095829260000000003 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155185+0000", 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012273795000000001 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167459+0000", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028113900000000002 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167740+0000", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4212999999999998e-05 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167784+0000", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6879000000000005e-05 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167881+0000", 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97061990799999998 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111975 7.a 7:5962aaaf:::benchmark_data_smithi053_107660_object22768:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.62051800199998, 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91135001299999996, 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111975 2024-02-22T19:38:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256772+0000", 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7850000000000002e-06 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256776+0000", 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0140000000000001e-06 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256783+0000", 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9890000000000003e-06 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256787+0000", 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6700999999999998e-05 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256843+0000", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8880000000000003e-06 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256852+0000", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89037246800000003 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147225+0000", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011598499999999999 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147341+0000", 2024-02-22T19:38:15.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013154 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147472+0000", 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077203439999999996 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155193+0000", 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012737145 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167930+0000", 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0123999999999997e-05 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167970+0000", 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3705000000000003e-05 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168014+0000", 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108915 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168123+0000", 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91135001299999996 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112025 7.a 7:5114742b:::benchmark_data_smithi053_107660_object22818:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.56239456100002, 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85347167700000004, 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112025 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:38:15.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314895+0000", 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5490000000000001e-06 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314902+0000", 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531e-05 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314913+0000", 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1970000000000004e-06 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314918+0000", 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1729999999999993e-05 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314980+0000", 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1537e-05 2024-02-22T19:38:15.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314991+0000", 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.83431657000000004 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149308+0000", 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011637900000000001 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149424+0000", 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000147207 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149571+0000", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011278806000000001 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160850+0000", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073124599999999998 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168162+0000", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2563000000000004e-05 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168225+0000", 2024-02-22T19:38:15.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5136999999999998e-05 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168270+0000", 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7520000000000001e-05 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168368+0000", 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85347167700000004 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.935 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112063 7.a 7:5ff85e4d:::benchmark_data_smithi053_107660_object22856:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.52033302199999, 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.811635777, 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112063 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356956+0000", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356962+0000", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8179999999999993e-06 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356971+0000", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4229999999999996e-06 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356977+0000", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1233000000000001e-05 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.357038+0000", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.357048+0000", 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.79440474900000002 2024-02-22T19:38:15.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151452+0000", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012225300000000001 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151575+0000", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8306999999999993e-05 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151663+0000", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091947409999999993 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160858+0000", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075599589999999998 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168418+0000", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2265999999999998e-05 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168460+0000", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0454999999999998e-05 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168500+0000", 2024-02-22T19:38:15.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2958e-05 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168593+0000", 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.811635777 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112096 7.a 7:5442534a:::benchmark_data_smithi053_107660_object22889:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.485988298, 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.777506645, 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.941 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112096 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:15.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.313e-06 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391306+0000", 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0849999999999999e-06 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391313+0000", 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0760000000000002e-06 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391318+0000", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1353999999999994e-05 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391399+0000", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0243e-05 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391410+0000", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.76178757200000002 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153197+0000", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109708 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153307+0000", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013573900000000001 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153443+0000", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011010321999999999 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164453+0000", 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041854379999999997 2024-02-22T19:38:15.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168638+0000", 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0509999999999997e-05 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168679+0000", 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8803000000000001e-05 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168718+0000", 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1186000000000004e-05 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168809+0000", 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.777506645 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112098 7.a 7:519628cb:::benchmark_data_smithi053_107660_object22891:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.48404919199999, 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.77578508499999999, 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112098 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393240+0000", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1999999999999997e-06 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393247+0000", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5688999999999998e-05 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393263+0000", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-06 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393270+0000", 2024-02-22T19:38:15.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4311000000000001e-05 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393334+0000", 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3839999999999997e-06 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393342+0000", 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.76188229600000001 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155225+0000", 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5840999999999993e-05 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155310+0000", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5882000000000002e-05 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155396+0000", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0090645329999999996 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164461+0000", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043933849999999997 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168854+0000", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0268000000000001e-05 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168895+0000", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9428000000000003e-05 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168934+0000", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2540000000000005e-05 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169026+0000", 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.77578508499999999 2024-02-22T19:38:15.952 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112130 7.a 7:5e909bd4:::benchmark_data_smithi053_107660_object22923:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.44549826500003, 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73752859400000004, 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112130 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431791+0000", 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.251e-06 2024-02-22T19:38:15.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431797+0000", 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0682999999999999e-05 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431808+0000", 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3590000000000002e-06 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431816+0000", 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5444999999999994e-05 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431881+0000", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0912e-05 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431892+0000", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.72501750099999995 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.156909+0000", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111295 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.157021+0000", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7911999999999994e-05 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.157119+0000", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073499309999999997 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164468+0000", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046038349999999997 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169072+0000", 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105296 2024-02-22T19:38:15.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169178+0000", 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4647999999999997e-05 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169222+0000", 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8640000000000004e-05 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169321+0000", 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73752859400000004 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112203 7.a 7:5c4dbf81:::benchmark_data_smithi053_107660_object22996:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.36104431400003, 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65325745000000002, 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112203 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516245+0000", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516253+0000", 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7668000000000001e-05 2024-02-22T19:38:15.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516270+0000", 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6799999999999993e-06 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516278+0000", 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8060000000000003e-05 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516336+0000", 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6299999999999993e-06 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516346+0000", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64245662999999997 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.158802+0000", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103798 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.158906+0000", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129794 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.159036+0000", 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071273559999999996 2024-02-22T19:38:15.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166163+0000", 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029783499999999998 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169142+0000", 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022555399999999999 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169367+0000", 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2079000000000001e-05 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169409+0000", 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4319000000000003e-05 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169504+0000", 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65325745000000002 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112210 7.a 7:56d9e136:::benchmark_data_smithi053_107660_object23003:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.35083924600002, 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64327634499999997, 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112210 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526450+0000", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2809999999999997e-06 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526456+0000", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0599e-05 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526467+0000", 2024-02-22T19:38:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4539999999999998e-06 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526472+0000", 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6777000000000003e-05 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526519+0000", 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3770000000000001e-06 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526526+0000", 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63437577199999995 2024-02-22T19:38:15.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160901+0000", 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106143 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.161007+0000", 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109844 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.161117+0000", 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050523790000000001 2024-02-22T19:38:15.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166170+0000", 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003381743 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169551+0000", 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1298999999999999e-05 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169593+0000", 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0349000000000003e-05 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169633+0000", 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4598000000000005e-05 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169728+0000", 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64327634499999997 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112222 7.a 7:55f40a1f:::benchmark_data_smithi053_107660_object23015:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.550201+0000", 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.32708991499999, 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.61974588799999997, 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112222 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550201+0000", 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550201+0000", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550199+0000", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.092e-06 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550206+0000", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.275e-05 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550219+0000", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6590000000000004e-06 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550225+0000", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2351000000000001e-05 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550277+0000", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.799e-06 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.550286+0000", 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.61258154899999995 2024-02-22T19:38:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.162867+0000", 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010998899999999999 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.162977+0000", 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7453000000000004e-05 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.163075+0000", 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033242739999999999 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166399+0000", 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033724670000000001 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169772+0000", 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0144000000000001e-05 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169812+0000", 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2228999999999998e-05 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169854+0000", 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2577000000000005e-05 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169947+0000", 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.61974588799999997 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112313 7.a 7:5a054abe:::benchmark_data_smithi053_107660_object23106:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.811504+0000", 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 274.06578633800001, 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.35866692999999999, 2024-02-22T19:38:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112313 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811504+0000", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811504+0000", 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811503+0000", 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9540000000000004e-06 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811511+0000", 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1779e-05 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811522+0000", 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6200000000000004e-06 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811528+0000", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7555000000000001e-05 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811576+0000", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0870000000000001e-06 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.811583+0000", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.35293079100000002 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164513+0000", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010971699999999999 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164623+0000", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015673400000000001 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164780+0000", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020604970000000001 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166840+0000", 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003159944 2024-02-22T19:38:15.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170000+0000", 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5516999999999997e-05 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170056+0000", 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0157000000000002e-05 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170096+0000", 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5141000000000004e-05 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.170171+0000", 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.35866692999999999 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:155158 8.0 8:06cc271a:::benchmark_data_smithi053_130774_object43424:head [delete] snapc 0=[] ondisk+write+known_if_redirected e206)", 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:46.336591+0000", 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.540699211, 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.102372581, 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:38:15.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 155158 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.336591+0000", 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.336591+0000", 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.336589+0000", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6500000000000002e-06 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.336598+0000", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1527000000000006e-05 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.336690+0000", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3350000000000001e-06 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.336697+0000", 2024-02-22T19:38:15.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029480130000000002 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.339645+0000", 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8753999999999997e-05 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.339744+0000", 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0733e-05 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,4", 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.339834+0000", 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004634483 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.344469+0000", 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0299999999999999e-06 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.344475+0000", 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.094400131999999998 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.438875+0000", 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8902e-05 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.438924+0000", 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0018999999999997e-05 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.438964+0000", 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.102372581 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:155168 8.f 8:f61fafe5:::benchmark_data_smithi053_130774_object43434:head [delete] snapc 0=[] ondisk+write+known_if_redirected e206)", 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:46.345341+0000", 2024-02-22T19:38:15.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.531949496999999, 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096548893999999996, 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 155168 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345341+0000", 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345341+0000", 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345341+0000", 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.552e-06 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345344+0000", 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5769999999999999e-06 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345351+0000", 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6700000000000002e-06 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345355+0000", 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9660999999999996e-05 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345425+0000", 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126585 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345552+0000", 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124533 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.345676+0000", 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096643049999999998 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.355340+0000", 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017342799999999999 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.355514+0000", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086286330999999994 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.441800+0000", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0311e-05 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.441850+0000", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9526000000000001e-05 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:15.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:46.441890+0000", 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096548893999999996 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:176257 8.c 8:3ba3fd9d:::benchmark_data_smithi053_130774_object64523:head [delete] snapc 0=[] ondisk+write+known_if_redirected e234)", 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:14.231109+0000", 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.646181973, 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.15741065300000001, 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 176257 2024-02-22T19:38:15.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231109+0000", 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231109+0000", 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231106+0000", 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231114+0000", 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2177999999999999e-05 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231126+0000", 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7640000000000002e-06 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231132+0000", 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3068999999999997e-05 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231175+0000", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9789999999999993e-06 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.231183+0000", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.10921831799999999 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.340401+0000", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011356799999999999 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:38:15.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.340515+0000", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3309999999999999e-05 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,5,7", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.340608+0000", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036748029999999999 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.344283+0000", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00289223 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.347175+0000", 2024-02-22T19:38:15.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2092000000000001e-05 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.347237+0000", 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041027872 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.388265+0000", 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9972e-05 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.388335+0000", 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000184241 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:38:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:14.388519+0000", 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.15741065300000001 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:38:16.000 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:38:16.019 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:16.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:16.055 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:16.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:16.235 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:16.341 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:16.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:16.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:16.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:16.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:15.486598+0000 mgr.y (mgr.14535) 357 : cluster [DBG] pgmap v439: 148 pgs: 5 active+remapped+backfilling, 1 activating+degraded+remapped, 7 active+remapped+backfill_toofull, 135 active+clean; 2.9 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 570 op/s; 6/142056 objects degraded (0.004%); 34976/142056 objects misplaced (24.621%); 1.4 MiB/s, 33 objects/s recovering 2024-02-22T19:38:16.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:16.200451+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e236: 8 total, 8 up, 7 in 2024-02-22T19:38:16.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:15.486598+0000 mgr.y (mgr.14535) 357 : cluster [DBG] pgmap v439: 148 pgs: 5 active+remapped+backfilling, 1 activating+degraded+remapped, 7 active+remapped+backfill_toofull, 135 active+clean; 2.9 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 570 op/s; 6/142056 objects degraded (0.004%); 34976/142056 objects misplaced (24.621%); 1.4 MiB/s, 33 objects/s recovering 2024-02-22T19:38:16.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:16.200451+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e236: 8 total, 8 up, 7 in 2024-02-22T19:38:16.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:38:16.821 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:16.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:16.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:15.486598+0000 mgr.y (mgr.14535) 357 : cluster [DBG] pgmap v439: 148 pgs: 5 active+remapped+backfilling, 1 activating+degraded+remapped, 7 active+remapped+backfill_toofull, 135 active+clean; 2.9 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 570 op/s; 6/142056 objects degraded (0.004%); 34976/142056 objects misplaced (24.621%); 1.4 MiB/s, 33 objects/s recovering 2024-02-22T19:38:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:16.200451+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e236: 8 total, 8 up, 7 in 2024-02-22T19:38:16.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:17.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:17.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:17.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:17.187 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:17.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:38:17.230 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:17.280 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:17.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:17.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:17.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:17.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:16.445214+0000 mon.a (mon.0) 1138 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:17.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:16.445265+0000 mon.a (mon.0) 1139 : cluster [WRN] Health check failed: Degraded data redundancy: 6/142056 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:38:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:17.503 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:17.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:17.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:17.677 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:17.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:16.445214+0000 mon.a (mon.0) 1138 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:17.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:16.445265+0000 mon.a (mon.0) 1139 : cluster [WRN] Health check failed: Degraded data redundancy: 6/142056 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:38:17.785 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:16.445214+0000 mon.a (mon.0) 1138 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:17.786 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:16.445265+0000 mon.a (mon.0) 1139 : cluster [WRN] Health check failed: Degraded data redundancy: 6/142056 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:38:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:17.487094+0000 mgr.y (mgr.14535) 358 : cluster [DBG] pgmap v441: 148 pgs: 5 active+remapped+backfilling, 1 activating+degraded+remapped, 7 active+remapped+backfill_toofull, 135 active+clean; 2.9 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 529 op/s; 6/142056 objects degraded (0.004%); 34976/142056 objects misplaced (24.621%); 1.3 MiB/s, 30 objects/s recovering 2024-02-22T19:38:18.673 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 6, 7, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:38:18.673 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:38:18.673 INFO:tasks.thrashosds.thrasher:fixing pg num pool iscsi 2024-02-22T19:38:18.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:38:19.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:17.487094+0000 mgr.y (mgr.14535) 358 : cluster [DBG] pgmap v441: 148 pgs: 5 active+remapped+backfilling, 1 activating+degraded+remapped, 7 active+remapped+backfill_toofull, 135 active+clean; 2.9 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 529 op/s; 6/142056 objects degraded (0.004%); 34976/142056 objects misplaced (24.621%); 1.3 MiB/s, 30 objects/s recovering 2024-02-22T19:38:19.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:17.487094+0000 mgr.y (mgr.14535) 358 : cluster [DBG] pgmap v441: 148 pgs: 5 active+remapped+backfilling, 1 activating+degraded+remapped, 7 active+remapped+backfill_toofull, 135 active+clean; 2.9 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 529 op/s; 6/142056 objects degraded (0.004%); 34976/142056 objects misplaced (24.621%); 1.3 MiB/s, 30 objects/s recovering 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15054.0:179699 8.e e236/233 8:70f635fa:::benchmark_data_smithi053_130774_object67965:head v 236'11260, mlcod=236'11258)", 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:19.264429+0000", 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.002521653, 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025494010000000002, 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:19.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264429+0000", 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264429+0000", 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264428+0000", 2024-02-22T19:38:19.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5420000000000005e-06 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264437+0000", 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3711e-05 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264450+0000", 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3910000000000003e-06 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264457+0000", 2024-02-22T19:38:19.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1080999999999993e-05 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264518+0000", 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0623000000000001e-05 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.264548+0000", 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000119373 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:179705 8.2 8:4d0e41f7:::benchmark_data_smithi053_130774_object67971:head [delete] snapc 0=[] ondisk+write+known_if_redirected e236)", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:19.265898+0000", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0010525829999999999, 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001162901, 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 179705 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.265898+0000", 2024-02-22T19:38:19.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.265898+0000", 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.265897+0000", 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6699999999999997e-06 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:19.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.265904+0000", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4953999999999999e-05 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.265919+0000", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7800000000000003e-06 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.265925+0000", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3973000000000004e-05 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266009+0000", 2024-02-22T19:38:19.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000139889 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266149+0000", 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0001719 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266321+0000", 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00042304700000000002 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.274 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1428 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 9] snapc 0=[] ondisk+write+known_if_redirected e236)", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:19.266204+0000", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00074671399999999999, 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00094774799999999999, 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1428 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266204+0000", 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266204+0000", 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266203+0000", 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2979999999999999e-06 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266208+0000", 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0662000000000001e-05 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:19.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266219+0000", 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4019999999999999e-06 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266225+0000", 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7570000000000001e-05 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266263+0000", 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2104e-05 2024-02-22T19:38:19.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:19.266325+0000", 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120964 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 3 2024-02-22T19:38:19.280 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:38:19.575 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:38:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:19] "GET /metrics HTTP/1.1" 200 175945 "" "Prometheus/2.33.4" 2024-02-22T19:38:19.577 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:19.507346+0000 mon.a (mon.0) 1140 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/142056 objects degraded (0.004%), 1 pg degraded) 2024-02-22T19:38:19.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:19.507346+0000 mon.a (mon.0) 1140 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/142056 objects degraded (0.004%), 1 pg degraded) 2024-02-22T19:38:19.578 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:38:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a[93504]: debug 2024-02-22T19:38:19.265+0000 7fb44dbeb700 -1 RGWWatcher::handle_error cookie 94129597099392 err (110) Connection timed out 2024-02-22T19:38:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:19.507346+0000 mon.a (mon.0) 1140 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/142056 objects degraded (0.004%), 1 pg degraded) 2024-02-22T19:38:20.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:38:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:20.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:19.487851+0000 mgr.y (mgr.14535) 359 : cluster [DBG] pgmap v442: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 818 op/s; 31664/134190 objects misplaced (23.596%); 13 MiB/s, 216 objects/s recovering 2024-02-22T19:38:21.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:19.487851+0000 mgr.y (mgr.14535) 359 : cluster [DBG] pgmap v442: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 818 op/s; 31664/134190 objects misplaced (23.596%); 13 MiB/s, 216 objects/s recovering 2024-02-22T19:38:21.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:19.487851+0000 mgr.y (mgr.14535) 359 : cluster [DBG] pgmap v442: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 818 op/s; 31664/134190 objects misplaced (23.596%); 13 MiB/s, 216 objects/s recovering 2024-02-22T19:38:21.803 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:20.926760+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:21.804 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:20.926817+0000 osd.1 (osd.1) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:20.926760+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:20.926817+0000 osd.1 (osd.1) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:22.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:20.926760+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:22.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:20.926817+0000 osd.1 (osd.1) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:22.427 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:38:22.427 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:38:22.427 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:38:22.427 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:38:22.427 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:38:22.667 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:22.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:22.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:22.770 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:22.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:22.820 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:21.488354+0000 mgr.y (mgr.14535) 360 : cluster [DBG] pgmap v443: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.0 GiB used, 617 GiB / 626 GiB avail; 699 B/s rd, 0 B/s wr, 789 op/s; 31182/131595 objects misplaced (23.695%); 11 MiB/s, 185 objects/s recovering 2024-02-22T19:38:22.820 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:21.604352+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e237: 8 total, 8 up, 7 in 2024-02-22T19:38:22.820 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:22.188280+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e238: 8 total, 8 up, 7 in 2024-02-22T19:38:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:22.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:22.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:22.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:23.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:21.488354+0000 mgr.y (mgr.14535) 360 : cluster [DBG] pgmap v443: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.0 GiB used, 617 GiB / 626 GiB avail; 699 B/s rd, 0 B/s wr, 789 op/s; 31182/131595 objects misplaced (23.695%); 11 MiB/s, 185 objects/s recovering 2024-02-22T19:38:23.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:21.604352+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e237: 8 total, 8 up, 7 in 2024-02-22T19:38:23.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:22.188280+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e238: 8 total, 8 up, 7 in 2024-02-22T19:38:23.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:21.488354+0000 mgr.y (mgr.14535) 360 : cluster [DBG] pgmap v443: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.0 GiB used, 617 GiB / 626 GiB avail; 699 B/s rd, 0 B/s wr, 789 op/s; 31182/131595 objects misplaced (23.695%); 11 MiB/s, 185 objects/s recovering 2024-02-22T19:38:23.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:21.604352+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e237: 8 total, 8 up, 7 in 2024-02-22T19:38:23.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:22.188280+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e238: 8 total, 8 up, 7 in 2024-02-22T19:38:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:23.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:23.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:23.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:38:23.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:23.336 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:23.372 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:23.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:23.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:23.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:23.548 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:23.920 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:23.440653+0000 mon.a (mon.0) 1143 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:38:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:23.440653+0000 mon.a (mon.0) 1143 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:38:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:23.440653+0000 mon.a (mon.0) 1143 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:38:25.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:23.495758+0000 mgr.y (mgr.14535) 361 : cluster [DBG] pgmap v446: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.0 GiB used, 617 GiB / 626 GiB avail; 702 B/s rd, 0 B/s wr, 479 op/s; 31182/131595 objects misplaced (23.695%); 10 MiB/s, 167 objects/s recovering 2024-02-22T19:38:25.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:23.729997+0000 mon.a (mon.0) 1144 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:23.495758+0000 mgr.y (mgr.14535) 361 : cluster [DBG] pgmap v446: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.0 GiB used, 617 GiB / 626 GiB avail; 702 B/s rd, 0 B/s wr, 479 op/s; 31182/131595 objects misplaced (23.695%); 10 MiB/s, 167 objects/s recovering 2024-02-22T19:38:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:23.729997+0000 mon.a (mon.0) 1144 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:25.037 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:38:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:38:25.038 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:38:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:38:25.038 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:23.495758+0000 mgr.y (mgr.14535) 361 : cluster [DBG] pgmap v446: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 9.0 GiB used, 617 GiB / 626 GiB avail; 702 B/s rd, 0 B/s wr, 479 op/s; 31182/131595 objects misplaced (23.695%); 10 MiB/s, 167 objects/s recovering 2024-02-22T19:38:25.038 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:23.729997+0000 mon.a (mon.0) 1144 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:25.308 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:25.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:25.312 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:25.315 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:25.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:25.459 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:25.498 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:25.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:25.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:25.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:694 5.e 5:7759931f:::notify.3:head [watch ping cookie 94129597099392] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:38:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:49.154376+0000", 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 396.50040168999999, 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85628850199999995, 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 694 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154376+0000", 2024-02-22T19:38:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154376+0000", 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154374+0000", 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0577000000000001e-05 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154385+0000", 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9412999999999999e-05 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154404+0000", 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6293000000000001e-05 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154421+0000", 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000222499 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154643+0000", 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4482e-05 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154678+0000", 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85175950099999997 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.006437+0000", 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00037694099999999999 2024-02-22T19:38:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for degraded object", 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.006814+0000", 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036508669999999999 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010465+0000", 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2737000000000003e-05 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010558+0000", 2024-02-22T19:38:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000106976 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010665+0000", 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85628850199999995 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:700 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:49.154699+0000", 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 396.50007896199998, 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86230325500000005, 2024-02-22T19:38:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 700 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154699+0000", 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154699+0000", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154698+0000", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2769999999999996e-06 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154703+0000", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8890000000000008e-06 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154712+0000", 2024-02-22T19:38:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1720000000000004e-06 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154718+0000", 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8034e-05 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154776+0000", 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2228999999999999e-05 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154789+0000", 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85235517100000002 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.007144+0000", 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1841000000000006e-05 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for degraded object", 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.007206+0000", 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035078069999999999 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010714+0000", 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5166000000000005e-05 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010789+0000", 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062134900000000003 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.017002+0000", 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86230325500000005 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:66838 8.2 8:4b0bb9c8:::benchmark_data_smithi053_130774_object66837:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e169)", 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:16.792557+0000", 2024-02-22T19:38:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 128.86222091400001, 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.060944979000000003, 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 66838 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792557+0000", 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792557+0000", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792555+0000", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1019000000000001e-05 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792606+0000", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2921e-05 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792629+0000", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3340000000000008e-06 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792638+0000", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016233799999999999 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792801+0000", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000136458 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.792937+0000", 2024-02-22T19:38:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018457799999999999 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.793122+0000", 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024924629999999999 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.795614+0000", 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000211337 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.795826+0000", 2024-02-22T19:38:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057603644000000002 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.853429+0000", 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4462000000000003e-05 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.853464+0000", 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8253000000000001e-05 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:16.853502+0000", 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.060944979000000003 2024-02-22T19:38:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71273 8.2 8:4d5bc216:::benchmark_data_smithi053_130774_object71272:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.014979+0000", 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.63979922599999, 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078888333000000005, 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71273 2024-02-22T19:38:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.014979+0000", 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.014979+0000", 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.014977+0000", 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4988000000000003e-05 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015032+0000", 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2476e-05 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015045+0000", 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.516e-06 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015051+0000", 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2370000000000003e-05 2024-02-22T19:38:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015094+0000", 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014868999999999999 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015242+0000", 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000250423 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015493+0000", 2024-02-22T19:38:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084955440000000007 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.023988+0000", 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00087941699999999996 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.024868+0000", 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068938761000000001 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093806+0000", 2024-02-22T19:38:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3099000000000003e-05 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093839+0000", 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7535999999999999e-05 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093867+0000", 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078888333000000005 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71277 8.2 8:41883358:::benchmark_data_smithi053_130774_object71276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.015237+0000", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.639541151, 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078755951000000005, 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71277 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015237+0000", 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015237+0000", 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015236+0000", 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2608999999999999e-05 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015269+0000", 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1059999999999996e-06 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015275+0000", 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9690000000000002e-06 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015280+0000", 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001987417 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.017267+0000", 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012988 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.017397+0000", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102525 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.017500+0000", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064942439999999997 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.023994+0000", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00095847700000000001 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.024952+0000", 2024-02-22T19:38:25.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068964619000000005 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093917+0000", 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7161999999999998e-05 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093944+0000", 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8488000000000001e-05 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093993+0000", 2024-02-22T19:38:25.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078755951000000005 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71278 8.2 8:451cdea5:::benchmark_data_smithi053_130774_object71277:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.015291+0000", 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.63948702099999, 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078881751, 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71278 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015291+0000", 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015291+0000", 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015290+0000", 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8708999999999995e-05 2024-02-22T19:38:25.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015369+0000", 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5310000000000004e-06 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015376+0000", 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6179999999999999e-06 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015380+0000", 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038524800000000001 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.019232+0000", 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116681 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.019349+0000", 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146007 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.019495+0000", 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045034209999999996 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.023999+0000", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032699700000000001 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.027269+0000", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.066793575999999993 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094062+0000", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3528000000000006e-05 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094126+0000", 2024-02-22T19:38:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6968999999999997e-05 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094173+0000", 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078881751 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71284 8.2 8:482c922c:::benchmark_data_smithi053_130774_object71283:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.018930+0000", 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.635848252, 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075350869000000001, 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71284 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018930+0000", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018930+0000", 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:25.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018928+0000", 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5718999999999999e-05 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018954+0000", 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5325999999999999e-05 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018969+0000", 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.182e-06 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018976+0000", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026448090000000001 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.021621+0000", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014431000000000001 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.021765+0000", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000181514 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.021947+0000", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002211073 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.024158+0000", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057578457999999999 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.081736+0000", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012481897 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094218+0000", 2024-02-22T19:38:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1065000000000002e-05 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094249+0000", 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1066999999999999e-05 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094281+0000", 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075350869000000001 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.699 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73406 8.2 8:445e69a0:::benchmark_data_smithi053_130774_object73405:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.189313+0000", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 122.465464849, 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.076342253999999998, 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73406 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189313+0000", 2024-02-22T19:38:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189313+0000", 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189312+0000", 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0809000000000002e-05 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189353+0000", 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8460000000000002e-06 2024-02-22T19:38:25.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189360+0000", 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.941e-06 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189364+0000", 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5580000000000006e-05 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189429+0000", 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1738999999999997e-05 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189521+0000", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107824 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189629+0000", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003956846 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.193586+0000", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071767559999999994 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265353+0000", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019441099999999999 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265548+0000", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1397999999999997e-05 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265609+0000", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6378000000000002e-05 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265655+0000", 2024-02-22T19:38:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.076342253999999998 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73408 8.2 8:4a6b68d5:::benchmark_data_smithi053_130774_object73407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 122.46527855399999, 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.079867652999999997, 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.705 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73408 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100202 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189599+0000", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1030000000000008e-06 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189608+0000", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6629999999999998e-06 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189614+0000", 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018859739999999999 2024-02-22T19:38:25.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.191500+0000", 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000125933 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.191626+0000", 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017495499999999999 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.191800+0000", 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019171380000000001 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.193718+0000", 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071980290000000002 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265698+0000", 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035836459999999998 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.269282+0000", 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7651999999999997e-05 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.269329+0000", 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7818999999999998e-05 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.269367+0000", 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.079867652999999997 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:156980 8.2 8:446b8e74:::benchmark_data_smithi053_130774_object45246:head [delete] snapc 0=[] ondisk+write+known_if_redirected e208)", 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.437573297, 2024-02-22T19:38:25.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064537973999999998, 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 156980 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217204+0000", 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6099999999999999e-06 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217208+0000", 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4339999999999997e-06 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217215+0000", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1739999999999999e-06 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217220+0000", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002322302 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219542+0000", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9874e-05 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219632+0000", 2024-02-22T19:38:25.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9351000000000006e-05 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219711+0000", 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012168019999999999 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.220928+0000", 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004561847 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.225490+0000", 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.056184824000000001 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281675+0000", 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2453999999999998e-05 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281707+0000", 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5172000000000001e-05 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281743+0000", 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064537973999999998 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:157456 8.2 8:47cb7be2:::benchmark_data_smithi053_130774_object45722:head [delete] snapc 0=[] ondisk+write+known_if_redirected e209)", 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 36.915341570999999, 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065090921999999996, 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 157456 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739435+0000", 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4889999999999998e-06 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739443+0000", 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6900000000000004e-06 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739453+0000", 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5810000000000002e-06 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739459+0000", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5296000000000001e-05 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739505+0000", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7620999999999994e-05 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739582+0000", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6128e-05 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739658+0000", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022481889999999998 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.741906+0000", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00070410199999999996 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.742611+0000", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061844286999999998 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804455+0000", 2024-02-22T19:38:25.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6128999999999998e-05 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804501+0000", 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6259e-05 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804527+0000", 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065090921999999996 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.721 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:158760 8.2 8:4b5b1757:::benchmark_data_smithi053_130774_object47026:head [delete] snapc 0=[] ondisk+write+known_if_redirected e210)", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 35.586177046000003, 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.070793354000000003, 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 158760 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068599+0000", 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3269999999999997e-06 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068605+0000", 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0729e-05 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068616+0000", 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9429999999999999e-06 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068621+0000", 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4260000000000001e-05 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068655+0000", 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110695 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068766+0000", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104616 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068871+0000", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019013000000000001 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.070772+0000", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065132173000000002 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.135904+0000", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033978099999999998 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139302+0000", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1437000000000002e-05 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139353+0000", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0763999999999998e-05 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139394+0000", 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.070793354000000003 2024-02-22T19:38:25.726 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165007 8.2 8:4b3d7df9:::benchmark_data_smithi053_130774_object53273:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 29.016176713, 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.689438121, 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165007 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638600+0000", 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8800000000000008e-06 2024-02-22T19:38:25.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638608+0000", 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0635999999999999e-05 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638618+0000", 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9420000000000003e-06 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638623+0000", 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5618999999999998e-05 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638669+0000", 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012975000000000001 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638799+0000", 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015391499999999999 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638953+0000", 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040805320000000004 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.643033+0000", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012758660000000001 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.644309+0000", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015138571999999999 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659448+0000", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0815999999999999e-05 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659458+0000", 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50944174600000003 2024-02-22T19:38:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168900+0000", 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.302e-06 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168909+0000", 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1590766800000001 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327986+0000", 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3182e-05 2024-02-22T19:38:25.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328039+0000", 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.689438121 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165013 8.2 8:42c6cb88:::benchmark_data_smithi053_130774_object53279:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 29.016013020999999, 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6893490839999998, 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:38:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165013 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638764+0000", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9380000000000003e-06 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638771+0000", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4669999999999993e-06 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638780+0000", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9320000000000001e-06 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638786+0000", 2024-02-22T19:38:25.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024245949999999999 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641211+0000", 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011383399999999999 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641324+0000", 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000133826 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641458+0000", 2024-02-22T19:38:25.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017005779999999999 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.643159+0000", 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005364382 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.648523+0000", 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010966531999999999 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659490+0000", 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3050000000000004e-06 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659497+0000", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50944507900000002 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168942+0000", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4430000000000003e-06 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168951+0000", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1591219230000001 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328073+0000", 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1409999999999998e-05 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328114+0000", 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6893490839999998 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1372 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 4] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.982491372999998, 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.542875494, 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1372 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672285+0000", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6209999999999998e-06 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672291+0000", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0467e-05 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672302+0000", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0189999999999995e-06 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672311+0000", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3849999999999993e-05 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672375+0000", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3922e-05 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672389+0000", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.496154551 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168543+0000", 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3271999999999999e-05 2024-02-22T19:38:25.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168556+0000", 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0231974429999999 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.191754+0000", 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000199951 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.191954+0000", 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023208168000000001 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.215162+0000", 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.542875494 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1374 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 5] snapc 0=[] ondisk+write+known_if_redirected e218)", 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.487042128999999, 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0476472020000001, 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1374 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167734+0000", 2024-02-22T19:38:25.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7470000000000005e-06 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167742+0000", 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5179999999999993e-06 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167751+0000", 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1430000000000001e-06 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167758+0000", 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00082432800000000004 2024-02-22T19:38:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168583+0000", 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3889999999999994e-06 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168592+0000", 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.023928116 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.192520+0000", 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8975999999999997e-05 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.192599+0000", 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022783739000000001 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.215383+0000", 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0476472020000001 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1382 5.e 5:7759931f:::notify.3:head [watch ping cookie 94129597099392 gen 5] snapc 0=[] ondisk+write+known_if_redirected e219)", 2024-02-22T19:38:25.748 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 26.390952631000001, 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95001531500000003, 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1382 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0230000000000001e-06 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263828+0000", 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3229999999999999e-06 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263832+0000", 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2860000000000001e-06 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263836+0000", 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4635000000000005e-05 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263921+0000", 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.167e-05 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263933+0000", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94966063199999995 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213593+0000", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000170743 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213764+0000", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6550000000000004e-05 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213841+0000", 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95001531500000003 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1387 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 6] snapc 0=[] ondisk+write+known_if_redirected e220)", 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 26.390934453, 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96457482999999999, 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.753 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1387 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:38:25.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5679999999999998e-06 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263846+0000", 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1229999999999997e-06 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263852+0000", 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1789999999999999e-06 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263857+0000", 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104499 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263961+0000", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6989999999999997e-06 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263968+0000", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95095029499999995 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.214918+0000", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7822999999999998e-05 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.214976+0000", 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01344215 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.228418+0000", 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96457482999999999 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1416 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 7] snapc 0=[] ondisk+write+known_if_redirected e234)", 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:14.264947+0000", 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 11.389831165, 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.062213602, 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1416 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264947+0000", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264947+0000", 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264945+0000", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0709999999999997e-06 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264953+0000", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5057e-05 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264968+0000", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2929999999999997e-06 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264974+0000", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5454000000000001e-05 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.265029+0000", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5899000000000001e-05 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for degraded object", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.265085+0000", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.048580248999999999 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.313665+0000", 2024-02-22T19:38:25.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7860000000000002e-05 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.313753+0000", 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013406998999999999 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.327160+0000", 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.062213602 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1418 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 8] snapc 0=[] ondisk+write+known_if_redirected e234)", 2024-02-22T19:38:25.761 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:14.264985+0000", 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 11.389792457, 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.062356013000000002, 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1418 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264985+0000", 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264985+0000", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264985+0000", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0730000000000001e-06 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264988+0000", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0980000000000001e-06 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264993+0000", 2024-02-22T19:38:25.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6440000000000002e-06 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.264997+0000", 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012880500000000001 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.265126+0000", 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5787999999999997e-05 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for degraded object", 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.265161+0000", 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.048885125000000001 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.314046+0000", 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6002999999999996e-05 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.314132+0000", 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013209039000000001 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:14.327341+0000", 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.062356013000000002 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.765 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.766 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.766 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:25.766 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:25.766 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:38:25.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:25.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:25.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:25.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:24.769664+0000 mgr.y (mgr.14535) 362 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:25.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:25.399618+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:38:25.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:25.400766+0000 mon.a (mon.0) 1146 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:38:25.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:25.415204+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:25.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:24.769664+0000 mgr.y (mgr.14535) 362 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:25.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:25.399618+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:38:25.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:25.400766+0000 mon.a (mon.0) 1146 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:38:25.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:25.415204+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:25.956 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:25.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:25.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:25.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:24.769664+0000 mgr.y (mgr.14535) 362 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:25.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:25.399618+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:38:25.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:25.400766+0000 mon.a (mon.0) 1146 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:38:25.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:25.415204+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:26.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:26.126 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:26.298 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:26.305 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:26.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:26.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:26.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:38:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:26.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:26.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:26.782 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:25.497990+0000 mgr.y (mgr.14535) 363 : cluster [DBG] pgmap v447: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.4 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 883 op/s; 27457/120861 objects misplaced (22.718%); 19 MiB/s, 302 objects/s recovering 2024-02-22T19:38:26.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:26.852 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:26.942 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:27.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:25.497990+0000 mgr.y (mgr.14535) 363 : cluster [DBG] pgmap v447: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.4 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 883 op/s; 27457/120861 objects misplaced (22.718%); 19 MiB/s, 302 objects/s recovering 2024-02-22T19:38:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:25.497990+0000 mgr.y (mgr.14535) 363 : cluster [DBG] pgmap v447: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.4 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 883 op/s; 27457/120861 objects misplaced (22.718%); 19 MiB/s, 302 objects/s recovering 2024-02-22T19:38:27.111 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:27.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:27.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:27.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:27.279 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:27.304 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:27.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:27.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:27.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:27.489 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:27.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:27.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:27.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:27.639 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:27.814 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:26.968831+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.53:0/2089502026' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:27.814 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:26.970970+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:27.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:26.968831+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.53:0/2089502026' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:27.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:26.970970+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:27.843 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:38:27.844 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:27.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:27.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:27.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:28.030 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:28.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:28.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:28.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:26.968831+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.53:0/2089502026' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:28.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:26.970970+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:38:28.431 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:28.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:28.433 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:38:28.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:28.594 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:29.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:29.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:29.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:27.498522+0000 mgr.y (mgr.14535) 364 : cluster [DBG] pgmap v448: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.4 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 555 op/s; 27457/120861 objects misplaced (22.718%); 9.8 MiB/s, 157 objects/s recovering 2024-02-22T19:38:29.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:27.779139+0000 mon.a (mon.0) 1149 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:29.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:27.800930+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:29.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:27.801002+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e239: 8 total, 8 up, 7 in 2024-02-22T19:38:29.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:27.498522+0000 mgr.y (mgr.14535) 364 : cluster [DBG] pgmap v448: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.4 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 555 op/s; 27457/120861 objects misplaced (22.718%); 9.8 MiB/s, 157 objects/s recovering 2024-02-22T19:38:29.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:27.779139+0000 mon.a (mon.0) 1149 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:29.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:27.800930+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:29.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:27.801002+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e239: 8 total, 8 up, 7 in 2024-02-22T19:38:29.098 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15054.0:186613 8.8 e239/218 8:1f24e3ec:::benchmark_data_smithi053_130774_object74879:head v 239'11957, mlcod=239'11955)", 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:29.094709+0000", 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0028230030000000001, 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028486829999999999, 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094709+0000", 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094709+0000", 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094708+0000", 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8599999999999993e-06 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094716+0000", 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7520000000000008e-06 2024-02-22T19:38:29.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094726+0000", 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7219999999999996e-06 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094731+0000", 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3297999999999998e-05 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094775+0000", 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2926999999999999e-05 2024-02-22T19:38:29.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.094798+0000", 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8725000000000002e-05 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15054.0:186605 8.a e239/218 8:5a207d90:::benchmark_data_smithi053_130774_object74871:head v 239'11595, mlcod=239'11592)", 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:29.095728+0000", 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0018042379999999999, 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00188905, 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:29.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095728+0000", 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095728+0000", 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095727+0000", 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095734+0000", 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3569e-05 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095748+0000", 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5150000000000003e-06 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095755+0000", 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3291e-05 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095828+0000", 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0239000000000001e-05 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095848+0000", 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120364 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15054.0:186609 8.a e239/218 8:5f0093a3:::benchmark_data_smithi053_130774_object74875:head v 239'11596, mlcod=239'11592)", 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:29.095768+0000", 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001763466, 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019117590000000001, 2024-02-22T19:38:29.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095768+0000", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095768+0000", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095768+0000", 2024-02-22T19:38:29.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8349999999999997e-06 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095772+0000", 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1569999999999999e-06 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095777+0000", 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9129999999999996e-06 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:29.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.095781+0000", 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001779077 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.097560+0000", 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9284e-05 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:29.097579+0000", 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001810975 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:29.108 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:38:29.109 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 3 2024-02-22T19:38:29.109 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:38:29.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:27.498522+0000 mgr.y (mgr.14535) 364 : cluster [DBG] pgmap v448: 148 pgs: 5 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 2.4 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 555 op/s; 27457/120861 objects misplaced (22.718%); 9.8 MiB/s, 157 objects/s recovering 2024-02-22T19:38:29.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:27.779139+0000 mon.a (mon.0) 1149 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:29.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:27.800930+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:29.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:27.801002+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e239: 8 total, 8 up, 7 in 2024-02-22T19:38:29.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:29.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:29.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:29.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:29.505 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:38:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:29] "GET /metrics HTTP/1.1" 200 175971 "" "Prometheus/2.33.4" 2024-02-22T19:38:29.508 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:29.571 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:29.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:29.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:29.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:29.747 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:29.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:38:30.026 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:30.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:30.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:30.039 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-02-22T19:38:27.797951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2627606204,"num_objects":40287,"num_object_clones":0,"num_object_copies":120861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27457,"num_objects_unfound":0,"num_objects_dirty":40287,"num_whiteouts":0,"num_read":1266,"num_read_kb":1018,"num_write":183797,"num_write_kb":7150826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2924,"num_bytes_recovered":181278326,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76251,"ondisk_log_size":76251,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":449,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":8938924,"kb_used_data":6792180,"kb_used_omap":0,"kb_used_meta":2146048,"kb_avail":647277140,"statfs":{"total":671965249536,"available":662811791360,"internally_reserved":0,"allocated":6955192320,"data_stored":6951583857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2197553152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,3,2,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-234487808,"num_objects":-3578,"num_object_clones":0,"num_object_copies":-10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3725,"num_objects_unfound":0,"num_objects_dirty":-3578,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3578,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":78708736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.309952"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":217,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606430+0000","last_change":"2024-02-22T19:37:57.180408+0000","last_active":"2024-02-22T19:38:21.606430+0000","last_peered":"2024-02-22T19:38:21.606430+0000","last_clean":"2024-02-22T19:38:21.606430+0000","last_became_active":"2024-02-22T19:37:57.180160+0000","last_became_peered":"2024-02-22T19:37:57.180160+0000","last_unstale":"2024-02-22T19:38:21.606430+0000","last_undegraded":"2024-02-22T19:38:21.606430+0000","last_fullsized":"2024-02-22T19:38:21.606430+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.806069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.806069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":214,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609984+0000","last_change":"2024-02-22T19:38:13.197918+0000","last_active":"2024-02-22T19:38:21.609984+0000","last_peered":"2024-02-22T19:38:21.609984+0000","last_clean":"2024-02-22T19:38:21.609984+0000","last_became_active":"2024-02-22T19:38:13.197285+0000","last_became_peered":"2024-02-22T19:38:13.197285+0000","last_unstale":"2024-02-22T19:38:21.609984+0000","last_undegraded":"2024-02-22T19:38:21.609984+0000","last_fullsized":"2024-02-22T19:38:21.609984+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":219,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605613+0000","last_change":"2024-02-22T19:36:10.678582+0000","last_active":"2024-02-22T19:38:21.605613+0000","last_peered":"2024-02-22T19:38:21.605613+0000","last_clean":"2024-02-22T19:38:21.605613+0000","last_became_active":"2024-02-22T19:29:03.082280+0000","last_became_peered":"2024-02-22T19:29:03.082280+0000","last_unstale":"2024-02-22T19:38:21.605613+0000","last_undegraded":"2024-02-22T19:38:21.605613+0000","last_fullsized":"2024-02-22T19:38:21.605613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:10.678491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:10.678491+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":196,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615077+0000","last_change":"2024-02-22T19:36:59.298132+0000","last_active":"2024-02-22T19:38:21.615077+0000","last_peered":"2024-02-22T19:38:21.615077+0000","last_clean":"2024-02-22T19:38:21.615077+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:38:21.615077+0000","last_undegraded":"2024-02-22T19:38:21.615077+0000","last_fullsized":"2024-02-22T19:38:21.615077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:59.298021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:36:59.298021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":197,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.620564+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:38:21.620564+0000","last_peered":"2024-02-22T19:38:21.620564+0000","last_clean":"2024-02-22T19:38:21.620564+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:38:21.620564+0000","last_undegraded":"2024-02-22T19:38:21.620564+0000","last_fullsized":"2024-02-22T19:38:21.620564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'15","reported_seq":245,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619158+0000","last_change":"2024-02-22T19:38:14.149263+0000","last_active":"2024-02-22T19:38:21.619158+0000","last_peered":"2024-02-22T19:38:21.619158+0000","last_clean":"2024-02-22T19:38:21.619158+0000","last_became_active":"2024-02-22T19:38:13.201143+0000","last_became_peered":"2024-02-22T19:38:13.201143+0000","last_unstale":"2024-02-22T19:38:21.619158+0000","last_undegraded":"2024-02-22T19:38:21.619158+0000","last_fullsized":"2024-02-22T19:38:21.619158+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:34:45.948358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:45.948358+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606159+0000","last_change":"2024-02-22T19:38:13.192534+0000","last_active":"2024-02-22T19:38:21.606159+0000","last_peered":"2024-02-22T19:38:21.606159+0000","last_clean":"2024-02-22T19:38:21.606159+0000","last_became_active":"2024-02-22T19:38:13.192041+0000","last_became_peered":"2024-02-22T19:38:13.192041+0000","last_unstale":"2024-02-22T19:38:21.606159+0000","last_undegraded":"2024-02-22T19:38:21.606159+0000","last_fullsized":"2024-02-22T19:38:21.606159+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:23.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:23.243945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610365+0000","last_change":"2024-02-22T19:37:57.190538+0000","last_active":"2024-02-22T19:38:21.610365+0000","last_peered":"2024-02-22T19:38:21.610365+0000","last_clean":"2024-02-22T19:38:21.610365+0000","last_became_active":"2024-02-22T19:37:57.189778+0000","last_became_peered":"2024-02-22T19:37:57.189778+0000","last_unstale":"2024-02-22T19:38:21.610365+0000","last_undegraded":"2024-02-22T19:38:21.610365+0000","last_fullsized":"2024-02-22T19:38:21.610365+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":238,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606547+0000","last_change":"2024-02-22T19:38:13.552285+0000","last_active":"2024-02-22T19:38:21.606547+0000","last_peered":"2024-02-22T19:38:21.606547+0000","last_clean":"2024-02-22T19:38:21.606547+0000","last_became_active":"2024-02-22T19:38:13.195310+0000","last_became_peered":"2024-02-22T19:38:13.195310+0000","last_unstale":"2024-02-22T19:38:21.606547+0000","last_undegraded":"2024-02-22T19:38:21.606547+0000","last_fullsized":"2024-02-22T19:38:21.606547+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:21.735513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:21.735513+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609936+0000","last_change":"2024-02-22T19:37:57.186178+0000","last_active":"2024-02-22T19:38:21.609936+0000","last_peered":"2024-02-22T19:38:21.609936+0000","last_clean":"2024-02-22T19:38:21.609936+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:38:21.609936+0000","last_undegraded":"2024-02-22T19:38:21.609936+0000","last_fullsized":"2024-02-22T19:38:21.609936+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.908571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.908571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":34,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.612696+0000","last_change":"2024-02-22T19:37:57.195074+0000","last_active":"2024-02-22T19:38:21.612696+0000","last_peered":"2024-02-22T19:38:21.612696+0000","last_clean":"2024-02-22T19:38:21.612696+0000","last_became_active":"2024-02-22T19:37:57.182010+0000","last_became_peered":"2024-02-22T19:37:57.182010+0000","last_unstale":"2024-02-22T19:38:21.612696+0000","last_undegraded":"2024-02-22T19:38:21.612696+0000","last_fullsized":"2024-02-22T19:38:21.612696+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:35:35.930654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.930654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'5","reported_seq":208,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606913+0000","last_change":"2024-02-22T19:34:46.118383+0000","last_active":"2024-02-22T19:38:21.606913+0000","last_peered":"2024-02-22T19:38:21.606913+0000","last_clean":"2024-02-22T19:38:21.606913+0000","last_became_active":"2024-02-22T19:29:03.081537+0000","last_became_peered":"2024-02-22T19:29:03.081537+0000","last_unstale":"2024-02-22T19:38:21.606913+0000","last_undegraded":"2024-02-22T19:38:21.606913+0000","last_fullsized":"2024-02-22T19:38:21.606913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:34:46.118282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:46.118282+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":212,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606999+0000","last_change":"2024-02-22T19:37:57.180194+0000","last_active":"2024-02-22T19:38:21.606999+0000","last_peered":"2024-02-22T19:38:21.606999+0000","last_clean":"2024-02-22T19:38:21.606999+0000","last_became_active":"2024-02-22T19:37:57.179814+0000","last_became_peered":"2024-02-22T19:37:57.179814+0000","last_unstale":"2024-02-22T19:38:21.606999+0000","last_undegraded":"2024-02-22T19:38:21.606999+0000","last_fullsized":"2024-02-22T19:38:21.606999+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:56.107558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:34:56.107558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606863+0000","last_change":"2024-02-22T19:38:13.195748+0000","last_active":"2024-02-22T19:38:21.606863+0000","last_peered":"2024-02-22T19:38:21.606863+0000","last_clean":"2024-02-22T19:38:21.606863+0000","last_became_active":"2024-02-22T19:38:13.194956+0000","last_became_peered":"2024-02-22T19:38:13.194956+0000","last_unstale":"2024-02-22T19:38:21.606863+0000","last_undegraded":"2024-02-22T19:38:21.606863+0000","last_fullsized":"2024-02-22T19:38:21.606863+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.732856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.732856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"172'22","reported_seq":237,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.613208+0000","last_change":"2024-02-22T19:36:24.021213+0000","last_active":"2024-02-22T19:38:21.613208+0000","last_peered":"2024-02-22T19:38:21.613208+0000","last_clean":"2024-02-22T19:38:21.613208+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:38:21.613208+0000","last_undegraded":"2024-02-22T19:38:21.613208+0000","last_fullsized":"2024-02-22T19:38:21.613208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'22","last_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.021111+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":214,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.627015+0000","last_change":"2024-02-22T19:37:57.180410+0000","last_active":"2024-02-22T19:38:21.627015+0000","last_peered":"2024-02-22T19:38:21.627015+0000","last_clean":"2024-02-22T19:38:21.627015+0000","last_became_active":"2024-02-22T19:37:57.180089+0000","last_became_peered":"2024-02-22T19:37:57.180089+0000","last_unstale":"2024-02-22T19:38:21.627015+0000","last_undegraded":"2024-02-22T19:38:21.627015+0000","last_fullsized":"2024-02-22T19:38:21.627015+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":205,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606588+0000","last_change":"2024-02-22T19:37:08.584019+0000","last_active":"2024-02-22T19:38:21.606588+0000","last_peered":"2024-02-22T19:38:21.606588+0000","last_clean":"2024-02-22T19:38:21.606588+0000","last_became_active":"2024-02-22T19:29:01.006228+0000","last_became_peered":"2024-02-22T19:29:01.006228+0000","last_unstale":"2024-02-22T19:38:21.606588+0000","last_undegraded":"2024-02-22T19:38:21.606588+0000","last_fullsized":"2024-02-22T19:38:21.606588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.583926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.583926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"87'17","reported_seq":223,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615509+0000","last_change":"2024-02-22T19:35:36.338814+0000","last_active":"2024-02-22T19:38:21.615509+0000","last_peered":"2024-02-22T19:38:21.615509+0000","last_clean":"2024-02-22T19:38:21.615509+0000","last_became_active":"2024-02-22T19:29:03.082690+0000","last_became_peered":"2024-02-22T19:29:03.082690+0000","last_unstale":"2024-02-22T19:38:21.615509+0000","last_undegraded":"2024-02-22T19:38:21.615509+0000","last_fullsized":"2024-02-22T19:38:21.615509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":231,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.618318+0000","last_change":"2024-02-22T19:38:05.361666+0000","last_active":"2024-02-22T19:38:21.618318+0000","last_peered":"2024-02-22T19:38:21.618318+0000","last_clean":"2024-02-22T19:38:21.618318+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:38:21.618318+0000","last_undegraded":"2024-02-22T19:38:21.618318+0000","last_fullsized":"2024-02-22T19:38:21.618318+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:36:06.575860+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:36:06.575860+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":210,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.608102+0000","last_change":"2024-02-22T19:37:50.144269+0000","last_active":"2024-02-22T19:38:21.608102+0000","last_peered":"2024-02-22T19:38:21.608102+0000","last_clean":"2024-02-22T19:38:21.608102+0000","last_became_active":"2024-02-22T19:29:01.008652+0000","last_became_peered":"2024-02-22T19:29:01.008652+0000","last_unstale":"2024-02-22T19:38:21.608102+0000","last_undegraded":"2024-02-22T19:38:21.608102+0000","last_fullsized":"2024-02-22T19:38:21.608102+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:50.144179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:50.144179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":56,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617742+0000","last_change":"2024-02-22T19:38:13.718776+0000","last_active":"2024-02-22T19:38:21.617742+0000","last_peered":"2024-02-22T19:38:21.617742+0000","last_clean":"2024-02-22T19:38:21.617742+0000","last_became_active":"2024-02-22T19:38:13.201224+0000","last_became_peered":"2024-02-22T19:38:13.201224+0000","last_unstale":"2024-02-22T19:38:21.617742+0000","last_undegraded":"2024-02-22T19:38:21.617742+0000","last_fullsized":"2024-02-22T19:38:21.617742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:36:04.546703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.546703+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":219,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606292+0000","last_change":"2024-02-22T19:37:57.174313+0000","last_active":"2024-02-22T19:38:21.606292+0000","last_peered":"2024-02-22T19:38:21.606292+0000","last_clean":"2024-02-22T19:38:21.606292+0000","last_became_active":"2024-02-22T19:37:57.173953+0000","last_became_peered":"2024-02-22T19:37:57.173953+0000","last_unstale":"2024-02-22T19:38:21.606292+0000","last_undegraded":"2024-02-22T19:38:21.606292+0000","last_fullsized":"2024-02-22T19:38:21.606292+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.995002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.995002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":208,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615128+0000","last_change":"2024-02-22T19:38:13.197395+0000","last_active":"2024-02-22T19:38:21.615128+0000","last_peered":"2024-02-22T19:38:21.615128+0000","last_clean":"2024-02-22T19:38:21.615128+0000","last_became_active":"2024-02-22T19:38:13.196388+0000","last_became_peered":"2024-02-22T19:38:13.196388+0000","last_unstale":"2024-02-22T19:38:21.615128+0000","last_undegraded":"2024-02-22T19:38:21.615128+0000","last_fullsized":"2024-02-22T19:38:21.615128+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:34:58.164960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":298,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611502+0000","last_change":"2024-02-22T19:37:57.212060+0000","last_active":"2024-02-22T19:38:21.611502+0000","last_peered":"2024-02-22T19:38:21.611502+0000","last_clean":"2024-02-22T19:38:21.611502+0000","last_became_active":"2024-02-22T19:37:57.190476+0000","last_became_peered":"2024-02-22T19:37:57.190476+0000","last_unstale":"2024-02-22T19:38:21.611502+0000","last_undegraded":"2024-02-22T19:38:21.611502+0000","last_fullsized":"2024-02-22T19:38:21.611502+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:52.432675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:52.432675+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":204,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617131+0000","last_change":"2024-02-22T19:37:56.406824+0000","last_active":"2024-02-22T19:38:21.617131+0000","last_peered":"2024-02-22T19:38:21.617131+0000","last_clean":"2024-02-22T19:38:21.617131+0000","last_became_active":"2024-02-22T19:29:07.097608+0000","last_became_peered":"2024-02-22T19:29:07.097608+0000","last_unstale":"2024-02-22T19:38:21.617131+0000","last_undegraded":"2024-02-22T19:38:21.617131+0000","last_fullsized":"2024-02-22T19:38:21.617131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":203,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611175+0000","last_change":"2024-02-22T19:37:53.400620+0000","last_active":"2024-02-22T19:38:21.611175+0000","last_peered":"2024-02-22T19:38:21.611175+0000","last_clean":"2024-02-22T19:38:21.611175+0000","last_became_active":"2024-02-22T19:29:05.091832+0000","last_became_peered":"2024-02-22T19:29:05.091832+0000","last_unstale":"2024-02-22T19:38:21.611175+0000","last_undegraded":"2024-02-22T19:38:21.611175+0000","last_fullsized":"2024-02-22T19:38:21.611175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.400534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.400534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":214,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610032+0000","last_change":"2024-02-22T19:38:13.199554+0000","last_active":"2024-02-22T19:38:21.610032+0000","last_peered":"2024-02-22T19:38:21.610032+0000","last_clean":"2024-02-22T19:38:21.610032+0000","last_became_active":"2024-02-22T19:38:13.199208+0000","last_became_peered":"2024-02-22T19:38:13.199208+0000","last_unstale":"2024-02-22T19:38:21.610032+0000","last_undegraded":"2024-02-22T19:38:21.610032+0000","last_fullsized":"2024-02-22T19:38:21.610032+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":206,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615206+0000","last_change":"2024-02-22T19:36:58.263929+0000","last_active":"2024-02-22T19:38:21.615206+0000","last_peered":"2024-02-22T19:38:21.615206+0000","last_clean":"2024-02-22T19:38:21.615206+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:38:21.615206+0000","last_undegraded":"2024-02-22T19:38:21.615206+0000","last_fullsized":"2024-02-22T19:38:21.615206+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:36:58.263834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.263834+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":207,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615249+0000","last_change":"2024-02-22T19:38:13.199595+0000","last_active":"2024-02-22T19:38:21.615249+0000","last_peered":"2024-02-22T19:38:21.615249+0000","last_clean":"2024-02-22T19:38:21.615249+0000","last_became_active":"2024-02-22T19:38:13.198701+0000","last_became_peered":"2024-02-22T19:38:13.198701+0000","last_unstale":"2024-02-22T19:38:21.615249+0000","last_undegraded":"2024-02-22T19:38:21.615249+0000","last_fullsized":"2024-02-22T19:38:21.615249+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":204,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617070+0000","last_change":"2024-02-22T19:37:54.419055+0000","last_active":"2024-02-22T19:38:21.617070+0000","last_peered":"2024-02-22T19:38:21.617070+0000","last_clean":"2024-02-22T19:38:21.617070+0000","last_became_active":"2024-02-22T19:29:05.098300+0000","last_became_peered":"2024-02-22T19:29:05.098300+0000","last_unstale":"2024-02-22T19:38:21.617070+0000","last_undegraded":"2024-02-22T19:38:21.617070+0000","last_fullsized":"2024-02-22T19:38:21.617070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:54.418958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:37:54.418958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609987+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:38:21.609987+0000","last_peered":"2024-02-22T19:38:21.609987+0000","last_clean":"2024-02-22T19:38:21.609987+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:38:21.609987+0000","last_undegraded":"2024-02-22T19:38:21.609987+0000","last_fullsized":"2024-02-22T19:38:21.609987+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":214,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619061+0000","last_change":"2024-02-22T19:34:43.915568+0000","last_active":"2024-02-22T19:38:21.619061+0000","last_peered":"2024-02-22T19:38:21.619061+0000","last_clean":"2024-02-22T19:38:21.619061+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:38:21.619061+0000","last_undegraded":"2024-02-22T19:38:21.619061+0000","last_fullsized":"2024-02-22T19:38:21.619061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:34:43.915463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.915463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":205,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609288+0000","last_change":"2024-02-22T19:38:13.197088+0000","last_active":"2024-02-22T19:38:21.609288+0000","last_peered":"2024-02-22T19:38:21.609288+0000","last_clean":"2024-02-22T19:38:21.609288+0000","last_became_active":"2024-02-22T19:38:13.196703+0000","last_became_peered":"2024-02-22T19:38:13.196703+0000","last_unstale":"2024-02-22T19:38:21.609288+0000","last_undegraded":"2024-02-22T19:38:21.609288+0000","last_fullsized":"2024-02-22T19:38:21.609288+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":202,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.614988+0000","last_change":"2024-02-22T19:38:13.201404+0000","last_active":"2024-02-22T19:38:21.614988+0000","last_peered":"2024-02-22T19:38:21.614988+0000","last_clean":"2024-02-22T19:38:21.614988+0000","last_became_active":"2024-02-22T19:38:13.201031+0000","last_became_peered":"2024-02-22T19:38:13.201031+0000","last_unstale":"2024-02-22T19:38:21.614988+0000","last_undegraded":"2024-02-22T19:38:21.614988+0000","last_fullsized":"2024-02-22T19:38:21.614988+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:41.961788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:41.961788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":212,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617374+0000","last_change":"2024-02-22T19:36:52.463891+0000","last_active":"2024-02-22T19:38:21.617374+0000","last_peered":"2024-02-22T19:38:21.617374+0000","last_clean":"2024-02-22T19:38:21.617374+0000","last_became_active":"2024-02-22T19:29:01.007449+0000","last_became_peered":"2024-02-22T19:29:01.007449+0000","last_unstale":"2024-02-22T19:38:21.617374+0000","last_undegraded":"2024-02-22T19:38:21.617374+0000","last_fullsized":"2024-02-22T19:38:21.617374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:52.463774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.463774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":219,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606958+0000","last_change":"2024-02-22T19:35:35.855558+0000","last_active":"2024-02-22T19:38:21.606958+0000","last_peered":"2024-02-22T19:38:21.606958+0000","last_clean":"2024-02-22T19:38:21.606958+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:38:21.606958+0000","last_undegraded":"2024-02-22T19:38:21.606958+0000","last_fullsized":"2024-02-22T19:38:21.606958+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:35.855449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.855449+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":223,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.618767+0000","last_change":"2024-02-22T19:38:13.202562+0000","last_active":"2024-02-22T19:38:21.618767+0000","last_peered":"2024-02-22T19:38:21.618767+0000","last_clean":"2024-02-22T19:38:21.618767+0000","last_became_active":"2024-02-22T19:38:13.202073+0000","last_became_peered":"2024-02-22T19:38:13.202073+0000","last_unstale":"2024-02-22T19:38:21.618767+0000","last_undegraded":"2024-02-22T19:38:21.618767+0000","last_fullsized":"2024-02-22T19:38:21.618767+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.229816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.229816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":219,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606803+0000","last_change":"2024-02-22T19:37:57.187682+0000","last_active":"2024-02-22T19:38:21.606803+0000","last_peered":"2024-02-22T19:38:21.606803+0000","last_clean":"2024-02-22T19:38:21.606803+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:38:21.606803+0000","last_undegraded":"2024-02-22T19:38:21.606803+0000","last_fullsized":"2024-02-22T19:38:21.606803+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:47.993328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:47.993328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":205,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605912+0000","last_change":"2024-02-22T19:36:27.685508+0000","last_active":"2024-02-22T19:38:21.605912+0000","last_peered":"2024-02-22T19:38:21.605912+0000","last_clean":"2024-02-22T19:38:21.605912+0000","last_became_active":"2024-02-22T19:29:01.002658+0000","last_became_peered":"2024-02-22T19:29:01.002658+0000","last_unstale":"2024-02-22T19:38:21.605912+0000","last_undegraded":"2024-02-22T19:38:21.605912+0000","last_fullsized":"2024-02-22T19:38:21.605912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.685418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.685418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":222,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606821+0000","last_change":"2024-02-22T19:37:53.461356+0000","last_active":"2024-02-22T19:38:21.606821+0000","last_peered":"2024-02-22T19:38:21.606821+0000","last_clean":"2024-02-22T19:38:21.606821+0000","last_became_active":"2024-02-22T19:29:03.083841+0000","last_became_peered":"2024-02-22T19:29:03.083841+0000","last_unstale":"2024-02-22T19:38:21.606821+0000","last_undegraded":"2024-02-22T19:38:21.606821+0000","last_fullsized":"2024-02-22T19:38:21.606821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:53.461286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.461286+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617476+0000","last_change":"2024-02-22T19:38:13.201215+0000","last_active":"2024-02-22T19:38:21.617476+0000","last_peered":"2024-02-22T19:38:21.617476+0000","last_clean":"2024-02-22T19:38:21.617476+0000","last_became_active":"2024-02-22T19:38:13.200680+0000","last_became_peered":"2024-02-22T19:38:13.200680+0000","last_unstale":"2024-02-22T19:38:21.617476+0000","last_undegraded":"2024-02-22T19:38:21.617476+0000","last_fullsized":"2024-02-22T19:38:21.617476+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.699319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.699319+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"234'12","reported_seq":54,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.199309+0000","last_change":"2024-02-22T19:38:13.205213+0000","last_active":"2024-02-22T19:38:22.199309+0000","last_peered":"2024-02-22T19:38:22.199309+0000","last_clean":"2024-02-22T19:38:22.199309+0000","last_became_active":"2024-02-22T19:38:13.191071+0000","last_became_peered":"2024-02-22T19:38:13.191071+0000","last_unstale":"2024-02-22T19:38:22.199309+0000","last_undegraded":"2024-02-22T19:38:22.199309+0000","last_fullsized":"2024-02-22T19:38:22.199309+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":203,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617228+0000","last_change":"2024-02-22T19:36:56.463572+0000","last_active":"2024-02-22T19:38:21.617228+0000","last_peered":"2024-02-22T19:38:21.617228+0000","last_clean":"2024-02-22T19:38:21.617228+0000","last_became_active":"2024-02-22T19:29:01.004315+0000","last_became_peered":"2024-02-22T19:29:01.004315+0000","last_unstale":"2024-02-22T19:38:21.617228+0000","last_undegraded":"2024-02-22T19:38:21.617228+0000","last_fullsized":"2024-02-22T19:38:21.617228+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:56.463502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:36:56.463502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"85'15","reported_seq":220,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619718+0000","last_change":"2024-02-22T19:37:09.345416+0000","last_active":"2024-02-22T19:38:21.619718+0000","last_peered":"2024-02-22T19:38:21.619718+0000","last_clean":"2024-02-22T19:38:21.619718+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:38:21.619718+0000","last_undegraded":"2024-02-22T19:38:21.619718+0000","last_fullsized":"2024-02-22T19:38:21.619718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:37:09.345319+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.345319+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":217,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606298+0000","last_change":"2024-02-22T19:37:57.180359+0000","last_active":"2024-02-22T19:38:21.606298+0000","last_peered":"2024-02-22T19:38:21.606298+0000","last_clean":"2024-02-22T19:38:21.606298+0000","last_became_active":"2024-02-22T19:37:57.179955+0000","last_became_peered":"2024-02-22T19:37:57.179955+0000","last_unstale":"2024-02-22T19:38:21.606298+0000","last_undegraded":"2024-02-22T19:38:21.606298+0000","last_fullsized":"2024-02-22T19:38:21.606298+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.761226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":314,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.219732+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:38:22.219732+0000","last_peered":"2024-02-22T19:38:22.219732+0000","last_clean":"2024-02-22T19:38:22.219732+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:38:22.219732+0000","last_undegraded":"2024-02-22T19:38:22.219732+0000","last_fullsized":"2024-02-22T19:38:22.219732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.616903+0000","last_change":"2024-02-22T19:38:13.200574+0000","last_active":"2024-02-22T19:38:21.616903+0000","last_peered":"2024-02-22T19:38:21.616903+0000","last_clean":"2024-02-22T19:38:21.616903+0000","last_became_active":"2024-02-22T19:38:13.198508+0000","last_became_peered":"2024-02-22T19:38:13.198508+0000","last_unstale":"2024-02-22T19:38:21.616903+0000","last_undegraded":"2024-02-22T19:38:21.616903+0000","last_fullsized":"2024-02-22T19:38:21.616903+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"88'13","reported_seq":40,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610591+0000","last_change":"2024-02-22T19:37:57.198915+0000","last_active":"2024-02-22T19:38:21.610591+0000","last_peered":"2024-02-22T19:38:21.610591+0000","last_clean":"2024-02-22T19:38:21.610591+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:38:21.610591+0000","last_undegraded":"2024-02-22T19:38:21.610591+0000","last_fullsized":"2024-02-22T19:38:21.610591+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":204,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.613385+0000","last_change":"2024-02-22T19:37:02.236048+0000","last_active":"2024-02-22T19:38:21.613385+0000","last_peered":"2024-02-22T19:38:21.613385+0000","last_clean":"2024-02-22T19:38:21.613385+0000","last_became_active":"2024-02-22T19:29:07.096185+0000","last_became_peered":"2024-02-22T19:29:07.096185+0000","last_unstale":"2024-02-22T19:38:21.613385+0000","last_undegraded":"2024-02-22T19:38:21.613385+0000","last_fullsized":"2024-02-22T19:38:21.613385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.235958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.235958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":208,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.616951+0000","last_change":"2024-02-22T19:37:09.470209+0000","last_active":"2024-02-22T19:38:21.616951+0000","last_peered":"2024-02-22T19:38:21.616951+0000","last_clean":"2024-02-22T19:38:21.616951+0000","last_became_active":"2024-02-22T19:29:01.010762+0000","last_became_peered":"2024-02-22T19:29:01.010762+0000","last_unstale":"2024-02-22T19:38:21.616951+0000","last_undegraded":"2024-02-22T19:38:21.616951+0000","last_fullsized":"2024-02-22T19:38:21.616951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.470149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.470149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":55,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.620707+0000","last_change":"2024-02-22T19:38:13.546539+0000","last_active":"2024-02-22T19:38:21.620707+0000","last_peered":"2024-02-22T19:38:21.620707+0000","last_clean":"2024-02-22T19:38:21.620707+0000","last_became_active":"2024-02-22T19:38:13.196752+0000","last_became_peered":"2024-02-22T19:38:13.196752+0000","last_unstale":"2024-02-22T19:38:21.620707+0000","last_undegraded":"2024-02-22T19:38:21.620707+0000","last_fullsized":"2024-02-22T19:38:21.620707+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":202,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617288+0000","last_change":"2024-02-22T19:37:53.392269+0000","last_active":"2024-02-22T19:38:21.617288+0000","last_peered":"2024-02-22T19:38:21.617288+0000","last_clean":"2024-02-22T19:38:21.617288+0000","last_became_active":"2024-02-22T19:29:07.199410+0000","last_became_peered":"2024-02-22T19:29:07.199410+0000","last_unstale":"2024-02-22T19:38:21.617288+0000","last_undegraded":"2024-02-22T19:38:21.617288+0000","last_fullsized":"2024-02-22T19:38:21.617288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":236,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610863+0000","last_change":"2024-02-22T19:38:13.196006+0000","last_active":"2024-02-22T19:38:21.610863+0000","last_peered":"2024-02-22T19:38:21.610863+0000","last_clean":"2024-02-22T19:38:21.610863+0000","last_became_active":"2024-02-22T19:38:13.195628+0000","last_became_peered":"2024-02-22T19:38:13.195628+0000","last_unstale":"2024-02-22T19:38:21.610863+0000","last_undegraded":"2024-02-22T19:38:21.610863+0000","last_fullsized":"2024-02-22T19:38:21.610863+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.431655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.431655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'6","reported_seq":226,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.627075+0000","last_change":"2024-02-22T19:36:26.647888+0000","last_active":"2024-02-22T19:38:21.627075+0000","last_peered":"2024-02-22T19:38:21.627075+0000","last_clean":"2024-02-22T19:38:21.627075+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:38:21.627075+0000","last_undegraded":"2024-02-22T19:38:21.627075+0000","last_fullsized":"2024-02-22T19:38:21.627075+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:36:25.296484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.296484+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"238'11316","reported_seq":11416,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.359333+0000","last_change":"2024-02-22T19:37:58.343330+0000","last_active":"2024-02-22T19:38:24.359333+0000","last_peered":"2024-02-22T19:38:24.359333+0000","last_clean":"2024-02-22T19:37:56.641804+0000","last_became_active":"2024-02-22T19:37:58.292350+0000","last_became_peered":"2024-02-22T19:37:58.292350+0000","last_unstale":"2024-02-22T19:38:24.359333+0000","last_undegraded":"2024-02-22T19:38:24.359333+0000","last_fullsized":"2024-02-22T19:38:24.359333+0000","mapping_epoch":218,"log_start":"181'6337","ondisk_log_start":"181'6337","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4979,"ondisk_log_size":4979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":162660352,"num_objects":2482,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2482,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11316,"num_write_kb":441536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[2,4,6],"avail_no_missing":["2","4","6"],"object_location_counts":[{"shards":"2,4,6","objects":2482}],"blocked_by":[],"up_primary":5,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609505+0000","last_change":"2024-02-22T19:38:13.192429+0000","last_active":"2024-02-22T19:38:21.609505+0000","last_peered":"2024-02-22T19:38:21.609505+0000","last_clean":"2024-02-22T19:38:21.609505+0000","last_became_active":"2024-02-22T19:38:13.192057+0000","last_became_peered":"2024-02-22T19:38:13.192057+0000","last_unstale":"2024-02-22T19:38:21.609505+0000","last_undegraded":"2024-02-22T19:38:21.609505+0000","last_fullsized":"2024-02-22T19:38:21.609505+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.151294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.151294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":233,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617709+0000","last_change":"2024-02-22T19:38:13.218151+0000","last_active":"2024-02-22T19:38:21.617709+0000","last_peered":"2024-02-22T19:38:21.617709+0000","last_clean":"2024-02-22T19:38:21.617709+0000","last_became_active":"2024-02-22T19:38:13.198125+0000","last_became_peered":"2024-02-22T19:38:13.198125+0000","last_unstale":"2024-02-22T19:38:21.617709+0000","last_undegraded":"2024-02-22T19:38:21.617709+0000","last_fullsized":"2024-02-22T19:38:21.617709+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:36:04.642361+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:36:04.642361+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":46,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609144+0000","last_change":"2024-02-22T19:37:58.397461+0000","last_active":"2024-02-22T19:38:21.609144+0000","last_peered":"2024-02-22T19:38:21.609144+0000","last_clean":"2024-02-22T19:38:21.609144+0000","last_became_active":"2024-02-22T19:37:57.181199+0000","last_became_peered":"2024-02-22T19:37:57.181199+0000","last_unstale":"2024-02-22T19:38:21.609144+0000","last_undegraded":"2024-02-22T19:38:21.609144+0000","last_fullsized":"2024-02-22T19:38:21.609144+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:36:09.623521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.623521+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"217'10","reported_seq":339,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.205439+0000","last_change":"2024-02-22T19:37:56.672147+0000","last_active":"2024-02-22T19:38:22.205439+0000","last_peered":"2024-02-22T19:38:22.205439+0000","last_clean":"2024-02-22T19:38:22.205439+0000","last_became_active":"2024-02-22T19:37:56.670665+0000","last_became_peered":"2024-02-22T19:37:56.670665+0000","last_unstale":"2024-02-22T19:38:22.205439+0000","last_undegraded":"2024-02-22T19:38:22.205439+0000","last_fullsized":"2024-02-22T19:38:22.205439+0000","mapping_epoch":60,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:36:53.503128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.503128+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"238'11601","reported_seq":11698,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:24.287738+0000","last_change":"2024-02-22T19:37:07.737898+0000","last_active":"2024-02-22T19:38:24.287738+0000","last_peered":"2024-02-22T19:38:24.287738+0000","last_clean":"2024-02-22T19:38:24.287738+0000","last_became_active":"2024-02-22T19:35:19.225793+0000","last_became_peered":"2024-02-22T19:35:19.225793+0000","last_unstale":"2024-02-22T19:38:24.287738+0000","last_undegraded":"2024-02-22T19:38:24.287738+0000","last_fullsized":"2024-02-22T19:38:24.287738+0000","mapping_epoch":154,"log_start":"180'6340","ondisk_log_start":"180'6340","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'7399","last_scrub_stamp":"2024-02-22T19:37:07.737806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:07.737806+0000","log_size":5261,"ondisk_log_size":5261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":166526976,"num_objects":2541,"num_object_clones":0,"num_object_copies":7623,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11601,"num_write_kb":452544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":191,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615168+0000","last_change":"2024-02-22T19:36:11.224751+0000","last_active":"2024-02-22T19:38:21.615168+0000","last_peered":"2024-02-22T19:38:21.615168+0000","last_clean":"2024-02-22T19:38:21.615168+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:38:21.615168+0000","last_undegraded":"2024-02-22T19:38:21.615168+0000","last_fullsized":"2024-02-22T19:38:21.615168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.224646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.224646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":220,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617653+0000","last_change":"2024-02-22T19:37:51.431591+0000","last_active":"2024-02-22T19:38:21.617653+0000","last_peered":"2024-02-22T19:38:21.617653+0000","last_clean":"2024-02-22T19:38:21.617653+0000","last_became_active":"2024-02-22T19:29:01.009916+0000","last_became_peered":"2024-02-22T19:29:01.009916+0000","last_unstale":"2024-02-22T19:38:21.617653+0000","last_undegraded":"2024-02-22T19:38:21.617653+0000","last_fullsized":"2024-02-22T19:38:21.617653+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.431527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.431527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":226,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605926+0000","last_change":"2024-02-22T19:37:05.645806+0000","last_active":"2024-02-22T19:38:21.605926+0000","last_peered":"2024-02-22T19:38:21.605926+0000","last_clean":"2024-02-22T19:38:21.605926+0000","last_became_active":"2024-02-22T19:29:03.080284+0000","last_became_peered":"2024-02-22T19:29:03.080284+0000","last_unstale":"2024-02-22T19:38:21.605926+0000","last_undegraded":"2024-02-22T19:38:21.605926+0000","last_fullsized":"2024-02-22T19:38:21.605926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:05.645703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.645703+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":228,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.612253+0000","last_change":"2024-02-22T19:38:13.183412+0000","last_active":"2024-02-22T19:38:21.612253+0000","last_peered":"2024-02-22T19:38:21.612253+0000","last_clean":"2024-02-22T19:38:21.612253+0000","last_became_active":"2024-02-22T19:38:13.183006+0000","last_became_peered":"2024-02-22T19:38:13.183006+0000","last_unstale":"2024-02-22T19:38:21.612253+0000","last_undegraded":"2024-02-22T19:38:21.612253+0000","last_fullsized":"2024-02-22T19:38:21.612253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.039457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.039457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"238'11654","reported_seq":13391,"reported_epoch":238,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:38:24.284498+0000","last_change":"2024-02-22T19:38:14.386980+0000","last_active":"2024-02-22T19:38:24.284498+0000","last_peered":"2024-02-22T19:38:24.284498+0000","last_clean":"2024-02-22T19:38:12.171359+0000","last_became_active":"2024-02-22T19:38:14.321076+0000","last_became_peered":"2024-02-22T19:38:14.321076+0000","last_unstale":"2024-02-22T19:38:24.284498+0000","last_undegraded":"2024-02-22T19:38:24.284498+0000","last_fullsized":"2024-02-22T19:38:24.284498+0000","mapping_epoch":233,"log_start":"180'6427","ondisk_log_start":"180'6427","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'10059","last_scrub_stamp":"2024-02-22T19:37:49.752288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:49.752288+0000","log_size":5227,"ondisk_log_size":5227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165675008,"num_objects":2528,"num_object_clones":0,"num_object_copies":7584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1787,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11654,"num_write_kb":453824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":52953088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,0],"avail_no_missing":["6","0","7"],"object_location_counts":[{"shards":"0,6,7","objects":2528}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":197,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.616649+0000","last_change":"2024-02-22T19:36:54.489612+0000","last_active":"2024-02-22T19:38:21.616649+0000","last_peered":"2024-02-22T19:38:21.616649+0000","last_clean":"2024-02-22T19:38:21.616649+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:38:21.616649+0000","last_undegraded":"2024-02-22T19:38:21.616649+0000","last_fullsized":"2024-02-22T19:38:21.616649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.489509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.489509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":254,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619597+0000","last_change":"2024-02-22T19:38:13.195590+0000","last_active":"2024-02-22T19:38:21.619597+0000","last_peered":"2024-02-22T19:38:21.619597+0000","last_clean":"2024-02-22T19:38:21.619597+0000","last_became_active":"2024-02-22T19:38:13.195206+0000","last_became_peered":"2024-02-22T19:38:13.195206+0000","last_unstale":"2024-02-22T19:38:21.619597+0000","last_undegraded":"2024-02-22T19:38:21.619597+0000","last_fullsized":"2024-02-22T19:38:21.619597+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.029389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":226,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617486+0000","last_change":"2024-02-22T19:37:57.185054+0000","last_active":"2024-02-22T19:38:21.617486+0000","last_peered":"2024-02-22T19:38:21.617486+0000","last_clean":"2024-02-22T19:38:21.617486+0000","last_became_active":"2024-02-22T19:37:57.184627+0000","last_became_peered":"2024-02-22T19:37:57.184627+0000","last_unstale":"2024-02-22T19:38:21.617486+0000","last_undegraded":"2024-02-22T19:38:21.617486+0000","last_fullsized":"2024-02-22T19:38:21.617486+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.438112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.438112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":236,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611272+0000","last_change":"2024-02-22T19:37:18.374920+0000","last_active":"2024-02-22T19:38:21.611272+0000","last_peered":"2024-02-22T19:38:21.611272+0000","last_clean":"2024-02-22T19:38:21.611272+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:38:21.611272+0000","last_undegraded":"2024-02-22T19:38:21.611272+0000","last_fullsized":"2024-02-22T19:38:21.611272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:37:18.374807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.374807+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.7","version":"238'11611","reported_seq":11708,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:24.367271+0000","last_change":"2024-02-22T19:37:17.093546+0000","last_active":"2024-02-22T19:38:24.367271+0000","last_peered":"2024-02-22T19:38:24.367271+0000","last_clean":"2024-02-22T19:38:24.367271+0000","last_became_active":"2024-02-22T19:35:19.227265+0000","last_became_peered":"2024-02-22T19:35:19.227265+0000","last_unstale":"2024-02-22T19:38:24.367271+0000","last_undegraded":"2024-02-22T19:38:24.367271+0000","last_fullsized":"2024-02-22T19:38:24.367271+0000","mapping_epoch":154,"log_start":"182'6950","ondisk_log_start":"182'6950","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'7976","last_scrub_stamp":"2024-02-22T19:37:17.093426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:17.093426+0000","log_size":4661,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":167706624,"num_objects":2559,"num_object_clones":0,"num_object_copies":7677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11611,"num_write_kb":453440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.616723+0000","last_change":"2024-02-22T19:38:13.201370+0000","last_active":"2024-02-22T19:38:21.616723+0000","last_peered":"2024-02-22T19:38:21.616723+0000","last_clean":"2024-02-22T19:38:21.616723+0000","last_became_active":"2024-02-22T19:38:13.201127+0000","last_became_peered":"2024-02-22T19:38:13.201127+0000","last_unstale":"2024-02-22T19:38:21.616723+0000","last_undegraded":"2024-02-22T19:38:21.616723+0000","last_fullsized":"2024-02-22T19:38:21.616723+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.690298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.690298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":224,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.612658+0000","last_change":"2024-02-22T19:37:52.617217+0000","last_active":"2024-02-22T19:38:21.612658+0000","last_peered":"2024-02-22T19:38:21.612658+0000","last_clean":"2024-02-22T19:38:21.612658+0000","last_became_active":"2024-02-22T19:37:52.616905+0000","last_became_peered":"2024-02-22T19:37:52.616905+0000","last_unstale":"2024-02-22T19:38:21.612658+0000","last_undegraded":"2024-02-22T19:38:21.612658+0000","last_fullsized":"2024-02-22T19:38:21.612658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.207753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.207753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.612614+0000","last_change":"2024-02-22T19:37:57.182314+0000","last_active":"2024-02-22T19:38:21.612614+0000","last_peered":"2024-02-22T19:38:21.612614+0000","last_clean":"2024-02-22T19:38:21.612614+0000","last_became_active":"2024-02-22T19:37:57.181963+0000","last_became_peered":"2024-02-22T19:37:57.181963+0000","last_unstale":"2024-02-22T19:38:21.612614+0000","last_undegraded":"2024-02-22T19:38:21.612614+0000","last_fullsized":"2024-02-22T19:38:21.612614+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.297621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.297621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":75,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619343+0000","last_change":"2024-02-22T19:38:13.551157+0000","last_active":"2024-02-22T19:38:21.619343+0000","last_peered":"2024-02-22T19:38:21.619343+0000","last_clean":"2024-02-22T19:38:21.619343+0000","last_became_active":"2024-02-22T19:38:13.193094+0000","last_became_peered":"2024-02-22T19:38:13.193094+0000","last_unstale":"2024-02-22T19:38:21.619343+0000","last_undegraded":"2024-02-22T19:38:21.619343+0000","last_fullsized":"2024-02-22T19:38:21.619343+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:18.621747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:18.621747+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.612682+0000","last_change":"2024-02-22T19:38:13.192763+0000","last_active":"2024-02-22T19:38:21.612682+0000","last_peered":"2024-02-22T19:38:21.612682+0000","last_clean":"2024-02-22T19:38:21.612682+0000","last_became_active":"2024-02-22T19:38:13.190598+0000","last_became_peered":"2024-02-22T19:38:13.190598+0000","last_unstale":"2024-02-22T19:38:21.612682+0000","last_undegraded":"2024-02-22T19:38:21.612682+0000","last_fullsized":"2024-02-22T19:38:21.612682+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"238'11759","reported_seq":11865,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.405238+0000","last_change":"2024-02-22T19:37:58.412582+0000","last_active":"2024-02-22T19:38:24.405238+0000","last_peered":"2024-02-22T19:38:24.405238+0000","last_clean":"2024-02-22T19:37:56.648601+0000","last_became_active":"2024-02-22T19:37:58.273546+0000","last_became_peered":"2024-02-22T19:37:58.273546+0000","last_unstale":"2024-02-22T19:38:24.405238+0000","last_undegraded":"2024-02-22T19:38:24.405238+0000","last_fullsized":"2024-02-22T19:38:24.405238+0000","mapping_epoch":218,"log_start":"181'6500","ondisk_log_start":"181'6500","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":5259,"ondisk_log_size":5259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160497664,"num_objects":2449,"num_object_clones":0,"num_object_copies":7347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2449,"num_objects_unfound":0,"num_objects_dirty":2449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11759,"num_write_kb":454656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[0,3,7],"avail_no_missing":["0","3","7"],"object_location_counts":[{"shards":"0,3,7","objects":2449}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":196,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.620785+0000","last_change":"2024-02-22T19:37:45.317538+0000","last_active":"2024-02-22T19:38:21.620785+0000","last_peered":"2024-02-22T19:38:21.620785+0000","last_clean":"2024-02-22T19:38:21.620785+0000","last_became_active":"2024-02-22T19:29:07.100963+0000","last_became_peered":"2024-02-22T19:29:07.100963+0000","last_unstale":"2024-02-22T19:38:21.620785+0000","last_undegraded":"2024-02-22T19:38:21.620785+0000","last_fullsized":"2024-02-22T19:38:21.620785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:45.317440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:45.317440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609448+0000","last_change":"2024-02-22T19:37:57.185637+0000","last_active":"2024-02-22T19:38:21.609448+0000","last_peered":"2024-02-22T19:38:21.609448+0000","last_clean":"2024-02-22T19:38:21.609448+0000","last_became_active":"2024-02-22T19:37:57.185312+0000","last_became_peered":"2024-02-22T19:37:57.185312+0000","last_unstale":"2024-02-22T19:38:21.609448+0000","last_undegraded":"2024-02-22T19:38:21.609448+0000","last_fullsized":"2024-02-22T19:38:21.609448+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":216,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606747+0000","last_change":"2024-02-22T19:38:13.194397+0000","last_active":"2024-02-22T19:38:21.606747+0000","last_peered":"2024-02-22T19:38:21.606747+0000","last_clean":"2024-02-22T19:38:21.606747+0000","last_became_active":"2024-02-22T19:38:13.194029+0000","last_became_peered":"2024-02-22T19:38:13.194029+0000","last_unstale":"2024-02-22T19:38:21.606747+0000","last_undegraded":"2024-02-22T19:38:21.606747+0000","last_fullsized":"2024-02-22T19:38:21.606747+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.578317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.578317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":372,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.193079+0000","last_change":"2024-02-22T19:38:07.259101+0000","last_active":"2024-02-22T19:38:22.193079+0000","last_peered":"2024-02-22T19:38:22.193079+0000","last_clean":"2024-02-22T19:38:22.193079+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:38:22.193079+0000","last_undegraded":"2024-02-22T19:38:22.193079+0000","last_fullsized":"2024-02-22T19:38:22.193079+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-02-22T19:35:20.843809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:20.843809+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606205+0000","last_change":"2024-02-22T19:38:13.191268+0000","last_active":"2024-02-22T19:38:21.606205+0000","last_peered":"2024-02-22T19:38:21.606205+0000","last_clean":"2024-02-22T19:38:21.606205+0000","last_became_active":"2024-02-22T19:38:13.190933+0000","last_became_peered":"2024-02-22T19:38:13.190933+0000","last_unstale":"2024-02-22T19:38:21.606205+0000","last_undegraded":"2024-02-22T19:38:21.606205+0000","last_fullsized":"2024-02-22T19:38:21.606205+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.590509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.590509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":229,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617052+0000","last_change":"2024-02-22T19:38:13.196721+0000","last_active":"2024-02-22T19:38:21.617052+0000","last_peered":"2024-02-22T19:38:21.617052+0000","last_clean":"2024-02-22T19:38:21.617052+0000","last_became_active":"2024-02-22T19:38:13.196313+0000","last_became_peered":"2024-02-22T19:38:13.196313+0000","last_unstale":"2024-02-22T19:38:21.617052+0000","last_undegraded":"2024-02-22T19:38:21.617052+0000","last_fullsized":"2024-02-22T19:38:21.617052+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:24.596531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.596531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"238'11109","reported_seq":11200,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:24.512133+0000","last_change":"2024-02-22T19:35:19.225962+0000","last_active":"2024-02-22T19:38:24.512133+0000","last_peered":"2024-02-22T19:38:24.512133+0000","last_clean":"2024-02-22T19:38:24.512133+0000","last_became_active":"2024-02-22T19:35:19.225604+0000","last_became_peered":"2024-02-22T19:35:19.225604+0000","last_unstale":"2024-02-22T19:38:24.512133+0000","last_undegraded":"2024-02-22T19:38:24.512133+0000","last_fullsized":"2024-02-22T19:38:24.512133+0000","mapping_epoch":154,"log_start":"183'6954","ondisk_log_start":"183'6954","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4155,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165871616,"num_objects":2531,"num_object_clones":0,"num_object_copies":7593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11109,"num_write_kb":436480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":197,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617560+0000","last_change":"2024-02-22T19:36:08.683152+0000","last_active":"2024-02-22T19:38:21.617560+0000","last_peered":"2024-02-22T19:38:21.617560+0000","last_clean":"2024-02-22T19:38:21.617560+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:38:21.617560+0000","last_undegraded":"2024-02-22T19:38:21.617560+0000","last_fullsized":"2024-02-22T19:38:21.617560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.683006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.683006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":205,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606367+0000","last_change":"2024-02-22T19:35:38.917238+0000","last_active":"2024-02-22T19:38:21.606367+0000","last_peered":"2024-02-22T19:38:21.606367+0000","last_clean":"2024-02-22T19:38:21.606367+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:38:21.606367+0000","last_undegraded":"2024-02-22T19:38:21.606367+0000","last_fullsized":"2024-02-22T19:38:21.606367+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:38.917177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.917177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610086+0000","last_change":"2024-02-22T19:37:57.186799+0000","last_active":"2024-02-22T19:38:21.610086+0000","last_peered":"2024-02-22T19:38:21.610086+0000","last_clean":"2024-02-22T19:38:21.610086+0000","last_became_active":"2024-02-22T19:37:57.186482+0000","last_became_peered":"2024-02-22T19:37:57.186482+0000","last_unstale":"2024-02-22T19:38:21.610086+0000","last_undegraded":"2024-02-22T19:38:21.610086+0000","last_fullsized":"2024-02-22T19:38:21.610086+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.185719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.185719+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":254,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.613214+0000","last_change":"2024-02-22T19:38:13.636670+0000","last_active":"2024-02-22T19:38:21.613214+0000","last_peered":"2024-02-22T19:38:21.613214+0000","last_clean":"2024-02-22T19:38:21.613214+0000","last_became_active":"2024-02-22T19:38:13.190303+0000","last_became_peered":"2024-02-22T19:38:13.190303+0000","last_unstale":"2024-02-22T19:38:21.613214+0000","last_undegraded":"2024-02-22T19:38:21.613214+0000","last_fullsized":"2024-02-22T19:38:21.613214+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-02-22T19:36:15.051953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.051953+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":245,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.614062+0000","last_change":"2024-02-22T19:38:09.219065+0000","last_active":"2024-02-22T19:38:21.614062+0000","last_peered":"2024-02-22T19:38:21.614062+0000","last_clean":"2024-02-22T19:38:21.614062+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:38:21.614062+0000","last_undegraded":"2024-02-22T19:38:21.614062+0000","last_fullsized":"2024-02-22T19:38:21.614062+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:49.141855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:49.141855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"238'11518","reported_seq":12189,"reported_epoch":238,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:38:24.281522+0000","last_change":"2024-02-22T19:38:14.276512+0000","last_active":"2024-02-22T19:38:24.281522+0000","last_peered":"2024-02-22T19:38:24.281522+0000","last_clean":"2024-02-22T19:37:56.653987+0000","last_became_active":"2024-02-22T19:38:14.189782+0000","last_became_peered":"2024-02-22T19:38:14.189782+0000","last_unstale":"2024-02-22T19:38:24.281522+0000","last_undegraded":"2024-02-22T19:38:24.281522+0000","last_fullsized":"2024-02-22T19:38:24.281522+0000","mapping_epoch":233,"log_start":"180'6284","ondisk_log_start":"180'6284","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'6381","last_scrub_stamp":"2024-02-22T19:36:52.260780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.260780+0000","log_size":5234,"ondisk_log_size":5234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":169213952,"num_objects":2582,"num_object_clones":0,"num_object_copies":7746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2354,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11518,"num_write_kb":451200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":18022400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,3,0],"avail_no_missing":["6","0","3"],"object_location_counts":[{"shards":"0,3,6","objects":2582}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610247+0000","last_change":"2024-02-22T19:37:57.174846+0000","last_active":"2024-02-22T19:38:21.610247+0000","last_peered":"2024-02-22T19:38:21.610247+0000","last_clean":"2024-02-22T19:38:21.610247+0000","last_became_active":"2024-02-22T19:37:57.174034+0000","last_became_peered":"2024-02-22T19:37:57.174034+0000","last_unstale":"2024-02-22T19:38:21.610247+0000","last_undegraded":"2024-02-22T19:38:21.610247+0000","last_fullsized":"2024-02-22T19:38:21.610247+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:22.280418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:22.280418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":229,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.616835+0000","last_change":"2024-02-22T19:38:13.197774+0000","last_active":"2024-02-22T19:38:21.616835+0000","last_peered":"2024-02-22T19:38:21.616835+0000","last_clean":"2024-02-22T19:38:21.616835+0000","last_became_active":"2024-02-22T19:38:13.197392+0000","last_became_peered":"2024-02-22T19:38:13.197392+0000","last_unstale":"2024-02-22T19:38:21.616835+0000","last_undegraded":"2024-02-22T19:38:21.616835+0000","last_fullsized":"2024-02-22T19:38:21.616835+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.598758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.598758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":215,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619972+0000","last_change":"2024-02-22T19:37:57.180939+0000","last_active":"2024-02-22T19:38:21.619972+0000","last_peered":"2024-02-22T19:38:21.619972+0000","last_clean":"2024-02-22T19:38:21.619972+0000","last_became_active":"2024-02-22T19:37:57.180638+0000","last_became_peered":"2024-02-22T19:37:57.180638+0000","last_unstale":"2024-02-22T19:38:21.619972+0000","last_undegraded":"2024-02-22T19:38:21.619972+0000","last_fullsized":"2024-02-22T19:38:21.619972+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:50.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:50.967435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"85'21","reported_seq":234,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611510+0000","last_change":"2024-02-22T19:37:47.421688+0000","last_active":"2024-02-22T19:38:21.611510+0000","last_peered":"2024-02-22T19:38:21.611510+0000","last_clean":"2024-02-22T19:38:21.611510+0000","last_became_active":"2024-02-22T19:29:03.447037+0000","last_became_peered":"2024-02-22T19:29:03.447037+0000","last_unstale":"2024-02-22T19:38:21.611510+0000","last_undegraded":"2024-02-22T19:38:21.611510+0000","last_fullsized":"2024-02-22T19:38:21.611510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:37:47.421622+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:47.421622+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":217,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.618390+0000","last_change":"2024-02-22T19:37:57.175982+0000","last_active":"2024-02-22T19:38:21.618390+0000","last_peered":"2024-02-22T19:38:21.618390+0000","last_clean":"2024-02-22T19:38:21.618390+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:38:21.618390+0000","last_undegraded":"2024-02-22T19:38:21.618390+0000","last_fullsized":"2024-02-22T19:38:21.618390+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:12.513648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:12.513648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":92,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.608887+0000","last_change":"2024-02-22T19:37:01.957962+0000","last_active":"2024-02-22T19:38:21.608887+0000","last_peered":"2024-02-22T19:38:21.608887+0000","last_clean":"2024-02-22T19:38:21.608887+0000","last_became_active":"2024-02-22T19:35:58.418408+0000","last_became_peered":"2024-02-22T19:35:58.418408+0000","last_unstale":"2024-02-22T19:38:21.608887+0000","last_undegraded":"2024-02-22T19:38:21.608887+0000","last_fullsized":"2024-02-22T19:38:21.608887+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:01.957854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:01.957854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"238'11384","reported_seq":11492,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.343996+0000","last_change":"2024-02-22T19:37:58.348143+0000","last_active":"2024-02-22T19:38:24.343996+0000","last_peered":"2024-02-22T19:38:24.343996+0000","last_clean":"2024-02-22T19:37:56.675559+0000","last_became_active":"2024-02-22T19:37:58.305838+0000","last_became_peered":"2024-02-22T19:37:58.305838+0000","last_unstale":"2024-02-22T19:38:24.343996+0000","last_undegraded":"2024-02-22T19:38:24.343996+0000","last_fullsized":"2024-02-22T19:38:24.343996+0000","mapping_epoch":218,"log_start":"181'6733","ondisk_log_start":"181'6733","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'7550","last_scrub_stamp":"2024-02-22T19:37:12.962216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:12.962216+0000","log_size":4651,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":158466048,"num_objects":2418,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2418,"num_objects_unfound":0,"num_objects_dirty":2418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11384,"num_write_kb":441664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":2418}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":217,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605977+0000","last_change":"2024-02-22T19:37:57.179467+0000","last_active":"2024-02-22T19:38:21.605977+0000","last_peered":"2024-02-22T19:38:21.605977+0000","last_clean":"2024-02-22T19:38:21.605977+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:38:21.605977+0000","last_undegraded":"2024-02-22T19:38:21.605977+0000","last_fullsized":"2024-02-22T19:38:21.605977+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.901512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.901512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605817+0000","last_change":"2024-02-22T19:38:13.194660+0000","last_active":"2024-02-22T19:38:21.605817+0000","last_peered":"2024-02-22T19:38:21.605817+0000","last_clean":"2024-02-22T19:38:21.605817+0000","last_became_active":"2024-02-22T19:38:13.194345+0000","last_became_peered":"2024-02-22T19:38:13.194345+0000","last_unstale":"2024-02-22T19:38:21.605817+0000","last_undegraded":"2024-02-22T19:38:21.605817+0000","last_fullsized":"2024-02-22T19:38:21.605817+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.342864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.342864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":781,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:23.227195+0000","last_change":"2024-02-22T19:38:13.805367+0000","last_active":"2024-02-22T19:38:23.227195+0000","last_peered":"2024-02-22T19:38:23.227195+0000","last_clean":"2024-02-22T19:38:23.227195+0000","last_became_active":"2024-02-22T19:38:13.205048+0000","last_became_peered":"2024-02-22T19:38:13.205048+0000","last_unstale":"2024-02-22T19:38:23.227195+0000","last_undegraded":"2024-02-22T19:38:23.227195+0000","last_fullsized":"2024-02-22T19:38:23.227195+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:36:20.553554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.553554+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":555,"num_read_kb":550,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":61,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.608454+0000","last_change":"2024-02-22T19:38:13.556278+0000","last_active":"2024-02-22T19:38:21.608454+0000","last_peered":"2024-02-22T19:38:21.608454+0000","last_clean":"2024-02-22T19:38:21.608454+0000","last_became_active":"2024-02-22T19:38:13.192027+0000","last_became_peered":"2024-02-22T19:38:13.192027+0000","last_unstale":"2024-02-22T19:38:21.608454+0000","last_undegraded":"2024-02-22T19:38:21.608454+0000","last_fullsized":"2024-02-22T19:38:21.608454+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:08.878452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.878452+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":57,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610305+0000","last_change":"2024-02-22T19:38:13.191347+0000","last_active":"2024-02-22T19:38:21.610305+0000","last_peered":"2024-02-22T19:38:21.610305+0000","last_clean":"2024-02-22T19:38:21.610305+0000","last_became_active":"2024-02-22T19:38:13.190347+0000","last_became_peered":"2024-02-22T19:38:13.190347+0000","last_unstale":"2024-02-22T19:38:21.610305+0000","last_undegraded":"2024-02-22T19:38:21.610305+0000","last_fullsized":"2024-02-22T19:38:21.610305+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.908714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.908714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611621+0000","last_change":"2024-02-22T19:38:13.202067+0000","last_active":"2024-02-22T19:38:21.611621+0000","last_peered":"2024-02-22T19:38:21.611621+0000","last_clean":"2024-02-22T19:38:21.611621+0000","last_became_active":"2024-02-22T19:38:13.201674+0000","last_became_peered":"2024-02-22T19:38:13.201674+0000","last_unstale":"2024-02-22T19:38:21.611621+0000","last_undegraded":"2024-02-22T19:38:21.611621+0000","last_fullsized":"2024-02-22T19:38:21.611621+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.230365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"238'11346","reported_seq":11438,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:24.490987+0000","last_change":"2024-02-22T19:35:19.745745+0000","last_active":"2024-02-22T19:38:24.490987+0000","last_peered":"2024-02-22T19:38:24.490987+0000","last_clean":"2024-02-22T19:38:24.490987+0000","last_became_active":"2024-02-22T19:35:19.745488+0000","last_became_peered":"2024-02-22T19:35:19.745488+0000","last_unstale":"2024-02-22T19:38:24.490987+0000","last_undegraded":"2024-02-22T19:38:24.490987+0000","last_fullsized":"2024-02-22T19:38:24.490987+0000","mapping_epoch":154,"log_start":"183'7179","ondisk_log_start":"183'7179","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4167,"ondisk_log_size":4167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165806080,"num_objects":2530,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":444032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":199,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.608014+0000","last_change":"2024-02-22T19:37:03.977945+0000","last_active":"2024-02-22T19:38:21.608014+0000","last_peered":"2024-02-22T19:38:21.608014+0000","last_clean":"2024-02-22T19:38:21.608014+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:38:21.608014+0000","last_undegraded":"2024-02-22T19:38:21.608014+0000","last_fullsized":"2024-02-22T19:38:21.608014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.977850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.977850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":223,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606200+0000","last_change":"2024-02-22T19:37:57.179302+0000","last_active":"2024-02-22T19:38:21.606200+0000","last_peered":"2024-02-22T19:38:21.606200+0000","last_clean":"2024-02-22T19:38:21.606200+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:38:21.606200+0000","last_undegraded":"2024-02-22T19:38:21.606200+0000","last_fullsized":"2024-02-22T19:38:21.606200+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.861921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.861921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":279,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609634+0000","last_change":"2024-02-22T19:38:13.887556+0000","last_active":"2024-02-22T19:38:21.609634+0000","last_peered":"2024-02-22T19:38:21.609634+0000","last_clean":"2024-02-22T19:38:21.609634+0000","last_became_active":"2024-02-22T19:38:13.191382+0000","last_became_peered":"2024-02-22T19:38:13.191382+0000","last_unstale":"2024-02-22T19:38:21.609634+0000","last_undegraded":"2024-02-22T19:38:21.609634+0000","last_fullsized":"2024-02-22T19:38:21.609634+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:35:35.426296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.426296+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":232,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611256+0000","last_change":"2024-02-22T19:38:13.197803+0000","last_active":"2024-02-22T19:38:21.611256+0000","last_peered":"2024-02-22T19:38:21.611256+0000","last_clean":"2024-02-22T19:38:21.611256+0000","last_became_active":"2024-02-22T19:38:13.197274+0000","last_became_peered":"2024-02-22T19:38:13.197274+0000","last_unstale":"2024-02-22T19:38:21.611256+0000","last_undegraded":"2024-02-22T19:38:21.611256+0000","last_fullsized":"2024-02-22T19:38:21.611256+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.441645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.441645+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":103,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611346+0000","last_change":"2024-02-22T19:36:07.161771+0000","last_active":"2024-02-22T19:38:21.611346+0000","last_peered":"2024-02-22T19:38:21.611346+0000","last_clean":"2024-02-22T19:38:21.611346+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:38:21.611346+0000","last_undegraded":"2024-02-22T19:38:21.611346+0000","last_fullsized":"2024-02-22T19:38:21.611346+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:07.161538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:07.161538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"238'11497","reported_seq":11838,"reported_epoch":238,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:38:24.389746+0000","last_change":"2024-02-22T19:38:14.320343+0000","last_active":"2024-02-22T19:38:24.389746+0000","last_peered":"2024-02-22T19:38:24.389746+0000","last_clean":"2024-02-22T19:38:12.115134+0000","last_became_active":"2024-02-22T19:38:14.271833+0000","last_became_peered":"2024-02-22T19:38:14.271833+0000","last_unstale":"2024-02-22T19:38:24.389746+0000","last_undegraded":"2024-02-22T19:38:24.389746+0000","last_fullsized":"2024-02-22T19:38:24.389746+0000","mapping_epoch":233,"log_start":"181'6521","ondisk_log_start":"181'6521","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'7961","last_scrub_stamp":"2024-02-22T19:37:18.264240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.264240+0000","log_size":4976,"ondisk_log_size":4976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":168230912,"num_objects":2567,"num_object_clones":0,"num_object_copies":7701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":2567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11497,"num_write_kb":450048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7667712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,0],"avail_no_missing":["2","0","4"],"object_location_counts":[{"shards":"0,2,4","objects":2567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615048+0000","last_change":"2024-02-22T19:38:13.201422+0000","last_active":"2024-02-22T19:38:21.615048+0000","last_peered":"2024-02-22T19:38:21.615048+0000","last_clean":"2024-02-22T19:38:21.615048+0000","last_became_active":"2024-02-22T19:38:13.201107+0000","last_became_peered":"2024-02-22T19:38:13.201107+0000","last_unstale":"2024-02-22T19:38:21.615048+0000","last_undegraded":"2024-02-22T19:38:21.615048+0000","last_fullsized":"2024-02-22T19:38:21.615048+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.610438+0000","last_change":"2024-02-22T19:37:57.190664+0000","last_active":"2024-02-22T19:38:21.610438+0000","last_peered":"2024-02-22T19:38:21.610438+0000","last_clean":"2024-02-22T19:38:21.610438+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:38:21.610438+0000","last_undegraded":"2024-02-22T19:38:21.610438+0000","last_fullsized":"2024-02-22T19:38:21.610438+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.261537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.261537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":260,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606881+0000","last_change":"2024-02-22T19:37:57.204273+0000","last_active":"2024-02-22T19:38:21.606881+0000","last_peered":"2024-02-22T19:38:21.606881+0000","last_clean":"2024-02-22T19:38:21.606881+0000","last_became_active":"2024-02-22T19:37:57.190027+0000","last_became_peered":"2024-02-22T19:37:57.190027+0000","last_unstale":"2024-02-22T19:38:21.606881+0000","last_undegraded":"2024-02-22T19:38:21.606881+0000","last_fullsized":"2024-02-22T19:38:21.606881+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:37.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.886787+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"234'11","reported_seq":43,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.198988+0000","last_change":"2024-02-22T19:38:13.288120+0000","last_active":"2024-02-22T19:38:22.198988+0000","last_peered":"2024-02-22T19:38:22.198988+0000","last_clean":"2024-02-22T19:38:22.198988+0000","last_became_active":"2024-02-22T19:38:13.196617+0000","last_became_peered":"2024-02-22T19:38:13.196617+0000","last_unstale":"2024-02-22T19:38:22.198988+0000","last_undegraded":"2024-02-22T19:38:22.198988+0000","last_fullsized":"2024-02-22T19:38:22.198988+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:36:23.658957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.658957+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619317+0000","last_change":"2024-02-22T19:37:57.179173+0000","last_active":"2024-02-22T19:38:21.619317+0000","last_peered":"2024-02-22T19:38:21.619317+0000","last_clean":"2024-02-22T19:38:21.619317+0000","last_became_active":"2024-02-22T19:37:57.178778+0000","last_became_peered":"2024-02-22T19:37:57.178778+0000","last_unstale":"2024-02-22T19:38:21.619317+0000","last_undegraded":"2024-02-22T19:38:21.619317+0000","last_fullsized":"2024-02-22T19:38:21.619317+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"238'11623","reported_seq":11730,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.398489+0000","last_change":"2024-02-22T19:37:58.324791+0000","last_active":"2024-02-22T19:38:24.398489+0000","last_peered":"2024-02-22T19:38:24.398489+0000","last_clean":"2024-02-22T19:37:56.634204+0000","last_became_active":"2024-02-22T19:37:58.288424+0000","last_became_peered":"2024-02-22T19:37:58.288424+0000","last_unstale":"2024-02-22T19:38:24.398489+0000","last_undegraded":"2024-02-22T19:38:24.398489+0000","last_fullsized":"2024-02-22T19:38:24.398489+0000","mapping_epoch":218,"log_start":"181'6664","ondisk_log_start":"181'6664","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'6967","last_scrub_stamp":"2024-02-22T19:37:01.648425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:01.648425+0000","log_size":4959,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":161742872,"num_objects":2469,"num_object_clones":0,"num_object_copies":7407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2469,"num_objects_unfound":0,"num_objects_dirty":2469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11623,"num_write_kb":450881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[2,7,6],"avail_no_missing":["2","6","7"],"object_location_counts":[{"shards":"2,6,7","objects":2469}],"blocked_by":[],"up_primary":5,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":197,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.617473+0000","last_change":"2024-02-22T19:36:11.554490+0000","last_active":"2024-02-22T19:38:21.617473+0000","last_peered":"2024-02-22T19:38:21.617473+0000","last_clean":"2024-02-22T19:38:21.617473+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:38:21.617473+0000","last_undegraded":"2024-02-22T19:38:21.617473+0000","last_fullsized":"2024-02-22T19:38:21.617473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.554382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.554382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.608253+0000","last_change":"2024-02-22T19:38:13.192992+0000","last_active":"2024-02-22T19:38:21.608253+0000","last_peered":"2024-02-22T19:38:21.608253+0000","last_clean":"2024-02-22T19:38:21.608253+0000","last_became_active":"2024-02-22T19:38:13.192492+0000","last_became_peered":"2024-02-22T19:38:13.192492+0000","last_unstale":"2024-02-22T19:38:21.608253+0000","last_undegraded":"2024-02-22T19:38:21.608253+0000","last_fullsized":"2024-02-22T19:38:21.608253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.680723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.680723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"85'21","reported_seq":276,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605796+0000","last_change":"2024-02-22T19:37:57.793821+0000","last_active":"2024-02-22T19:38:21.605796+0000","last_peered":"2024-02-22T19:38:21.605796+0000","last_clean":"2024-02-22T19:38:21.605796+0000","last_became_active":"2024-02-22T19:37:57.183291+0000","last_became_peered":"2024-02-22T19:37:57.183291+0000","last_unstale":"2024-02-22T19:38:21.605796+0000","last_undegraded":"2024-02-22T19:38:21.605796+0000","last_fullsized":"2024-02-22T19:38:21.605796+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:35:26.969827+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:35:26.969827+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":314,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.223192+0000","last_change":"2024-02-22T19:35:25.316903+0000","last_active":"2024-02-22T19:38:22.223192+0000","last_peered":"2024-02-22T19:38:22.223192+0000","last_clean":"2024-02-22T19:38:22.223192+0000","last_became_active":"2024-02-22T19:29:05.097110+0000","last_became_peered":"2024-02-22T19:29:05.097110+0000","last_unstale":"2024-02-22T19:38:22.223192+0000","last_undegraded":"2024-02-22T19:38:22.223192+0000","last_fullsized":"2024-02-22T19:38:22.223192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606034+0000","last_change":"2024-02-22T19:37:18.583096+0000","last_active":"2024-02-22T19:38:21.606034+0000","last_peered":"2024-02-22T19:38:21.606034+0000","last_clean":"2024-02-22T19:38:21.606034+0000","last_became_active":"2024-02-22T19:35:58.417154+0000","last_became_peered":"2024-02-22T19:35:58.417154+0000","last_unstale":"2024-02-22T19:38:21.606034+0000","last_undegraded":"2024-02-22T19:38:21.606034+0000","last_fullsized":"2024-02-22T19:38:21.606034+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.583038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.583038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"238'11475","reported_seq":11810,"reported_epoch":238,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:38:24.496322+0000","last_change":"2024-02-22T19:38:14.347287+0000","last_active":"2024-02-22T19:38:24.496322+0000","last_peered":"2024-02-22T19:38:24.496322+0000","last_clean":"2024-02-22T19:38:12.154617+0000","last_became_active":"2024-02-22T19:38:14.277323+0000","last_became_peered":"2024-02-22T19:38:14.277323+0000","last_unstale":"2024-02-22T19:38:24.496322+0000","last_undegraded":"2024-02-22T19:38:24.496322+0000","last_fullsized":"2024-02-22T19:38:24.496322+0000","mapping_epoch":233,"log_start":"183'7318","ondisk_log_start":"183'7318","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4157,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160628736,"num_objects":2451,"num_object_clones":0,"num_object_copies":7353,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2354,"num_objects_unfound":0,"num_objects_dirty":2451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":445632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":7733248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,0],"avail_no_missing":["3","0","7"],"object_location_counts":[{"shards":"0,3,7","objects":2451}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":199,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606473+0000","last_change":"2024-02-22T19:36:20.739814+0000","last_active":"2024-02-22T19:38:21.606473+0000","last_peered":"2024-02-22T19:38:21.606473+0000","last_clean":"2024-02-22T19:38:21.606473+0000","last_became_active":"2024-02-22T19:29:07.094120+0000","last_became_peered":"2024-02-22T19:29:07.094120+0000","last_unstale":"2024-02-22T19:38:21.606473+0000","last_undegraded":"2024-02-22T19:38:21.606473+0000","last_fullsized":"2024-02-22T19:38:21.606473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":211,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.626981+0000","last_change":"2024-02-22T19:38:13.197433+0000","last_active":"2024-02-22T19:38:21.626981+0000","last_peered":"2024-02-22T19:38:21.626981+0000","last_clean":"2024-02-22T19:38:21.626981+0000","last_became_active":"2024-02-22T19:38:13.197072+0000","last_became_peered":"2024-02-22T19:38:13.197072+0000","last_unstale":"2024-02-22T19:38:21.626981+0000","last_undegraded":"2024-02-22T19:38:21.626981+0000","last_fullsized":"2024-02-22T19:38:21.626981+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:37:06.317125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":261,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615360+0000","last_change":"2024-02-22T19:38:14.244177+0000","last_active":"2024-02-22T19:38:21.615360+0000","last_peered":"2024-02-22T19:38:21.615360+0000","last_clean":"2024-02-22T19:38:21.615360+0000","last_became_active":"2024-02-22T19:38:13.198160+0000","last_became_peered":"2024-02-22T19:38:13.198160+0000","last_unstale":"2024-02-22T19:38:21.615360+0000","last_undegraded":"2024-02-22T19:38:21.615360+0000","last_fullsized":"2024-02-22T19:38:21.615360+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":201,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605684+0000","last_change":"2024-02-22T19:35:36.900633+0000","last_active":"2024-02-22T19:38:21.605684+0000","last_peered":"2024-02-22T19:38:21.605684+0000","last_clean":"2024-02-22T19:38:21.605684+0000","last_became_active":"2024-02-22T19:29:05.100567+0000","last_became_peered":"2024-02-22T19:29:05.100567+0000","last_unstale":"2024-02-22T19:38:21.605684+0000","last_undegraded":"2024-02-22T19:38:21.605684+0000","last_fullsized":"2024-02-22T19:38:21.605684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609603+0000","last_change":"2024-02-22T19:37:57.190227+0000","last_active":"2024-02-22T19:38:21.609603+0000","last_peered":"2024-02-22T19:38:21.609603+0000","last_clean":"2024-02-22T19:38:21.609603+0000","last_became_active":"2024-02-22T19:37:57.189767+0000","last_became_peered":"2024-02-22T19:37:57.189767+0000","last_unstale":"2024-02-22T19:38:21.609603+0000","last_undegraded":"2024-02-22T19:38:21.609603+0000","last_fullsized":"2024-02-22T19:38:21.609603+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.071500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.071500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3","version":"238'11257","reported_seq":11372,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.353584+0000","last_change":"2024-02-22T19:37:58.345676+0000","last_active":"2024-02-22T19:38:24.353584+0000","last_peered":"2024-02-22T19:38:24.353584+0000","last_clean":"2024-02-22T19:37:56.636687+0000","last_became_active":"2024-02-22T19:37:58.288736+0000","last_became_peered":"2024-02-22T19:37:58.288736+0000","last_unstale":"2024-02-22T19:38:24.353584+0000","last_undegraded":"2024-02-22T19:38:24.353584+0000","last_fullsized":"2024-02-22T19:38:24.353584+0000","mapping_epoch":218,"log_start":"181'6603","ondisk_log_start":"181'6603","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'9564","last_scrub_stamp":"2024-02-22T19:37:47.213288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:47.213288+0000","log_size":4654,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":163643392,"num_objects":2497,"num_object_clones":0,"num_object_copies":7491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2497,"num_objects_unfound":0,"num_objects_dirty":2497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11257,"num_write_kb":440128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,6,2],"avail_no_missing":["1","2","6"],"object_location_counts":[{"shards":"1,2,6","objects":2497}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609628+0000","last_change":"2024-02-22T19:38:13.188904+0000","last_active":"2024-02-22T19:38:21.609628+0000","last_peered":"2024-02-22T19:38:21.609628+0000","last_clean":"2024-02-22T19:38:21.609628+0000","last_became_active":"2024-02-22T19:38:13.188424+0000","last_became_peered":"2024-02-22T19:38:13.188424+0000","last_unstale":"2024-02-22T19:38:21.609628+0000","last_undegraded":"2024-02-22T19:38:21.609628+0000","last_fullsized":"2024-02-22T19:38:21.609628+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.979469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.979469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"236'19","reported_seq":412,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.188850+0000","last_change":"2024-02-22T19:38:16.222206+0000","last_active":"2024-02-22T19:38:22.188850+0000","last_peered":"2024-02-22T19:38:22.188850+0000","last_clean":"2024-02-22T19:38:22.188850+0000","last_became_active":"2024-02-22T19:38:16.221894+0000","last_became_peered":"2024-02-22T19:38:16.221894+0000","last_unstale":"2024-02-22T19:38:22.188850+0000","last_undegraded":"2024-02-22T19:38:22.188850+0000","last_fullsized":"2024-02-22T19:38:22.188850+0000","mapping_epoch":235,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":202,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.615281+0000","last_change":"2024-02-22T19:35:27.352610+0000","last_active":"2024-02-22T19:38:21.615281+0000","last_peered":"2024-02-22T19:38:21.615281+0000","last_clean":"2024-02-22T19:38:21.615281+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:38:21.615281+0000","last_undegraded":"2024-02-22T19:38:21.615281+0000","last_fullsized":"2024-02-22T19:38:21.615281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:27.352545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.352545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":95,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.607772+0000","last_change":"2024-02-22T19:36:16.249180+0000","last_active":"2024-02-22T19:38:21.607772+0000","last_peered":"2024-02-22T19:38:21.607772+0000","last_clean":"2024-02-22T19:38:21.607772+0000","last_became_active":"2024-02-22T19:36:16.154859+0000","last_became_peered":"2024-02-22T19:36:16.154859+0000","last_unstale":"2024-02-22T19:38:21.607772+0000","last_undegraded":"2024-02-22T19:38:21.607772+0000","last_fullsized":"2024-02-22T19:38:21.607772+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:35:24.795027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:24.795027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"238'11328","reported_seq":11442,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.196307+0000","last_change":"2024-02-22T19:38:14.226701+0000","last_active":"2024-02-22T19:38:24.196307+0000","last_peered":"2024-02-22T19:38:24.196307+0000","last_clean":"2024-02-22T19:37:56.619086+0000","last_became_active":"2024-02-22T19:38:14.184964+0000","last_became_peered":"2024-02-22T19:38:14.184964+0000","last_unstale":"2024-02-22T19:38:24.196307+0000","last_undegraded":"2024-02-22T19:38:24.196307+0000","last_fullsized":"2024-02-22T19:38:24.196307+0000","mapping_epoch":233,"log_start":"181'6781","ondisk_log_start":"181'6781","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4547,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165281792,"num_objects":2522,"num_object_clones":0,"num_object_copies":7566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2522,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":443200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,3,0],"avail_no_missing":["4","0","3"],"object_location_counts":[{"shards":"0,3,4","objects":2522}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":209,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.619641+0000","last_change":"2024-02-22T19:37:57.180115+0000","last_active":"2024-02-22T19:38:21.619641+0000","last_peered":"2024-02-22T19:38:21.619641+0000","last_clean":"2024-02-22T19:38:21.619641+0000","last_became_active":"2024-02-22T19:37:57.179488+0000","last_became_peered":"2024-02-22T19:37:57.179488+0000","last_unstale":"2024-02-22T19:38:21.619641+0000","last_undegraded":"2024-02-22T19:38:21.619641+0000","last_fullsized":"2024-02-22T19:38:21.619641+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"221'14","reported_seq":162,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.199276+0000","last_change":"2024-02-22T19:38:00.199707+0000","last_active":"2024-02-22T19:38:22.199276+0000","last_peered":"2024-02-22T19:38:22.199276+0000","last_clean":"2024-02-22T19:38:22.199276+0000","last_became_active":"2024-02-22T19:38:00.199317+0000","last_became_peered":"2024-02-22T19:38:00.199317+0000","last_unstale":"2024-02-22T19:38:22.199276+0000","last_undegraded":"2024-02-22T19:38:22.199276+0000","last_fullsized":"2024-02-22T19:38:22.199276+0000","mapping_epoch":220,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:36:53.427951+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.427951+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":217,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.605707+0000","last_change":"2024-02-22T19:36:08.709226+0000","last_active":"2024-02-22T19:38:21.605707+0000","last_peered":"2024-02-22T19:38:21.605707+0000","last_clean":"2024-02-22T19:38:21.605707+0000","last_became_active":"2024-02-22T19:29:01.002636+0000","last_became_peered":"2024-02-22T19:29:01.002636+0000","last_unstale":"2024-02-22T19:38:21.605707+0000","last_undegraded":"2024-02-22T19:38:21.605707+0000","last_fullsized":"2024-02-22T19:38:21.605707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.709163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.709163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":222,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606897+0000","last_change":"2024-02-22T19:36:11.687594+0000","last_active":"2024-02-22T19:38:21.606897+0000","last_peered":"2024-02-22T19:38:21.606897+0000","last_clean":"2024-02-22T19:38:21.606897+0000","last_became_active":"2024-02-22T19:29:03.081361+0000","last_became_peered":"2024-02-22T19:29:03.081361+0000","last_unstale":"2024-02-22T19:38:21.606897+0000","last_undegraded":"2024-02-22T19:38:21.606897+0000","last_fullsized":"2024-02-22T19:38:21.606897+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:36:11.687522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.687522+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":30,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.609227+0000","last_change":"2024-02-22T19:37:57.189860+0000","last_active":"2024-02-22T19:38:21.609227+0000","last_peered":"2024-02-22T19:38:21.609227+0000","last_clean":"2024-02-22T19:38:21.609227+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:38:21.609227+0000","last_undegraded":"2024-02-22T19:38:21.609227+0000","last_fullsized":"2024-02-22T19:38:21.609227+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.022678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.022678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"238'11522","reported_seq":14180,"reported_epoch":238,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:38:24.396262+0000","last_change":"2024-02-22T19:38:14.331923+0000","last_active":"2024-02-22T19:38:24.396262+0000","last_peered":"2024-02-22T19:38:24.396262+0000","last_clean":"2024-02-22T19:38:12.171312+0000","last_became_active":"2024-02-22T19:38:14.276783+0000","last_became_peered":"2024-02-22T19:38:14.276783+0000","last_unstale":"2024-02-22T19:38:24.396262+0000","last_undegraded":"2024-02-22T19:38:24.396262+0000","last_fullsized":"2024-02-22T19:38:24.396262+0000","mapping_epoch":233,"log_start":"181'6824","ondisk_log_start":"181'6824","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"216'10279","last_scrub_stamp":"2024-02-22T19:37:56.009518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.009518+0000","log_size":4698,"ondisk_log_size":4698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":163577856,"num_objects":2496,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1329,"num_objects_unfound":0,"num_objects_dirty":2496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11522,"num_write_kb":448576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1275,"num_bytes_recovered":83558400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,0],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"0,1,2","objects":2496}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":204,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.613052+0000","last_change":"2024-02-22T19:37:56.312793+0000","last_active":"2024-02-22T19:38:21.613052+0000","last_peered":"2024-02-22T19:38:21.613052+0000","last_clean":"2024-02-22T19:38:21.613052+0000","last_became_active":"2024-02-22T19:29:07.102726+0000","last_became_peered":"2024-02-22T19:29:07.102726+0000","last_unstale":"2024-02-22T19:38:21.613052+0000","last_undegraded":"2024-02-22T19:38:21.613052+0000","last_fullsized":"2024-02-22T19:38:21.613052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.312692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.312692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":228,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606424+0000","last_change":"2024-02-22T19:38:13.190616+0000","last_active":"2024-02-22T19:38:21.606424+0000","last_peered":"2024-02-22T19:38:21.606424+0000","last_clean":"2024-02-22T19:38:21.606424+0000","last_became_active":"2024-02-22T19:38:13.190198+0000","last_became_peered":"2024-02-22T19:38:13.190198+0000","last_unstale":"2024-02-22T19:38:21.606424+0000","last_undegraded":"2024-02-22T19:38:21.606424+0000","last_fullsized":"2024-02-22T19:38:21.606424+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:09.885998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.885998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":207,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.611084+0000","last_change":"2024-02-22T19:36:28.180370+0000","last_active":"2024-02-22T19:38:21.611084+0000","last_peered":"2024-02-22T19:38:21.611084+0000","last_clean":"2024-02-22T19:38:21.611084+0000","last_became_active":"2024-02-22T19:29:01.005390+0000","last_became_peered":"2024-02-22T19:29:01.005390+0000","last_unstale":"2024-02-22T19:38:21.611084+0000","last_undegraded":"2024-02-22T19:38:21.611084+0000","last_fullsized":"2024-02-22T19:38:21.611084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.180297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.180297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":233,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606415+0000","last_change":"2024-02-22T19:37:48.379855+0000","last_active":"2024-02-22T19:38:21.606415+0000","last_peered":"2024-02-22T19:38:21.606415+0000","last_clean":"2024-02-22T19:38:21.606415+0000","last_became_active":"2024-02-22T19:29:03.080664+0000","last_became_peered":"2024-02-22T19:29:03.080664+0000","last_unstale":"2024-02-22T19:38:21.606415+0000","last_undegraded":"2024-02-22T19:38:21.606415+0000","last_fullsized":"2024-02-22T19:38:21.606415+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:37:48.379789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:48.379789+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":220,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.616626+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:38:21.616626+0000","last_peered":"2024-02-22T19:38:21.616626+0000","last_clean":"2024-02-22T19:38:21.616626+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:38:21.616626+0000","last_undegraded":"2024-02-22T19:38:21.616626+0000","last_fullsized":"2024-02-22T19:38:21.616626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"238'11369","reported_seq":11823,"reported_epoch":238,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:38:24.462576+0000","last_change":"2024-02-22T19:38:13.892663+0000","last_active":"2024-02-22T19:38:24.462576+0000","last_peered":"2024-02-22T19:38:24.462576+0000","last_clean":"2024-02-22T19:37:56.624892+0000","last_became_active":"2024-02-22T19:37:58.267019+0000","last_became_peered":"2024-02-22T19:37:58.267019+0000","last_unstale":"2024-02-22T19:38:24.462576+0000","last_undegraded":"2024-02-22T19:38:24.462576+0000","last_fullsized":"2024-02-22T19:38:24.462576+0000","mapping_epoch":218,"log_start":"183'7204","ondisk_log_start":"183'7204","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:18.211860+0000","last_clean_scrub_stamp":"2024-02-22T19:35:18.211860+0000","log_size":4165,"ondisk_log_size":4165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":162070528,"num_objects":2473,"num_object_clones":0,"num_object_copies":7419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2338,"num_objects_unfound":0,"num_objects_dirty":2473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11369,"num_write_kb":442944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":11337728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,1,4],"avail_no_missing":["3","1","4"],"object_location_counts":[{"shards":"1,3,4","objects":2473}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":198,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.607081+0000","last_change":"2024-02-22T19:34:54.019363+0000","last_active":"2024-02-22T19:38:21.607081+0000","last_peered":"2024-02-22T19:38:21.607081+0000","last_clean":"2024-02-22T19:38:21.607081+0000","last_became_active":"2024-02-22T19:29:07.099277+0000","last_became_peered":"2024-02-22T19:29:07.099277+0000","last_unstale":"2024-02-22T19:38:21.607081+0000","last_undegraded":"2024-02-22T19:38:21.607081+0000","last_fullsized":"2024-02-22T19:38:21.607081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:34:54.019270+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:34:54.019270+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":389,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-02-22T19:38:22.198475+0000","last_change":"2024-02-22T19:38:00.221092+0000","last_active":"2024-02-22T19:38:22.198475+0000","last_peered":"2024-02-22T19:38:22.198475+0000","last_clean":"2024-02-22T19:38:22.198475+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:38:22.198475+0000","last_undegraded":"2024-02-22T19:38:22.198475+0000","last_fullsized":"2024-02-22T19:38:22.198475+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:37:13.243499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:13.243499+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":224,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.614908+0000","last_change":"2024-02-22T19:38:13.892378+0000","last_active":"2024-02-22T19:38:21.614908+0000","last_peered":"2024-02-22T19:38:21.614908+0000","last_clean":"2024-02-22T19:38:21.614908+0000","last_became_active":"2024-02-22T19:38:13.199183+0000","last_became_peered":"2024-02-22T19:38:13.199183+0000","last_unstale":"2024-02-22T19:38:21.614908+0000","last_undegraded":"2024-02-22T19:38:21.614908+0000","last_fullsized":"2024-02-22T19:38:21.614908+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:37:05.357880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.357880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":213,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-02-22T19:38:21.606052+0000","last_change":"2024-02-22T19:35:33.971551+0000","last_active":"2024-02-22T19:38:21.606052+0000","last_peered":"2024-02-22T19:38:21.606052+0000","last_clean":"2024-02-22T19:38:21.606052+0000","last_became_active":"2024-02-22T19:29:03.081334+0000","last_became_peered":"2024-02-22T19:29:03.081334+0000","last_unstale":"2024-02-22T19:38:21.606052+0000","last_undegraded":"2024-02-22T19:38:21.606052+0000","last_fullsized":"2024-02-22T19:38:21.606052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:33.971446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.971446+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":2627600408,"num_objects":40095,"num_object_clones":0,"num_object_copies":120285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27457,"num_objects_unfound":0,"num_objects_dirty":40095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183369,"num_write_kb":7150785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2771,"num_bytes_recovered":181272576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8046653440,"data_stored":8046641224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75750,"ondisk_log_size":75750,"up":48,"acting":48,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":555,"num_read_kb":550,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266804,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1118008,"kb_used_data":812528,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92627144,"statfs":{"total":95995035648,"available":94850195456,"internally_reserved":0,"allocated":832028672,"data_stored":831509045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.472,"15min":1.1839999999999999},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":5.4279999999999999,"5min":7.0700000000000003,"15min":42.597999999999999},"last":2.4100000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.734,"15min":1.1379999999999999},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":4.3499999999999996,"5min":9.8870000000000005,"15min":42.155999999999999},"last":0.223}]},{"osd":1,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.593,"15min":1.1140000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":5.5549999999999997,"5min":8.0679999999999996,"15min":38.188000000000002},"last":3.5329999999999999},{"interface":"front","average":{"1min":2.625,"5min":1.948,"15min":1.2669999999999999},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.219},"max":{"1min":8.3109999999999999,"5min":9.7520000000000007,"15min":37.938000000000002},"last":2.2999999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":1.819,"5min":1.258,"15min":1.0149999999999999},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.1550000000000002,"5min":6.5419999999999998,"15min":41.640999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":2.173,"5min":1.4590000000000001,"15min":1.139},"min":{"1min":0.25700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":7.3579999999999997,"5min":8.2929999999999993,"15min":41.722000000000001},"last":7.8609999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":1.9870000000000001,"5min":1.607,"15min":1.1799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.8120000000000003,"5min":8.0570000000000004,"15min":41.457000000000001},"last":2.0470000000000002},{"interface":"front","average":{"1min":2.3959999999999999,"5min":1.7290000000000001,"15min":1.1859999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":7.6950000000000003,"5min":7.6950000000000003,"15min":41.311999999999998},"last":2.286}]},{"osd":4,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67200000000000004,"15min":0.52400000000000002},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.23999999999999999},"max":{"1min":2.653,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.48999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":1.9390000000000001,"5min":3.4780000000000002,"15min":3.4780000000000002},"last":0.29399999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.69499999999999995,"15min":0.55200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":5.5010000000000003,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.69999999999999996,"15min":0.57399999999999995},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.5310000000000001,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.317}]},{"osd":6,"last update":"Thu Feb 22 19:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.78100000000000003,"15min":0.65000000000000002},"min":{"1min":0.32000000000000001,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":3.3180000000000001,"5min":4.7160000000000002,"15min":12.199999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.974,"5min":3.4430000000000001,"15min":12.067},"last":0.35299999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528441,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1442964,"kb_used_data":1133260,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92302188,"statfs":{"total":95995035648,"available":94517440512,"internally_reserved":0,"allocated":1160458240,"data_stored":1159974077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.718,"15min":0.92900000000000005},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":3.6749999999999998,"5min":17.885000000000002,"15min":17.885000000000002},"last":1.875},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.369,"15min":0.86499999999999999},"min":{"1min":0.218,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.9399999999999999,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":2.0790000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":2.2469999999999999,"5min":1.5129999999999999,"15min":0.88100000000000001},"min":{"1min":0.25900000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":12.808999999999999,"5min":12.808999999999999,"15min":12.808999999999999},"last":3.415},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.2729999999999999,"15min":0.80600000000000005},"min":{"1min":0.27300000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.1470000000000002,"5min":7.8410000000000002,"15min":7.8410000000000002},"last":1.8380000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":1.7450000000000001,"15min":1.014},"min":{"1min":0.253,"5min":0.249,"15min":0.249},"max":{"1min":6.0119999999999996,"5min":9.2319999999999993,"15min":12.859999999999999},"last":2.2210000000000001},{"interface":"front","average":{"1min":1.998,"5min":1.6080000000000001,"15min":0.93000000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.9589999999999996,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.5740000000000001,"15min":1.0800000000000001},"min":{"1min":0.34100000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.3339999999999996,"5min":7.327,"15min":38.762},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.956,"5min":1.913,"15min":1.2090000000000001},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.4269999999999996,"5min":24.242999999999999,"15min":35.612000000000002},"last":2.0390000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48299999999999998,"15min":0.44},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.907},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.54800000000000004,"15min":0.47399999999999998},"min":{"1min":0.251,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":2.1419999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.61499999999999999,"15min":0.5},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.302,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":2.1139999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.50800000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27800000000000002},"max":{"1min":1.752,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":2.1829999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63200000000000001,"15min":0.54800000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.736,"5min":3.488,"15min":3.488},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72199999999999998,"15min":0.623},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.36199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1436064,"kb_used_data":1126488,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92309088,"statfs":{"total":95995035648,"available":94524506112,"internally_reserved":0,"allocated":1153523712,"data_stored":1152964840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.98,"5min":2.444,"15min":1.1699999999999999},"min":{"1min":0.32600000000000001,"5min":0.20699999999999999,"15min":0.20499999999999999},"max":{"1min":5.3769999999999998,"5min":15.613,"15min":15.613},"last":0.185},{"interface":"front","average":{"1min":1.9890000000000001,"5min":2.5350000000000001,"15min":1.276},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.6859999999999999,"5min":13.515000000000001,"15min":13.515000000000001},"last":0.17499999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:38:07 2024","interfaces":[{"interface":"back","average":{"1min":1.8380000000000001,"5min":2.129,"15min":1.2809999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.0830000000000002,"5min":16.503,"15min":16.503},"last":0.30599999999999999},{"interface":"front","average":{"1min":1.9470000000000001,"5min":2.395,"15min":1.3759999999999999},"min":{"1min":0.22700000000000001,"5min":0.192,"15min":0.192},"max":{"1min":5.7549999999999999,"5min":14.994,"15min":14.994},"last":9.9559999999999995}]},{"osd":3,"last update":"Thu Feb 22 19:37:40 2024","interfaces":[{"interface":"back","average":{"1min":1.855,"5min":2.5150000000000001,"15min":1.3540000000000001},"min":{"1min":0.253,"5min":0.214,"15min":0.214},"max":{"1min":6.3449999999999998,"5min":11.932,"15min":25.145},"last":5.476},{"interface":"front","average":{"1min":3.1779999999999999,"5min":2.5419999999999998,"15min":1.431},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":7.9029999999999996,"5min":12.154999999999999,"15min":22.25},"last":4.9820000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.571,"15min":1.0509999999999999},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.3380000000000001,"5min":8.1449999999999996,"15min":15.542999999999999},"last":0.24399999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.6279999999999999,"15min":0.95999999999999996},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":3.7090000000000001,"5min":10.706,"15min":10.706},"last":2.3159999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:38:21 2024","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.51,"15min":1.1000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":8.0579999999999998,"5min":8.2189999999999994,"15min":15.587999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.5509999999999999,"15min":1.048},"min":{"1min":0.34300000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.0469999999999997,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.34300000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.7330000000000001,"15min":0.98199999999999998},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.7410000000000001,"5min":10.631,"15min":10.631},"last":2.3530000000000002},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.643,"15min":1.0669999999999999},"min":{"1min":0.40300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3799999999999999,"5min":8.1880000000000006,"15min":15.651999999999999},"last":0.379}]},{"osd":7,"last update":"Thu Feb 22 19:38:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.798,"15min":1.0980000000000001},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":8.2279999999999998,"5min":13.169,"15min":13.169},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.6830000000000001,"15min":1.018},"min":{"1min":0.45000000000000001,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":6.3789999999999996,"5min":10.673,"15min":10.673},"last":2.3849999999999998}]}]},{"osd":0,"up_from":0,"seq":34359738519,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411468,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1485168,"kb_used_data":1171560,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":92259984,"statfs":{"total":95995035648,"available":94474223616,"internally_reserved":0,"allocated":1199677440,"data_stored":1199165411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:38:09 2024","interfaces":[{"interface":"back","average":{"1min":2.4129999999999998,"5min":1.875,"15min":1.1379999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":13.954000000000001,"5min":22.338999999999999,"15min":22.338999999999999},"last":6.4009999999999998},{"interface":"front","average":{"1min":2.6619999999999999,"5min":2.3490000000000002,"15min":1.3779999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.4009999999999998,"5min":22.170000000000002,"15min":22.170000000000002},"last":2.4740000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:38:09 2024","interfaces":[{"interface":"back","average":{"1min":2.1480000000000001,"5min":2.2200000000000002,"15min":1.1870000000000001},"min":{"1min":0.24299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.109,"5min":9.9030000000000005,"15min":10.183},"last":3.863},{"interface":"front","average":{"1min":2.786,"5min":2.1259999999999999,"15min":1.0900000000000001},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.193},"max":{"1min":9.3520000000000003,"5min":12.393000000000001,"15min":12.393000000000001},"last":3.2160000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:37:47 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":2.5049999999999999,"15min":1.3049999999999999},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":22.302,"5min":30.071000000000002,"15min":30.071000000000002},"last":11.038},{"interface":"front","average":{"1min":2.5249999999999999,"5min":2.7120000000000002,"15min":1.393},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":22.204000000000001,"5min":32.119,"15min":32.119},"last":2.387}]},{"osd":4,"last update":"Thu Feb 22 19:38:00 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.5980000000000001,"15min":1.024},"min":{"1min":0.37,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.5910000000000002,"5min":6.3890000000000002,"15min":6.3890000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.476,"5min":1.224,"15min":0.82999999999999996},"min":{"1min":0.28000000000000003,"5min":0.253,"15min":0.253},"max":{"1min":6.9850000000000003,"5min":6.9850000000000003,"15min":9.5199999999999996},"last":3.2440000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:38:17 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.373,"15min":0.91500000000000004},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":7.2859999999999996,"5min":7.2859999999999996,"15min":9.6300000000000008},"last":3.274},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.681,"15min":1.085},"min":{"1min":0.309,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.6840000000000002,"5min":6.4320000000000004,"15min":6.4320000000000004},"last":3.8980000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.8260000000000001,"15min":1.0960000000000001},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":5.6660000000000004,"5min":9.9689999999999994,"15min":9.9689999999999994},"last":6.4100000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.2470000000000001,"15min":0.86299999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":9.5990000000000002},"last":3.2970000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:38:09 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.8320000000000001,"15min":1.119},"min":{"1min":0.42099999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":5.6340000000000003,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":0.80300000000000005},{"interface":"front","average":{"1min":2.153,"5min":1.5549999999999999,"15min":0.96199999999999997},"min":{"1min":0.438,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.2560000000000002,"5min":9.2810000000000006,"15min":9.5660000000000007},"last":6.367}]}]},{"osd":3,"up_from":26,"seq":111669149830,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1763712,"kb_used_data":1462200,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":91981440,"statfs":{"total":95995035648,"available":94188994560,"internally_reserved":0,"allocated":1497292800,"data_stored":1496765793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:32 2024","interfaces":[{"interface":"back","average":{"1min":3.6850000000000001,"5min":2.7050000000000001,"15min":1.3089999999999999},"min":{"1min":0.498,"5min":0.19400000000000001,"15min":0.193},"max":{"1min":23.48,"5min":23.48,"15min":23.48},"last":0.753},{"interface":"front","average":{"1min":3.4529999999999998,"5min":3.04,"15min":1.47},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":23.411999999999999,"5min":23.411999999999999,"15min":23.411999999999999},"last":2.387}]},{"osd":1,"last update":"Thu Feb 22 19:37:32 2024","interfaces":[{"interface":"back","average":{"1min":3.1230000000000002,"5min":2.895,"15min":1.452},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":11.204000000000001,"5min":18.259,"15min":18.259},"last":3.6779999999999999},{"interface":"front","average":{"1min":3.0009999999999999,"5min":2.2090000000000001,"15min":1.1759999999999999},"min":{"1min":0.31900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":16.943999999999999,"5min":16.943999999999999,"15min":16.943999999999999},"last":3.4900000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:37:32 2024","interfaces":[{"interface":"back","average":{"1min":2.714,"5min":2.6419999999999999,"15min":1.369},"min":{"1min":0.312,"5min":0.29699999999999999,"15min":0.27400000000000002},"max":{"1min":10.893000000000001,"5min":16.271000000000001,"15min":16.271000000000001},"last":2.9980000000000002},{"interface":"front","average":{"1min":2.8159999999999998,"5min":2.4510000000000001,"15min":1.2949999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.217},"max":{"1min":9.2799999999999994,"5min":15.143000000000001,"15min":15.143000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Thu Feb 22 19:37:55 2024","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":2.028,"15min":1.214},"min":{"1min":0.26300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":11.148999999999999,"5min":13.680999999999999,"15min":13.680999999999999},"last":2.5190000000000001},{"interface":"front","average":{"1min":2.0579999999999998,"5min":1.6839999999999999,"15min":1.0820000000000001},"min":{"1min":0.29799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":10.756,"5min":11.366,"15min":11.366},"last":3.0299999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:37:22 2024","interfaces":[{"interface":"back","average":{"1min":2.266,"5min":1.9039999999999999,"15min":1.083},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":8.0289999999999999,"5min":12.879,"15min":12.879},"last":0.82199999999999995},{"interface":"front","average":{"1min":2.016,"5min":2.1030000000000002,"15min":1.135},"min":{"1min":0.441,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":11.167999999999999,"5min":13.707000000000001,"15min":13.707000000000001},"last":1.216}]},{"osd":6,"last update":"Thu Feb 22 19:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.8009999999999999,"15min":1.0760000000000001},"min":{"1min":0.39300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":6.5279999999999996,"5min":12.898999999999999,"15min":12.898999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":2.3929999999999998,"5min":2.2109999999999999,"15min":1.2809999999999999},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":11.121,"5min":13.803000000000001,"15min":13.803000000000001},"last":1.3029999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.6140000000000001,"15min":1.089},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":10.708,"5min":11.348000000000001,"15min":11.348000000000001},"last":2.9580000000000002},{"interface":"front","average":{"1min":1.758,"5min":1.7749999999999999,"15min":1.135},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.4939999999999998,"5min":12.846,"15min":12.846},"last":0.84999999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953604,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1288052,"kb_used_data":982508,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92457100,"statfs":{"total":95995035648,"available":94676070400,"internally_reserved":0,"allocated":1006088192,"data_stored":1005573013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:58 2024","interfaces":[{"interface":"back","average":{"1min":5.1589999999999998,"5min":2.367,"15min":1.2549999999999999},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":69.251999999999995,"5min":69.251999999999995,"15min":69.251999999999995},"last":0.25},{"interface":"front","average":{"1min":5.056,"5min":2.1459999999999999,"15min":1.2250000000000001},"min":{"1min":0.26100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":69.373000000000005,"5min":69.373000000000005,"15min":69.373000000000005},"last":1.2030000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:37:58 2024","interfaces":[{"interface":"back","average":{"1min":2.7989999999999999,"5min":1.865,"15min":1.032},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":27.234999999999999,"5min":27.234999999999999,"15min":27.234999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":2.8740000000000001,"5min":1.6379999999999999,"15min":0.91300000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":28.879000000000001,"5min":28.879000000000001,"15min":28.879000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3540000000000001,"15min":0.93000000000000005},"min":{"1min":0.308,"5min":0.247,"15min":0.247},"max":{"1min":5.4859999999999998,"5min":8.3369999999999997,"15min":23.082000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":2.081,"5min":1.712,"15min":1.071},"min":{"1min":0.317,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":6.6980000000000004,"5min":7.992,"15min":23.138999999999999},"last":1.2629999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:37:58 2024","interfaces":[{"interface":"back","average":{"1min":2.915,"5min":2.2389999999999999,"15min":1.181},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":25.547000000000001,"5min":25.547000000000001,"15min":25.547000000000001},"last":1.7490000000000001},{"interface":"front","average":{"1min":2.718,"5min":1.8500000000000001,"15min":1.0389999999999999},"min":{"1min":0.30099999999999999,"5min":0.29199999999999998,"15min":0.28100000000000003},"max":{"1min":25.623000000000001,"5min":25.623000000000001,"15min":25.623000000000001},"last":10.499000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57699999999999996,"15min":0.47299999999999998},"min":{"1min":0.30499999999999999,"5min":0.253,"15min":0.253},"max":{"1min":2.266,"5min":2.835,"15min":2.835},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.66700000000000004,"15min":0.51700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":5.3899999999999997,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":2.3450000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.61299999999999999,"15min":0.51800000000000002},"min":{"1min":0.29099999999999998,"5min":0.25900000000000001,"15min":0.25600000000000001},"max":{"1min":5.4160000000000004,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56599999999999995,"15min":0.51500000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.238}]},{"osd":7,"last update":"Thu Feb 22 19:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":2.1930000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.63100000000000001,"15min":0.59799999999999998},"min":{"1min":0.32100000000000001,"5min":0.313,"15min":0.307},"max":{"1min":2.2999999999999998,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.0409999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724670,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":404956,"kb_used_data":103636,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93340196,"statfs":{"total":95995035648,"available":95580360704,"internally_reserved":0,"allocated":106123264,"data_stored":105631678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.6719999999999999,"15min":0.94499999999999995},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":4.1790000000000003,"5min":10.041,"15min":10.041},"last":2.7160000000000002},{"interface":"front","average":{"1min":2.1640000000000001,"5min":1.8859999999999999,"15min":1.0269999999999999},"min":{"1min":0.28699999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":8.9540000000000006,"5min":11.340999999999999,"15min":11.340999999999999},"last":1.7989999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:37:25 2024","interfaces":[{"interface":"back","average":{"1min":3.234,"5min":1.915,"15min":1.046},"min":{"1min":0.254,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":29.390999999999998,"5min":29.390999999999998,"15min":29.390999999999998},"last":1.718},{"interface":"front","average":{"1min":2.6179999999999999,"5min":1.9930000000000001,"15min":1.034},"min":{"1min":0.27300000000000002,"5min":0.218,"15min":0.218},"max":{"1min":29.367000000000001,"5min":29.367000000000001,"15min":29.367000000000001},"last":1.899}]},{"osd":2,"last update":"Thu Feb 22 19:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.7450000000000001,"15min":0.94199999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.9239999999999999,"5min":14.613,"15min":14.613},"last":2.5760000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.865,"15min":1.0269999999999999},"min":{"1min":0.27000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.7549999999999999,"5min":14.509,"15min":14.509},"last":3.2749999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:37:25 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":2.383,"15min":1.1970000000000001},"min":{"1min":0.374,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":7.0309999999999997,"5min":13.366,"15min":13.366},"last":4.3949999999999996},{"interface":"front","average":{"1min":1.8620000000000001,"5min":2.0150000000000001,"15min":1.131},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.851,"5min":10.16,"15min":10.16},"last":1.5249999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.54400000000000004,"15min":0.44800000000000001},"min":{"1min":0.25700000000000001,"5min":0.254,"15min":0.219},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.97},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.504,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.23499999999999999},"max":{"1min":1.98,"5min":3.258,"15min":3.258},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.49199999999999999},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.25800000000000001},"max":{"1min":1.5109999999999999,"5min":2.5950000000000002,"15min":5.6040000000000001},"last":1.49},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.309},"max":{"1min":2.0099999999999998,"5min":2.2650000000000001,"15min":3.8239999999999998},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56399999999999995,"15min":0.63300000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.28499999999999998},"max":{"1min":1.446,"5min":2.7269999999999999,"15min":5.5789999999999997},"last":1.452},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.67100000000000004,"15min":0.57399999999999995},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.27100000000000002},"max":{"1min":7.6379999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":1.9990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1151598592,"data_stored":1151598592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1148125184,"data_stored":1148125184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1193021440,"data_stored":1193017368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1472397312,"data_stored":1472397312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":993525760,"data_stored":993525760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":105644032,"data_stored":105644032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1158811648,"data_stored":1158807576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":823529472,"data_stored":823525400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:38:30.045 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set iscsi pgp_num 3 2024-02-22T19:38:30.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:28.407755+0000 mgr.y (mgr.14535) 365 : audit [DBG] from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:38:30.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:30.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:28.407755+0000 mgr.y (mgr.14535) 365 : audit [DBG] from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:38:30.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:28.407755+0000 mgr.y (mgr.14535) 365 : audit [DBG] from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:38:30.302 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:30.360 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:30.373 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:30.374 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:30.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:30.627 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:30.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:30.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:30.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:30.844 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:30.875 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:38:30.885 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:30.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:30.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:30.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:29.500755+0000 mgr.y (mgr.14535) 366 : cluster [DBG] pgmap v450: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.3 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 785 op/s; 24389/113010 objects misplaced (21.581%); 18 MiB/s, 295 objects/s recovering 2024-02-22T19:38:30.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:30.825366+0000 mon.a (mon.0) 1152 : audit [DBG] from='client.? 172.21.15.53:0/2369310865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:30.921 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:38:30.970 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:38:30.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:31.020 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:38:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:31.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:38:31.068 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:38:31.126 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:38:31.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:29.500755+0000 mgr.y (mgr.14535) 366 : cluster [DBG] pgmap v450: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.3 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 785 op/s; 24389/113010 objects misplaced (21.581%); 18 MiB/s, 295 objects/s recovering 2024-02-22T19:38:31.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:30.825366+0000 mon.a (mon.0) 1152 : audit [DBG] from='client.? 172.21.15.53:0/2369310865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:31.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:29.500755+0000 mgr.y (mgr.14535) 366 : cluster [DBG] pgmap v450: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.3 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 785 op/s; 24389/113010 objects misplaced (21.581%); 18 MiB/s, 295 objects/s recovering 2024-02-22T19:38:31.153 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:30.825366+0000 mon.a (mon.0) 1152 : audit [DBG] from='client.? 172.21.15.53:0/2369310865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:31.164 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:38:31.190 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:31.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:31.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:31.201 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:38:31.294 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:31.484 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:31.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:31.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:31.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:31.640 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:31.729 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:31.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:31.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:31.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:32.656 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:32.660 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:32.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:32.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:32.823 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:32.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:31.503712+0000 mgr.y (mgr.14535) 367 : cluster [DBG] pgmap v451: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 758 op/s; 23921/110439 objects misplaced (21.660%); 16 MiB/s, 256 objects/s recovering 2024-02-22T19:38:32.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:32.201783+0000 mon.a (mon.0) 1153 : cluster [DBG] osdmap e240: 8 total, 8 up, 7 in 2024-02-22T19:38:32.936 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:32.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:32.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:32.970 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:32.970 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:33.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:33.056 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:33.082 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:33.220 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:33.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:33.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:33.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:31.503712+0000 mgr.y (mgr.14535) 367 : cluster [DBG] pgmap v451: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 758 op/s; 23921/110439 objects misplaced (21.660%); 16 MiB/s, 256 objects/s recovering 2024-02-22T19:38:33.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:32.201783+0000 mon.a (mon.0) 1153 : cluster [DBG] osdmap e240: 8 total, 8 up, 7 in 2024-02-22T19:38:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:31.503712+0000 mgr.y (mgr.14535) 367 : cluster [DBG] pgmap v451: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 758 op/s; 23921/110439 objects misplaced (21.660%); 16 MiB/s, 256 objects/s recovering 2024-02-22T19:38:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:32.201783+0000 mon.a (mon.0) 1153 : cluster [DBG] osdmap e240: 8 total, 8 up, 7 in 2024-02-22T19:38:33.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:33.428 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:33.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:38:33.995 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:33.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:34.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:34.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:34.133 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:34.187 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:34.357 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:34.362 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:34.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:34.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:38:34.495 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:34.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:38:34.625 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:38:34.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:34.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:38:34.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:34.757 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:38:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:38:34.760 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:34.920 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 2 pgp_num to 3 2024-02-22T19:38:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:34.967 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:34.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:35.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:35.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:38:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:38:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:33.506072+0000 mgr.y (mgr.14535) 368 : cluster [DBG] pgmap v453: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 434 op/s; 23921/110439 objects misplaced (21.660%); 9.3 MiB/s, 148 objects/s recovering 2024-02-22T19:38:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:33.895874+0000 mon.a (mon.0) 1154 : cluster [DBG] osdmap e241: 8 total, 8 up, 7 in 2024-02-22T19:38:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:34.230665+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? 172.21.15.53:0/1049917747' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "iscsi", "var": "pgp_num", "val": "3"}]: dispatch 2024-02-22T19:38:35.146 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:33.506072+0000 mgr.y (mgr.14535) 368 : cluster [DBG] pgmap v453: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 434 op/s; 23921/110439 objects misplaced (21.660%); 9.3 MiB/s, 148 objects/s recovering 2024-02-22T19:38:35.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:33.895874+0000 mon.a (mon.0) 1154 : cluster [DBG] osdmap e241: 8 total, 8 up, 7 in 2024-02-22T19:38:35.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:34.230665+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? 172.21.15.53:0/1049917747' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "iscsi", "var": "pgp_num", "val": "3"}]: dispatch 2024-02-22T19:38:35.261 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:33.506072+0000 mgr.y (mgr.14535) 368 : cluster [DBG] pgmap v453: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 434 op/s; 23921/110439 objects misplaced (21.660%); 9.3 MiB/s, 148 objects/s recovering 2024-02-22T19:38:35.261 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:33.895874+0000 mon.a (mon.0) 1154 : cluster [DBG] osdmap e241: 8 total, 8 up, 7 in 2024-02-22T19:38:35.261 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:34.230665+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? 172.21.15.53:0/1049917747' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "iscsi", "var": "pgp_num", "val": "3"}]: dispatch 2024-02-22T19:38:35.553 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:35.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:35.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:35.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:36.066 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:34.781118+0000 mgr.y (mgr.14535) 369 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:36.066 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:34.896595+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? 172.21.15.53:0/1049917747' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "iscsi", "var": "pgp_num", "val": "3"}]': finished 2024-02-22T19:38:36.066 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:34.896650+0000 mon.a (mon.0) 1157 : cluster [DBG] osdmap e242: 8 total, 8 up, 7 in 2024-02-22T19:38:36.066 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:34.781118+0000 mgr.y (mgr.14535) 369 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:36.066 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:34.896595+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? 172.21.15.53:0/1049917747' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "iscsi", "var": "pgp_num", "val": "3"}]': finished 2024-02-22T19:38:36.066 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:34.896650+0000 mon.a (mon.0) 1157 : cluster [DBG] osdmap e242: 8 total, 8 up, 7 in 2024-02-22T19:38:36.189 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:36.189 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:36.193 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:36.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:36.196 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:36.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:34.781118+0000 mgr.y (mgr.14535) 369 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:36.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:34.896595+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? 172.21.15.53:0/1049917747' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "iscsi", "var": "pgp_num", "val": "3"}]': finished 2024-02-22T19:38:36.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:34.896650+0000 mon.a (mon.0) 1157 : cluster [DBG] osdmap e242: 8 total, 8 up, 7 in 2024-02-22T19:38:36.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:36.378 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:36.694 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:36.697 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:36.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:36.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:36.887 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:36.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:38:36.951 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:36.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:36.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:35.506538+0000 mgr.y (mgr.14535) 370 : cluster [DBG] pgmap v456: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.0 GiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 685 op/s; 20717/100677 objects misplaced (20.578%); 13 MiB/s, 201 objects/s recovering 2024-02-22T19:38:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:35.884322+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:35.884381+0000 osd.0 (osd.0) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:35.903219+0000 mon.a (mon.0) 1158 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:35.917168+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e243: 8 total, 8 up, 7 in 2024-02-22T19:38:36.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:36.165124+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.53:0/2386142364' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:38:36.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:36.165618+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:38:36.960 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:35.506538+0000 mgr.y (mgr.14535) 370 : cluster [DBG] pgmap v456: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.0 GiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 685 op/s; 20717/100677 objects misplaced (20.578%); 13 MiB/s, 201 objects/s recovering 2024-02-22T19:38:36.961 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:35.884322+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:36.961 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:35.884381+0000 osd.0 (osd.0) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:36.961 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:35.903219+0000 mon.a (mon.0) 1158 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:36.961 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:35.917168+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e243: 8 total, 8 up, 7 in 2024-02-22T19:38:36.961 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:36.165124+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.53:0/2386142364' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:38:36.961 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:36.165618+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:38:36.965 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:38:37.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:37.109 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:37.139 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:37.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:37.145 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:35.506538+0000 mgr.y (mgr.14535) 370 : cluster [DBG] pgmap v456: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.0 GiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 685 op/s; 20717/100677 objects misplaced (20.578%); 13 MiB/s, 201 objects/s recovering 2024-02-22T19:38:37.145 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:35.884322+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:37.145 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:35.884381+0000 osd.0 (osd.0) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:37.146 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:35.903219+0000 mon.a (mon.0) 1158 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:37.146 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:35.917168+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e243: 8 total, 8 up, 7 in 2024-02-22T19:38:37.146 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:36.165124+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.53:0/2386142364' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:38:37.146 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:36.165618+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:38:37.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:37.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:37.371 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:36.922100+0000 mon.a (mon.0) 1161 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:36.951202+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:38:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:36.951279+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e244: 8 total, 8 up, 7 in 2024-02-22T19:38:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:36.922100+0000 mon.a (mon.0) 1161 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:36.951202+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:38:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:36.951279+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e244: 8 total, 8 up, 7 in 2024-02-22T19:38:38.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:36.922100+0000 mon.a (mon.0) 1161 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:38:38.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:36.951202+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:38:38.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:36.951279+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e244: 8 total, 8 up, 7 in 2024-02-22T19:38:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:38.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:38.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:38.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:38.651 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:38.769 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:38.775 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:38.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:38.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:38.918 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:38.966 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:37.508667+0000 mgr.y (mgr.14535) 371 : cluster [DBG] pgmap v459: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.0 GiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 814 op/s; 20717/100677 objects misplaced (20.578%); 18 MiB/s, 288 objects/s recovering 2024-02-22T19:38:38.966 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:38.925158+0000 mon.c (mon.2) 51 : audit [DBG] from='client.? 172.21.15.53:0/4189186961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:38.978 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:38.981 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:38.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:39.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:39.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:37.508667+0000 mgr.y (mgr.14535) 371 : cluster [DBG] pgmap v459: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.0 GiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 814 op/s; 20717/100677 objects misplaced (20.578%); 18 MiB/s, 288 objects/s recovering 2024-02-22T19:38:39.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:38.925158+0000 mon.c (mon.2) 51 : audit [DBG] from='client.? 172.21.15.53:0/4189186961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:39.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:37.508667+0000 mgr.y (mgr.14535) 371 : cluster [DBG] pgmap v459: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 2.0 GiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 814 op/s; 20717/100677 objects misplaced (20.578%); 18 MiB/s, 288 objects/s recovering 2024-02-22T19:38:39.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:38.925158+0000 mon.c (mon.2) 51 : audit [DBG] from='client.? 172.21.15.53:0/4189186961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:39.204 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:39.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:39.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:39.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:193081 8.9 8:9b0678c3:::benchmark_data_smithi053_130774_object81347:head [delete] snapc 0=[] ondisk+write+known_if_redirected e244)", 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:39.320459+0000", 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011376661999999999, 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01140706, 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 193081 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320459+0000", 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320459+0000", 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320458+0000", 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7109999999999998e-06 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320464+0000", 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1060000000000001e-05 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320475+0000", 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4450000000000001e-06 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320482+0000", 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8733999999999997e-05 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320541+0000", 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011917999999999999 2024-02-22T19:38:39.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320660+0000", 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012564799999999999 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.320785+0000", 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029907190000000002 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.323776+0000", 2024-02-22T19:38:39.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038222669999999999 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.327598+0000", 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00048414299999999998 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.328083+0000", 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076232119999999999 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15054.0:193082 8.d e244/218 8:b87a80e7:::benchmark_data_smithi053_130774_object81348:head v 244'12223, mlcod=244'12220)", 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:39.324815+0000", 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0070212720000000003, 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0071366249999999997, 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324815+0000", 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:39.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324815+0000", 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324814+0000", 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0270000000000003e-06 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324819+0000", 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0180000000000004e-06 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324826+0000", 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8879999999999996e-06 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324831+0000", 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8636000000000001e-05 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324870+0000", 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7501999999999999e-05 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.324897+0000", 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2645000000000006e-05 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:193086 8.5 8:af8e43dc:::benchmark_data_smithi053_130774_object81352:head [delete] snapc 0=[] ondisk+write+known_if_redirected e244)", 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:39.325402+0000", 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0064338809999999998, 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006593074, 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:38:39.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 193086 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325402+0000", 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325402+0000", 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:38:39.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325400+0000", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7600000000000002e-06 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325408+0000", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1362999999999999e-05 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325419+0000", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.156e-06 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325425+0000", 2024-02-22T19:38:39.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9731000000000003e-05 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325465+0000", 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000112691 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325577+0000", 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000131655 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:39.325709+0000", 2024-02-22T19:38:39.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030704400000000002 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 3 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:38:39.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:38:39.362 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:39.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:38:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:39] "GET /metrics HTTP/1.1" 200 175977 "" "Prometheus/2.33.4" 2024-02-22T19:38:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:40.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:38:40.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:40.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:40.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:40.305 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:40.344 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:40.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:40.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:40.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:40.566 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:40.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:40.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:40.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:40.710 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:40.879 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:40.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:40.887 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:40.988 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:40.990 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:39.509584+0000 mgr.y (mgr.14535) 372 : cluster [DBG] pgmap v460: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 989 op/s; 18499/93813 objects misplaced (19.719%); 22 MiB/s, 348 objects/s recovering 2024-02-22T19:38:40.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:39.509584+0000 mgr.y (mgr.14535) 372 : cluster [DBG] pgmap v460: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 989 op/s; 18499/93813 objects misplaced (19.719%); 22 MiB/s, 348 objects/s recovering 2024-02-22T19:38:41.076 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:41.103 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:41.109 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 6, 7, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:38:41.109 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:38:41.109 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [1, 2, 3, 4, 6, 7, 5] 2024-02-22T19:38:41.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 4 2024-02-22T19:38:41.111 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:41.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:41.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:38:41.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:41.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:39.509584+0000 mgr.y (mgr.14535) 372 : cluster [DBG] pgmap v460: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 989 op/s; 18499/93813 objects misplaced (19.719%); 22 MiB/s, 348 objects/s recovering 2024-02-22T19:38:41.301 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:41.338 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:41.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:41.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:41.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:41.601 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:41.669 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:41.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:41.677 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:41.731 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:41.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:41.869 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:42.082 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:42.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:42.089 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:42.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:42.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:38:42.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:42.228 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:42.375 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:42.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:42.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:42.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:42.500 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:38:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:38:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:38:42.500 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:38:42.500 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:38:42.566 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:42.866 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:42.874 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:42.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:42.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:43.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:41.511676+0000 mgr.y (mgr.14535) 373 : cluster [DBG] pgmap v461: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 773 B/s rd, 0 B/s wr, 459 op/s; 18079/91575 objects misplaced (19.742%); 8.1 MiB/s, 129 objects/s recovering 2024-02-22T19:38:43.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:42.039728+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e245: 8 total, 8 up, 7 in 2024-02-22T19:38:43.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:42.222172+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e246: 8 total, 8 up, 7 in 2024-02-22T19:38:43.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:41.511676+0000 mgr.y (mgr.14535) 373 : cluster [DBG] pgmap v461: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 773 B/s rd, 0 B/s wr, 459 op/s; 18079/91575 objects misplaced (19.742%); 8.1 MiB/s, 129 objects/s recovering 2024-02-22T19:38:43.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:42.039728+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e245: 8 total, 8 up, 7 in 2024-02-22T19:38:43.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:42.222172+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e246: 8 total, 8 up, 7 in 2024-02-22T19:38:43.334 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:43.345 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:43.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:41.511676+0000 mgr.y (mgr.14535) 373 : cluster [DBG] pgmap v461: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 773 B/s rd, 0 B/s wr, 459 op/s; 18079/91575 objects misplaced (19.742%); 8.1 MiB/s, 129 objects/s recovering 2024-02-22T19:38:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:42.039728+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e245: 8 total, 8 up, 7 in 2024-02-22T19:38:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:42.222172+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e246: 8 total, 8 up, 7 in 2024-02-22T19:38:43.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:43.841 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:43.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:43.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:43.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:44.065 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:44.369 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:44.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:44.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:44.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:38:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:44.797 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:38:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:38:45.063 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:38:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:38:45.063 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:43.512786+0000 mgr.y (mgr.14535) 374 : cluster [DBG] pgmap v464: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 780 B/s rd, 0 B/s wr, 463 op/s; 18079/91575 objects misplaced (19.742%); 8.2 MiB/s, 130 objects/s recovering 2024-02-22T19:38:45.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:43.512786+0000 mgr.y (mgr.14535) 374 : cluster [DBG] pgmap v464: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 780 B/s rd, 0 B/s wr, 463 op/s; 18079/91575 objects misplaced (19.742%); 8.2 MiB/s, 130 objects/s recovering 2024-02-22T19:38:45.261 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:43.512786+0000 mgr.y (mgr.14535) 374 : cluster [DBG] pgmap v464: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 5 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 137 active+clean; 1.9 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 780 B/s rd, 0 B/s wr, 463 op/s; 18079/91575 objects misplaced (19.742%); 8.2 MiB/s, 130 objects/s recovering 2024-02-22T19:38:45.460 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:45.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:38:45.377+0000 7fe2d794e700 -1 mon.a@0(leader).osd e246 definitely_dead 0 2024-02-22T19:38:46.072 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:44.796194+0000 mgr.y (mgr.14535) 375 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:46.072 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:45.377808+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.53:0/3443918863' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:38:46.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:45.378506+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:38:46.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:45.378597+0000 mon.a (mon.0) 1167 : cluster [INF] Client client.admin marked osd.4 out, while it was still marked up 2024-02-22T19:38:46.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:45.736561+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.53:0/2285488146' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:46.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:45.736948+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:46.086 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:38:46.110 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.4. 2024-02-22T19:38:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:44.796194+0000 mgr.y (mgr.14535) 375 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:45.377808+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.53:0/3443918863' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:38:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:45.378506+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:38:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:45.378597+0000 mon.a (mon.0) 1167 : cluster [INF] Client client.admin marked osd.4 out, while it was still marked up 2024-02-22T19:38:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:45.736561+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.53:0/2285488146' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:45.736948+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:44.796194+0000 mgr.y (mgr.14535) 375 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:45.377808+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.53:0/3443918863' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:38:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:45.378506+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:38:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:45.378597+0000 mon.a (mon.0) 1167 : cluster [INF] Client client.admin marked osd.4 out, while it was still marked up 2024-02-22T19:38:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:45.736561+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.53:0/2285488146' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:46.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:45.736948+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:38:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:45.517312+0000 mgr.y (mgr.14535) 376 : cluster [DBG] pgmap v465: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 138 active+clean; 1.6 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s; 15765/81597 objects misplaced (19.321%); 9.5 MiB/s, 152 objects/s recovering 2024-02-22T19:38:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:46.058806+0000 mon.a (mon.0) 1169 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:38:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:46.070627+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-02-22T19:38:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:46.070694+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:46.070734+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e247: 8 total, 8 up, 6 in 2024-02-22T19:38:47.480 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:45.517312+0000 mgr.y (mgr.14535) 376 : cluster [DBG] pgmap v465: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 138 active+clean; 1.6 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s; 15765/81597 objects misplaced (19.321%); 9.5 MiB/s, 152 objects/s recovering 2024-02-22T19:38:47.480 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:46.058806+0000 mon.a (mon.0) 1169 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:38:47.480 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:46.070627+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-02-22T19:38:47.480 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:46.070694+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:47.480 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:46.070734+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e247: 8 total, 8 up, 6 in 2024-02-22T19:38:47.481 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:45.517312+0000 mgr.y (mgr.14535) 376 : cluster [DBG] pgmap v465: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 138 active+clean; 1.6 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s; 15765/81597 objects misplaced (19.321%); 9.5 MiB/s, 152 objects/s recovering 2024-02-22T19:38:47.481 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:46.058806+0000 mon.a (mon.0) 1169 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:38:47.481 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:46.070627+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-02-22T19:38:47.481 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:46.070694+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:38:47.481 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:46.070734+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e247: 8 total, 8 up, 6 in 2024-02-22T19:38:48.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:38:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:38:48.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:48.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:38:48.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:38:48.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:47.073136+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e248: 8 total, 8 up, 6 in 2024-02-22T19:38:48.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:47.074913+0000 osd.4 (osd.4) 145 : cluster [INF] 8.2 continuing backfill to osd.5 from (185'7681,246'12242] MIN to 246'12242 2024-02-22T19:38:48.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:47.074943+0000 osd.4 (osd.4) 146 : cluster [DBG] 8.2 starting backfill to osd.6 from (0'0,0'0] MAX to 246'12242 2024-02-22T19:38:48.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:47.082047+0000 osd.2 (osd.2) 123 : cluster [DBG] 8.e starting backfill to osd.1 from (0'0,0'0] MAX to 246'12412 2024-02-22T19:38:48.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:47.102397+0000 osd.2 (osd.2) 124 : cluster [INF] 8.e continuing backfill to osd.7 from (183'7421,246'12412] 8:72cb1154:::benchmark_data_smithi053_130774_object110750:head to 246'12412 2024-02-22T19:38:48.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:47.073136+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e248: 8 total, 8 up, 6 in 2024-02-22T19:38:48.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:47.074913+0000 osd.4 (osd.4) 145 : cluster [INF] 8.2 continuing backfill to osd.5 from (185'7681,246'12242] MIN to 246'12242 2024-02-22T19:38:48.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:47.074943+0000 osd.4 (osd.4) 146 : cluster [DBG] 8.2 starting backfill to osd.6 from (0'0,0'0] MAX to 246'12242 2024-02-22T19:38:48.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:47.082047+0000 osd.2 (osd.2) 123 : cluster [DBG] 8.e starting backfill to osd.1 from (0'0,0'0] MAX to 246'12412 2024-02-22T19:38:48.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:47.102397+0000 osd.2 (osd.2) 124 : cluster [INF] 8.e continuing backfill to osd.7 from (183'7421,246'12412] 8:72cb1154:::benchmark_data_smithi053_130774_object110750:head to 246'12412 2024-02-22T19:38:48.289 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:48.335 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:38:48.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:48.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:38:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:47.073136+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e248: 8 total, 8 up, 6 in 2024-02-22T19:38:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:47.074913+0000 osd.4 (osd.4) 145 : cluster [INF] 8.2 continuing backfill to osd.5 from (185'7681,246'12242] MIN to 246'12242 2024-02-22T19:38:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:47.074943+0000 osd.4 (osd.4) 146 : cluster [DBG] 8.2 starting backfill to osd.6 from (0'0,0'0] MAX to 246'12242 2024-02-22T19:38:48.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:47.082047+0000 osd.2 (osd.2) 123 : cluster [DBG] 8.e starting backfill to osd.1 from (0'0,0'0] MAX to 246'12412 2024-02-22T19:38:48.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:47.102397+0000 osd.2 (osd.2) 124 : cluster [INF] 8.e continuing backfill to osd.7 from (183'7421,246'12412] 8:72cb1154:::benchmark_data_smithi053_130774_object110750:head to 246'12412 2024-02-22T19:38:48.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:48.521 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:48.930 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:48.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:48.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:49.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:47.517694+0000 mgr.y (mgr.14535) 377 : cluster [DBG] pgmap v468: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 138 active+clean; 1.6 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 934 B/s rd, 0 B/s wr, 608 op/s; 15765/81597 objects misplaced (19.321%); 4.1 MiB/s, 66 objects/s recovering 2024-02-22T19:38:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.074988+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e249: 8 total, 8 up, 6 in 2024-02-22T19:38:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.077496+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.e starting backfill to osd.3 from (0'0,0'0] MAX to 236'19 2024-02-22T19:38:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.077501+0000 osd.5 (osd.5) 62 : cluster [DBG] 5.f starting backfill to osd.2 from (0'0,0'0] MAX to 221'14 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.078066+0000 osd.7 (osd.7) 165 : cluster [DBG] 5.9 starting backfill to osd.1 from (0'0,0'0] MAX to 217'10 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.080303+0000 osd.3 (osd.3) 157 : cluster [DBG] 8.b starting backfill to osd.2 from (0'0,0'0] MAX to 246'11964 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.081317+0000 osd.6 (osd.6) 104 : cluster [DBG] 8.5 starting backfill to osd.1 from (0'0,0'0] MAX to 246'12563 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:48.086633+0000 osd.2 (osd.2) 125 : cluster [DBG] 8.4 starting backfill to osd.5 from (183'7137,246'12154] MIN to 246'12155 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:47.517694+0000 mgr.y (mgr.14535) 377 : cluster [DBG] pgmap v468: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 138 active+clean; 1.6 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 934 B/s rd, 0 B/s wr, 608 op/s; 15765/81597 objects misplaced (19.321%); 4.1 MiB/s, 66 objects/s recovering 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.074988+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e249: 8 total, 8 up, 6 in 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.077496+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.e starting backfill to osd.3 from (0'0,0'0] MAX to 236'19 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.077501+0000 osd.5 (osd.5) 62 : cluster [DBG] 5.f starting backfill to osd.2 from (0'0,0'0] MAX to 221'14 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.078066+0000 osd.7 (osd.7) 165 : cluster [DBG] 5.9 starting backfill to osd.1 from (0'0,0'0] MAX to 217'10 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.080303+0000 osd.3 (osd.3) 157 : cluster [DBG] 8.b starting backfill to osd.2 from (0'0,0'0] MAX to 246'11964 2024-02-22T19:38:49.102 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.081317+0000 osd.6 (osd.6) 104 : cluster [DBG] 8.5 starting backfill to osd.1 from (0'0,0'0] MAX to 246'12563 2024-02-22T19:38:49.134 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:49.205 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:49.214 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:49.215 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:49.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:49.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:48.086633+0000 osd.2 (osd.2) 125 : cluster [DBG] 8.4 starting backfill to osd.5 from (183'7137,246'12154] MIN to 246'12155 2024-02-22T19:38:49.407 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:49] "GET /metrics HTTP/1.1" 200 175977 "" "Prometheus/2.33.4" 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:47.517694+0000 mgr.y (mgr.14535) 377 : cluster [DBG] pgmap v468: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 138 active+clean; 1.6 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 934 B/s rd, 0 B/s wr, 608 op/s; 15765/81597 objects misplaced (19.321%); 4.1 MiB/s, 66 objects/s recovering 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.074988+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e249: 8 total, 8 up, 6 in 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.077496+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.e starting backfill to osd.3 from (0'0,0'0] MAX to 236'19 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.077501+0000 osd.5 (osd.5) 62 : cluster [DBG] 5.f starting backfill to osd.2 from (0'0,0'0] MAX to 221'14 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.078066+0000 osd.7 (osd.7) 165 : cluster [DBG] 5.9 starting backfill to osd.1 from (0'0,0'0] MAX to 217'10 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.080303+0000 osd.3 (osd.3) 157 : cluster [DBG] 8.b starting backfill to osd.2 from (0'0,0'0] MAX to 246'11964 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.081317+0000 osd.6 (osd.6) 104 : cluster [DBG] 8.5 starting backfill to osd.1 from (0'0,0'0] MAX to 246'12563 2024-02-22T19:38:49.408 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:48.086633+0000 osd.2 (osd.2) 125 : cluster [DBG] 8.4 starting backfill to osd.5 from (183'7137,246'12154] MIN to 246'12155 2024-02-22T19:38:49.412 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:49.455 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:49.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:49.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:49.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:49.651 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:38:49.696 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:38:49.743 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:38:49.783 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:38:49.819 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:38:49.862 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:38:49.865 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:49.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:49.871 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:49.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:38:49.904 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:38:49.953 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:38:49.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:49.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.976 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:38:49.976 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:38:50.005 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:50.040 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:50.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:50.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:50.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:38:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:50.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:49.111974+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e250: 8 total, 8 up, 6 in 2024-02-22T19:38:50.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:49.518523+0000 mgr.y (mgr.14535) 378 : cluster [DBG] pgmap v471: 148 pgs: 2 peering, 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 134 active+clean; 1.6 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 369 op/s; 19340/77166 objects misplaced (25.063%); 3.2 MiB/s, 65 objects/s recovering 2024-02-22T19:38:50.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:49.612476+0000 mon.b (mon.1) 64 : audit [DBG] from='client.? 172.21.15.53:0/3102099641' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:50.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:49.111974+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e250: 8 total, 8 up, 6 in 2024-02-22T19:38:50.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:49.518523+0000 mgr.y (mgr.14535) 378 : cluster [DBG] pgmap v471: 148 pgs: 2 peering, 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 134 active+clean; 1.6 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 369 op/s; 19340/77166 objects misplaced (25.063%); 3.2 MiB/s, 65 objects/s recovering 2024-02-22T19:38:50.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:49.612476+0000 mon.b (mon.1) 64 : audit [DBG] from='client.? 172.21.15.53:0/3102099641' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:38:50.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:50.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:38:50.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:38:50.455 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:49.111974+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e250: 8 total, 8 up, 6 in 2024-02-22T19:38:50.455 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:49.518523+0000 mgr.y (mgr.14535) 378 : cluster [DBG] pgmap v471: 148 pgs: 2 peering, 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 134 active+clean; 1.6 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 369 op/s; 19340/77166 objects misplaced (25.063%); 3.2 MiB/s, 65 objects/s recovering 2024-02-22T19:38:50.455 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:49.612476+0000 mon.b (mon.1) 64 : audit [DBG] from='client.? 172.21.15.53:0/3102099641' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:38:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:50.468 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:50.468 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:50.491 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:38:50.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:50.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:38:50.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:38:50.704 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:50.844 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:38:50.849 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:50.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:38:50.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:51.021 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:51.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:51.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:51.182 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:38:51.253 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:51.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:38:51.303 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:38:51.306 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:51.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:38:51.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:49.287208+0000 osd.6 (osd.6) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:51.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:49.287234+0000 osd.6 (osd.6) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:51.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:50.118777+0000 mon.a (mon.0) 1176 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:51.309 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:50.133760+0000 mon.a (mon.0) 1177 : cluster [DBG] osdmap e251: 8 total, 8 up, 6 in 2024-02-22T19:38:51.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:49.287208+0000 osd.6 (osd.6) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:51.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:49.287234+0000 osd.6 (osd.6) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:51.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:50.118777+0000 mon.a (mon.0) 1176 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:51.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:50.133760+0000 mon.a (mon.0) 1177 : cluster [DBG] osdmap e251: 8 total, 8 up, 6 in 2024-02-22T19:38:51.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:51.494 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:51.565 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:51.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:51.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:49.287208+0000 osd.6 (osd.6) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:49.287234+0000 osd.6 (osd.6) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:50.118777+0000 mon.a (mon.0) 1176 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:38:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:50.133760+0000 mon.a (mon.0) 1177 : cluster [DBG] osdmap e251: 8 total, 8 up, 6 in 2024-02-22T19:38:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:51.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:38:51.747 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:38:52.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:51.518944+0000 mgr.y (mgr.14535) 379 : cluster [DBG] pgmap v473: 148 pgs: 2 peering, 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 133 active+clean; 1.5 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 440 op/s; 20687/75723 objects misplaced (27.319%); 3.8 MiB/s, 76 objects/s recovering 2024-02-22T19:38:52.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:51.518944+0000 mgr.y (mgr.14535) 379 : cluster [DBG] pgmap v473: 148 pgs: 2 peering, 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 133 active+clean; 1.5 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 440 op/s; 20687/75723 objects misplaced (27.319%); 3.8 MiB/s, 76 objects/s recovering 2024-02-22T19:38:52.558 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:51.518944+0000 mgr.y (mgr.14535) 379 : cluster [DBG] pgmap v473: 148 pgs: 2 peering, 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 133 active+clean; 1.5 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 440 op/s; 20687/75723 objects misplaced (27.319%); 3.8 MiB/s, 76 objects/s recovering 2024-02-22T19:38:52.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:38:53.216 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:52.216009+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e252: 8 total, 8 up, 6 in 2024-02-22T19:38:53.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:52.216009+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e252: 8 total, 8 up, 6 in 2024-02-22T19:38:53.359 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:38:53.359 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:38:53.360 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-02-22T19:38:53.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 0 2024-02-22T19:38:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:52.216009+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e252: 8 total, 8 up, 6 in 2024-02-22T19:38:53.573 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:53.574 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:54.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:38:54.662 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:53.380982+0000 mon.a (mon.0) 1179 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:54.662 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:53.522468+0000 mgr.y (mgr.14535) 380 : cluster [DBG] pgmap v475: 148 pgs: 2 peering, 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 133 active+clean; 1.5 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 359 op/s; 20687/75723 objects misplaced (27.319%); 3.1 MiB/s, 62 objects/s recovering 2024-02-22T19:38:54.662 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:53.690850+0000 osd.5 (osd.5) 63 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:54.662 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:53.690893+0000 osd.5 (osd.5) 64 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:54.663 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:38:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:38:54.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:53.380982+0000 mon.a (mon.0) 1179 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:54.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:53.522468+0000 mgr.y (mgr.14535) 380 : cluster [DBG] pgmap v475: 148 pgs: 2 peering, 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 133 active+clean; 1.5 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 359 op/s; 20687/75723 objects misplaced (27.319%); 3.1 MiB/s, 62 objects/s recovering 2024-02-22T19:38:54.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:53.690850+0000 osd.5 (osd.5) 63 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:54.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:53.690893+0000 osd.5 (osd.5) 64 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:54.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:53.380982+0000 mon.a (mon.0) 1179 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:38:54.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:53.522468+0000 mgr.y (mgr.14535) 380 : cluster [DBG] pgmap v475: 148 pgs: 2 peering, 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 133 active+clean; 1.5 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 359 op/s; 20687/75723 objects misplaced (27.319%); 3.1 MiB/s, 62 objects/s recovering 2024-02-22T19:38:54.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:53.690850+0000 osd.5 (osd.5) 63 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:38:54.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:53.690893+0000 osd.5 (osd.5) 64 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:38:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:55.032 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:38:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:38:55.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:54.411517+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e253: 8 total, 8 up, 6 in 2024-02-22T19:38:55.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:54.806217+0000 mgr.y (mgr.14535) 381 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:55.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:54.411517+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e253: 8 total, 8 up, 6 in 2024-02-22T19:38:55.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:54.806217+0000 mgr.y (mgr.14535) 381 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:54.411517+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e253: 8 total, 8 up, 6 in 2024-02-22T19:38:55.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:54.806217+0000 mgr.y (mgr.14535) 381 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:38:56.705 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:56.705 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:56.728 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:55.524726+0000 mgr.y (mgr.14535) 382 : cluster [DBG] pgmap v477: 148 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 135 active+clean; 1.3 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 340 B/s rd, 0 B/s wr, 644 op/s; 17160/65559 objects misplaced (26.175%); 9.0 MiB/s, 146 objects/s recovering 2024-02-22T19:38:56.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:55.524726+0000 mgr.y (mgr.14535) 382 : cluster [DBG] pgmap v477: 148 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 135 active+clean; 1.3 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 340 B/s rd, 0 B/s wr, 644 op/s; 17160/65559 objects misplaced (26.175%); 9.0 MiB/s, 146 objects/s recovering 2024-02-22T19:38:56.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:55.524726+0000 mgr.y (mgr.14535) 382 : cluster [DBG] pgmap v477: 148 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 135 active+clean; 1.3 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 340 B/s rd, 0 B/s wr, 644 op/s; 17160/65559 objects misplaced (26.175%); 9.0 MiB/s, 146 objects/s recovering 2024-02-22T19:38:57.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:38:57.911 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:38:58.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:57.219359+0000 mon.a (mon.0) 1181 : cluster [DBG] osdmap e254: 8 total, 8 up, 6 in 2024-02-22T19:38:58.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:57.525219+0000 mgr.y (mgr.14535) 383 : cluster [DBG] pgmap v479: 148 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 135 active+clean; 1.3 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 564 op/s; 17160/65559 objects misplaced (26.175%); 8.3 MiB/s, 132 objects/s recovering 2024-02-22T19:38:58.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:57.219359+0000 mon.a (mon.0) 1181 : cluster [DBG] osdmap e254: 8 total, 8 up, 6 in 2024-02-22T19:38:58.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:57.525219+0000 mgr.y (mgr.14535) 383 : cluster [DBG] pgmap v479: 148 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 135 active+clean; 1.3 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 564 op/s; 17160/65559 objects misplaced (26.175%); 8.3 MiB/s, 132 objects/s recovering 2024-02-22T19:38:58.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:57.219359+0000 mon.a (mon.0) 1181 : cluster [DBG] osdmap e254: 8 total, 8 up, 6 in 2024-02-22T19:38:58.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:57.525219+0000 mgr.y (mgr.14535) 383 : cluster [DBG] pgmap v479: 148 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 135 active+clean; 1.3 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 564 op/s; 17160/65559 objects misplaced (26.175%); 8.3 MiB/s, 132 objects/s recovering 2024-02-22T19:38:59.702 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:38:59.703 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:38:59.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:38:59] "GET /metrics HTTP/1.1" 200 175996 "" "Prometheus/2.33.4" 2024-02-22T19:38:59.956 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:38:59.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:38:59.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:38:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:38:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:00.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:00.066 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:38:58.796872+0000 mon.a (mon.0) 1182 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:00.066 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:59.447346+0000 osd.7 (osd.7) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:00.066 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:38:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:59.447393+0000 osd.7 (osd.7) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:38:58.796872+0000 mon.a (mon.0) 1182 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:59.447346+0000 osd.7 (osd.7) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:59.447393+0000 osd.7 (osd.7) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:00.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:38:58.796872+0000 mon.a (mon.0) 1182 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:00.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:59.447346+0000 osd.7 (osd.7) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:00.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:38:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:59.447393+0000 osd.7 (osd.7) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:00.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:39:00.452 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:00.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:00.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:00.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:00.586 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:00.611 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:00.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:00.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:59.526394+0000 mgr.y (mgr.14535) 384 : cluster [DBG] pgmap v480: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.2 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 837 op/s; 14146/57375 objects misplaced (24.655%); 14 MiB/s, 217 objects/s recovering 2024-02-22T19:39:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:59.794328+0000 mon.a (mon.0) 1183 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:38:59.814413+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e255: 8 total, 8 up, 6 in 2024-02-22T19:39:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:00.274653+0000 osd.4 (osd.4) 147 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:00.274695+0000 osd.4 (osd.4) 148 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:59.526394+0000 mgr.y (mgr.14535) 384 : cluster [DBG] pgmap v480: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.2 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 837 op/s; 14146/57375 objects misplaced (24.655%); 14 MiB/s, 217 objects/s recovering 2024-02-22T19:39:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:59.794328+0000 mon.a (mon.0) 1183 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:38:59.814413+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e255: 8 total, 8 up, 6 in 2024-02-22T19:39:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:00.274653+0000 osd.4 (osd.4) 147 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:01.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:00.274695+0000 osd.4 (osd.4) 148 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:59.526394+0000 mgr.y (mgr.14535) 384 : cluster [DBG] pgmap v480: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.2 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 837 op/s; 14146/57375 objects misplaced (24.655%); 14 MiB/s, 217 objects/s recovering 2024-02-22T19:39:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:59.794328+0000 mon.a (mon.0) 1183 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:38:59.814413+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e255: 8 total, 8 up, 6 in 2024-02-22T19:39:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:00.274653+0000 osd.4 (osd.4) 147 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:00.274695+0000 osd.4 (osd.4) 148 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:02.035 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:02.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:02.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:00.821693+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e256: 8 total, 8 up, 6 in 2024-02-22T19:39:02.123 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:00.821693+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e256: 8 total, 8 up, 6 in 2024-02-22T19:39:02.126 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:00.821693+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e256: 8 total, 8 up, 6 in 2024-02-22T19:39:02.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:02.171 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:02.199 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:02.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:02.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:02.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:02.358 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:02.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:02.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:02.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:02.555 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:02.671 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:02.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:02.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:02.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:02.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:02.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:02.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:02.969 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:02.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:02.999 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:03.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:01.526967+0000 mgr.y (mgr.14535) 385 : cluster [DBG] pgmap v483: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.1 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 605 op/s; 13407/54666 objects misplaced (24.525%); 9.4 MiB/s, 151 objects/s recovering 2024-02-22T19:39:03.134 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:03.164 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:01.526967+0000 mgr.y (mgr.14535) 385 : cluster [DBG] pgmap v483: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.1 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 605 op/s; 13407/54666 objects misplaced (24.525%); 9.4 MiB/s, 151 objects/s recovering 2024-02-22T19:39:03.164 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:01.526967+0000 mgr.y (mgr.14535) 385 : cluster [DBG] pgmap v483: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.1 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 605 op/s; 13407/54666 objects misplaced (24.525%); 9.4 MiB/s, 151 objects/s recovering 2024-02-22T19:39:03.165 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:03.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:03.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:03.345 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:03.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:03.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:03.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:03.615 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:03.934 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:39:04.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:02.861233+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.53:0/1731450550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:04.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:02.863582+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:04.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:02.861233+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.53:0/1731450550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:04.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:02.863582+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:04.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:04.308 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:04.309 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:04.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:02.861233+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.53:0/1731450550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:04.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:02.863582+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:04.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:04.521 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:05.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:39:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:39:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:39:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:39:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:03.527444+0000 mgr.y (mgr.14535) 386 : cluster [DBG] pgmap v484: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.1 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 811 B/s rd, 0 B/s wr, 576 op/s; 13407/54666 objects misplaced (24.525%); 9.0 MiB/s, 143 objects/s recovering 2024-02-22T19:39:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:03.859731+0000 mon.a (mon.0) 1187 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:03.887928+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:39:05.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:03.887969+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e257: 8 total, 8 up, 6 in 2024-02-22T19:39:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:03.527444+0000 mgr.y (mgr.14535) 386 : cluster [DBG] pgmap v484: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.1 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 811 B/s rd, 0 B/s wr, 576 op/s; 13407/54666 objects misplaced (24.525%); 9.0 MiB/s, 143 objects/s recovering 2024-02-22T19:39:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:03.859731+0000 mon.a (mon.0) 1187 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:03.887928+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:39:05.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:03.887969+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e257: 8 total, 8 up, 6 in 2024-02-22T19:39:05.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:03.527444+0000 mgr.y (mgr.14535) 386 : cluster [DBG] pgmap v484: 148 pgs: 2 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 135 active+clean; 1.1 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 811 B/s rd, 0 B/s wr, 576 op/s; 13407/54666 objects misplaced (24.525%); 9.0 MiB/s, 143 objects/s recovering 2024-02-22T19:39:05.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:03.859731+0000 mon.a (mon.0) 1187 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:05.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:03.887928+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:39:05.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:03.887969+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e257: 8 total, 8 up, 6 in 2024-02-22T19:39:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:05.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:05.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:05.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:05.667 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:05.669 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:05.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:05.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:05.809 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:05.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:04.818808+0000 mgr.y (mgr.14535) 387 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:05.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:04.928990+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e258: 8 total, 8 up, 6 in 2024-02-22T19:39:06.231 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:06.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:06.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:06.242 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:04.818808+0000 mgr.y (mgr.14535) 387 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:06.243 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:04.928990+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e258: 8 total, 8 up, 6 in 2024-02-22T19:39:06.243 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:04.818808+0000 mgr.y (mgr.14535) 387 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:06.243 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:04.928990+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e258: 8 total, 8 up, 6 in 2024-02-22T19:39:06.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:06.410 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:06.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:06.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:06.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:06.539 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:06.546 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:06.567 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:06.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:06.571 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:06.671 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:06.689 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:39:06.747 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:39:06.760 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:06.801 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:39:06.856 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:39:06.861 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:06.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:06.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:06.903 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:39:06.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:39:06.670+0000 7fe2d794e700 -1 mon.a@0(leader).osd e259 definitely_dead 0 2024-02-22T19:39:06.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:05.527879+0000 mgr.y (mgr.14535) 388 : cluster [DBG] pgmap v487: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 136 active+clean; 941 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 716 B/s rd, 0 B/s wr, 522 op/s; 10176/45726 objects misplaced (22.254%); 13 MiB/s, 202 objects/s recovering 2024-02-22T19:39:06.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:05.921075+0000 mon.a (mon.0) 1191 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:06.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:05.929326+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e259: 8 total, 8 up, 6 in 2024-02-22T19:39:06.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:06.622970+0000 mon.a (mon.0) 1193 : audit [DBG] from='client.? 172.21.15.53:0/1360042223' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:06.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:06.671368+0000 mon.a (mon.0) 1194 : audit [DBG] from='client.? 172.21.15.53:0/2088145075' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:39:06.937 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:39:06.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:06.983 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.0. 2024-02-22T19:39:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:07.004 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:39:07.024 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:07.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:07.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:07.035 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:39:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:39:07.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:07.155 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:07.176 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:07.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:07.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:05.527879+0000 mgr.y (mgr.14535) 388 : cluster [DBG] pgmap v487: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 136 active+clean; 941 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 716 B/s rd, 0 B/s wr, 522 op/s; 10176/45726 objects misplaced (22.254%); 13 MiB/s, 202 objects/s recovering 2024-02-22T19:39:07.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:05.921075+0000 mon.a (mon.0) 1191 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:07.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:05.929326+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e259: 8 total, 8 up, 6 in 2024-02-22T19:39:07.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:06.622970+0000 mon.a (mon.0) 1193 : audit [DBG] from='client.? 172.21.15.53:0/1360042223' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:07.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:06.671368+0000 mon.a (mon.0) 1194 : audit [DBG] from='client.? 172.21.15.53:0/2088145075' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:39:07.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:07.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:05.527879+0000 mgr.y (mgr.14535) 388 : cluster [DBG] pgmap v487: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 136 active+clean; 941 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 716 B/s rd, 0 B/s wr, 522 op/s; 10176/45726 objects misplaced (22.254%); 13 MiB/s, 202 objects/s recovering 2024-02-22T19:39:07.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:05.921075+0000 mon.a (mon.0) 1191 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:07.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:05.929326+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e259: 8 total, 8 up, 6 in 2024-02-22T19:39:07.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:06.622970+0000 mon.a (mon.0) 1193 : audit [DBG] from='client.? 172.21.15.53:0/1360042223' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:07.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:06.671368+0000 mon.a (mon.0) 1194 : audit [DBG] from='client.? 172.21.15.53:0/2088145075' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:39:07.309 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:07.331 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:07.332 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:07.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:07.459 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:07.477 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:07.480 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:07.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:07.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:39:07.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:07.606 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:07.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:07.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:07.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:07.764 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:06.944469+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? 172.21.15.53:0/2088145075' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:06.944549+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e260: 8 total, 8 up, 7 in 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:07.219498+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e261: 8 total, 8 up, 7 in 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:07.223762+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.e starting backfill to osd.0 from (0'0,0'0] MAX to 250'21 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:06.944469+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? 172.21.15.53:0/2088145075' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:06.944549+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e260: 8 total, 8 up, 7 in 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:07.219498+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e261: 8 total, 8 up, 7 in 2024-02-22T19:39:08.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:07.223762+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.e starting backfill to osd.0 from (0'0,0'0] MAX to 250'21 2024-02-22T19:39:08.216 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:08.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:08.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:08.231 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-02-22T19:39:08.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:06.944469+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? 172.21.15.53:0/2088145075' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-22T19:39:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:06.944549+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e260: 8 total, 8 up, 7 in 2024-02-22T19:39:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:07.219498+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e261: 8 total, 8 up, 7 in 2024-02-22T19:39:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:07.223762+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.e starting backfill to osd.0 from (0'0,0'0] MAX to 250'21 2024-02-22T19:39:08.365 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:08.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:08.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:08.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:08.584 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:08.856 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:08.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:08.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:08.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:09.034 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:09.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:09.039 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:09.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:09.212 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:09.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:07.528349+0000 mgr.y (mgr.14535) 389 : cluster [DBG] pgmap v491: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 136 active+clean; 941 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 819 op/s; 10176/45726 objects misplaced (22.254%); 20 MiB/s, 318 objects/s recovering 2024-02-22T19:39:09.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:08.222910+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e262: 8 total, 8 up, 7 in 2024-02-22T19:39:09.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:08.367948+0000 mon.a (mon.0) 1199 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:09.219 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:09.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:07.528349+0000 mgr.y (mgr.14535) 389 : cluster [DBG] pgmap v491: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 136 active+clean; 941 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 819 op/s; 10176/45726 objects misplaced (22.254%); 20 MiB/s, 318 objects/s recovering 2024-02-22T19:39:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:08.222910+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e262: 8 total, 8 up, 7 in 2024-02-22T19:39:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:08.367948+0000 mon.a (mon.0) 1199 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:09.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:07.528349+0000 mgr.y (mgr.14535) 389 : cluster [DBG] pgmap v491: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 136 active+clean; 941 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 819 op/s; 10176/45726 objects misplaced (22.254%); 20 MiB/s, 318 objects/s recovering 2024-02-22T19:39:09.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:08.222910+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e262: 8 total, 8 up, 7 in 2024-02-22T19:39:09.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:08.367948+0000 mon.a (mon.0) 1199 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:09.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:09.425 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:09.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:09] "GET /metrics HTTP/1.1" 200 175966 "" "Prometheus/2.33.4" 2024-02-22T19:39:09.801 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:09.811 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:09.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:09.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:09.986 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:10.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:10.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:10.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:08.222431+0000 osd.6 (osd.6) 107 : cluster [DBG] 8.5 starting backfill to osd.0 from (0'0,0'0] MAX to 259'13323 2024-02-22T19:39:10.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:08.222764+0000 osd.6 (osd.6) 108 : cluster [INF] 8.9 continuing backfill to osd.5 from (192'8847,259'13196] 8:93a59b7a:::benchmark_data_smithi053_130774_object105021:head to 259'13196 2024-02-22T19:39:10.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:08.225333+0000 osd.1 (osd.1) 143 : cluster [DBG] 8.1 starting backfill to osd.0 from (183'7224,240'11930] MIN to 259'13152 2024-02-22T19:39:10.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:08.227047+0000 osd.2 (osd.2) 126 : cluster [DBG] 8.6 starting backfill to osd.0 from (183'7127,244'12390] MIN to 259'13277 2024-02-22T19:39:10.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:08.231994+0000 osd.0 (osd.0) 160 : cluster [INF] 8.2 continuing backfill to osd.5 from (185'7681,259'12967] MIN to 259'12967 2024-02-22T19:39:10.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:09.374169+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e263: 8 total, 8 up, 7 in 2024-02-22T19:39:10.101 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:08.222431+0000 osd.6 (osd.6) 107 : cluster [DBG] 8.5 starting backfill to osd.0 from (0'0,0'0] MAX to 259'13323 2024-02-22T19:39:10.101 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:08.222764+0000 osd.6 (osd.6) 108 : cluster [INF] 8.9 continuing backfill to osd.5 from (192'8847,259'13196] 8:93a59b7a:::benchmark_data_smithi053_130774_object105021:head to 259'13196 2024-02-22T19:39:10.101 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:08.225333+0000 osd.1 (osd.1) 143 : cluster [DBG] 8.1 starting backfill to osd.0 from (183'7224,240'11930] MIN to 259'13152 2024-02-22T19:39:10.101 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:08.227047+0000 osd.2 (osd.2) 126 : cluster [DBG] 8.6 starting backfill to osd.0 from (183'7127,244'12390] MIN to 259'13277 2024-02-22T19:39:10.101 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:08.231994+0000 osd.0 (osd.0) 160 : cluster [INF] 8.2 continuing backfill to osd.5 from (185'7681,259'12967] MIN to 259'12967 2024-02-22T19:39:10.101 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:09.374169+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e263: 8 total, 8 up, 7 in 2024-02-22T19:39:10.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:10.261 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:10.313 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:10.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:08.222431+0000 osd.6 (osd.6) 107 : cluster [DBG] 8.5 starting backfill to osd.0 from (0'0,0'0] MAX to 259'13323 2024-02-22T19:39:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:08.222764+0000 osd.6 (osd.6) 108 : cluster [INF] 8.9 continuing backfill to osd.5 from (192'8847,259'13196] 8:93a59b7a:::benchmark_data_smithi053_130774_object105021:head to 259'13196 2024-02-22T19:39:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:08.225333+0000 osd.1 (osd.1) 143 : cluster [DBG] 8.1 starting backfill to osd.0 from (183'7224,240'11930] MIN to 259'13152 2024-02-22T19:39:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:08.227047+0000 osd.2 (osd.2) 126 : cluster [DBG] 8.6 starting backfill to osd.0 from (183'7127,244'12390] MIN to 259'13277 2024-02-22T19:39:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:08.231994+0000 osd.0 (osd.0) 160 : cluster [INF] 8.2 continuing backfill to osd.5 from (185'7681,259'12967] MIN to 259'12967 2024-02-22T19:39:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:09.374169+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e263: 8 total, 8 up, 7 in 2024-02-22T19:39:10.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:10.443 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:210942 8.9 e264/261 8:99efced1:::benchmark_data_smithi053_130774_object99208:head v 264'13291, mlcod=264'13289)", 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502619+0000", 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010215510000000001, 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010244669, 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502619+0000", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502619+0000", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502618+0000", 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-02-22T19:39:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502624+0000", 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2320000000000001e-06 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502629+0000", 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8890000000000004e-06 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502633+0000", 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091626720000000002 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.511796+0000", 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6268999999999999e-05 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.511822+0000", 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0092031700000000001 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:210945 8.9 e264/261)", 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502643+0000", 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010191254, 2024-02-22T19:39:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010290013000000001, 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502643+0000", 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502643+0000", 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502643+0000", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4029999999999996e-06 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502652+0000", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5729999999999998e-06 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502658+0000", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3379999999999998e-06 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502662+0000", 2024-02-22T19:39:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8983000000000001e-05 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:210946 8.9 e264/261)", 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502666+0000", 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010167871, 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010317262000000001, 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502666+0000", 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502666+0000", 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502666+0000", 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502670+0000", 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9649999999999999e-06 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502675+0000", 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5530000000000001e-06 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502679+0000", 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2418e-05 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24371.0:727 4.6 4:63d917ed:::obj_delete_at_hint.0000000065:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502692+0000", 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010141902, 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010349819999999999, 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24371", 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/65330745", 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 727 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502692+0000", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502692+0000", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502690+0000", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2089999999999999e-06 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502697+0000", 2024-02-22T19:39:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7649999999999999e-06 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502706+0000", 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1989999999999997e-06 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502711+0000", 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030237299999999999 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.503014+0000", 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3444999999999997e-05 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.503107+0000", 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018382699999999999 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.503291+0000", 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060247790000000001 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509316+0000", 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066232260000000003 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:210951 8.8 8:104e3626:::benchmark_data_smithi053_130774_object99217:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502741+0000", 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010093682, 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01040458, 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 210951 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502741+0000", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502741+0000", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502740+0000", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502744+0000", 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9789999999999996e-06 2024-02-22T19:39:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502751+0000", 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5709999999999999e-06 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502756+0000", 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021492730000000002 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.504905+0000", 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000119283 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.505024+0000", 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015086799999999999 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,5,7", 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.505175+0000", 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002434514 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:210955 8.e 8:7d77ecf6:::benchmark_data_smithi053_130774_object99221:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502771+0000", 2024-02-22T19:39:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010063701, 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010471223, 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 210955 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502771+0000", 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502771+0000", 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502770+0000", 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8320000000000002e-06 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502773+0000", 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.575e-06 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502779+0000", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4530000000000004e-06 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502783+0000", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071091100000000005 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.503494+0000", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012511900000000001 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.503619+0000", 2024-02-22T19:39:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014920499999999999 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.503768+0000", 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053580859999999998 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509126+0000", 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022587309999999999 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.511385+0000", 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0086144480000000002 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:210956 8.8 8:1c096131:::benchmark_data_smithi053_130774_object99222:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.502788+0000", 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.010046641, 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010554795, 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 210956 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502788+0000", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502788+0000", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502787+0000", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5490000000000001e-06 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502790+0000", 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6959999999999996e-06 2024-02-22T19:39:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502794+0000", 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7699999999999999e-06 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.502798+0000", 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048029589999999999 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.507601+0000", 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012364400000000001 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.507725+0000", 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014599 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,5,7", 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.507871+0000", 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054378489999999998 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513309+0000", 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010521056000000001 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:210954 8.1 e264/261)", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.509513+0000", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0033213349999999999, 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039140240000000003, 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509513+0000", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509513+0000", 2024-02-22T19:39:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509512+0000", 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4819999999999999e-06 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509519+0000", 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7389999999999999e-06 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509527+0000", 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5090000000000004e-06 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509531+0000", 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8085e-05 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:210959 8.2 8.7b27cca2 (undecoded) ondisk+write+known_if_redirected e264)", 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.509702+0000", 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0031318819999999999, 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037759880000000001, 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:39:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 210959 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509702+0000", 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509702+0000", 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509701+0000", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2789999999999998e-06 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509706+0000", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7860000000000002e-06 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509715+0000", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7740000000000002e-06 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.509719+0000", 2024-02-22T19:39:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6328999999999999e-05 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15054.0:210952 8.6 e264/261 8:6b9cad0c:::benchmark_data_smithi053_130774_object99218:head v 264'13392, mlcod=264'13390)", 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.512353+0000", 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00048163299999999997, 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001185385, 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512353+0000", 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512353+0000", 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512352+0000", 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9989000000000001e-05 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512372+0000", 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4606e-05 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512386+0000", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1809999999999998e-06 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512392+0000", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5958000000000003e-05 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512438+0000", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1920999999999999e-05 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.512460+0000", 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107719 2024-02-22T19:39:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.15054.0:210955 8.e e264/261)", 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.513118+0000", 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9997168, 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000483773, 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513118+0000", 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513118+0000", 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513117+0000", 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513123+0000", 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0232e-05 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513133+0000", 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513139+0000", 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1206999999999999e-05 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.15054.0:210956 8.8 e264/218)", 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:10.513352+0000", 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9994822, 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029973899999999998, 2024-02-22T19:39:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513352+0000", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513352+0000", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513351+0000", 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0866999999999999e-05 2024-02-22T19:39:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513362+0000", 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513371+0000", 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5509999999999999e-06 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:10.513376+0000", 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4451e-05 2024-02-22T19:39:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 12 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:39:10.557 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:10.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:39:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:09.533635+0000 mgr.y (mgr.14535) 390 : cluster [DBG] pgmap v494: 148 pgs: 1 active+clean+remapped, 6 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 849 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 29 KiB/s rd, 0 B/s wr, 456 op/s; 8279/41307 objects misplaced (20.043%); 18 MiB/s, 306 objects/s recovering 2024-02-22T19:39:11.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:10.411861+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e264: 8 total, 8 up, 7 in 2024-02-22T19:39:11.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:09.533635+0000 mgr.y (mgr.14535) 390 : cluster [DBG] pgmap v494: 148 pgs: 1 active+clean+remapped, 6 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 849 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 29 KiB/s rd, 0 B/s wr, 456 op/s; 8279/41307 objects misplaced (20.043%); 18 MiB/s, 306 objects/s recovering 2024-02-22T19:39:11.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:10.411861+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e264: 8 total, 8 up, 7 in 2024-02-22T19:39:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:09.533635+0000 mgr.y (mgr.14535) 390 : cluster [DBG] pgmap v494: 148 pgs: 1 active+clean+remapped, 6 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 136 active+clean; 849 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 29 KiB/s rd, 0 B/s wr, 456 op/s; 8279/41307 objects misplaced (20.043%); 18 MiB/s, 306 objects/s recovering 2024-02-22T19:39:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:10.411861+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e264: 8 total, 8 up, 7 in 2024-02-22T19:39:11.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:39:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:12.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:12.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:12.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:12.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:11.964178+0000 mon.a (mon.0) 1202 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:12.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:11.964178+0000 mon.a (mon.0) 1202 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:12.447 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:11.964178+0000 mon.a (mon.0) 1202 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:12.450 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:12.721 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:12.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:12.731 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:12.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:13.059 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:13.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:13.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:13.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:13.227 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:13.232 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:39:13.232 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:39:13.233 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [1, 2, 3, 6, 7, 5, 0] 2024-02-22T19:39:13.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 0 2024-02-22T19:39:13.287 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:13.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:13.294 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:11.534125+0000 mgr.y (mgr.14535) 391 : cluster [DBG] pgmap v496: 148 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 827 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 34 KiB/s rd, 0 B/s wr, 478 op/s; 8060/40260 objects misplaced (20.020%); 16 MiB/s, 272 objects/s recovering 2024-02-22T19:39:13.294 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:11.534125+0000 mgr.y (mgr.14535) 391 : cluster [DBG] pgmap v496: 148 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 827 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 34 KiB/s rd, 0 B/s wr, 478 op/s; 8060/40260 objects misplaced (20.020%); 16 MiB/s, 272 objects/s recovering 2024-02-22T19:39:13.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:11.534125+0000 mgr.y (mgr.14535) 391 : cluster [DBG] pgmap v496: 148 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 827 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 34 KiB/s rd, 0 B/s wr, 478 op/s; 8060/40260 objects misplaced (20.020%); 16 MiB/s, 272 objects/s recovering 2024-02-22T19:39:13.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:13.428 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:13.442 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:13.492 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:13.778 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:13.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:13.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:13.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:13.912 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:14.260 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:39:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:13.249794+0000 mon.a (mon.0) 1203 : cluster [DBG] osdmap e265: 8 total, 8 up, 7 in 2024-02-22T19:39:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:13.535715+0000 mgr.y (mgr.14535) 392 : cluster [DBG] pgmap v498: 148 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 827 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 28 KiB/s rd, 0 B/s wr, 388 op/s; 8060/40260 objects misplaced (20.020%); 13 MiB/s, 221 objects/s recovering 2024-02-22T19:39:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:13.792139+0000 mon.a (mon.0) 1204 : audit [INF] from='client.? 172.21.15.53:0/2146497067' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:14.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:13.249794+0000 mon.a (mon.0) 1203 : cluster [DBG] osdmap e265: 8 total, 8 up, 7 in 2024-02-22T19:39:14.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:13.535715+0000 mgr.y (mgr.14535) 392 : cluster [DBG] pgmap v498: 148 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 827 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 28 KiB/s rd, 0 B/s wr, 388 op/s; 8060/40260 objects misplaced (20.020%); 13 MiB/s, 221 objects/s recovering 2024-02-22T19:39:14.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:13.792139+0000 mon.a (mon.0) 1204 : audit [INF] from='client.? 172.21.15.53:0/2146497067' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:13.249794+0000 mon.a (mon.0) 1203 : cluster [DBG] osdmap e265: 8 total, 8 up, 7 in 2024-02-22T19:39:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:13.535715+0000 mgr.y (mgr.14535) 392 : cluster [DBG] pgmap v498: 148 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 136 active+clean; 827 MiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 28 KiB/s rd, 0 B/s wr, 388 op/s; 8060/40260 objects misplaced (20.020%); 13 MiB/s, 221 objects/s recovering 2024-02-22T19:39:14.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:13.792139+0000 mon.a (mon.0) 1204 : audit [INF] from='client.? 172.21.15.53:0/2146497067' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:14.869 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:14.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:14.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:15.018 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:39:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:39:15.019 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:39:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:39:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:15.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:15.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:15.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:15.197 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:15.227 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:15.234 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:15.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:15.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:13.900417+0000 osd.3 (osd.3) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:13.900483+0000 osd.3 (osd.3) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:14.246658+0000 mon.a (mon.0) 1205 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:14.254841+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? 172.21.15.53:0/2146497067' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:39:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:14.254902+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e266: 8 total, 8 up, 7 in 2024-02-22T19:39:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:14.829176+0000 mgr.y (mgr.14535) 393 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:15.391 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:15.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:13.900417+0000 osd.3 (osd.3) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:15.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:13.900483+0000 osd.3 (osd.3) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:15.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:14.246658+0000 mon.a (mon.0) 1205 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:15.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:14.254841+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? 172.21.15.53:0/2146497067' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:39:15.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:14.254902+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e266: 8 total, 8 up, 7 in 2024-02-22T19:39:15.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:14.829176+0000 mgr.y (mgr.14535) 393 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:15.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:13.900417+0000 osd.3 (osd.3) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:15.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:13.900483+0000 osd.3 (osd.3) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:15.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:14.246658+0000 mon.a (mon.0) 1205 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:15.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:14.254841+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? 172.21.15.53:0/2146497067' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:39:15.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:14.254902+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e266: 8 total, 8 up, 7 in 2024-02-22T19:39:15.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:14.829176+0000 mgr.y (mgr.14535) 393 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:15.787 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:15.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:15.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:15.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:15.974 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:15.977 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:15.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:16.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:16.177 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:16.194 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:15.536510+0000 mgr.y (mgr.14535) 394 : cluster [DBG] pgmap v500: 148 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 138 active+clean; 634 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 55 KiB/s rd, 0 B/s wr, 662 op/s; 5589/31029 objects misplaced (18.012%); 7.7 MiB/s, 127 objects/s recovering 2024-02-22T19:39:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:15.536510+0000 mgr.y (mgr.14535) 394 : cluster [DBG] pgmap v500: 148 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 138 active+clean; 634 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 55 KiB/s rd, 0 B/s wr, 662 op/s; 5589/31029 objects misplaced (18.012%); 7.7 MiB/s, 127 objects/s recovering 2024-02-22T19:39:16.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:15.536510+0000 mgr.y (mgr.14535) 394 : cluster [DBG] pgmap v500: 148 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 138 active+clean; 634 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 55 KiB/s rd, 0 B/s wr, 662 op/s; 5589/31029 objects misplaced (18.012%); 7.7 MiB/s, 127 objects/s recovering 2024-02-22T19:39:17.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:39:18.078 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:39:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:39:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:39:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:39:18.079 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:39:18.726 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:18.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:18.736 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:18.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:18.885 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:17.537204+0000 mgr.y (mgr.14535) 395 : cluster [DBG] pgmap v501: 148 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 138 active+clean; 634 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 41 KiB/s rd, 0 B/s wr, 499 op/s; 5589/31029 objects misplaced (18.012%); 6.1 MiB/s, 97 objects/s recovering 2024-02-22T19:39:18.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:17.537204+0000 mgr.y (mgr.14535) 395 : cluster [DBG] pgmap v501: 148 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 138 active+clean; 634 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 41 KiB/s rd, 0 B/s wr, 499 op/s; 5589/31029 objects misplaced (18.012%); 6.1 MiB/s, 97 objects/s recovering 2024-02-22T19:39:18.918 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:17.537204+0000 mgr.y (mgr.14535) 395 : cluster [DBG] pgmap v501: 148 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 138 active+clean; 634 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 41 KiB/s rd, 0 B/s wr, 499 op/s; 5589/31029 objects misplaced (18.012%); 6.1 MiB/s, 97 objects/s recovering 2024-02-22T19:39:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:19.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:19.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:19.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:39:19.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:19.583 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:39:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:19] "GET /metrics HTTP/1.1" 200 175966 "" "Prometheus/2.33.4" 2024-02-22T19:39:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:19.744 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:19.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:19.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:19.775 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:19.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:19.626116+0000 mon.a (mon.0) 1208 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:19.836 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:19.626116+0000 mon.a (mon.0) 1208 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:19.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:19.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:19.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:20.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:20.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:19.626116+0000 mon.a (mon.0) 1208 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:20.185 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:20.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:20.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:20.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:20.433 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:20.437 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:20.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:20.538 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:20.570 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:20.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:20.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:20.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:20.777 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:20.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:19.538206+0000 mgr.y (mgr.14535) 396 : cluster [DBG] pgmap v502: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 437 MiB data, 3.5 GiB used, 622 GiB / 626 GiB avail; 37 KiB/s rd, 0 B/s wr, 839 op/s; 3692/21567 objects misplaced (17.119%); 6.8 MiB/s, 108 objects/s recovering 2024-02-22T19:39:20.986 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:19.538206+0000 mgr.y (mgr.14535) 396 : cluster [DBG] pgmap v502: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 437 MiB data, 3.5 GiB used, 622 GiB / 626 GiB avail; 37 KiB/s rd, 0 B/s wr, 839 op/s; 3692/21567 objects misplaced (17.119%); 6.8 MiB/s, 108 objects/s recovering 2024-02-22T19:39:20.987 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:19.538206+0000 mgr.y (mgr.14535) 396 : cluster [DBG] pgmap v502: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 437 MiB data, 3.5 GiB used, 622 GiB / 626 GiB avail; 37 KiB/s rd, 0 B/s wr, 839 op/s; 3692/21567 objects misplaced (17.119%); 6.8 MiB/s, 108 objects/s recovering 2024-02-22T19:39:21.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:39:21.111+0000 7fe2d794e700 -1 mon.a@0(leader).osd e266 definitely_dead 0 2024-02-22T19:39:21.738 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:21.105884+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.53:0/772142367' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:39:21.738 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:21.112375+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:39:21.738 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:21.112479+0000 mon.a (mon.0) 1210 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-22T19:39:21.738 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:21.380172+0000 mon.b (mon.1) 67 : audit [DBG] from='client.? 172.21.15.53:0/3921923924' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:21.749 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.0. 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:39:21.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:39:21.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:39:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:21.105884+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.53:0/772142367' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:39:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:21.112375+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:39:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:21.112479+0000 mon.a (mon.0) 1210 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-22T19:39:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:21.380172+0000 mon.b (mon.1) 67 : audit [DBG] from='client.? 172.21.15.53:0/3921923924' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:22.081 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:22.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:22.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:21.105884+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.53:0/772142367' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:39:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:21.112375+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-22T19:39:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:21.112479+0000 mon.a (mon.0) 1210 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-22T19:39:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:21.380172+0000 mon.b (mon.1) 67 : audit [DBG] from='client.? 172.21.15.53:0/3921923924' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:22.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:22.213 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:22.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:22.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:22.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:22.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:22.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:22.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:22.536 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:22.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:22.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:22.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:22.689 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:22.714 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:22.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:22.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:21.542347+0000 mgr.y (mgr.14535) 397 : cluster [DBG] pgmap v503: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 393 MiB data, 3.4 GiB used, 622 GiB / 626 GiB avail; 36 KiB/s rd, 0 B/s wr, 895 op/s; 3299/19434 objects misplaced (16.975%); 6.6 MiB/s, 104 objects/s recovering 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:21.730177+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:21.730247+0000 mon.a (mon.0) 1212 : cluster [DBG] osdmap e267: 8 total, 8 up, 6 in 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:22.224548+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e268: 8 total, 8 up, 6 in 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:21.542347+0000 mgr.y (mgr.14535) 397 : cluster [DBG] pgmap v503: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 393 MiB data, 3.4 GiB used, 622 GiB / 626 GiB avail; 36 KiB/s rd, 0 B/s wr, 895 op/s; 3299/19434 objects misplaced (16.975%); 6.6 MiB/s, 104 objects/s recovering 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:21.730177+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-22T19:39:22.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:21.730247+0000 mon.a (mon.0) 1212 : cluster [DBG] osdmap e267: 8 total, 8 up, 6 in 2024-02-22T19:39:22.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:22.224548+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e268: 8 total, 8 up, 6 in 2024-02-22T19:39:22.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:22.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:21.542347+0000 mgr.y (mgr.14535) 397 : cluster [DBG] pgmap v503: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 393 MiB data, 3.4 GiB used, 622 GiB / 626 GiB avail; 36 KiB/s rd, 0 B/s wr, 895 op/s; 3299/19434 objects misplaced (16.975%); 6.6 MiB/s, 104 objects/s recovering 2024-02-22T19:39:22.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:21.730177+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-22T19:39:22.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:21.730247+0000 mon.a (mon.0) 1212 : cluster [DBG] osdmap e267: 8 total, 8 up, 6 in 2024-02-22T19:39:22.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:22.224548+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e268: 8 total, 8 up, 6 in 2024-02-22T19:39:22.844 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:22.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:22.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:22.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:23.005 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:23.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:22.239583+0000 osd.0 (osd.0) 161 : cluster [INF] 8.2 continuing backfill to osd.5 from (197'9056,266'13526] MIN to 266'13526 2024-02-22T19:39:23.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:22.239615+0000 osd.0 (osd.0) 162 : cluster [DBG] 8.2 starting backfill to osd.6 from (185'7681,259'12967] MIN to 266'13526 2024-02-22T19:39:23.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:23.225403+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e269: 8 total, 8 up, 6 in 2024-02-22T19:39:23.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:23.227360+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.e starting backfill to osd.3 from (0'0,0'0] MAX to 264'23 2024-02-22T19:39:23.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:23.230911+0000 osd.3 (osd.3) 160 : cluster [DBG] 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 266'13636 2024-02-22T19:39:23.966 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:22.239583+0000 osd.0 (osd.0) 161 : cluster [INF] 8.2 continuing backfill to osd.5 from (197'9056,266'13526] MIN to 266'13526 2024-02-22T19:39:23.966 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:22.239615+0000 osd.0 (osd.0) 162 : cluster [DBG] 8.2 starting backfill to osd.6 from (185'7681,259'12967] MIN to 266'13526 2024-02-22T19:39:23.966 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:23.225403+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e269: 8 total, 8 up, 6 in 2024-02-22T19:39:23.966 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:23.227360+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.e starting backfill to osd.3 from (0'0,0'0] MAX to 264'23 2024-02-22T19:39:23.966 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:23.230911+0000 osd.3 (osd.3) 160 : cluster [DBG] 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 266'13636 2024-02-22T19:39:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:22.239583+0000 osd.0 (osd.0) 161 : cluster [INF] 8.2 continuing backfill to osd.5 from (197'9056,266'13526] MIN to 266'13526 2024-02-22T19:39:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:22.239615+0000 osd.0 (osd.0) 162 : cluster [DBG] 8.2 starting backfill to osd.6 from (185'7681,259'12967] MIN to 266'13526 2024-02-22T19:39:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:23.225403+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e269: 8 total, 8 up, 6 in 2024-02-22T19:39:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:23.227360+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.e starting backfill to osd.3 from (0'0,0'0] MAX to 264'23 2024-02-22T19:39:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:23.230911+0000 osd.3 (osd.3) 160 : cluster [DBG] 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 266'13636 2024-02-22T19:39:24.485 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:24.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:24.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:24.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:24.629 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:24.654 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:24.656 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:24.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:24.758 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:24.793 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:24.823 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:24.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:24.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:24.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:24.964 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:23.224738+0000 osd.6 (osd.6) 109 : cluster [INF] 8.9 continuing backfill to osd.5 from (197'9147,266'13807] 8:97fd7221:::benchmark_data_smithi053_130774_object110418:head to 266'13807 2024-02-22T19:39:24.964 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:23.234979+0000 osd.2 (osd.2) 127 : cluster [DBG] 8.e starting backfill to osd.1 from (0'0,0'0] MAX to 266'13751 2024-02-22T19:39:24.964 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:23.542821+0000 mgr.y (mgr.14535) 398 : cluster [DBG] pgmap v507: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 393 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 644 op/s; 3299/19434 objects misplaced (16.975%); 1.9 MiB/s, 29 objects/s recovering 2024-02-22T19:39:24.964 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:23.839570+0000 mon.a (mon.0) 1215 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:24.964 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:24.718558+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.53:0/1126805274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:39:24.965 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:24.719088+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:39:24.965 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:39:24.965 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:39:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:39:24.968 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:24.999 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:25.003 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:25.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:25.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:23.224738+0000 osd.6 (osd.6) 109 : cluster [INF] 8.9 continuing backfill to osd.5 from (197'9147,266'13807] 8:97fd7221:::benchmark_data_smithi053_130774_object110418:head to 266'13807 2024-02-22T19:39:25.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:23.234979+0000 osd.2 (osd.2) 127 : cluster [DBG] 8.e starting backfill to osd.1 from (0'0,0'0] MAX to 266'13751 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:23.542821+0000 mgr.y (mgr.14535) 398 : cluster [DBG] pgmap v507: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 393 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 644 op/s; 3299/19434 objects misplaced (16.975%); 1.9 MiB/s, 29 objects/s recovering 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:23.839570+0000 mon.a (mon.0) 1215 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:24.718558+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.53:0/1126805274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:24.719088+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:23.224738+0000 osd.6 (osd.6) 109 : cluster [INF] 8.9 continuing backfill to osd.5 from (197'9147,266'13807] 8:97fd7221:::benchmark_data_smithi053_130774_object110418:head to 266'13807 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:23.234979+0000 osd.2 (osd.2) 127 : cluster [DBG] 8.e starting backfill to osd.1 from (0'0,0'0] MAX to 266'13751 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:23.542821+0000 mgr.y (mgr.14535) 398 : cluster [DBG] pgmap v507: 148 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 393 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 644 op/s; 3299/19434 objects misplaced (16.975%); 1.9 MiB/s, 29 objects/s recovering 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:23.839570+0000 mon.a (mon.0) 1215 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:24.718558+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.53:0/1126805274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:39:25.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:24.719088+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:39:25.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:25.146 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:25.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:25.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:25.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:39:25.362 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:25.535 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:25.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:25.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:25.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:25.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:39:25.812 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:26.034 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:26.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:26.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:24.839172+0000 mgr.y (mgr.14535) 399 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:25.231482+0000 mon.a (mon.0) 1217 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.255890+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:25.255952+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e270: 8 total, 8 up, 6 in 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.422724+0000 mon.a (mon.0) 1220 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:24.839172+0000 mgr.y (mgr.14535) 399 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:25.231482+0000 mon.a (mon.0) 1217 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.255890+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:25.255952+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e270: 8 total, 8 up, 6 in 2024-02-22T19:39:26.070 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.422724+0000 mon.a (mon.0) 1220 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:39:26.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:26.181 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:24.839172+0000 mgr.y (mgr.14535) 399 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:26.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:25.231482+0000 mon.a (mon.0) 1217 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:26.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.255890+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:39:26.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:25.255952+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e270: 8 total, 8 up, 6 in 2024-02-22T19:39:26.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.422724+0000 mon.a (mon.0) 1220 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:39:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:26.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:26.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:26.416 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:26.803 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:26.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:26.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:26.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:26.949 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:39:26.949 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:39:26.970 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:25.543446+0000 mgr.y (mgr.14535) 400 : cluster [DBG] pgmap v509: 148 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 588 op/s; 1996/12375 objects misplaced (16.129%); 2.6 MiB/s, 61 objects/s recovering 2024-02-22T19:39:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.953833+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.959080+0000 mon.a (mon.0) 1222 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.960297+0000 mon.a (mon.0) 1223 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.961348+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:39:25.962052+0000 mgr.y (mgr.14535) 401 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 3257M 2024-02-22T19:39:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.971049+0000 mon.a (mon.0) 1225 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.972704+0000 mon.a (mon.0) 1226 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:39:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.973829+0000 mon.a (mon.0) 1227 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:39:27.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:25.988529+0000 mon.a (mon.0) 1228 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:25.543446+0000 mgr.y (mgr.14535) 400 : cluster [DBG] pgmap v509: 148 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 588 op/s; 1996/12375 objects misplaced (16.129%); 2.6 MiB/s, 61 objects/s recovering 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.953833+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.959080+0000 mon.a (mon.0) 1222 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.960297+0000 mon.a (mon.0) 1223 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.961348+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:39:25.962052+0000 mgr.y (mgr.14535) 401 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 3257M 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.971049+0000 mon.a (mon.0) 1225 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.972704+0000 mon.a (mon.0) 1226 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.973829+0000 mon.a (mon.0) 1227 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:39:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:25.988529+0000 mon.a (mon.0) 1228 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:25.543446+0000 mgr.y (mgr.14535) 400 : cluster [DBG] pgmap v509: 148 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 588 op/s; 1996/12375 objects misplaced (16.129%); 2.6 MiB/s, 61 objects/s recovering 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.953833+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.959080+0000 mon.a (mon.0) 1222 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.960297+0000 mon.a (mon.0) 1223 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.961348+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:39:25.962052+0000 mgr.y (mgr.14535) 401 : cephadm [INF] Adjusting osd_memory_target on smithi053 to 3257M 2024-02-22T19:39:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.971049+0000 mon.a (mon.0) 1225 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:27.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.972704+0000 mon.a (mon.0) 1226 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:39:27.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.973829+0000 mon.a (mon.0) 1227 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:39:27.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:25.988529+0000 mon.a (mon.0) 1228 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:27.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:39:27.726 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:27.749 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:27.759 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:27.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:27.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:27.984 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:28.042 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:28.050 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:28.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:28.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:28.213 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:28.220 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:28.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:28.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:28.379 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:39:28.380 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:39:28.380 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-02-22T19:39:28.380 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:39:28.380 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:39:28.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-02-22T19:39:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:28.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:28.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:28.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:28.520 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:27.543933+0000 mgr.y (mgr.14535) 402 : cluster [DBG] pgmap v510: 148 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 404 op/s; 1996/12375 objects misplaced (16.129%); 1.8 MiB/s, 42 objects/s recovering 2024-02-22T19:39:28.526 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:28.549 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:28.553 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:28.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:28.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:28.683 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:28.708 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:28.711 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:28.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:28.711 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:39:28.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:27.543933+0000 mgr.y (mgr.14535) 402 : cluster [DBG] pgmap v510: 148 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 404 op/s; 1996/12375 objects misplaced (16.129%); 1.8 MiB/s, 42 objects/s recovering 2024-02-22T19:39:28.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:27.543933+0000 mgr.y (mgr.14535) 402 : cluster [DBG] pgmap v510: 148 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 404 op/s; 1996/12375 objects misplaced (16.129%); 1.8 MiB/s, 42 objects/s recovering 2024-02-22T19:39:28.812 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:28.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:39:28.840 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:28.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:28.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:29.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:29.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:29.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:29.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:29.304 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:29.505 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:29.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:28.266957+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e271: 8 total, 8 up, 6 in 2024-02-22T19:39:29.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:29.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:29.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:29.644 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:28.266957+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e271: 8 total, 8 up, 6 in 2024-02-22T19:39:29.644 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:39:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:29] "GET /metrics HTTP/1.1" 200 176013 "" "Prometheus/2.33.4" 2024-02-22T19:39:29.645 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:28.266957+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e271: 8 total, 8 up, 6 in 2024-02-22T19:39:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:30.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:29.267497+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e272: 8 total, 8 up, 6 in 2024-02-22T19:39:30.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:29.544743+0000 mgr.y (mgr.14535) 403 : cluster [DBG] pgmap v513: 148 pgs: 1 active, 1 peering, 1 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 793 op/s; 768/5142 objects misplaced (14.936%); 3.3 MiB/s, 66 objects/s recovering 2024-02-22T19:39:30.365 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:30.585 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:39:30.585 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-22T19:39:30.585 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:39:30.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:29.267497+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e272: 8 total, 8 up, 6 in 2024-02-22T19:39:30.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:29.544743+0000 mgr.y (mgr.14535) 403 : cluster [DBG] pgmap v513: 148 pgs: 1 active, 1 peering, 1 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 793 op/s; 768/5142 objects misplaced (14.936%); 3.3 MiB/s, 66 objects/s recovering 2024-02-22T19:39:30.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:29.267497+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e272: 8 total, 8 up, 6 in 2024-02-22T19:39:30.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:29.544743+0000 mgr.y (mgr.14535) 403 : cluster [DBG] pgmap v513: 148 pgs: 1 active, 1 peering, 1 active+remapped+backfilling, 1 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 138 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 793 op/s; 768/5142 objects misplaced (14.936%); 3.3 MiB/s, 66 objects/s recovering 2024-02-22T19:39:31.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:30.276363+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e273: 8 total, 8 up, 6 in 2024-02-22T19:39:31.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:30.276363+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e273: 8 total, 8 up, 6 in 2024-02-22T19:39:31.560 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Removed 111731 objects 2024-02-22T19:39:31.560 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Clean up completed and total clean up time :148.511 2024-02-22T19:39:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:30.276363+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e273: 8 total, 8 up, 6 in 2024-02-22T19:39:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:31.277796+0000 mon.a (mon.0) 1232 : cluster [DBG] osdmap e274: 8 total, 8 up, 6 in 2024-02-22T19:39:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:31.314467+0000 osd.2 (osd.2) 128 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:31.314508+0000 osd.2 (osd.2) 129 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:31.545218+0000 mgr.y (mgr.14535) 404 : cluster [DBG] pgmap v516: 148 pgs: 1 active, 1 peering, 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 810 op/s; 258/2658 objects misplaced (9.707%); 2.6 MiB/s, 41 objects/s recovering 2024-02-22T19:39:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.228189+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e275: 8 total, 8 up, 6 in 2024-02-22T19:39:32.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:31.277796+0000 mon.a (mon.0) 1232 : cluster [DBG] osdmap e274: 8 total, 8 up, 6 in 2024-02-22T19:39:32.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:31.314467+0000 osd.2 (osd.2) 128 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:32.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:31.314508+0000 osd.2 (osd.2) 129 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:32.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:31.545218+0000 mgr.y (mgr.14535) 404 : cluster [DBG] pgmap v516: 148 pgs: 1 active, 1 peering, 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 810 op/s; 258/2658 objects misplaced (9.707%); 2.6 MiB/s, 41 objects/s recovering 2024-02-22T19:39:32.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.228189+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e275: 8 total, 8 up, 6 in 2024-02-22T19:39:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:31.277796+0000 mon.a (mon.0) 1232 : cluster [DBG] osdmap e274: 8 total, 8 up, 6 in 2024-02-22T19:39:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:31.314467+0000 osd.2 (osd.2) 128 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:31.314508+0000 osd.2 (osd.2) 129 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:31.545218+0000 mgr.y (mgr.14535) 404 : cluster [DBG] pgmap v516: 148 pgs: 1 active, 1 peering, 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 810 op/s; 258/2658 objects misplaced (9.707%); 2.6 MiB/s, 41 objects/s recovering 2024-02-22T19:39:32.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.228189+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e275: 8 total, 8 up, 6 in 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:31.768333+0000 osd.3 (osd.3) 161 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:31.771241+0000 osd.3 (osd.3) 162 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.385282+0000 osd.1 (osd.1) 144 : cluster [DBG] 6.e scrub starts 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.393312+0000 osd.1 (osd.1) 145 : cluster [DBG] 6.e scrub ok 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.444643+0000 osd.2 (osd.2) 130 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.447642+0000 osd.2 (osd.2) 131 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:39:33.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:33.229801+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e276: 8 total, 8 up, 6 in 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:31.768333+0000 osd.3 (osd.3) 161 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:31.771241+0000 osd.3 (osd.3) 162 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.385282+0000 osd.1 (osd.1) 144 : cluster [DBG] 6.e scrub starts 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.393312+0000 osd.1 (osd.1) 145 : cluster [DBG] 6.e scrub ok 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.444643+0000 osd.2 (osd.2) 130 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.447642+0000 osd.2 (osd.2) 131 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:33.229801+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e276: 8 total, 8 up, 6 in 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:31.768333+0000 osd.3 (osd.3) 161 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:39:33.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:31.771241+0000 osd.3 (osd.3) 162 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:39:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.385282+0000 osd.1 (osd.1) 144 : cluster [DBG] 6.e scrub starts 2024-02-22T19:39:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.393312+0000 osd.1 (osd.1) 145 : cluster [DBG] 6.e scrub ok 2024-02-22T19:39:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.444643+0000 osd.2 (osd.2) 130 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:39:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.447642+0000 osd.2 (osd.2) 131 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:39:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:33.229801+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e276: 8 total, 8 up, 6 in 2024-02-22T19:39:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.698832+0000 osd.7 (osd.7) 168 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:39:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:32.702530+0000 osd.7 (osd.7) 169 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:39:34.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:33.545626+0000 mgr.y (mgr.14535) 405 : cluster [DBG] pgmap v519: 148 pgs: 1 active, 1 peering, 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 206 op/s; 258/2658 objects misplaced (9.707%); 272 KiB/s, 4 objects/s recovering 2024-02-22T19:39:34.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.698832+0000 osd.7 (osd.7) 168 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:39:34.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:32.702530+0000 osd.7 (osd.7) 169 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:39:34.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:33.545626+0000 mgr.y (mgr.14535) 405 : cluster [DBG] pgmap v519: 148 pgs: 1 active, 1 peering, 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 206 op/s; 258/2658 objects misplaced (9.707%); 272 KiB/s, 4 objects/s recovering 2024-02-22T19:39:34.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.698832+0000 osd.7 (osd.7) 168 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:39:34.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:32.702530+0000 osd.7 (osd.7) 169 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:39:34.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:33.545626+0000 mgr.y (mgr.14535) 405 : cluster [DBG] pgmap v519: 148 pgs: 1 active, 1 peering, 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 138 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 206 op/s; 258/2658 objects misplaced (9.707%); 272 KiB/s, 4 objects/s recovering 2024-02-22T19:39:35.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:39:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:39:35.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:39:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:39:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:33.733470+0000 osd.7 (osd.7) 170 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:39:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:33.737063+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:39:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:34.407079+0000 osd.2 (osd.2) 132 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:39:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:34.410543+0000 osd.2 (osd.2) 133 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:39:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:34.553326+0000 osd.6 (osd.6) 110 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:39:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:34.555293+0000 osd.6 (osd.6) 111 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:39:35.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:34.849361+0000 mgr.y (mgr.14535) 406 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:35.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:33.733470+0000 osd.7 (osd.7) 170 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:33.737063+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:34.407079+0000 osd.2 (osd.2) 132 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:34.410543+0000 osd.2 (osd.2) 133 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:34.553326+0000 osd.6 (osd.6) 110 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:34.555293+0000 osd.6 (osd.6) 111 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:34.849361+0000 mgr.y (mgr.14535) 406 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:33.733470+0000 osd.7 (osd.7) 170 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:33.737063+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:39:35.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:34.407079+0000 osd.2 (osd.2) 132 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:39:35.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:34.410543+0000 osd.2 (osd.2) 133 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:39:35.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:34.553326+0000 osd.6 (osd.6) 110 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:39:35.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:34.555293+0000 osd.6 (osd.6) 111 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:39:35.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:34.849361+0000 mgr.y (mgr.14535) 406 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:35.691 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:35.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:35.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:35.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:35.833 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:35.858 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:35.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:35.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:35.998 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:36.291 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:39:36.291 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:39:36.291 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-02-22T19:39:36.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-22T19:39:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:34.809731+0000 osd.3 (osd.3) 163 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:39:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:34.813219+0000 osd.3 (osd.3) 164 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:39:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:35.328878+0000 osd.1 (osd.1) 146 : cluster [DBG] 4.f scrub starts 2024-02-22T19:39:36.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:35.330909+0000 osd.1 (osd.1) 147 : cluster [DBG] 4.f scrub ok 2024-02-22T19:39:36.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:35.546615+0000 mgr.y (mgr.14535) 407 : cluster [DBG] pgmap v520: 148 pgs: 6 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 971 B/s rd, 0 B/s wr, 132 op/s; 1/576 objects misplaced (0.174%); 291 KiB/s, 4 objects/s recovering 2024-02-22T19:39:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:36.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:34.809731+0000 osd.3 (osd.3) 163 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:39:36.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:34.813219+0000 osd.3 (osd.3) 164 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:39:36.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:35.328878+0000 osd.1 (osd.1) 146 : cluster [DBG] 4.f scrub starts 2024-02-22T19:39:36.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:35.330909+0000 osd.1 (osd.1) 147 : cluster [DBG] 4.f scrub ok 2024-02-22T19:39:36.655 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:35.546615+0000 mgr.y (mgr.14535) 407 : cluster [DBG] pgmap v520: 148 pgs: 6 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 971 B/s rd, 0 B/s wr, 132 op/s; 1/576 objects misplaced (0.174%); 291 KiB/s, 4 objects/s recovering 2024-02-22T19:39:36.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:34.809731+0000 osd.3 (osd.3) 163 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:39:36.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:34.813219+0000 osd.3 (osd.3) 164 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:39:36.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:35.328878+0000 osd.1 (osd.1) 146 : cluster [DBG] 4.f scrub starts 2024-02-22T19:39:36.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:35.330909+0000 osd.1 (osd.1) 147 : cluster [DBG] 4.f scrub ok 2024-02-22T19:39:36.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:35.546615+0000 mgr.y (mgr.14535) 407 : cluster [DBG] pgmap v520: 148 pgs: 6 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 971 B/s rd, 0 B/s wr, 132 op/s; 1/576 objects misplaced (0.174%); 291 KiB/s, 4 objects/s recovering 2024-02-22T19:39:36.656 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:36.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:36.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:36.813 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:36.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:36.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:36.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:36.945 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:36.946 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-02-22T19:39:36.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-02-22T19:39:37.145 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:37.148 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:37.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:37.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:37.276 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:37.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:37.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:37.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:37.560 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:36.294820+0000 osd.1 (osd.1) 148 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:39:37.560 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:36.295983+0000 osd.1 (osd.1) 149 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:39:37.560 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:36.384202+0000 osd.2 (osd.2) 134 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:39:37.560 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:36.387852+0000 osd.2 (osd.2) 135 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:39:37.560 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:36.526057+0000 osd.6 (osd.6) 112 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:39:37.561 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:36.528170+0000 osd.6 (osd.6) 113 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:39:37.563 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:37.584 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:37.587 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:37.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:37.688 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:39:37.689 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:39:37.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:37.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:36.294820+0000 osd.1 (osd.1) 148 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:39:37.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:36.295983+0000 osd.1 (osd.1) 149 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:36.384202+0000 osd.2 (osd.2) 134 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:36.387852+0000 osd.2 (osd.2) 135 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:36.526057+0000 osd.6 (osd.6) 112 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:36.528170+0000 osd.6 (osd.6) 113 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:36.294820+0000 osd.1 (osd.1) 148 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:36.295983+0000 osd.1 (osd.1) 149 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:36.384202+0000 osd.2 (osd.2) 134 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:36.387852+0000 osd.2 (osd.2) 135 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:36.526057+0000 osd.6 (osd.6) 112 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:39:37.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:36.528170+0000 osd.6 (osd.6) 113 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:39:37.719 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:37.740 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:37.742 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:37.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:37.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:37.876 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:38.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.300540+0000 osd.1 (osd.1) 150 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:39:38.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.302886+0000 osd.1 (osd.1) 151 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:39:38.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.547214+0000 mgr.y (mgr.14535) 408 : cluster [DBG] pgmap v521: 148 pgs: 6 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 816 B/s rd, 0 B/s wr, 111 op/s; 1/576 objects misplaced (0.174%); 245 KiB/s, 3 objects/s recovering 2024-02-22T19:39:38.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.853031+0000 osd.1 (osd.1) 152 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:38.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.853080+0000 osd.1 (osd.1) 153 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:38.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:37.917519+0000 mon.a (mon.0) 1235 : audit [DBG] from='client.? 172.21.15.53:0/3158407961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:38.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.300540+0000 osd.1 (osd.1) 150 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:39:38.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.302886+0000 osd.1 (osd.1) 151 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:39:38.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.547214+0000 mgr.y (mgr.14535) 408 : cluster [DBG] pgmap v521: 148 pgs: 6 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 816 B/s rd, 0 B/s wr, 111 op/s; 1/576 objects misplaced (0.174%); 245 KiB/s, 3 objects/s recovering 2024-02-22T19:39:38.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.853031+0000 osd.1 (osd.1) 152 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:38.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.853080+0000 osd.1 (osd.1) 153 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:38.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:37.917519+0000 mon.a (mon.0) 1235 : audit [DBG] from='client.? 172.21.15.53:0/3158407961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:38.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:39:38.528 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:38.530 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:38.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.300540+0000 osd.1 (osd.1) 150 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:39:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.302886+0000 osd.1 (osd.1) 151 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:39:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.547214+0000 mgr.y (mgr.14535) 408 : cluster [DBG] pgmap v521: 148 pgs: 6 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 816 B/s rd, 0 B/s wr, 111 op/s; 1/576 objects misplaced (0.174%); 245 KiB/s, 3 objects/s recovering 2024-02-22T19:39:38.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.853031+0000 osd.1 (osd.1) 152 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:38.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.853080+0000 osd.1 (osd.1) 153 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:38.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:37.917519+0000 mon.a (mon.0) 1235 : audit [DBG] from='client.? 172.21.15.53:0/3158407961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:38.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:38.654 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:39:38.654 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:39:38.654 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:39:38.654 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:39:38.658 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:38.856 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:38.861 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:38.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:38.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:39.520 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:39.520 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:39.523 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:39] "GET /metrics HTTP/1.1" 200 175982 "" "Prometheus/2.33.4" 2024-02-22T19:39:39.523 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.800538+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:39:39.523 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.803470+0000 osd.7 (osd.7) 173 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.812792+0000 osd.3 (osd.3) 165 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:37.816652+0000 osd.3 (osd.3) 166 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:38.308670+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e277: 8 total, 8 up, 6 in 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:38.561456+0000 osd.6 (osd.6) 114 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:38.562494+0000 osd.6 (osd.6) 115 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:38.874855+0000 mon.a (mon.0) 1237 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.800538+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.803470+0000 osd.7 (osd.7) 173 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.812792+0000 osd.3 (osd.3) 165 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:37.816652+0000 osd.3 (osd.3) 166 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:39:39.524 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:38.308670+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e277: 8 total, 8 up, 6 in 2024-02-22T19:39:39.525 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:38.561456+0000 osd.6 (osd.6) 114 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:39:39.525 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:38.562494+0000 osd.6 (osd.6) 115 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:39:39.525 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:38.874855+0000 mon.a (mon.0) 1237 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:39.538 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:39.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:39.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.800538+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.803470+0000 osd.7 (osd.7) 173 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.812792+0000 osd.3 (osd.3) 165 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:37.816652+0000 osd.3 (osd.3) 166 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:38.308670+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e277: 8 total, 8 up, 6 in 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:38.561456+0000 osd.6 (osd.6) 114 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:39:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:38.562494+0000 osd.6 (osd.6) 115 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:39:39.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:38.874855+0000 mon.a (mon.0) 1237 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:39.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:39.699 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:39.699 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:39.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:39.852 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:39.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:39.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:39.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:39.984 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:40.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:40.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:40.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:40.472 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:40.475 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:40.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:40.576 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:39.315533+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e278: 8 total, 8 up, 6 in 2024-02-22T19:39:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:39.400761+0000 osd.2 (osd.2) 136 : cluster [DBG] 5.b scrub starts 2024-02-22T19:39:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:39.403771+0000 osd.2 (osd.2) 137 : cluster [DBG] 5.b scrub ok 2024-02-22T19:39:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:39.547740+0000 mgr.y (mgr.14535) 409 : cluster [DBG] pgmap v524: 148 pgs: 1 active, 5 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 111 op/s; 1/576 objects misplaced (0.174%); 243 KiB/s, 3 objects/s recovering 2024-02-22T19:39:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:39.864871+0000 osd.0 (osd.0) 163 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:39.864919+0000 osd.0 (osd.0) 164 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:40.608 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:40.629 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:40.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:40.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:40.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:39.315533+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e278: 8 total, 8 up, 6 in 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:39.400761+0000 osd.2 (osd.2) 136 : cluster [DBG] 5.b scrub starts 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:39.403771+0000 osd.2 (osd.2) 137 : cluster [DBG] 5.b scrub ok 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:39.547740+0000 mgr.y (mgr.14535) 409 : cluster [DBG] pgmap v524: 148 pgs: 1 active, 5 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 111 op/s; 1/576 objects misplaced (0.174%); 243 KiB/s, 3 objects/s recovering 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:39.864871+0000 osd.0 (osd.0) 163 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:39.864919+0000 osd.0 (osd.0) 164 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:39.315533+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e278: 8 total, 8 up, 6 in 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:39.400761+0000 osd.2 (osd.2) 136 : cluster [DBG] 5.b scrub starts 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:39.403771+0000 osd.2 (osd.2) 137 : cluster [DBG] 5.b scrub ok 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:39.547740+0000 mgr.y (mgr.14535) 409 : cluster [DBG] pgmap v524: 148 pgs: 1 active, 5 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 111 op/s; 1/576 objects misplaced (0.174%); 243 KiB/s, 3 objects/s recovering 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:39.864871+0000 osd.0 (osd.0) 163 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:39:40.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:39.864919+0000 osd.0 (osd.0) 164 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:39:40.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:40.765 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:41.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:39:41.317 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:41.326 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:41.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:41.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:41.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:40.313224+0000 mon.a (mon.0) 1239 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:41.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:40.319595+0000 osd.1 (osd.1) 154 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:39:41.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:40.321841+0000 osd.1 (osd.1) 155 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:39:41.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:40.323866+0000 mon.a (mon.0) 1240 : cluster [DBG] osdmap e279: 8 total, 8 up, 6 in 2024-02-22T19:39:41.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:40.313224+0000 mon.a (mon.0) 1239 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:41.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:40.319595+0000 osd.1 (osd.1) 154 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:39:41.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:40.321841+0000 osd.1 (osd.1) 155 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:39:41.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:40.323866+0000 mon.a (mon.0) 1240 : cluster [DBG] osdmap e279: 8 total, 8 up, 6 in 2024-02-22T19:39:41.456 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:40.313224+0000 mon.a (mon.0) 1239 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:40.319595+0000 osd.1 (osd.1) 154 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:39:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:40.321841+0000 osd.1 (osd.1) 155 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:39:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:40.323866+0000 mon.a (mon.0) 1240 : cluster [DBG] osdmap e279: 8 total, 8 up, 6 in 2024-02-22T19:39:41.614 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:41.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:41.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:41.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:41.750 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:42.101 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:42.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:42.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:42.205 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:42.242 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.325171+0000 osd.2 (osd.2) 138 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:39:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.365871+0000 osd.2 (osd.2) 139 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:39:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.548138+0000 mgr.y (mgr.14535) 410 : cluster [DBG] pgmap v526: 148 pgs: 1 active, 5 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.562727+0000 osd.6 (osd.6) 116 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:39:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.566514+0000 osd.6 (osd.6) 117 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.793178+0000 osd.3 (osd.3) 167 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.796934+0000 osd.3 (osd.3) 168 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:42.231913+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e280: 8 total, 8 up, 6 in 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.325171+0000 osd.2 (osd.2) 138 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.365871+0000 osd.2 (osd.2) 139 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.548138+0000 mgr.y (mgr.14535) 410 : cluster [DBG] pgmap v526: 148 pgs: 1 active, 5 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.562727+0000 osd.6 (osd.6) 116 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.566514+0000 osd.6 (osd.6) 117 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.793178+0000 osd.3 (osd.3) 167 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.796934+0000 osd.3 (osd.3) 168 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:39:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:42.231913+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e280: 8 total, 8 up, 6 in 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.325171+0000 osd.2 (osd.2) 138 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.365871+0000 osd.2 (osd.2) 139 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.548138+0000 mgr.y (mgr.14535) 410 : cluster [DBG] pgmap v526: 148 pgs: 1 active, 5 active+remapped+backfill_toofull, 142 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.562727+0000 osd.6 (osd.6) 116 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.566514+0000 osd.6 (osd.6) 117 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.793178+0000 osd.3 (osd.3) 167 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:39:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.796934+0000 osd.3 (osd.3) 168 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:39:42.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:42.231913+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e280: 8 total, 8 up, 6 in 2024-02-22T19:39:43.268 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:43.382 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.752675+0000 osd.7 (osd.7) 174 : cluster [DBG] 3.f scrub starts 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:41.753985+0000 osd.7 (osd.7) 175 : cluster [DBG] 3.f scrub ok 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:42.350301+0000 osd.5 (osd.5) 67 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:42.355174+0000 osd.5 (osd.5) 68 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:42.556457+0000 osd.6 (osd.6) 118 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:42.557784+0000 osd.6 (osd.6) 119 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:42.738153+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.53:0/2680707291' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:42.738454+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:43.257284+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? 172.21.15.53:0/114609287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.752675+0000 osd.7 (osd.7) 174 : cluster [DBG] 3.f scrub starts 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:41.753985+0000 osd.7 (osd.7) 175 : cluster [DBG] 3.f scrub ok 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:42.350301+0000 osd.5 (osd.5) 67 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:42.355174+0000 osd.5 (osd.5) 68 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:42.556457+0000 osd.6 (osd.6) 118 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:42.557784+0000 osd.6 (osd.6) 119 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:39:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:42.738153+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.53:0/2680707291' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:42.738454+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:43.257284+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? 172.21.15.53:0/114609287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.752675+0000 osd.7 (osd.7) 174 : cluster [DBG] 3.f scrub starts 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:41.753985+0000 osd.7 (osd.7) 175 : cluster [DBG] 3.f scrub ok 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:42.350301+0000 osd.5 (osd.5) 67 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:42.355174+0000 osd.5 (osd.5) 68 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:42.556457+0000 osd.6 (osd.6) 118 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:42.557784+0000 osd.6 (osd.6) 119 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:42.738153+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.53:0/2680707291' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:42.738454+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:39:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:43.257284+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? 172.21.15.53:0/114609287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:39:44.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:39:44.361 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:39:44.361 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 281, 2024-02-22T19:39:44.361 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:39:44.361 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-02-22T19:22:59.746063+0000", 2024-02-22T19:39:44.361 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-02-22T19:39:43.326283+0000", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-02-22T19:28:53.274045+0000", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-02-22T19:39:21.706990+0000", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5799936, 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 18, 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-02-22T19:39:44.362 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 8, 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "pacific", 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "device_health_metrics", 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:26:54.342689+0000", 2024-02-22T19:39:44.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:39:44.364 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 11, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 11, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 11, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 11, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 11, 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:39:44.365 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "164", 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "163", 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:39:44.366 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:39:44.367 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:39:44.368 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_devicehealth": {} 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "iscsi", 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.555543+0000", 2024-02-22T19:39:44.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 3, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 3, 2024-02-22T19:39:44.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 3, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 3, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 3, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "242", 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:39:44.371 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:39:44.372 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:39:44.373 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".rgw.root", 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.986190+0000", 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:39:44.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:39:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "58", 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:39:44.377 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:39:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:39:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 4, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.log", 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:01.494051+0000", 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:39:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:39:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "60", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:39:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:39:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:39:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 5, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.control", 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:03.518927+0000", 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:39:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 22, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 22, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 22, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 22, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 22, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "5.16", 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 229, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 228, 2024-02-22T19:39:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 228, 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "229", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "229", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "227", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:39:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:39:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:39:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 6, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.meta", 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:05.596341+0000", 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:39:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:39:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "64", 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:39:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:39:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:39:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_bias": 4 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab", 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:39:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 8, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 276, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:39:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/162271081", 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6802/162271081", 2024-02-22T19:39:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6804/162271081", 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6803/162271081", 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2", 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 267, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6805/2973403044", 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6806/2973403044", 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6808/2973403044", 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6807/2973403044", 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83", 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 18, 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 272, 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/936980935", 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6810/936980935", 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6812/936980935", 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6811/936980935", 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993", 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:39:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 26, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 275, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-02-22T19:39:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6813/3878043918", 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6814/3878043918", 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6816/3878043918", 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6815/3878043918", 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211", 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 32, 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 260, 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6800", 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6801", 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6803", 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6802", 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6800/2373040976", 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6801/2373040976", 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6803/2373040976", 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6802/2373040976", 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07", 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:39:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 39, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 277, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6804", 2024-02-22T19:39:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6805", 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6807", 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6806", 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6804/2413553331", 2024-02-22T19:39:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6805/2413553331", 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6807/2413553331", 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6806/2413553331", 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e", 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 45, 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 272, 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6808", 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6809", 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6811", 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6810", 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6808/3043274732", 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6809/3043274732", 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6811/3043274732", 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6810/3043274732", 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a", 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:39:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 53, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 267, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6812", 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6813", 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6815", 2024-02-22T19:39:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6814", 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6812/2605082260", 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6813/2605082260", 2024-02-22T19:39:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6815/2605082260", 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6814/2605082260", 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:39.864921+0000", 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:37.853088+0000", 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:31.314509+0000", 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:13.900490+0000", 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:00.274700+0000", 2024-02-22T19:39:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:38:53.690895+0000", 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:39:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:38:49.287236+0000", 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:39:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:38:59.447395+0000", 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [ 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.1", 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.7", 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.b", 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.17", 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [ 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.e", 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:39:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: 0 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": { 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4161439976": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2578736337": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/762516580": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3676085495": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3516945073": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/828421682": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4127851809": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/1906929179": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:39:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2640420245": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/2854906057": "2024-02-23T19:23:32.485117+0000" 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "ruleset-failure-domain": "osd", 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-02-22T19:39:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:39:44.438 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:43.326456+0000 mon.a (mon.0) 1244 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:43.334881+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:43.334963+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.53:0/114609287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:43.335011+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e281: 8 total, 8 up, 6 in 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:43.548478+0000 mgr.y (mgr.14535) 411 : cluster [DBG] pgmap v529: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:43.587791+0000 osd.6 (osd.6) 120 : cluster [DBG] 4.e scrub starts 2024-02-22T19:39:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:43.589471+0000 osd.6 (osd.6) 121 : cluster [DBG] 4.e scrub ok 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:43.326456+0000 mon.a (mon.0) 1244 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:43.334881+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:43.334963+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.53:0/114609287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:43.335011+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e281: 8 total, 8 up, 6 in 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:43.548478+0000 mgr.y (mgr.14535) 411 : cluster [DBG] pgmap v529: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:43.587791+0000 osd.6 (osd.6) 120 : cluster [DBG] 4.e scrub starts 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:43.589471+0000 osd.6 (osd.6) 121 : cluster [DBG] 4.e scrub ok 2024-02-22T19:39:44.747 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:43.326456+0000 mon.a (mon.0) 1244 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:39:44.748 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:43.334881+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:39:44.748 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:43.334963+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.53:0/114609287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:39:44.748 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:43.335011+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e281: 8 total, 8 up, 6 in 2024-02-22T19:39:44.748 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:43.548478+0000 mgr.y (mgr.14535) 411 : cluster [DBG] pgmap v529: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:44.748 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:43.587791+0000 osd.6 (osd.6) 120 : cluster [DBG] 4.e scrub starts 2024-02-22T19:39:44.748 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:43.589471+0000 osd.6 (osd.6) 121 : cluster [DBG] 4.e scrub ok 2024-02-22T19:39:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:45.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:45.001 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:45.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:45.103 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:39:45.104 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:39:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:39:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:45.136 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_1' removed 2024-02-22T19:39:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:44.295195+0000 osd.1 (osd.1) 156 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:39:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:44.296966+0000 osd.1 (osd.1) 157 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:39:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:44.359823+0000 mon.a (mon.0) 1248 : audit [DBG] from='client.? 172.21.15.53:0/2087120899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:39:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:44.860380+0000 mgr.y (mgr.14535) 412 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:45.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:44.295195+0000 osd.1 (osd.1) 156 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:39:45.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:44.296966+0000 osd.1 (osd.1) 157 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:39:45.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:44.359823+0000 mon.a (mon.0) 1248 : audit [DBG] from='client.? 172.21.15.53:0/2087120899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:39:45.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:44.860380+0000 mgr.y (mgr.14535) 412 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:45.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:44.295195+0000 osd.1 (osd.1) 156 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:39:45.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:44.296966+0000 osd.1 (osd.1) 157 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:39:45.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:44.359823+0000 mon.a (mon.0) 1248 : audit [DBG] from='client.? 172.21.15.53:0/2087120899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:39:45.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:44.860380+0000 mgr.y (mgr.14535) 412 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:45.945 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:45.948 INFO:tasks.thrashosds.thrasher:j is {'epoch': 281, 'fsid': 'b6783888-d1b7-11ee-95bf-87774f69a715', 'created': '2024-02-22T19:22:59.746063+0000', 'modified': '2024-02-22T19:39:43.326283+0000', 'last_up_change': '2024-02-22T19:28:53.274045+0000', 'last_in_change': '2024-02-22T19:39:21.706990+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '163', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '5.16', 'ready_epoch': 229, 'last_epoch_started': 228, 'last_epoch_clean': 228, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '229', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '229', 'last_force_op_resend_preluminous': '227', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}], 'osds': [{'osd': 0, 'uuid': '4326a76a-6144-4859-9b40-5ff69edbbcab', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 276, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6801', 'nonce': 162271081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 162271081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 162271081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6803', 'nonce': 162271081}]}, 'public_addr': '172.21.15.53:6801/162271081', 'cluster_addr': '172.21.15.53:6802/162271081', 'heartbeat_back_addr': '172.21.15.53:6804/162271081', 'heartbeat_front_addr': '172.21.15.53:6803/162271081', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 267, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6805', 'nonce': 2973403044}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2973403044}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2973403044}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6807', 'nonce': 2973403044}]}, 'public_addr': '172.21.15.53:6805/2973403044', 'cluster_addr': '172.21.15.53:6806/2973403044', 'heartbeat_back_addr': '172.21.15.53:6808/2973403044', 'heartbeat_front_addr': '172.21.15.53:6807/2973403044', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 272, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6809', 'nonce': 936980935}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 936980935}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 936980935}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6811', 'nonce': 936980935}]}, 'public_addr': '172.21.15.53:6809/936980935', 'cluster_addr': '172.21.15.53:6810/936980935', 'heartbeat_back_addr': '172.21.15.53:6812/936980935', 'heartbeat_front_addr': '172.21.15.53:6811/936980935', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3f285320-aa28-4fc1-b1c7-456c9aecd993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 275, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6813', 'nonce': 3878043918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 3878043918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 3878043918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6815', 'nonce': 3878043918}]}, 'public_addr': '172.21.15.53:6813/3878043918', 'cluster_addr': '172.21.15.53:6814/3878043918', 'heartbeat_back_addr': '172.21.15.53:6816/3878043918', 'heartbeat_front_addr': '172.21.15.53:6815/3878043918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '773e8080-1028-48fe-8f6e-ed6beca60211', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 260, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2373040976}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6801', 'nonce': 2373040976}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 2373040976}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2373040976}]}, 'public_addr': '172.21.15.64:6800/2373040976', 'cluster_addr': '172.21.15.64:6801/2373040976', 'heartbeat_back_addr': '172.21.15.64:6803/2373040976', 'heartbeat_front_addr': '172.21.15.64:6802/2373040976', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 277, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6804', 'nonce': 2413553331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6805', 'nonce': 2413553331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 2413553331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2413553331}]}, 'public_addr': '172.21.15.64:6804/2413553331', 'cluster_addr': '172.21.15.64:6805/2413553331', 'heartbeat_back_addr': '172.21.15.64:6807/2413553331', 'heartbeat_front_addr': '172.21.15.64:6806/2413553331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 272, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6808', 'nonce': 3043274732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6809', 'nonce': 3043274732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 3043274732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 3043274732}]}, 'public_addr': '172.21.15.64:6808/3043274732', 'cluster_addr': '172.21.15.64:6809/3043274732', 'heartbeat_back_addr': '172.21.15.64:6811/3043274732', 'heartbeat_front_addr': '172.21.15.64:6810/3043274732', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'cde36201-cb77-4967-9402-6fdbe633b40a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 267, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6812', 'nonce': 2605082260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6813', 'nonce': 2605082260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6815', 'nonce': 2605082260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 2605082260}]}, 'public_addr': '172.21.15.64:6812/2605082260', 'cluster_addr': '172.21.15.64:6813/2605082260', 'heartbeat_back_addr': '172.21.15.64:6815/2605082260', 'heartbeat_front_addr': '172.21.15.64:6814/2605082260', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:39:39.864921+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:39:37.853088+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:39:31.314509+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:39:13.900490+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:39:00.274700+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:38:53.690895+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:38:49.287236+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:38:59.447395+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.1', 'osds': [5, 3, 4]}], 'pg_upmap_items': [{'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [{'pgid': '5.e', 'osds': [5, 2, 0]}], 'primary_temp': [], 'blocklist': {'172.21.15.53:0/4161439976': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/2578736337': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/762516580': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:6800/3676085495': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:6800/3516945073': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/828421682': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/4127851809': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/1906929179': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:0/2640420245': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:6800/2854906057': '2024-02-23T19:23:32.485117+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-22T19:39:45.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:39:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:46.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:46.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:46.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:46.421 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.252596+0000 osd.1 (osd.1) 158 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:39:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.253847+0000 osd.1 (osd.1) 159 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:39:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.549353+0000 mgr.y (mgr.14535) 413 : cluster [DBG] pgmap v530: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.612315+0000 osd.6 (osd.6) 122 : cluster [DBG] 3.a scrub starts 2024-02-22T19:39:46.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.615341+0000 osd.6 (osd.6) 123 : cluster [DBG] 3.a scrub ok 2024-02-22T19:39:46.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.252596+0000 osd.1 (osd.1) 158 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:39:46.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.253847+0000 osd.1 (osd.1) 159 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:39:46.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.549353+0000 mgr.y (mgr.14535) 413 : cluster [DBG] pgmap v530: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:46.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.612315+0000 osd.6 (osd.6) 122 : cluster [DBG] 3.a scrub starts 2024-02-22T19:39:46.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.615341+0000 osd.6 (osd.6) 123 : cluster [DBG] 3.a scrub ok 2024-02-22T19:39:46.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.252596+0000 osd.1 (osd.1) 158 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:39:46.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.253847+0000 osd.1 (osd.1) 159 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:39:46.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.549353+0000 mgr.y (mgr.14535) 413 : cluster [DBG] pgmap v530: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:46.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.612315+0000 osd.6 (osd.6) 122 : cluster [DBG] 3.a scrub starts 2024-02-22T19:39:46.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.615341+0000 osd.6 (osd.6) 123 : cluster [DBG] 3.a scrub ok 2024-02-22T19:39:46.729 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-02-22T19:39:46.729 INFO:tasks.radosbench:Beginning radosbench... 2024-02-22T19:39:46.730 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-02-22T19:39:46.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool create unique_pool_2 16 2024-02-22T19:39:47.272 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:47.275 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:47.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:47.289 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:39:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:39:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:39:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:39:47.289 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:39:47.336 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:47.337 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:47.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:47.406 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:47.577 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.655578+0000 osd.7 (osd.7) 176 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:39:47.577 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:45.659139+0000 osd.7 (osd.7) 177 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:39:47.577 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:46.251309+0000 osd.2 (osd.2) 140 : cluster [DBG] 4.b scrub starts 2024-02-22T19:39:47.577 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:46.254895+0000 osd.2 (osd.2) 141 : cluster [DBG] 4.b scrub ok 2024-02-22T19:39:47.577 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:46.298357+0000 osd.1 (osd.1) 160 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:46.299681+0000 osd.1 (osd.1) 161 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.228129+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.232414+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e282: 8 total, 8 up, 6 in 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.655578+0000 osd.7 (osd.7) 176 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:45.659139+0000 osd.7 (osd.7) 177 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:46.251309+0000 osd.2 (osd.2) 140 : cluster [DBG] 4.b scrub starts 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:46.254895+0000 osd.2 (osd.2) 141 : cluster [DBG] 4.b scrub ok 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:46.298357+0000 osd.1 (osd.1) 160 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:46.299681+0000 osd.1 (osd.1) 161 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.228129+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:47.578 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.232414+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e282: 8 total, 8 up, 6 in 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.655578+0000 osd.7 (osd.7) 176 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:45.659139+0000 osd.7 (osd.7) 177 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:46.251309+0000 osd.2 (osd.2) 140 : cluster [DBG] 4.b scrub starts 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:46.254895+0000 osd.2 (osd.2) 141 : cluster [DBG] 4.b scrub ok 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:46.298357+0000 osd.1 (osd.1) 160 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:46.299681+0000 osd.1 (osd.1) 161 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.228129+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:39:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.232414+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e282: 8 total, 8 up, 6 in 2024-02-22T19:39:48.234 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:48.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:48.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:48.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:48.394 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:48.397 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:48.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:48.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:46.426286+0000 osd.5 (osd.5) 69 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:46.430005+0000 osd.5 (osd.5) 70 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.549987+0000 mgr.y (mgr.14535) 414 : cluster [DBG] pgmap v532: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.702822+0000 osd.3 (osd.3) 169 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.705734+0000 osd.3 (osd.3) 170 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:46.426286+0000 osd.5 (osd.5) 69 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:46.430005+0000 osd.5 (osd.5) 70 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:39:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.549987+0000 mgr.y (mgr.14535) 414 : cluster [DBG] pgmap v532: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.702822+0000 osd.3 (osd.3) 169 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:39:48.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.705734+0000 osd.3 (osd.3) 170 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:39:48.528 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:39:48.552 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:48.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:39:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:46.426286+0000 osd.5 (osd.5) 69 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:39:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:46.430005+0000 osd.5 (osd.5) 70 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:39:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.549987+0000 mgr.y (mgr.14535) 414 : cluster [DBG] pgmap v532: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.702822+0000 osd.3 (osd.3) 169 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:39:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.705734+0000 osd.3 (osd.3) 170 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:39:48.654 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:49.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:39:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.624354+0000 osd.7 (osd.7) 178 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:39:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:47.644907+0000 osd.7 (osd.7) 179 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:39:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:48.224218+0000 osd.2 (osd.2) 142 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:39:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:48.227430+0000 osd.2 (osd.2) 143 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:39:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:48.360430+0000 osd.1 (osd.1) 162 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:39:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:48.361423+0000 osd.1 (osd.1) 163 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:39:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:49.635 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.624354+0000 osd.7 (osd.7) 178 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:39:49.635 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:47.644907+0000 osd.7 (osd.7) 179 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:48.224218+0000 osd.2 (osd.2) 142 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:48.227430+0000 osd.2 (osd.2) 143 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:48.360430+0000 osd.1 (osd.1) 162 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:48.361423+0000 osd.1 (osd.1) 163 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.624354+0000 osd.7 (osd.7) 178 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:47.644907+0000 osd.7 (osd.7) 179 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:48.224218+0000 osd.2 (osd.2) 142 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:48.227430+0000 osd.2 (osd.2) 143 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:48.360430+0000 osd.1 (osd.1) 162 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:48.361423+0000 osd.1 (osd.1) 163 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:39:49.636 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:39:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:49] "GET /metrics HTTP/1.1" 200 175982 "" "Prometheus/2.33.4" 2024-02-22T19:39:49.637 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:49.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:49.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:49.771 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:49.792 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:49.795 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:49.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:49.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:50.027 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:50.030 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:50.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:50.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:50.162 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:50.264 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:39:50.264 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:39:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:39:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:39:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:48.450665+0000 osd.5 (osd.5) 71 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:39:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:48.479125+0000 osd.5 (osd.5) 72 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:39:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:49.401967+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.c scrub starts 2024-02-22T19:39:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:49.402936+0000 osd.1 (osd.1) 165 : cluster [DBG] 5.c scrub ok 2024-02-22T19:39:50.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:49.550540+0000 mgr.y (mgr.14535) 415 : cluster [DBG] pgmap v533: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:49.625721+0000 osd.6 (osd.6) 124 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:49.629116+0000 osd.6 (osd.6) 125 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:50.024332+0000 mon.a (mon.0) 1251 : audit [DBG] from='client.? 172.21.15.53:0/4215689713' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:48.450665+0000 osd.5 (osd.5) 71 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:48.479125+0000 osd.5 (osd.5) 72 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:49.401967+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.c scrub starts 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:49.402936+0000 osd.1 (osd.1) 165 : cluster [DBG] 5.c scrub ok 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:49.550540+0000 mgr.y (mgr.14535) 415 : cluster [DBG] pgmap v533: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:49.625721+0000 osd.6 (osd.6) 124 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:49.629116+0000 osd.6 (osd.6) 125 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:39:50.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:50.024332+0000 mon.a (mon.0) 1251 : audit [DBG] from='client.? 172.21.15.53:0/4215689713' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:48.450665+0000 osd.5 (osd.5) 71 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:39:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:48.479125+0000 osd.5 (osd.5) 72 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:39:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:49.401967+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.c scrub starts 2024-02-22T19:39:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:49.402936+0000 osd.1 (osd.1) 165 : cluster [DBG] 5.c scrub ok 2024-02-22T19:39:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:49.550540+0000 mgr.y (mgr.14535) 415 : cluster [DBG] pgmap v533: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:49.625721+0000 osd.6 (osd.6) 124 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:39:50.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:49.629116+0000 osd.6 (osd.6) 125 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:39:50.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:50.024332+0000 mon.a (mon.0) 1251 : audit [DBG] from='client.? 172.21.15.53:0/4215689713' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:39:51.116 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:51.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:51.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:51.136 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-02-22T19:39:49.550142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":1757,"num_read_kb":1509,"num_write":714,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13666,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":806,"ondisk_log_size":806,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":401,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":1817568,"kb_used_data":9648,"kb_used_omap":0,"kb_used_meta":1807872,"kb_avail":560653344,"statfs":{"total":575970213888,"available":574109024256,"internally_reserved":0,"allocated":9879552,"data_stored":5365099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1851260928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":-223464,"num_write_kb":-7150785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-11818,"num_bytes_recovered":-773521432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.317628"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":234,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336019+0000","last_change":"2024-02-22T19:38:47.158144+0000","last_active":"2024-02-22T19:39:43.336019+0000","last_peered":"2024-02-22T19:39:43.336019+0000","last_clean":"2024-02-22T19:39:43.336019+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:39:43.336019+0000","last_undegraded":"2024-02-22T19:39:43.336019+0000","last_fullsized":"2024-02-22T19:39:43.336019+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.806069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.806069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":260,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336570+0000","last_change":"2024-02-22T19:39:22.268923+0000","last_active":"2024-02-22T19:39:43.336570+0000","last_peered":"2024-02-22T19:39:43.336570+0000","last_clean":"2024-02-22T19:39:43.336570+0000","last_became_active":"2024-02-22T19:39:22.268587+0000","last_became_peered":"2024-02-22T19:39:22.268587+0000","last_unstale":"2024-02-22T19:39:43.336570+0000","last_undegraded":"2024-02-22T19:39:43.336570+0000","last_fullsized":"2024-02-22T19:39:43.336570+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":97,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.411907+0000","last_change":"2024-02-22T19:39:36.528260+0000","last_active":"2024-02-22T19:39:43.411907+0000","last_peered":"2024-02-22T19:39:43.411907+0000","last_clean":"2024-02-22T19:39:43.411907+0000","last_became_active":"2024-02-22T19:38:47.164599+0000","last_became_peered":"2024-02-22T19:38:47.164599+0000","last_unstale":"2024-02-22T19:39:43.411907+0000","last_undegraded":"2024-02-22T19:39:43.411907+0000","last_fullsized":"2024-02-22T19:39:43.411907+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:39:36.528175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.528175+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":239,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237902+0000","last_change":"2024-02-22T19:36:59.298132+0000","last_active":"2024-02-22T19:39:42.237902+0000","last_peered":"2024-02-22T19:39:42.237902+0000","last_clean":"2024-02-22T19:39:42.237902+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:39:42.237902+0000","last_undegraded":"2024-02-22T19:39:42.237902+0000","last_fullsized":"2024-02-22T19:39:42.237902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:59.298021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:36:59.298021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":240,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235033+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:39:42.235033+0000","last_peered":"2024-02-22T19:39:42.235033+0000","last_clean":"2024-02-22T19:39:42.235033+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:39:42.235033+0000","last_undegraded":"2024-02-22T19:39:42.235033+0000","last_fullsized":"2024-02-22T19:39:42.235033+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":361,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.236169+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:39:42.236169+0000","last_peered":"2024-02-22T19:39:42.236169+0000","last_clean":"2024-02-22T19:39:42.236169+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:39:42.236169+0000","last_undegraded":"2024-02-22T19:39:42.236169+0000","last_fullsized":"2024-02-22T19:39:42.236169+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":78,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.333531+0000","last_change":"2024-02-22T19:39:36.296064+0000","last_active":"2024-02-22T19:39:43.333531+0000","last_peered":"2024-02-22T19:39:43.333531+0000","last_clean":"2024-02-22T19:39:43.333531+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:39:43.333531+0000","last_undegraded":"2024-02-22T19:39:43.333531+0000","last_fullsized":"2024-02-22T19:39:43.333531+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:36.295988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.295988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":83,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335449+0000","last_change":"2024-02-22T19:38:47.169310+0000","last_active":"2024-02-22T19:39:43.335449+0000","last_peered":"2024-02-22T19:39:43.335449+0000","last_clean":"2024-02-22T19:39:43.335449+0000","last_became_active":"2024-02-22T19:38:47.168764+0000","last_became_peered":"2024-02-22T19:38:47.168764+0000","last_unstale":"2024-02-22T19:39:43.335449+0000","last_undegraded":"2024-02-22T19:39:43.335449+0000","last_fullsized":"2024-02-22T19:39:43.335449+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"264'15","reported_seq":130,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.236603+0000","last_change":"2024-02-22T19:39:22.646901+0000","last_active":"2024-02-22T19:39:42.236603+0000","last_peered":"2024-02-22T19:39:42.236603+0000","last_clean":"2024-02-22T19:39:42.236603+0000","last_became_active":"2024-02-22T19:39:22.269400+0000","last_became_peered":"2024-02-22T19:39:22.269400+0000","last_unstale":"2024-02-22T19:39:42.236603+0000","last_undegraded":"2024-02-22T19:39:42.236603+0000","last_fullsized":"2024-02-22T19:39:42.236603+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:21.735513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:21.735513+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334849+0000","last_change":"2024-02-22T19:37:57.186178+0000","last_active":"2024-02-22T19:39:43.334849+0000","last_peered":"2024-02-22T19:39:43.334849+0000","last_clean":"2024-02-22T19:39:43.334849+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:39:43.334849+0000","last_undegraded":"2024-02-22T19:39:43.334849+0000","last_fullsized":"2024-02-22T19:39:43.334849+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.908571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.908571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":91,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335582+0000","last_change":"2024-02-22T19:38:47.443968+0000","last_active":"2024-02-22T19:39:43.335582+0000","last_peered":"2024-02-22T19:39:43.335582+0000","last_clean":"2024-02-22T19:39:43.335582+0000","last_became_active":"2024-02-22T19:38:47.179485+0000","last_became_peered":"2024-02-22T19:38:47.179485+0000","last_unstale":"2024-02-22T19:39:43.335582+0000","last_undegraded":"2024-02-22T19:39:43.335582+0000","last_fullsized":"2024-02-22T19:39:43.335582+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:35:35.930654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:35.930654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":81,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237521+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:39:42.237521+0000","last_peered":"2024-02-22T19:39:42.237521+0000","last_clean":"2024-02-22T19:39:42.237521+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:39:42.237521+0000","last_undegraded":"2024-02-22T19:39:42.237521+0000","last_fullsized":"2024-02-22T19:39:42.237521+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":235,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412902+0000","last_change":"2024-02-22T19:39:34.555380+0000","last_active":"2024-02-22T19:39:43.412902+0000","last_peered":"2024-02-22T19:39:43.412902+0000","last_clean":"2024-02-22T19:39:43.412902+0000","last_became_active":"2024-02-22T19:38:47.168230+0000","last_became_peered":"2024-02-22T19:38:47.168230+0000","last_unstale":"2024-02-22T19:39:43.412902+0000","last_undegraded":"2024-02-22T19:39:43.412902+0000","last_fullsized":"2024-02-22T19:39:43.412902+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:34.555298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.555298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":51,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387290+0000","last_change":"2024-02-22T19:39:22.245523+0000","last_active":"2024-02-22T19:39:43.387290+0000","last_peered":"2024-02-22T19:39:43.387290+0000","last_clean":"2024-02-22T19:39:43.387290+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:39:43.387290+0000","last_undegraded":"2024-02-22T19:39:43.387290+0000","last_fullsized":"2024-02-22T19:39:43.387290+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.732856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.732856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":304,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.409305+0000","last_change":"2024-02-22T19:39:36.387949+0000","last_active":"2024-02-22T19:39:43.409305+0000","last_peered":"2024-02-22T19:39:43.409305+0000","last_clean":"2024-02-22T19:39:43.409305+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:39:43.409305+0000","last_undegraded":"2024-02-22T19:39:43.409305+0000","last_fullsized":"2024-02-22T19:39:43.409305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:39:36.387857+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.387857+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":257,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235899+0000","last_change":"2024-02-22T19:37:57.180410+0000","last_active":"2024-02-22T19:39:42.235899+0000","last_peered":"2024-02-22T19:39:42.235899+0000","last_clean":"2024-02-22T19:39:42.235899+0000","last_became_active":"2024-02-22T19:37:57.180089+0000","last_became_peered":"2024-02-22T19:37:57.180089+0000","last_unstale":"2024-02-22T19:39:42.235899+0000","last_undegraded":"2024-02-22T19:39:42.235899+0000","last_fullsized":"2024-02-22T19:39:42.235899+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":64,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336542+0000","last_change":"2024-02-22T19:39:22.252423+0000","last_active":"2024-02-22T19:39:43.336542+0000","last_peered":"2024-02-22T19:39:43.336542+0000","last_clean":"2024-02-22T19:39:43.336542+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:39:43.336542+0000","last_undegraded":"2024-02-22T19:39:43.336542+0000","last_fullsized":"2024-02-22T19:39:43.336542+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.583926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.583926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":296,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235049+0000","last_change":"2024-02-22T19:38:48.219376+0000","last_active":"2024-02-22T19:39:42.235049+0000","last_peered":"2024-02-22T19:39:42.235049+0000","last_clean":"2024-02-22T19:39:42.235049+0000","last_became_active":"2024-02-22T19:38:47.180453+0000","last_became_peered":"2024-02-22T19:38:47.180453+0000","last_unstale":"2024-02-22T19:39:42.235049+0000","last_undegraded":"2024-02-22T19:39:42.235049+0000","last_fullsized":"2024-02-22T19:39:42.235049+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":288,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387506+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:39:43.387506+0000","last_peered":"2024-02-22T19:39:43.387506+0000","last_clean":"2024-02-22T19:39:43.387506+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:39:43.387506+0000","last_undegraded":"2024-02-22T19:39:43.387506+0000","last_fullsized":"2024-02-22T19:39:43.387506+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":263,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.333820+0000","last_change":"2024-02-22T19:38:47.168587+0000","last_active":"2024-02-22T19:39:43.333820+0000","last_peered":"2024-02-22T19:39:43.333820+0000","last_clean":"2024-02-22T19:39:43.333820+0000","last_became_active":"2024-02-22T19:38:47.168149+0000","last_became_peered":"2024-02-22T19:38:47.168149+0000","last_unstale":"2024-02-22T19:39:43.333820+0000","last_undegraded":"2024-02-22T19:39:43.333820+0000","last_fullsized":"2024-02-22T19:39:43.333820+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:50.144179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:50.144179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":89,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.388631+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:39:43.388631+0000","last_peered":"2024-02-22T19:39:43.388631+0000","last_clean":"2024-02-22T19:39:43.388631+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:39:43.388631+0000","last_undegraded":"2024-02-22T19:39:43.388631+0000","last_fullsized":"2024-02-22T19:39:43.388631+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":272,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334496+0000","last_change":"2024-02-22T19:38:47.150417+0000","last_active":"2024-02-22T19:39:43.334496+0000","last_peered":"2024-02-22T19:39:43.334496+0000","last_clean":"2024-02-22T19:39:43.334496+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:39:43.334496+0000","last_undegraded":"2024-02-22T19:39:43.334496+0000","last_fullsized":"2024-02-22T19:39:43.334496+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.995002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.995002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":269,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238857+0000","last_change":"2024-02-22T19:39:22.271611+0000","last_active":"2024-02-22T19:39:42.238857+0000","last_peered":"2024-02-22T19:39:42.238857+0000","last_clean":"2024-02-22T19:39:42.238857+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:39:42.238857+0000","last_undegraded":"2024-02-22T19:39:42.238857+0000","last_fullsized":"2024-02-22T19:39:42.238857+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:34:58.164960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":353,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.413562+0000","last_change":"2024-02-22T19:37:57.212060+0000","last_active":"2024-02-22T19:39:43.413562+0000","last_peered":"2024-02-22T19:39:43.413562+0000","last_clean":"2024-02-22T19:39:43.413562+0000","last_became_active":"2024-02-22T19:37:57.190476+0000","last_became_peered":"2024-02-22T19:37:57.190476+0000","last_unstale":"2024-02-22T19:39:43.413562+0000","last_undegraded":"2024-02-22T19:39:43.413562+0000","last_fullsized":"2024-02-22T19:39:43.413562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:52.432675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:52.432675+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":257,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387254+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:39:43.387254+0000","last_peered":"2024-02-22T19:39:43.387254+0000","last_clean":"2024-02-22T19:39:43.387254+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:39:43.387254+0000","last_undegraded":"2024-02-22T19:39:43.387254+0000","last_fullsized":"2024-02-22T19:39:43.387254+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":273,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412171+0000","last_change":"2024-02-22T19:39:22.256927+0000","last_active":"2024-02-22T19:39:43.412171+0000","last_peered":"2024-02-22T19:39:43.412171+0000","last_clean":"2024-02-22T19:39:43.412171+0000","last_became_active":"2024-02-22T19:39:22.256576+0000","last_became_peered":"2024-02-22T19:39:22.256576+0000","last_unstale":"2024-02-22T19:39:43.412171+0000","last_undegraded":"2024-02-22T19:39:43.412171+0000","last_fullsized":"2024-02-22T19:39:43.412171+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.400534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.400534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":260,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336489+0000","last_change":"2024-02-22T19:39:22.246647+0000","last_active":"2024-02-22T19:39:43.336489+0000","last_peered":"2024-02-22T19:39:43.336489+0000","last_clean":"2024-02-22T19:39:43.336489+0000","last_became_active":"2024-02-22T19:39:22.246323+0000","last_became_peered":"2024-02-22T19:39:22.246323+0000","last_unstale":"2024-02-22T19:39:43.336489+0000","last_undegraded":"2024-02-22T19:39:43.336489+0000","last_fullsized":"2024-02-22T19:39:43.336489+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":255,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238548+0000","last_change":"2024-02-22T19:36:58.263929+0000","last_active":"2024-02-22T19:39:42.238548+0000","last_peered":"2024-02-22T19:39:42.238548+0000","last_clean":"2024-02-22T19:39:42.238548+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:39:42.238548+0000","last_undegraded":"2024-02-22T19:39:42.238548+0000","last_fullsized":"2024-02-22T19:39:42.238548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:36:58.263834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.263834+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":268,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237788+0000","last_change":"2024-02-22T19:39:22.250184+0000","last_active":"2024-02-22T19:39:42.237788+0000","last_peered":"2024-02-22T19:39:42.237788+0000","last_clean":"2024-02-22T19:39:42.237788+0000","last_became_active":"2024-02-22T19:39:22.249890+0000","last_became_peered":"2024-02-22T19:39:22.249890+0000","last_unstale":"2024-02-22T19:39:42.237788+0000","last_undegraded":"2024-02-22T19:39:42.237788+0000","last_fullsized":"2024-02-22T19:39:42.237788+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":275,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387337+0000","last_change":"2024-02-22T19:39:22.253288+0000","last_active":"2024-02-22T19:39:43.387337+0000","last_peered":"2024-02-22T19:39:43.387337+0000","last_clean":"2024-02-22T19:39:43.387337+0000","last_became_active":"2024-02-22T19:39:22.252705+0000","last_became_peered":"2024-02-22T19:39:22.252705+0000","last_unstale":"2024-02-22T19:39:43.387337+0000","last_undegraded":"2024-02-22T19:39:43.387337+0000","last_fullsized":"2024-02-22T19:39:43.387337+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:54.418958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:37:54.418958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336189+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:39:43.336189+0000","last_peered":"2024-02-22T19:39:43.336189+0000","last_clean":"2024-02-22T19:39:43.336189+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:39:43.336189+0000","last_undegraded":"2024-02-22T19:39:43.336189+0000","last_fullsized":"2024-02-22T19:39:43.336189+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":274,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.236075+0000","last_change":"2024-02-22T19:39:37.816765+0000","last_active":"2024-02-22T19:39:42.236075+0000","last_peered":"2024-02-22T19:39:42.236075+0000","last_clean":"2024-02-22T19:39:42.236075+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:39:42.236075+0000","last_undegraded":"2024-02-22T19:39:42.236075+0000","last_fullsized":"2024-02-22T19:39:42.236075+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":251,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334194+0000","last_change":"2024-02-22T19:39:22.247453+0000","last_active":"2024-02-22T19:39:43.334194+0000","last_peered":"2024-02-22T19:39:43.334194+0000","last_clean":"2024-02-22T19:39:43.334194+0000","last_became_active":"2024-02-22T19:39:22.247266+0000","last_became_peered":"2024-02-22T19:39:22.247266+0000","last_unstale":"2024-02-22T19:39:43.334194+0000","last_undegraded":"2024-02-22T19:39:43.334194+0000","last_fullsized":"2024-02-22T19:39:43.334194+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":268,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237613+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:39:42.237613+0000","last_peered":"2024-02-22T19:39:42.237613+0000","last_clean":"2024-02-22T19:39:42.237613+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:39:42.237613+0000","last_undegraded":"2024-02-22T19:39:42.237613+0000","last_fullsized":"2024-02-22T19:39:42.237613+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":283,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.388304+0000","last_change":"2024-02-22T19:39:22.254738+0000","last_active":"2024-02-22T19:39:43.388304+0000","last_peered":"2024-02-22T19:39:43.388304+0000","last_clean":"2024-02-22T19:39:43.388304+0000","last_became_active":"2024-02-22T19:39:22.254411+0000","last_became_peered":"2024-02-22T19:39:22.254411+0000","last_unstale":"2024-02-22T19:39:43.388304+0000","last_undegraded":"2024-02-22T19:39:43.388304+0000","last_fullsized":"2024-02-22T19:39:43.388304+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:52.463774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.463774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":277,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:44.297149+0000","last_change":"2024-02-22T19:39:44.297110+0000","last_active":"2024-02-22T19:39:44.297149+0000","last_peered":"2024-02-22T19:39:44.297149+0000","last_clean":"2024-02-22T19:39:44.297149+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:39:44.297149+0000","last_undegraded":"2024-02-22T19:39:44.297149+0000","last_fullsized":"2024-02-22T19:39:44.297149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:39:44.296975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:44.296975+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":284,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235730+0000","last_change":"2024-02-22T19:39:22.270145+0000","last_active":"2024-02-22T19:39:42.235730+0000","last_peered":"2024-02-22T19:39:42.235730+0000","last_clean":"2024-02-22T19:39:42.235730+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:39:42.235730+0000","last_undegraded":"2024-02-22T19:39:42.235730+0000","last_fullsized":"2024-02-22T19:39:42.235730+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.229816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.229816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":263,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336831+0000","last_change":"2024-02-22T19:37:57.187682+0000","last_active":"2024-02-22T19:39:43.336831+0000","last_peered":"2024-02-22T19:39:43.336831+0000","last_clean":"2024-02-22T19:39:43.336831+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:39:43.336831+0000","last_undegraded":"2024-02-22T19:39:43.336831+0000","last_fullsized":"2024-02-22T19:39:43.336831+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:47.993328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:47.993328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387382+0000","last_change":"2024-02-22T19:38:47.162281+0000","last_active":"2024-02-22T19:39:43.387382+0000","last_peered":"2024-02-22T19:39:43.387382+0000","last_clean":"2024-02-22T19:39:43.387382+0000","last_became_active":"2024-02-22T19:38:47.161791+0000","last_became_peered":"2024-02-22T19:38:47.161791+0000","last_unstale":"2024-02-22T19:39:43.387382+0000","last_undegraded":"2024-02-22T19:39:43.387382+0000","last_fullsized":"2024-02-22T19:39:43.387382+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.685418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.685418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":100,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412403+0000","last_change":"2024-02-22T19:38:47.783561+0000","last_active":"2024-02-22T19:39:43.412403+0000","last_peered":"2024-02-22T19:39:43.412403+0000","last_clean":"2024-02-22T19:39:43.412403+0000","last_became_active":"2024-02-22T19:38:47.168328+0000","last_became_peered":"2024-02-22T19:38:47.168328+0000","last_unstale":"2024-02-22T19:39:43.412403+0000","last_undegraded":"2024-02-22T19:39:43.412403+0000","last_fullsized":"2024-02-22T19:39:43.412403+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:53.461286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.461286+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":77,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.388472+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:39:43.388472+0000","last_peered":"2024-02-22T19:39:43.388472+0000","last_clean":"2024-02-22T19:39:43.388472+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:39:43.388472+0000","last_undegraded":"2024-02-22T19:39:43.388472+0000","last_fullsized":"2024-02-22T19:39:43.388472+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":143,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.432654+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:39:47.432654+0000","last_peered":"2024-02-22T19:39:47.432654+0000","last_clean":"2024-02-22T19:39:47.432654+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:39:47.432654+0000","last_undegraded":"2024-02-22T19:39:47.432654+0000","last_fullsized":"2024-02-22T19:39:47.432654+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":256,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387136+0000","last_change":"2024-02-22T19:38:47.162564+0000","last_active":"2024-02-22T19:39:43.387136+0000","last_peered":"2024-02-22T19:39:43.387136+0000","last_clean":"2024-02-22T19:39:43.387136+0000","last_became_active":"2024-02-22T19:38:47.162182+0000","last_became_peered":"2024-02-22T19:38:47.162182+0000","last_unstale":"2024-02-22T19:39:43.387136+0000","last_undegraded":"2024-02-22T19:39:43.387136+0000","last_fullsized":"2024-02-22T19:39:43.387136+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:56.463502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:36:56.463502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":278,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238042+0000","last_change":"2024-02-22T19:37:09.345416+0000","last_active":"2024-02-22T19:39:42.238042+0000","last_peered":"2024-02-22T19:39:42.238042+0000","last_clean":"2024-02-22T19:39:42.238042+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:39:42.238042+0000","last_undegraded":"2024-02-22T19:39:42.238042+0000","last_fullsized":"2024-02-22T19:39:42.238042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:37:09.345319+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.345319+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":257,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.411925+0000","last_change":"2024-02-22T19:39:41.365968+0000","last_active":"2024-02-22T19:39:43.411925+0000","last_peered":"2024-02-22T19:39:43.411925+0000","last_clean":"2024-02-22T19:39:43.411925+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:39:43.411925+0000","last_undegraded":"2024-02-22T19:39:43.411925+0000","last_fullsized":"2024-02-22T19:39:43.411925+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:41.365877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.365877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":373,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.338283+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:39:43.338283+0000","last_peered":"2024-02-22T19:39:43.338283+0000","last_clean":"2024-02-22T19:39:43.338283+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:39:43.338283+0000","last_undegraded":"2024-02-22T19:39:43.338283+0000","last_fullsized":"2024-02-22T19:39:43.338283+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":63,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.386992+0000","last_change":"2024-02-22T19:39:22.246092+0000","last_active":"2024-02-22T19:39:43.386992+0000","last_peered":"2024-02-22T19:39:43.386992+0000","last_clean":"2024-02-22T19:39:43.386992+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:39:43.386992+0000","last_undegraded":"2024-02-22T19:39:43.386992+0000","last_fullsized":"2024-02-22T19:39:43.386992+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":93,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335653+0000","last_change":"2024-02-22T19:37:57.198915+0000","last_active":"2024-02-22T19:39:43.335653+0000","last_peered":"2024-02-22T19:39:43.335653+0000","last_clean":"2024-02-22T19:39:43.335653+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:39:43.335653+0000","last_undegraded":"2024-02-22T19:39:43.335653+0000","last_fullsized":"2024-02-22T19:39:43.335653+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":275,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.409483+0000","last_change":"2024-02-22T19:39:22.259504+0000","last_active":"2024-02-22T19:39:43.409483+0000","last_peered":"2024-02-22T19:39:43.409483+0000","last_clean":"2024-02-22T19:39:43.409483+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:39:43.409483+0000","last_undegraded":"2024-02-22T19:39:43.409483+0000","last_fullsized":"2024-02-22T19:39:43.409483+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.235958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.235958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":279,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387147+0000","last_change":"2024-02-22T19:39:22.255233+0000","last_active":"2024-02-22T19:39:43.387147+0000","last_peered":"2024-02-22T19:39:43.387147+0000","last_clean":"2024-02-22T19:39:43.387147+0000","last_became_active":"2024-02-22T19:39:22.254888+0000","last_became_peered":"2024-02-22T19:39:22.254888+0000","last_unstale":"2024-02-22T19:39:43.387147+0000","last_undegraded":"2024-02-22T19:39:43.387147+0000","last_fullsized":"2024-02-22T19:39:43.387147+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.470149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.470149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":64,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235201+0000","last_change":"2024-02-22T19:39:23.200612+0000","last_active":"2024-02-22T19:39:42.235201+0000","last_peered":"2024-02-22T19:39:42.235201+0000","last_clean":"2024-02-22T19:39:42.235201+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:39:42.235201+0000","last_undegraded":"2024-02-22T19:39:42.235201+0000","last_fullsized":"2024-02-22T19:39:42.235201+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":255,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387677+0000","last_change":"2024-02-22T19:38:47.162050+0000","last_active":"2024-02-22T19:39:43.387677+0000","last_peered":"2024-02-22T19:39:43.387677+0000","last_clean":"2024-02-22T19:39:43.387677+0000","last_became_active":"2024-02-22T19:38:47.161672+0000","last_became_peered":"2024-02-22T19:38:47.161672+0000","last_unstale":"2024-02-22T19:39:43.387677+0000","last_undegraded":"2024-02-22T19:39:43.387677+0000","last_fullsized":"2024-02-22T19:39:43.387677+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":297,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.411615+0000","last_change":"2024-02-22T19:39:22.253731+0000","last_active":"2024-02-22T19:39:43.411615+0000","last_peered":"2024-02-22T19:39:43.411615+0000","last_clean":"2024-02-22T19:39:43.411615+0000","last_became_active":"2024-02-22T19:39:22.253387+0000","last_became_peered":"2024-02-22T19:39:22.253387+0000","last_unstale":"2024-02-22T19:39:43.411615+0000","last_undegraded":"2024-02-22T19:39:43.411615+0000","last_fullsized":"2024-02-22T19:39:43.411615+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.431655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.431655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":278,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235984+0000","last_change":"2024-02-22T19:36:26.647888+0000","last_active":"2024-02-22T19:39:42.235984+0000","last_peered":"2024-02-22T19:39:42.235984+0000","last_clean":"2024-02-22T19:39:42.235984+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:39:42.235984+0000","last_undegraded":"2024-02-22T19:39:42.235984+0000","last_fullsized":"2024-02-22T19:39:42.235984+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:36:25.296484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.296484+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":101,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336691+0000","last_change":"2024-02-22T19:39:22.272688+0000","last_active":"2024-02-22T19:39:43.336691+0000","last_peered":"2024-02-22T19:39:43.336691+0000","last_clean":"2024-02-22T19:39:43.336691+0000","last_became_active":"2024-02-22T19:39:22.272214+0000","last_became_peered":"2024-02-22T19:39:22.272214+0000","last_unstale":"2024-02-22T19:39:43.336691+0000","last_undegraded":"2024-02-22T19:39:43.336691+0000","last_fullsized":"2024-02-22T19:39:43.336691+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.151294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.151294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":327,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.388306+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:39:43.388306+0000","last_peered":"2024-02-22T19:39:43.388306+0000","last_clean":"2024-02-22T19:39:43.388306+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:39:43.388306+0000","last_undegraded":"2024-02-22T19:39:43.388306+0000","last_fullsized":"2024-02-22T19:39:43.388306+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":108,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.333919+0000","last_change":"2024-02-22T19:37:58.397461+0000","last_active":"2024-02-22T19:39:43.333919+0000","last_peered":"2024-02-22T19:39:43.333919+0000","last_clean":"2024-02-22T19:39:43.333919+0000","last_became_active":"2024-02-22T19:37:57.181199+0000","last_became_peered":"2024-02-22T19:37:57.181199+0000","last_unstale":"2024-02-22T19:39:43.333919+0000","last_undegraded":"2024-02-22T19:39:43.333919+0000","last_fullsized":"2024-02-22T19:39:43.333919+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:36:09.623521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.623521+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"251'13","reported_seq":427,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.625690+0000","last_change":"2024-02-22T19:38:50.149948+0000","last_active":"2024-02-22T19:39:47.625690+0000","last_peered":"2024-02-22T19:39:47.625690+0000","last_clean":"2024-02-22T19:39:47.625690+0000","last_became_active":"2024-02-22T19:38:50.148285+0000","last_became_peered":"2024-02-22T19:38:50.148285+0000","last_unstale":"2024-02-22T19:39:47.625690+0000","last_undegraded":"2024-02-22T19:39:47.625690+0000","last_fullsized":"2024-02-22T19:39:47.625690+0000","mapping_epoch":250,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:36:53.503128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.503128+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":234,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237706+0000","last_change":"2024-02-22T19:36:11.224751+0000","last_active":"2024-02-22T19:39:42.237706+0000","last_peered":"2024-02-22T19:39:42.237706+0000","last_clean":"2024-02-22T19:39:42.237706+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:39:42.237706+0000","last_undegraded":"2024-02-22T19:39:42.237706+0000","last_fullsized":"2024-02-22T19:39:42.237706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.224646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.224646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":291,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.388739+0000","last_change":"2024-02-22T19:39:22.248740+0000","last_active":"2024-02-22T19:39:43.388739+0000","last_peered":"2024-02-22T19:39:43.388739+0000","last_clean":"2024-02-22T19:39:43.388739+0000","last_became_active":"2024-02-22T19:39:22.248120+0000","last_became_peered":"2024-02-22T19:39:22.248120+0000","last_unstale":"2024-02-22T19:39:43.388739+0000","last_undegraded":"2024-02-22T19:39:43.388739+0000","last_fullsized":"2024-02-22T19:39:43.388739+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.431527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.431527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":102,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.333593+0000","last_change":"2024-02-22T19:38:47.208139+0000","last_active":"2024-02-22T19:39:43.333593+0000","last_peered":"2024-02-22T19:39:43.333593+0000","last_clean":"2024-02-22T19:39:43.333593+0000","last_became_active":"2024-02-22T19:38:47.167662+0000","last_became_peered":"2024-02-22T19:38:47.167662+0000","last_unstale":"2024-02-22T19:39:43.333593+0000","last_undegraded":"2024-02-22T19:39:43.333593+0000","last_fullsized":"2024-02-22T19:39:43.333593+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:05.645703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.645703+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":295,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:48.227541+0000","last_change":"2024-02-22T19:39:48.227541+0000","last_active":"2024-02-22T19:39:48.227541+0000","last_peered":"2024-02-22T19:39:48.227541+0000","last_clean":"2024-02-22T19:39:48.227541+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:39:48.227541+0000","last_undegraded":"2024-02-22T19:39:48.227541+0000","last_fullsized":"2024-02-22T19:39:48.227541+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.227439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.227439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":241,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.386709+0000","last_change":"2024-02-22T19:36:54.489612+0000","last_active":"2024-02-22T19:39:43.386709+0000","last_peered":"2024-02-22T19:39:43.386709+0000","last_clean":"2024-02-22T19:39:43.386709+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:39:43.386709+0000","last_undegraded":"2024-02-22T19:39:43.386709+0000","last_fullsized":"2024-02-22T19:39:43.386709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.489509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.489509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":321,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.408185+0000","last_change":"2024-02-22T19:39:39.403851+0000","last_active":"2024-02-22T19:39:43.408185+0000","last_peered":"2024-02-22T19:39:43.408185+0000","last_clean":"2024-02-22T19:39:43.408185+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:39:43.408185+0000","last_undegraded":"2024-02-22T19:39:43.408185+0000","last_fullsized":"2024-02-22T19:39:43.408185+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:39.403775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:39:39.403775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":270,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.388433+0000","last_change":"2024-02-22T19:37:57.185054+0000","last_active":"2024-02-22T19:39:43.388433+0000","last_peered":"2024-02-22T19:39:43.388433+0000","last_clean":"2024-02-22T19:39:43.388433+0000","last_became_active":"2024-02-22T19:37:57.184627+0000","last_became_peered":"2024-02-22T19:37:57.184627+0000","last_unstale":"2024-02-22T19:39:43.388433+0000","last_undegraded":"2024-02-22T19:39:43.388433+0000","last_fullsized":"2024-02-22T19:39:43.388433+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.438112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.438112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":303,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412733+0000","last_change":"2024-02-22T19:37:18.374920+0000","last_active":"2024-02-22T19:39:43.412733+0000","last_peered":"2024-02-22T19:39:43.412733+0000","last_clean":"2024-02-22T19:39:43.412733+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:39:43.412733+0000","last_undegraded":"2024-02-22T19:39:43.412733+0000","last_fullsized":"2024-02-22T19:39:43.412733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:37:18.374807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.374807+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":63,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387004+0000","last_change":"2024-02-22T19:39:22.254050+0000","last_active":"2024-02-22T19:39:43.387004+0000","last_peered":"2024-02-22T19:39:43.387004+0000","last_clean":"2024-02-22T19:39:43.387004+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:39:43.387004+0000","last_undegraded":"2024-02-22T19:39:43.387004+0000","last_fullsized":"2024-02-22T19:39:43.387004+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.690298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.690298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":295,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.409386+0000","last_change":"2024-02-22T19:39:22.267512+0000","last_active":"2024-02-22T19:39:43.409386+0000","last_peered":"2024-02-22T19:39:43.409386+0000","last_clean":"2024-02-22T19:39:43.409386+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:39:43.409386+0000","last_undegraded":"2024-02-22T19:39:43.409386+0000","last_fullsized":"2024-02-22T19:39:43.409386+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.207753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.207753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335399+0000","last_change":"2024-02-22T19:37:57.182314+0000","last_active":"2024-02-22T19:39:43.335399+0000","last_peered":"2024-02-22T19:39:43.335399+0000","last_clean":"2024-02-22T19:39:43.335399+0000","last_became_active":"2024-02-22T19:37:57.181963+0000","last_became_peered":"2024-02-22T19:37:57.181963+0000","last_unstale":"2024-02-22T19:39:43.335399+0000","last_undegraded":"2024-02-22T19:39:43.335399+0000","last_fullsized":"2024-02-22T19:39:43.335399+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.297621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.297621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"264'15","reported_seq":71,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:46.254999+0000","last_change":"2024-02-22T19:39:46.254971+0000","last_active":"2024-02-22T19:39:46.254999+0000","last_peered":"2024-02-22T19:39:46.254999+0000","last_clean":"2024-02-22T19:39:46.254999+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:39:46.254999+0000","last_undegraded":"2024-02-22T19:39:46.254999+0000","last_fullsized":"2024-02-22T19:39:46.254999+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:39:46.254899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.254899+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334525+0000","last_change":"2024-02-22T19:39:22.262082+0000","last_active":"2024-02-22T19:39:43.334525+0000","last_peered":"2024-02-22T19:39:43.334525+0000","last_clean":"2024-02-22T19:39:43.334525+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:39:43.334525+0000","last_undegraded":"2024-02-22T19:39:43.334525+0000","last_fullsized":"2024-02-22T19:39:43.334525+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":248,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235289+0000","last_change":"2024-02-22T19:38:47.171603+0000","last_active":"2024-02-22T19:39:42.235289+0000","last_peered":"2024-02-22T19:39:42.235289+0000","last_clean":"2024-02-22T19:39:42.235289+0000","last_became_active":"2024-02-22T19:38:47.171002+0000","last_became_peered":"2024-02-22T19:38:47.171002+0000","last_unstale":"2024-02-22T19:39:42.235289+0000","last_undegraded":"2024-02-22T19:39:42.235289+0000","last_fullsized":"2024-02-22T19:39:42.235289+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:45.317440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:45.317440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335808+0000","last_change":"2024-02-22T19:37:57.185637+0000","last_active":"2024-02-22T19:39:43.335808+0000","last_peered":"2024-02-22T19:39:43.335808+0000","last_clean":"2024-02-22T19:39:43.335808+0000","last_became_active":"2024-02-22T19:37:57.185312+0000","last_became_peered":"2024-02-22T19:37:57.185312+0000","last_unstale":"2024-02-22T19:39:43.335808+0000","last_undegraded":"2024-02-22T19:39:43.335808+0000","last_fullsized":"2024-02-22T19:39:43.335808+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412268+0000","last_change":"2024-02-22T19:39:22.252057+0000","last_active":"2024-02-22T19:39:43.412268+0000","last_peered":"2024-02-22T19:39:43.412268+0000","last_clean":"2024-02-22T19:39:43.412268+0000","last_became_active":"2024-02-22T19:39:22.251742+0000","last_became_peered":"2024-02-22T19:39:22.251742+0000","last_unstale":"2024-02-22T19:39:43.412268+0000","last_undegraded":"2024-02-22T19:39:43.412268+0000","last_fullsized":"2024-02-22T19:39:43.412268+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.578317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.578317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":456,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.233310+0000","last_change":"2024-02-22T19:39:37.302965+0000","last_active":"2024-02-22T19:39:47.233310+0000","last_peered":"2024-02-22T19:39:47.233310+0000","last_clean":"2024-02-22T19:39:47.233310+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:39:47.233310+0000","last_undegraded":"2024-02-22T19:39:47.233310+0000","last_fullsized":"2024-02-22T19:39:47.233310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:39:37.302890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.302890+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":77,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:46.299786+0000","last_change":"2024-02-22T19:39:46.299764+0000","last_active":"2024-02-22T19:39:46.299786+0000","last_peered":"2024-02-22T19:39:46.299786+0000","last_clean":"2024-02-22T19:39:46.299786+0000","last_became_active":"2024-02-22T19:39:22.251614+0000","last_became_peered":"2024-02-22T19:39:22.251614+0000","last_unstale":"2024-02-22T19:39:46.299786+0000","last_undegraded":"2024-02-22T19:39:46.299786+0000","last_fullsized":"2024-02-22T19:39:46.299786+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":300,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387143+0000","last_change":"2024-02-22T19:39:22.253396+0000","last_active":"2024-02-22T19:39:43.387143+0000","last_peered":"2024-02-22T19:39:43.387143+0000","last_clean":"2024-02-22T19:39:43.387143+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:39:43.387143+0000","last_undegraded":"2024-02-22T19:39:43.387143+0000","last_fullsized":"2024-02-22T19:39:43.387143+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:24.596531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.596531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":246,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.645026+0000","last_change":"2024-02-22T19:39:47.645026+0000","last_active":"2024-02-22T19:39:47.645026+0000","last_peered":"2024-02-22T19:39:47.645026+0000","last_clean":"2024-02-22T19:39:47.645026+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:39:47.645026+0000","last_undegraded":"2024-02-22T19:39:47.645026+0000","last_fullsized":"2024-02-22T19:39:47.645026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":254,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:45.253990+0000","last_change":"2024-02-22T19:39:45.253946+0000","last_active":"2024-02-22T19:39:45.253990+0000","last_peered":"2024-02-22T19:39:45.253990+0000","last_clean":"2024-02-22T19:39:45.253990+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:39:45.253990+0000","last_undegraded":"2024-02-22T19:39:45.253990+0000","last_fullsized":"2024-02-22T19:39:45.253990+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.253853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.253853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.410944+0000","last_change":"2024-02-22T19:39:22.318935+0000","last_active":"2024-02-22T19:39:43.410944+0000","last_peered":"2024-02-22T19:39:43.410944+0000","last_clean":"2024-02-22T19:39:43.410944+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:39:43.410944+0000","last_undegraded":"2024-02-22T19:39:43.410944+0000","last_fullsized":"2024-02-22T19:39:43.410944+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.185719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.117111+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.185719+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":351,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412007+0000","last_change":"2024-02-22T19:39:34.410627+0000","last_active":"2024-02-22T19:39:43.412007+0000","last_peered":"2024-02-22T19:39:43.412007+0000","last_clean":"2024-02-22T19:39:43.412007+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:39:43.412007+0000","last_undegraded":"2024-02-22T19:39:43.412007+0000","last_fullsized":"2024-02-22T19:39:43.412007+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:39:34.410547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.410547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":289,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.418161+0000","last_change":"2024-02-22T19:38:09.219065+0000","last_active":"2024-02-22T19:39:43.418161+0000","last_peered":"2024-02-22T19:39:43.418161+0000","last_clean":"2024-02-22T19:39:43.418161+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:39:43.418161+0000","last_undegraded":"2024-02-22T19:39:43.418161+0000","last_fullsized":"2024-02-22T19:39:43.418161+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:49.141855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:49.141855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":88,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:48.479212+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:39:48.479212+0000","last_peered":"2024-02-22T19:39:48.479212+0000","last_clean":"2024-02-22T19:39:48.479212+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:39:48.479212+0000","last_undegraded":"2024-02-22T19:39:48.479212+0000","last_fullsized":"2024-02-22T19:39:48.479212+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":291,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.386888+0000","last_change":"2024-02-22T19:39:22.252856+0000","last_active":"2024-02-22T19:39:43.386888+0000","last_peered":"2024-02-22T19:39:43.386888+0000","last_clean":"2024-02-22T19:39:43.386888+0000","last_became_active":"2024-02-22T19:39:22.252166+0000","last_became_peered":"2024-02-22T19:39:22.252166+0000","last_unstale":"2024-02-22T19:39:43.386888+0000","last_undegraded":"2024-02-22T19:39:43.386888+0000","last_fullsized":"2024-02-22T19:39:43.386888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.598758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.598758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":258,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.236324+0000","last_change":"2024-02-22T19:37:57.180939+0000","last_active":"2024-02-22T19:39:42.236324+0000","last_peered":"2024-02-22T19:39:42.236324+0000","last_clean":"2024-02-22T19:39:42.236324+0000","last_became_active":"2024-02-22T19:37:57.180638+0000","last_became_peered":"2024-02-22T19:37:57.180638+0000","last_unstale":"2024-02-22T19:39:42.236324+0000","last_undegraded":"2024-02-22T19:39:42.236324+0000","last_fullsized":"2024-02-22T19:39:42.236324+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:34:50.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:34:50.967435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":370,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.414433+0000","last_change":"2024-02-22T19:39:22.439994+0000","last_active":"2024-02-22T19:39:43.414433+0000","last_peered":"2024-02-22T19:39:43.414433+0000","last_clean":"2024-02-22T19:39:43.414433+0000","last_became_active":"2024-02-22T19:39:22.256569+0000","last_became_peered":"2024-02-22T19:39:22.256569+0000","last_unstale":"2024-02-22T19:39:43.414433+0000","last_undegraded":"2024-02-22T19:39:43.414433+0000","last_fullsized":"2024-02-22T19:39:43.414433+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:37:47.421622+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:47.421622+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":261,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387551+0000","last_change":"2024-02-22T19:37:57.175982+0000","last_active":"2024-02-22T19:39:43.387551+0000","last_peered":"2024-02-22T19:39:43.387551+0000","last_clean":"2024-02-22T19:39:43.387551+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:39:43.387551+0000","last_undegraded":"2024-02-22T19:39:43.387551+0000","last_fullsized":"2024-02-22T19:39:43.387551+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:12.513648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:12.513648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":145,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335164+0000","last_change":"2024-02-22T19:38:47.168318+0000","last_active":"2024-02-22T19:39:43.335164+0000","last_peered":"2024-02-22T19:39:43.335164+0000","last_clean":"2024-02-22T19:39:43.335164+0000","last_became_active":"2024-02-22T19:38:47.167994+0000","last_became_peered":"2024-02-22T19:38:47.167994+0000","last_unstale":"2024-02-22T19:39:43.335164+0000","last_undegraded":"2024-02-22T19:39:43.335164+0000","last_fullsized":"2024-02-22T19:39:43.335164+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:01.957854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:01.957854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":261,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334665+0000","last_change":"2024-02-22T19:37:57.179467+0000","last_active":"2024-02-22T19:39:43.334665+0000","last_peered":"2024-02-22T19:39:43.334665+0000","last_clean":"2024-02-22T19:39:43.334665+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:39:43.334665+0000","last_undegraded":"2024-02-22T19:39:43.334665+0000","last_fullsized":"2024-02-22T19:39:43.334665+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.901512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.901512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":50,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235861+0000","last_change":"2024-02-22T19:39:22.247828+0000","last_active":"2024-02-22T19:39:42.235861+0000","last_peered":"2024-02-22T19:39:42.235861+0000","last_clean":"2024-02-22T19:39:42.235861+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:39:42.235861+0000","last_undegraded":"2024-02-22T19:39:42.235861+0000","last_fullsized":"2024-02-22T19:39:42.235861+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:30.342864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:35:30.342864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":940,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.625255+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:39:47.625255+0000","last_peered":"2024-02-22T19:39:47.625255+0000","last_clean":"2024-02-22T19:39:47.625255+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:39:47.625255+0000","last_undegraded":"2024-02-22T19:39:47.625255+0000","last_fullsized":"2024-02-22T19:39:47.625255+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":631,"num_read_kb":626,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":83,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335840+0000","last_change":"2024-02-22T19:39:22.301413+0000","last_active":"2024-02-22T19:39:43.335840+0000","last_peered":"2024-02-22T19:39:43.335840+0000","last_clean":"2024-02-22T19:39:43.335840+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:39:43.335840+0000","last_undegraded":"2024-02-22T19:39:43.335840+0000","last_fullsized":"2024-02-22T19:39:43.335840+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:08.878452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.878452+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":128,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.337015+0000","last_change":"2024-02-22T19:39:22.248529+0000","last_active":"2024-02-22T19:39:43.337015+0000","last_peered":"2024-02-22T19:39:43.337015+0000","last_clean":"2024-02-22T19:39:43.337015+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:39:43.337015+0000","last_undegraded":"2024-02-22T19:39:43.337015+0000","last_fullsized":"2024-02-22T19:39:43.337015+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.908714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.908714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":191,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412617+0000","last_change":"2024-02-22T19:39:38.562584+0000","last_active":"2024-02-22T19:39:43.412617+0000","last_peered":"2024-02-22T19:39:43.412617+0000","last_clean":"2024-02-22T19:39:43.412617+0000","last_became_active":"2024-02-22T19:39:22.256349+0000","last_became_peered":"2024-02-22T19:39:22.256349+0000","last_unstale":"2024-02-22T19:39:43.412617+0000","last_undegraded":"2024-02-22T19:39:43.412617+0000","last_fullsized":"2024-02-22T19:39:43.412617+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:38.562500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:39:38.562500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.333688+0000","last_change":"2024-02-22T19:37:03.977945+0000","last_active":"2024-02-22T19:39:43.333688+0000","last_peered":"2024-02-22T19:39:43.333688+0000","last_clean":"2024-02-22T19:39:43.333688+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:39:43.333688+0000","last_undegraded":"2024-02-22T19:39:43.333688+0000","last_fullsized":"2024-02-22T19:39:43.333688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.977850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.977850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":272,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:48.361494+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:39:48.361494+0000","last_peered":"2024-02-22T19:39:48.361494+0000","last_clean":"2024-02-22T19:39:48.361494+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:39:48.361494+0000","last_undegraded":"2024-02-22T19:39:48.361494+0000","last_fullsized":"2024-02-22T19:39:48.361494+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":93,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334717+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:39:43.334717+0000","last_peered":"2024-02-22T19:39:43.334717+0000","last_clean":"2024-02-22T19:39:43.334717+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:39:43.334717+0000","last_undegraded":"2024-02-22T19:39:43.334717+0000","last_fullsized":"2024-02-22T19:39:43.334717+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":293,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.413904+0000","last_change":"2024-02-22T19:39:22.264303+0000","last_active":"2024-02-22T19:39:43.413904+0000","last_peered":"2024-02-22T19:39:43.413904+0000","last_clean":"2024-02-22T19:39:43.413904+0000","last_became_active":"2024-02-22T19:39:22.263999+0000","last_became_peered":"2024-02-22T19:39:22.263999+0000","last_unstale":"2024-02-22T19:39:43.413904+0000","last_undegraded":"2024-02-22T19:39:43.413904+0000","last_fullsized":"2024-02-22T19:39:43.413904+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.441645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.441645+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.412928+0000","last_change":"2024-02-22T19:39:42.557957+0000","last_active":"2024-02-22T19:39:43.412928+0000","last_peered":"2024-02-22T19:39:43.412928+0000","last_clean":"2024-02-22T19:39:43.412928+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:39:43.412928+0000","last_undegraded":"2024-02-22T19:39:43.412928+0000","last_fullsized":"2024-02-22T19:39:43.412928+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:42.557792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.557792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":62,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238460+0000","last_change":"2024-02-22T19:39:22.271772+0000","last_active":"2024-02-22T19:39:42.238460+0000","last_peered":"2024-02-22T19:39:42.238460+0000","last_clean":"2024-02-22T19:39:42.238460+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:39:42.238460+0000","last_undegraded":"2024-02-22T19:39:42.238460+0000","last_fullsized":"2024-02-22T19:39:42.238460+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335510+0000","last_change":"2024-02-22T19:37:57.190664+0000","last_active":"2024-02-22T19:39:43.335510+0000","last_peered":"2024-02-22T19:39:43.335510+0000","last_clean":"2024-02-22T19:39:43.335510+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:39:43.335510+0000","last_undegraded":"2024-02-22T19:39:43.335510+0000","last_fullsized":"2024-02-22T19:39:43.335510+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.261537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.261537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":387,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336914+0000","last_change":"2024-02-22T19:39:40.321965+0000","last_active":"2024-02-22T19:39:43.336914+0000","last_peered":"2024-02-22T19:39:43.336914+0000","last_clean":"2024-02-22T19:39:43.336914+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:39:43.336914+0000","last_undegraded":"2024-02-22T19:39:43.336914+0000","last_fullsized":"2024-02-22T19:39:43.336914+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:40.321851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:40.321851+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"268'13","reported_seq":112,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.485150+0000","last_change":"2024-02-22T19:39:41.566614+0000","last_active":"2024-02-22T19:39:47.485150+0000","last_peered":"2024-02-22T19:39:47.485150+0000","last_clean":"2024-02-22T19:39:47.485150+0000","last_became_active":"2024-02-22T19:39:22.252945+0000","last_became_peered":"2024-02-22T19:39:22.252945+0000","last_unstale":"2024-02-22T19:39:47.485150+0000","last_undegraded":"2024-02-22T19:39:47.485150+0000","last_fullsized":"2024-02-22T19:39:47.485150+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:39:41.566527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.566527+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":140,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237708+0000","last_change":"2024-02-22T19:37:57.179173+0000","last_active":"2024-02-22T19:39:42.237708+0000","last_peered":"2024-02-22T19:39:42.237708+0000","last_clean":"2024-02-22T19:39:42.237708+0000","last_became_active":"2024-02-22T19:37:57.178778+0000","last_became_peered":"2024-02-22T19:37:57.178778+0000","last_unstale":"2024-02-22T19:39:42.237708+0000","last_undegraded":"2024-02-22T19:39:42.237708+0000","last_fullsized":"2024-02-22T19:39:42.237708+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":241,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.387384+0000","last_change":"2024-02-22T19:36:11.554490+0000","last_active":"2024-02-22T19:39:43.387384+0000","last_peered":"2024-02-22T19:39:43.387384+0000","last_clean":"2024-02-22T19:39:43.387384+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:39:43.387384+0000","last_undegraded":"2024-02-22T19:39:43.387384+0000","last_fullsized":"2024-02-22T19:39:43.387384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.554382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.554382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":72,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335635+0000","last_change":"2024-02-22T19:39:22.245754+0000","last_active":"2024-02-22T19:39:43.335635+0000","last_peered":"2024-02-22T19:39:43.335635+0000","last_clean":"2024-02-22T19:39:43.335635+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:39:43.335635+0000","last_undegraded":"2024-02-22T19:39:43.335635+0000","last_fullsized":"2024-02-22T19:39:43.335635+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.680723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.680723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":369,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:46.430127+0000","last_change":"2024-02-22T19:39:46.430095+0000","last_active":"2024-02-22T19:39:46.430127+0000","last_peered":"2024-02-22T19:39:46.430127+0000","last_clean":"2024-02-22T19:39:46.430127+0000","last_became_active":"2024-02-22T19:39:22.248416+0000","last_became_peered":"2024-02-22T19:39:22.248416+0000","last_unstale":"2024-02-22T19:39:46.430127+0000","last_undegraded":"2024-02-22T19:39:46.430127+0000","last_fullsized":"2024-02-22T19:39:46.430127+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":414,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336746+0000","last_change":"2024-02-22T19:39:23.137035+0000","last_active":"2024-02-22T19:39:43.336746+0000","last_peered":"2024-02-22T19:39:43.336746+0000","last_clean":"2024-02-22T19:39:43.336746+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:39:43.336746+0000","last_undegraded":"2024-02-22T19:39:43.336746+0000","last_fullsized":"2024-02-22T19:39:43.336746+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.408115+0000","last_change":"2024-02-22T19:38:47.167886+0000","last_active":"2024-02-22T19:39:43.408115+0000","last_peered":"2024-02-22T19:39:43.408115+0000","last_clean":"2024-02-22T19:39:43.408115+0000","last_became_active":"2024-02-22T19:38:47.167514+0000","last_became_peered":"2024-02-22T19:38:47.167514+0000","last_unstale":"2024-02-22T19:39:43.408115+0000","last_undegraded":"2024-02-22T19:39:43.408115+0000","last_fullsized":"2024-02-22T19:39:43.408115+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.583038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.583038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":45,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.236466+0000","last_change":"2024-02-22T19:38:47.170822+0000","last_active":"2024-02-22T19:39:42.236466+0000","last_peered":"2024-02-22T19:39:42.236466+0000","last_clean":"2024-02-22T19:39:42.236466+0000","last_became_active":"2024-02-22T19:38:47.170472+0000","last_became_peered":"2024-02-22T19:38:47.170472+0000","last_unstale":"2024-02-22T19:39:42.236466+0000","last_undegraded":"2024-02-22T19:39:42.236466+0000","last_fullsized":"2024-02-22T19:39:42.236466+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":272,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235699+0000","last_change":"2024-02-22T19:39:22.243142+0000","last_active":"2024-02-22T19:39:42.235699+0000","last_peered":"2024-02-22T19:39:42.235699+0000","last_clean":"2024-02-22T19:39:42.235699+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:39:42.235699+0000","last_undegraded":"2024-02-22T19:39:42.235699+0000","last_fullsized":"2024-02-22T19:39:42.235699+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:37:06.317125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":376,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238720+0000","last_change":"2024-02-22T19:39:23.568647+0000","last_active":"2024-02-22T19:39:42.238720+0000","last_peered":"2024-02-22T19:39:42.238720+0000","last_clean":"2024-02-22T19:39:42.238720+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:39:42.238720+0000","last_undegraded":"2024-02-22T19:39:42.238720+0000","last_fullsized":"2024-02-22T19:39:42.238720+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":63,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.235994+0000","last_change":"2024-02-22T19:39:22.248060+0000","last_active":"2024-02-22T19:39:42.235994+0000","last_peered":"2024-02-22T19:39:42.235994+0000","last_clean":"2024-02-22T19:39:42.235994+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:39:42.235994+0000","last_undegraded":"2024-02-22T19:39:42.235994+0000","last_fullsized":"2024-02-22T19:39:42.235994+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.336172+0000","last_change":"2024-02-22T19:37:57.190227+0000","last_active":"2024-02-22T19:39:43.336172+0000","last_peered":"2024-02-22T19:39:43.336172+0000","last_clean":"2024-02-22T19:39:43.336172+0000","last_became_active":"2024-02-22T19:37:57.189767+0000","last_became_peered":"2024-02-22T19:37:57.189767+0000","last_unstale":"2024-02-22T19:39:43.336172+0000","last_undegraded":"2024-02-22T19:39:43.336172+0000","last_fullsized":"2024-02-22T19:39:43.336172+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.071500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.071500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":101,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335317+0000","last_change":"2024-02-22T19:39:22.269372+0000","last_active":"2024-02-22T19:39:43.335317+0000","last_peered":"2024-02-22T19:39:43.335317+0000","last_clean":"2024-02-22T19:39:43.335317+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:39:43.335317+0000","last_undegraded":"2024-02-22T19:39:43.335317+0000","last_fullsized":"2024-02-22T19:39:43.335317+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.979469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.979469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"269'25","reported_seq":510,"reported_epoch":282,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:39:47.431439+0000","last_change":"2024-02-22T19:39:23.249991+0000","last_active":"2024-02-22T19:39:47.431439+0000","last_peered":"2024-02-22T19:39:47.431439+0000","last_clean":"2024-02-22T19:39:19.281649+0000","last_became_active":"2024-02-22T19:39:23.246947+0000","last_became_peered":"2024-02-22T19:39:23.246947+0000","last_unstale":"2024-02-22T19:39:47.431439+0000","last_undegraded":"2024-02-22T19:39:47.431439+0000","last_fullsized":"2024-02-22T19:39:47.431439+0000","mapping_epoch":268,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,0],"avail_no_missing":["5","0","2"],"object_location_counts":[{"shards":"0,2,5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":245,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238639+0000","last_change":"2024-02-22T19:35:27.352610+0000","last_active":"2024-02-22T19:39:42.238639+0000","last_peered":"2024-02-22T19:39:42.238639+0000","last_clean":"2024-02-22T19:39:42.238639+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:39:42.238639+0000","last_undegraded":"2024-02-22T19:39:42.238639+0000","last_fullsized":"2024-02-22T19:39:42.238639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:27.352545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.352545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":187,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334038+0000","last_change":"2024-02-22T19:39:35.331000+0000","last_active":"2024-02-22T19:39:43.334038+0000","last_peered":"2024-02-22T19:39:43.334038+0000","last_clean":"2024-02-22T19:39:43.334038+0000","last_became_active":"2024-02-22T19:38:47.093284+0000","last_became_peered":"2024-02-22T19:38:47.093284+0000","last_unstale":"2024-02-22T19:39:43.334038+0000","last_undegraded":"2024-02-22T19:39:43.334038+0000","last_fullsized":"2024-02-22T19:39:43.334038+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:35.330913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:35.330913+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237956+0000","last_change":"2024-02-22T19:37:57.180115+0000","last_active":"2024-02-22T19:39:42.237956+0000","last_peered":"2024-02-22T19:39:42.237956+0000","last_clean":"2024-02-22T19:39:42.237956+0000","last_became_active":"2024-02-22T19:37:57.179488+0000","last_became_peered":"2024-02-22T19:37:57.179488+0000","last_unstale":"2024-02-22T19:39:42.237956+0000","last_undegraded":"2024-02-22T19:39:42.237956+0000","last_fullsized":"2024-02-22T19:39:42.237956+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"251'17","reported_seq":250,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.432853+0000","last_change":"2024-02-22T19:38:50.146296+0000","last_active":"2024-02-22T19:39:47.432853+0000","last_peered":"2024-02-22T19:39:47.432853+0000","last_clean":"2024-02-22T19:39:47.432853+0000","last_became_active":"2024-02-22T19:38:50.145632+0000","last_became_peered":"2024-02-22T19:38:50.145632+0000","last_unstale":"2024-02-22T19:39:47.432853+0000","last_undegraded":"2024-02-22T19:39:47.432853+0000","last_fullsized":"2024-02-22T19:39:47.432853+0000","mapping_epoch":250,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:36:53.427951+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.427951+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":51,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.411186+0000","last_change":"2024-02-22T19:39:32.447723+0000","last_active":"2024-02-22T19:39:43.411186+0000","last_peered":"2024-02-22T19:39:43.411186+0000","last_clean":"2024-02-22T19:39:43.411186+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:39:43.411186+0000","last_undegraded":"2024-02-22T19:39:43.411186+0000","last_fullsized":"2024-02-22T19:39:43.411186+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:32.447647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.447647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":105,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.589594+0000","last_change":"2024-02-22T19:39:43.589553+0000","last_active":"2024-02-22T19:39:43.589594+0000","last_peered":"2024-02-22T19:39:43.589594+0000","last_clean":"2024-02-22T19:39:43.589594+0000","last_became_active":"2024-02-22T19:38:47.183168+0000","last_became_peered":"2024-02-22T19:38:47.183168+0000","last_unstale":"2024-02-22T19:39:43.589594+0000","last_undegraded":"2024-02-22T19:39:43.589594+0000","last_fullsized":"2024-02-22T19:39:43.589594+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":74,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.334054+0000","last_change":"2024-02-22T19:37:57.189860+0000","last_active":"2024-02-22T19:39:43.334054+0000","last_peered":"2024-02-22T19:39:43.334054+0000","last_clean":"2024-02-22T19:39:43.334054+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:39:43.334054+0000","last_undegraded":"2024-02-22T19:39:43.334054+0000","last_fullsized":"2024-02-22T19:39:43.334054+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.022678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.022678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":257,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.411845+0000","last_change":"2024-02-22T19:38:47.172343+0000","last_active":"2024-02-22T19:39:43.411845+0000","last_peered":"2024-02-22T19:39:43.411845+0000","last_clean":"2024-02-22T19:39:43.411845+0000","last_became_active":"2024-02-22T19:38:47.171941+0000","last_became_peered":"2024-02-22T19:38:47.171941+0000","last_unstale":"2024-02-22T19:39:43.411845+0000","last_undegraded":"2024-02-22T19:39:43.411845+0000","last_fullsized":"2024-02-22T19:39:43.411845+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.312692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.312692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":304,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:49.403011+0000","last_change":"2024-02-22T19:39:49.403011+0000","last_active":"2024-02-22T19:39:49.403011+0000","last_peered":"2024-02-22T19:39:49.403011+0000","last_clean":"2024-02-22T19:39:49.403011+0000","last_became_active":"2024-02-22T19:39:22.244710+0000","last_became_peered":"2024-02-22T19:39:22.244710+0000","last_unstale":"2024-02-22T19:39:49.403011+0000","last_undegraded":"2024-02-22T19:39:49.403011+0000","last_fullsized":"2024-02-22T19:39:49.403011+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":264,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:45.615459+0000","last_change":"2024-02-22T19:39:45.615429+0000","last_active":"2024-02-22T19:39:45.615459+0000","last_peered":"2024-02-22T19:39:45.615459+0000","last_clean":"2024-02-22T19:39:45.615459+0000","last_became_active":"2024-02-22T19:38:47.164172+0000","last_became_peered":"2024-02-22T19:38:47.164172+0000","last_unstale":"2024-02-22T19:39:45.615459+0000","last_undegraded":"2024-02-22T19:39:45.615459+0000","last_fullsized":"2024-02-22T19:39:45.615459+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":125,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.408453+0000","last_change":"2024-02-22T19:38:48.843443+0000","last_active":"2024-02-22T19:39:43.408453+0000","last_peered":"2024-02-22T19:39:43.408453+0000","last_clean":"2024-02-22T19:39:43.408453+0000","last_became_active":"2024-02-22T19:38:47.181392+0000","last_became_peered":"2024-02-22T19:38:47.181392+0000","last_unstale":"2024-02-22T19:39:43.408453+0000","last_undegraded":"2024-02-22T19:39:43.408453+0000","last_fullsized":"2024-02-22T19:39:43.408453+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:37:48.379789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:48.379789+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":264,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.386886+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:39:43.386886+0000","last_peered":"2024-02-22T19:39:43.386886+0000","last_clean":"2024-02-22T19:39:43.386886+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:39:43.386886+0000","last_undegraded":"2024-02-22T19:39:43.386886+0000","last_fullsized":"2024-02-22T19:39:43.386886+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":90,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-22T19:39:43.335495+0000","last_change":"2024-02-22T19:39:32.393413+0000","last_active":"2024-02-22T19:39:43.335495+0000","last_peered":"2024-02-22T19:39:43.335495+0000","last_clean":"2024-02-22T19:39:43.335495+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:39:43.335495+0000","last_undegraded":"2024-02-22T19:39:43.335495+0000","last_fullsized":"2024-02-22T19:39:43.335495+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:39:32.393320+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.393320+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":450,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-22T19:39:47.233667+0000","last_change":"2024-02-22T19:38:00.221092+0000","last_active":"2024-02-22T19:39:47.233667+0000","last_peered":"2024-02-22T19:39:47.233667+0000","last_clean":"2024-02-22T19:39:47.233667+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:39:47.233667+0000","last_undegraded":"2024-02-22T19:39:47.233667+0000","last_fullsized":"2024-02-22T19:39:47.233667+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:37:13.243499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:13.243499+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":306,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.237588+0000","last_change":"2024-02-22T19:39:22.400760+0000","last_active":"2024-02-22T19:39:42.237588+0000","last_peered":"2024-02-22T19:39:42.237588+0000","last_clean":"2024-02-22T19:39:42.237588+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:39:42.237588+0000","last_undegraded":"2024-02-22T19:39:42.237588+0000","last_fullsized":"2024-02-22T19:39:42.237588+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:37:05.357880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.357880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":99,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-22T19:39:42.238229+0000","last_change":"2024-02-22T19:38:48.304832+0000","last_active":"2024-02-22T19:39:42.238229+0000","last_peered":"2024-02-22T19:39:42.238229+0000","last_clean":"2024-02-22T19:39:42.238229+0000","last_became_active":"2024-02-22T19:38:47.169571+0000","last_became_peered":"2024-02-22T19:38:47.169571+0000","last_unstale":"2024-02-22T19:39:42.238229+0000","last_undegraded":"2024-02-22T19:39:42.238229+0000","last_fullsized":"2024-02-22T19:39:42.238229+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-22T19:35:33.971446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.971446+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":631,"num_read_kb":626,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":6257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1109,"num_read_kb":866,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":7172,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266821,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307024,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93438128,"statfs":{"total":95995035648,"available":95680643072,"internally_reserved":0,"allocated":1646592,"data_stored":895769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.456,"15min":1.244},"min":{"1min":0.27400000000000002,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":5.4539999999999997,"5min":7.0700000000000003,"15min":42.597999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":2.387,"5min":1.8779999999999999,"15min":1.2609999999999999},"min":{"1min":0.223,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":8.0489999999999995,"5min":9.8870000000000005,"15min":42.155999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":3.9180000000000001,"5min":2.0470000000000002,"15min":1.337},"min":{"1min":0.245,"5min":0.223,"15min":0.223},"max":{"1min":54.951000000000001,"5min":54.951000000000001,"15min":54.951000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":7.2450000000000001,"5min":3.0049999999999999,"15min":1.714},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.219},"max":{"1min":57.012,"5min":57.012,"15min":57.012},"last":0.755}]},{"osd":2,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":3.8820000000000001,"5min":1.8200000000000001,"15min":1.236},"min":{"1min":0.28199999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":40.606000000000002,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":5.0039999999999996,"5min":2.1480000000000001,"15min":1.4319999999999999},"min":{"1min":0.33700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":38.411000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.501}]},{"osd":3,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":4.5229999999999997,"5min":2.1059999999999999,"15min":1.4410000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":38.692999999999998,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":4.2539999999999996,"5min":2.2280000000000002,"15min":1.4279999999999999},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":38.219000000000001,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":0.80500000000000005,"15min":0.57799999999999996},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.23999999999999999},"max":{"1min":5.5199999999999996,"5min":5.5199999999999996,"15min":5.5199999999999996},"last":0.623},{"interface":"front","average":{"1min":1.179,"5min":0.68400000000000005,"15min":0.53800000000000003},"min":{"1min":0.27500000000000002,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":7.6219999999999999,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.75600000000000001,"15min":0.57799999999999996},"min":{"1min":0.28399999999999997,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":3.254,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.2909999999999999,"5min":0.83399999999999996,"15min":0.626},"min":{"1min":0.317,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":0.96899999999999997,"15min":0.71499999999999997},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":12.199999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.444,"5min":0.75900000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.6390000000000002,"5min":7.6390000000000002,"15min":12.067},"last":0.68700000000000006}]}]},{"osd":6,"up_from":45,"seq":193273528458,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298816,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93446336,"statfs":{"total":95995035648,"available":95689048064,"internally_reserved":0,"allocated":1630208,"data_stored":894404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:38:52 2024","interfaces":[{"interface":"back","average":{"1min":2.0190000000000001,"5min":1.5669999999999999,"15min":1.0289999999999999},"min":{"1min":0.222,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":6.6660000000000004,"5min":6.9939999999999998,"15min":17.885000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.2629999999999999,"15min":0.96299999999999997},"min":{"1min":0.24299999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.2699999999999996},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Feb 22 19:38:52 2024","interfaces":[{"interface":"back","average":{"1min":3.0249999999999999,"5min":1.7749999999999999,"15min":1.046},"min":{"1min":0.45200000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":11.946999999999999,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":2.3050000000000002,"5min":1.4379999999999999,"15min":0.92500000000000004},"min":{"1min":0.26500000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.381,"5min":11.381,"15min":11.381},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:38:52 2024","interfaces":[{"interface":"back","average":{"1min":3.0569999999999999,"5min":1.9179999999999999,"15min":1.1819999999999999},"min":{"1min":0.28599999999999998,"5min":0.249,"15min":0.249},"max":{"1min":11.198,"5min":11.198,"15min":12.859999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":2.5539999999999998,"5min":1.8109999999999999,"15min":1.0640000000000001},"min":{"1min":0.30299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":11.565,"5min":11.565,"15min":11.565},"last":0.745}]},{"osd":3,"last update":"Thu Feb 22 19:38:52 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":1.464,"15min":1.1719999999999999},"min":{"1min":0.27000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":10.135,"5min":10.135,"15min":38.762},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.5149999999999999,"15min":1.286},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":7.4390000000000001,"5min":9.7070000000000007,"15min":35.612000000000002},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Feb 22 19:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51400000000000001,"15min":0.45400000000000001},"min":{"1min":0.254,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.4119999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.1419999999999999,"5min":0.69499999999999995,"15min":0.52200000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":9.3949999999999996,"5min":9.3949999999999996,"15min":9.3949999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.71099999999999997,"15min":0.53500000000000003},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.3780000000000001,"5min":7.3780000000000001,"15min":7.3780000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.70399999999999996,"15min":0.53700000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.0230000000000001,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":0.73799999999999999,"15min":0.58699999999999997},"min":{"1min":0.32200000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":3.2709999999999999,"5min":3.488,"15min":3.488},"last":0.495},{"interface":"front","average":{"1min":1.294,"5min":0.86399999999999999,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":5.7939999999999996,"5min":5.7939999999999996,"15min":5.7939999999999996},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315252,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93429900,"statfs":{"total":95995035648,"available":95672217600,"internally_reserved":0,"allocated":1683456,"data_stored":896525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:38:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.9710000000000001,"15min":1.2609999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":5.9000000000000004,"5min":10.6,"15min":15.613},"last":0.41899999999999998},{"interface":"front","average":{"1min":1.875,"5min":2.202,"15min":1.377},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":9.1259999999999994,"5min":13.515000000000001,"15min":13.515000000000001},"last":0.441}]},{"osd":2,"last update":"Thu Feb 22 19:39:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3809999999999998,"5min":1.766,"15min":1.4139999999999999},"min":{"1min":0.215,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.682,"5min":11.682,"15min":16.503},"last":0.50700000000000001},{"interface":"front","average":{"1min":2.8100000000000001,"5min":2.4700000000000002,"15min":1.5369999999999999},"min":{"1min":0.23200000000000001,"5min":0.20200000000000001,"15min":0.192},"max":{"1min":9.9559999999999995,"5min":14.994,"15min":14.994},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:39:44 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":2.0619999999999998,"15min":1.6180000000000001},"min":{"1min":0.27000000000000002,"5min":0.23100000000000001,"15min":0.214},"max":{"1min":9.7650000000000006,"5min":10.734999999999999,"15min":25.145},"last":0.53400000000000003},{"interface":"front","average":{"1min":2.8039999999999998,"5min":2.1779999999999999,"15min":1.6839999999999999},"min":{"1min":0.32000000000000001,"5min":0.251,"15min":0.251},"max":{"1min":9.0939999999999994,"5min":9.0939999999999994,"15min":22.25},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:38:59 2024","interfaces":[{"interface":"back","average":{"1min":1.8560000000000001,"5min":1.4570000000000001,"15min":1.133},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":5.9619999999999997,"5min":8.1449999999999996,"15min":15.542999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4910000000000001,"15min":1.022},"min":{"1min":0.33200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":4.5670000000000002,"5min":10.706,"15min":10.706},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:39:21 2024","interfaces":[{"interface":"back","average":{"1min":2.6699999999999999,"5min":1.6990000000000001,"15min":1.2370000000000001},"min":{"1min":0.442,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.976,"5min":8.2189999999999994,"15min":15.587999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.514,"15min":1.115},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":4.4059999999999997,"5min":6.6790000000000003,"15min":9.4719999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:38:49 2024","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.631,"15min":1.056},"min":{"1min":0.376,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.6139999999999999,"5min":10.631,"15min":10.631},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.464,"15min":1.1299999999999999},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.5800000000000001,"5min":8.1880000000000006,"15min":15.651999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:39:11 2024","interfaces":[{"interface":"back","average":{"1min":2.258,"5min":1.8680000000000001,"15min":1.214},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":8.9800000000000004,"5min":13.169,"15min":13.169},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.5900000000000001,"15min":1.073},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":4.5910000000000002,"5min":10.673,"15min":10.673},"last":0.5}]}]},{"osd":0,"up_from":0,"seq":34359738536,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411485,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298820,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93446332,"statfs":{"total":95995035648,"available":95689043968,"internally_reserved":0,"allocated":1634304,"data_stored":894302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:10 2024","interfaces":[{"interface":"back","average":{"1min":4.0750000000000002,"5min":2.2909999999999999,"15min":1.3859999999999999},"min":{"1min":0.20799999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":16.763999999999999,"5min":22.338999999999999,"15min":22.338999999999999},"last":0.315},{"interface":"front","average":{"1min":4.9349999999999996,"5min":2.8919999999999999,"15min":1.6830000000000001},"min":{"1min":0.251,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":30.427,"5min":30.427,"15min":30.427},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:39:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7759999999999998,"5min":2.194,"15min":1.3460000000000001},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":6.9249999999999998,"5min":9.9030000000000005,"15min":10.183},"last":0.35599999999999998},{"interface":"front","average":{"1min":2.927,"5min":2.242,"15min":1.258},"min":{"1min":0.314,"5min":0.22500000000000001,"15min":0.193},"max":{"1min":13.782,"5min":13.782,"15min":13.782},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:39:48 2024","interfaces":[{"interface":"back","average":{"1min":3.589,"5min":2.6269999999999998,"15min":1.722},"min":{"1min":0.307,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":16.076000000000001,"5min":22.302,"15min":30.071000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":3.79,"5min":2.7829999999999999,"15min":1.865},"min":{"1min":0.29399999999999998,"5min":0.184,"15min":0.184},"max":{"1min":11.768000000000001,"5min":30.282,"15min":32.119},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:39:00 2024","interfaces":[{"interface":"back","average":{"1min":2.1259999999999999,"5min":1.681,"15min":1.1279999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.5129999999999999,"5min":7.5129999999999999,"15min":7.5129999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":2.7050000000000001,"5min":1.5409999999999999,"15min":0.97099999999999997},"min":{"1min":0.34999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":7.9989999999999997,"5min":7.9989999999999997,"15min":9.5199999999999996},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:39:18 2024","interfaces":[{"interface":"back","average":{"1min":3.3149999999999999,"5min":1.7709999999999999,"15min":1.0940000000000001},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":10.300000000000001,"5min":10.300000000000001,"15min":10.300000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":2.774,"5min":1.7629999999999999,"15min":1.23},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":12.391999999999999,"5min":12.391999999999999,"15min":12.391999999999999},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:38:53 2024","interfaces":[{"interface":"back","average":{"1min":1.8819999999999999,"5min":1.7809999999999999,"15min":1.181},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":7.0220000000000002,"5min":7.0220000000000002,"15min":9.9689999999999994},"last":0.58599999999999997},{"interface":"front","average":{"1min":2.6930000000000001,"5min":1.5589999999999999,"15min":1.0009999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":7.2679999999999998,"5min":7.2679999999999998,"15min":9.5990000000000002},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:39:10 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":1.8109999999999999,"15min":1.214},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":7.0540000000000003,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":0.63300000000000001},{"interface":"front","average":{"1min":3.1099999999999999,"5min":1.9039999999999999,"15min":1.1299999999999999},"min":{"1min":0.51500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":9.8889999999999993,"5min":9.8889999999999993,"15min":9.8889999999999993},"last":0.499}]}]},{"osd":3,"up_from":26,"seq":111669149846,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298836,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93446316,"statfs":{"total":95995035648,"available":95689027584,"internally_reserved":0,"allocated":1650688,"data_stored":889595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:39 2024","interfaces":[{"interface":"back","average":{"1min":3.0339999999999998,"5min":2.3109999999999999,"15min":1.573},"min":{"1min":0.21199999999999999,"5min":0.19400000000000001,"15min":0.193},"max":{"1min":8.4339999999999993,"5min":23.48,"15min":23.48},"last":3.657},{"interface":"front","average":{"1min":2.919,"5min":2.6640000000000001,"15min":1.8069999999999999},"min":{"1min":0.23599999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.352,"5min":23.411999999999999,"15min":23.411999999999999},"last":18.172000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:39:39 2024","interfaces":[{"interface":"back","average":{"1min":4.7839999999999998,"5min":3.2869999999999999,"15min":2.0499999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":23.077999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":18.213999999999999},{"interface":"front","average":{"1min":4.2789999999999999,"5min":2.8260000000000001,"15min":1.6970000000000001},"min":{"1min":0.23499999999999999,"5min":0.214,"15min":0.214},"max":{"1min":22.187000000000001,"5min":22.187000000000001,"15min":22.187000000000001},"last":12.234}]},{"osd":2,"last update":"Thu Feb 22 19:39:39 2024","interfaces":[{"interface":"back","average":{"1min":3.8690000000000002,"5min":2.4100000000000001,"15min":1.7430000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.27400000000000002},"max":{"1min":13.532,"5min":13.532,"15min":16.271000000000001},"last":20.105},{"interface":"front","average":{"1min":5.0819999999999999,"5min":2.5720000000000001,"15min":1.7230000000000001},"min":{"1min":0.29999999999999999,"5min":0.245,"15min":0.217},"max":{"1min":35.939999999999998,"5min":35.939999999999998,"15min":35.939999999999998},"last":7.617}]},{"osd":4,"last update":"Thu Feb 22 19:38:55 2024","interfaces":[{"interface":"back","average":{"1min":2.8679999999999999,"5min":2.0470000000000002,"15min":1.3640000000000001},"min":{"1min":0.34000000000000002,"5min":0.249,"15min":0.249},"max":{"1min":10.967000000000001,"5min":13.680999999999999,"15min":13.680999999999999},"last":12.416},{"interface":"front","average":{"1min":4.181,"5min":2.0299999999999998,"15min":1.3200000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":13.548,"5min":13.548,"15min":13.548},"last":12.273}]},{"osd":5,"last update":"Thu Feb 22 19:39:27 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":1.6919999999999999,"15min":1.2549999999999999},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":7.9779999999999998,"5min":8.0289999999999999,"15min":12.879},"last":3.6789999999999998},{"interface":"front","average":{"1min":3.4079999999999999,"5min":2.1120000000000001,"15min":1.431},"min":{"1min":0.39700000000000002,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":10.446999999999999,"5min":11.167999999999999,"15min":13.707000000000001},"last":12.372999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:38:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.478,"15min":1.143},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.9560000000000004,"5min":7.9640000000000004,"15min":12.898999999999999},"last":3.5819999999999999},{"interface":"front","average":{"1min":2.0369999999999999,"5min":2.0110000000000001,"15min":1.373},"min":{"1min":0.42999999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":10.913,"5min":13.803000000000001,"15min":13.803000000000001},"last":12.396000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:39:04 2024","interfaces":[{"interface":"back","average":{"1min":3.9100000000000001,"5min":2.0169999999999999,"15min":1.3120000000000001},"min":{"1min":0.45100000000000001,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":13.483000000000001,"5min":13.483000000000001,"15min":13.483000000000001},"last":12.327},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.583,"15min":1.2150000000000001},"min":{"1min":0.47799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":7.9320000000000004,"5min":7.9909999999999997,"15min":12.846},"last":3.6320000000000001}]}]},{"osd":4,"up_from":0,"seq":137438953621,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724687,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298820,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93446332,"statfs":{"total":95995035648,"available":95689043968,"internally_reserved":0,"allocated":1634304,"data_stored":894504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":2.3490000000000002,"5min":1.6879999999999999,"15min":1.147},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.222},"max":{"1min":17.684000000000001,"5min":17.684000000000001,"15min":17.684000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":2.1789999999999998,"5min":1.7829999999999999,"15min":1.2130000000000001},"min":{"1min":0.252,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":17.402999999999999,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.827,"15min":1.202},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.22500000000000001},"max":{"1min":11.202,"5min":29.390999999999998,"15min":29.390999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.7549999999999999,"5min":1.6200000000000001,"15min":1.173},"min":{"1min":0.30599999999999999,"5min":0.26400000000000001,"15min":0.218},"max":{"1min":8.7490000000000006,"5min":29.367000000000001,"15min":29.367000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":3.3069999999999999,"5min":1.8069999999999999,"15min":1.218},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":11.367000000000001,"5min":11.367000000000001,"15min":14.613},"last":0.49199999999999999},{"interface":"front","average":{"1min":3.2989999999999999,"5min":1.847,"15min":1.284},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":16.099,"5min":16.099,"15min":16.099},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":1.766,"5min":1.79,"15min":1.361},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.5780000000000003,"5min":9.6029999999999998,"15min":13.366},"last":0.434},{"interface":"front","average":{"1min":2.0409999999999999,"5min":1.73,"15min":1.3440000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.28299999999999997},"max":{"1min":8.6069999999999993,"5min":9.4649999999999999,"15min":10.16},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65400000000000003,"15min":0.501},"min":{"1min":0.223,"5min":0.223,"15min":0.219},"max":{"1min":2.403,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.67300000000000004,"15min":0.51100000000000001},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Feb 22 19:39:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.58399999999999996,"15min":0.51700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.76600000000000001,"5min":3.246,"15min":5.6040000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.72899999999999998,"15min":0.58699999999999997},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.309},"max":{"1min":3.0590000000000002,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61299999999999999,"15min":0.63100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.6420000000000003,"5min":4.6420000000000003,"15min":5.5789999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.59599999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":3.2829999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:39:51.142 INFO:tasks.thrashosds.thrasher:Setting 1.4 to [3, 5, 6] 2024-02-22T19:39:51.142 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.4', '3', '5', '6'] 2024-02-22T19:39:51.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pg-upmap 1.4 3 5 6 2024-02-22T19:39:51.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:51.250 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:50.326032+0000 mgr.y (mgr.14535) 416 : audit [DBG] from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:39:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:50.386036+0000 osd.1 (osd.1) 166 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:39:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:50.386912+0000 osd.1 (osd.1) 167 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:39:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:50.693792+0000 osd.3 (osd.3) 171 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:39:51.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:50.696672+0000 osd.3 (osd.3) 172 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:39:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:50.326032+0000 mgr.y (mgr.14535) 416 : audit [DBG] from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:39:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:50.386036+0000 osd.1 (osd.1) 166 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:39:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:50.386912+0000 osd.1 (osd.1) 167 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:39:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:50.693792+0000 osd.3 (osd.3) 171 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:39:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:50.696672+0000 osd.3 (osd.3) 172 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:39:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:50.326032+0000 mgr.y (mgr.14535) 416 : audit [DBG] from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:39:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:50.386036+0000 osd.1 (osd.1) 166 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:39:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:50.386912+0000 osd.1 (osd.1) 167 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:39:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:50.693792+0000 osd.3 (osd.3) 171 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:39:51.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:50.696672+0000 osd.3 (osd.3) 172 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:39:52.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:39:52.300 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:52.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:52.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:52.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:52.435 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:52.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.125890+0000 osd.2 (osd.2) 144 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:39:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.129001+0000 osd.2 (osd.2) 145 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:39:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.551172+0000 mgr.y (mgr.14535) 417 : cluster [DBG] pgmap v534: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.566250+0000 osd.6 (osd.6) 126 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:39:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.569150+0000 osd.6 (osd.6) 127 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:39:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.677947+0000 osd.3 (osd.3) 173 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:51.680813+0000 osd.3 (osd.3) 174 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.125890+0000 osd.2 (osd.2) 144 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.129001+0000 osd.2 (osd.2) 145 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.551172+0000 mgr.y (mgr.14535) 417 : cluster [DBG] pgmap v534: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.566250+0000 osd.6 (osd.6) 126 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.569150+0000 osd.6 (osd.6) 127 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.677947+0000 osd.3 (osd.3) 173 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:39:52.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:51.680813+0000 osd.3 (osd.3) 174 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:39:52.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:52.591 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:52.610 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:52.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:52.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:52.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.125890+0000 osd.2 (osd.2) 144 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.129001+0000 osd.2 (osd.2) 145 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.551172+0000 mgr.y (mgr.14535) 417 : cluster [DBG] pgmap v534: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.566250+0000 osd.6 (osd.6) 126 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.569150+0000 osd.6 (osd.6) 127 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.677947+0000 osd.3 (osd.3) 173 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:39:52.742 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:51.680813+0000 osd.3 (osd.3) 174 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:39:52.744 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:52.766 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:52.768 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:52.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:52.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:39:52.898 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:53.095 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:53.097 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:39:53.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:53.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:39:53.136 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:53.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:53.232 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:53.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:53.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:53.349 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:53.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:39:53.466 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:53.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:52.172175+0000 osd.2 (osd.2) 146 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:39:53.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:52.175139+0000 osd.2 (osd.2) 147 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:39:53.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:52.172175+0000 osd.2 (osd.2) 146 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:39:53.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:52.175139+0000 osd.2 (osd.2) 147 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:39:53.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:39:53.814 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:39:53.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:53.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:39:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:52.172175+0000 osd.2 (osd.2) 146 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:39:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:52.175139+0000 osd.2 (osd.2) 147 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:39:53.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:53.974 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:53.977 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:53.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:54.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:39:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:54.131 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:39:54.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:54.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:39:54.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:54.288 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:54.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:54.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:54.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:54.420 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:54.440 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:54.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:54.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:54.544 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:54.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.123157+0000 osd.2 (osd.2) 148 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:39:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.126486+0000 osd.2 (osd.2) 149 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:39:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:53.363984+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.551535+0000 mgr.y (mgr.14535) 418 : cluster [DBG] pgmap v535: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.643022+0000 osd.3 (osd.3) 175 : cluster [DBG] 4.c scrub starts 2024-02-22T19:39:54.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.646664+0000 osd.3 (osd.3) 176 : cluster [DBG] 4.c scrub ok 2024-02-22T19:39:54.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.123157+0000 osd.2 (osd.2) 148 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:39:54.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.126486+0000 osd.2 (osd.2) 149 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:39:54.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:53.363984+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:54.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.551535+0000 mgr.y (mgr.14535) 418 : cluster [DBG] pgmap v535: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.643022+0000 osd.3 (osd.3) 175 : cluster [DBG] 4.c scrub starts 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.646664+0000 osd.3 (osd.3) 176 : cluster [DBG] 4.c scrub ok 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.123157+0000 osd.2 (osd.2) 148 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.126486+0000 osd.2 (osd.2) 149 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:53.363984+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.551535+0000 mgr.y (mgr.14535) 418 : cluster [DBG] pgmap v535: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.643022+0000 osd.3 (osd.3) 175 : cluster [DBG] 4.c scrub starts 2024-02-22T19:39:54.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.646664+0000 osd.3 (osd.3) 176 : cluster [DBG] 4.c scrub ok 2024-02-22T19:39:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:39:55.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:39:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:39:55.424 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:55.426 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:55.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:39:55.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.436668+0000 osd.5 (osd.5) 73 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:53.438288+0000 osd.5 (osd.5) 74 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.124980+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.a scrub starts 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.127872+0000 osd.2 (osd.2) 151 : cluster [DBG] 5.a scrub ok 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.559061+0000 osd.6 (osd.6) 128 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.559788+0000 osd.6 (osd.6) 129 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:39:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.617454+0000 osd.3 (osd.3) 177 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.620503+0000 osd.3 (osd.3) 178 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:54.863402+0000 mgr.y (mgr.14535) 419 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:55.181147+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? 172.21.15.53:0/2440417547' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.436668+0000 osd.5 (osd.5) 73 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:53.438288+0000 osd.5 (osd.5) 74 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.124980+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.a scrub starts 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.127872+0000 osd.2 (osd.2) 151 : cluster [DBG] 5.a scrub ok 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.559061+0000 osd.6 (osd.6) 128 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.559788+0000 osd.6 (osd.6) 129 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.617454+0000 osd.3 (osd.3) 177 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.620503+0000 osd.3 (osd.3) 178 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:39:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:54.863402+0000 mgr.y (mgr.14535) 419 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:55.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:55.181147+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? 172.21.15.53:0/2440417547' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:55.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:55.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.436668+0000 osd.5 (osd.5) 73 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:53.438288+0000 osd.5 (osd.5) 74 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.124980+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.a scrub starts 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.127872+0000 osd.2 (osd.2) 151 : cluster [DBG] 5.a scrub ok 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.559061+0000 osd.6 (osd.6) 128 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.559788+0000 osd.6 (osd.6) 129 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.617454+0000 osd.3 (osd.3) 177 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.620503+0000 osd.3 (osd.3) 178 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:39:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:54.863402+0000 mgr.y (mgr.14535) 419 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:39:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:55.181147+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? 172.21.15.53:0/2440417547' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:39:55.861 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:39:55.893 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:39:55.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:55.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:39:56.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:56.050 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:56.073 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:39:56.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:56.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:39:56.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:56.207 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:56.488 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.475957+0000 osd.5 (osd.5) 75 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:54.478844+0000 osd.5 (osd.5) 76 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.156337+0000 osd.2 (osd.2) 152 : cluster [DBG] 5.d scrub starts 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.160153+0000 osd.2 (osd.2) 153 : cluster [DBG] 5.d scrub ok 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.367134+0000 mon.a (mon.0) 1254 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:55.371668+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? 172.21.15.53:0/2440417547' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.371742+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e283: 8 total, 8 up, 6 in 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.454765+0000 osd.5 (osd.5) 77 : cluster [DBG] 1.a scrub starts 2024-02-22T19:39:56.491 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.551899+0000 mgr.y (mgr.14535) 420 : cluster [DBG] pgmap v537: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.475957+0000 osd.5 (osd.5) 75 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:54.478844+0000 osd.5 (osd.5) 76 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.156337+0000 osd.2 (osd.2) 152 : cluster [DBG] 5.d scrub starts 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.160153+0000 osd.2 (osd.2) 153 : cluster [DBG] 5.d scrub ok 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.367134+0000 mon.a (mon.0) 1254 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:55.371668+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? 172.21.15.53:0/2440417547' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.371742+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e283: 8 total, 8 up, 6 in 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.454765+0000 osd.5 (osd.5) 77 : cluster [DBG] 1.a scrub starts 2024-02-22T19:39:56.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.551899+0000 mgr.y (mgr.14535) 420 : cluster [DBG] pgmap v537: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:56.757 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:56.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:56.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.475957+0000 osd.5 (osd.5) 75 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:54.478844+0000 osd.5 (osd.5) 76 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.156337+0000 osd.2 (osd.2) 152 : cluster [DBG] 5.d scrub starts 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.160153+0000 osd.2 (osd.2) 153 : cluster [DBG] 5.d scrub ok 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.367134+0000 mon.a (mon.0) 1254 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:55.371668+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? 172.21.15.53:0/2440417547' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:39:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.371742+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e283: 8 total, 8 up, 6 in 2024-02-22T19:39:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.454765+0000 osd.5 (osd.5) 77 : cluster [DBG] 1.a scrub starts 2024-02-22T19:39:56.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.551899+0000 mgr.y (mgr.14535) 420 : cluster [DBG] pgmap v537: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:56.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:57.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:55.639173+0000 osd.7 (osd.7) 180 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:39:57.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:57.235701+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e284: 8 total, 8 up, 6 in 2024-02-22T19:39:57.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:55.639173+0000 osd.7 (osd.7) 180 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:39:57.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:57.235701+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e284: 8 total, 8 up, 6 in 2024-02-22T19:39:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:55.639173+0000 osd.7 (osd.7) 180 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:39:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:57.235701+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e284: 8 total, 8 up, 6 in 2024-02-22T19:39:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:58.247 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:39:58.247 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:39:58.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:58.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:58.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:58.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:57.552462+0000 mgr.y (mgr.14535) 421 : cluster [DBG] pgmap v539: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:58.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:57.552462+0000 mgr.y (mgr.14535) 421 : cluster [DBG] pgmap v539: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:58.533 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:58.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:58.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:39:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:57.552462+0000 mgr.y (mgr.14535) 421 : cluster [DBG] pgmap v539: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:39:58.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:39:59.599 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:39:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:39:59] "GET /metrics HTTP/1.1" 200 173618 "" "Prometheus/2.33.4" 2024-02-22T19:39:59.611 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:39:59.614 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:59.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:39:59.716 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:39:59.749 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:39:59.849 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:39:59.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:39:59.852 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:39:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_2' created 2024-02-22T19:39:59.883 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:39:59.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:39:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:00.010 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:00.013 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:00.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:00.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:00.142 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:59.370137+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.53:0/4174736023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-02-22T19:40:00.142 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:39:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:59.370420+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-02-22T19:40:00.147 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:00.168 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:00.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:00.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:59.370137+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.53:0/4174736023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-02-22T19:40:00.368 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:39:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:59.370420+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-02-22T19:40:00.369 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:59.370137+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.53:0/4174736023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-02-22T19:40:00.369 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:39:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:59.370420+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-02-22T19:40:00.369 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:00.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:00.422 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:00.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:00.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:00.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:00.574 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:00.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:00.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:00.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:set 1.4 pg_upmap mapping to [3,5,6] 2024-02-22T19:40:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:59.552986+0000 mgr.y (mgr.14535) 422 : cluster [DBG] pgmap v540: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:39:59.869344+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-02-22T19:40:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:39:59.869436+0000 mon.a (mon.0) 1260 : cluster [DBG] osdmap e285: 8 total, 8 up, 6 in 2024-02-22T19:40:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:00.000171+0000 mon.a (mon.0) 1261 : cluster [WRN] Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:40:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:00.000246+0000 mon.a (mon.0) 1262 : cluster [WRN] [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:00.000296+0000 mon.a (mon.0) 1263 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:00.000343+0000 mon.a (mon.0) 1264 : cluster [WRN] pg 5.e is active+remapped+backfill_toofull, acting [5,2,0] 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:00.495854+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.53:0/1920165501' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "1.4", "id": [3, 5, 6]}]: dispatch 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:00.870256+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? 172.21.15.53:0/1920165501' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.4", "id": [3, 5, 6]}]': finished 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:00.870323+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e286: 8 total, 8 up, 6 in 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:59.552986+0000 mgr.y (mgr.14535) 422 : cluster [DBG] pgmap v540: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:39:59.869344+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:39:59.869436+0000 mon.a (mon.0) 1260 : cluster [DBG] osdmap e285: 8 total, 8 up, 6 in 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:00.000171+0000 mon.a (mon.0) 1261 : cluster [WRN] Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:00.000246+0000 mon.a (mon.0) 1262 : cluster [WRN] [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:00.000296+0000 mon.a (mon.0) 1263 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:00.000343+0000 mon.a (mon.0) 1264 : cluster [WRN] pg 5.e is active+remapped+backfill_toofull, acting [5,2,0] 2024-02-22T19:40:01.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:00.495854+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.53:0/1920165501' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "1.4", "id": [3, 5, 6]}]: dispatch 2024-02-22T19:40:01.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:00.870256+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? 172.21.15.53:0/1920165501' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.4", "id": [3, 5, 6]}]': finished 2024-02-22T19:40:01.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:00.870323+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e286: 8 total, 8 up, 6 in 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:59.552986+0000 mgr.y (mgr.14535) 422 : cluster [DBG] pgmap v540: 132 pgs: 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:39:59.869344+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:39:59.869436+0000 mon.a (mon.0) 1260 : cluster [DBG] osdmap e285: 8 total, 8 up, 6 in 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:00.000171+0000 mon.a (mon.0) 1261 : cluster [WRN] Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:00.000246+0000 mon.a (mon.0) 1262 : cluster [WRN] [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:00.000296+0000 mon.a (mon.0) 1263 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:40:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:00.000343+0000 mon.a (mon.0) 1264 : cluster [WRN] pg 5.e is active+remapped+backfill_toofull, acting [5,2,0] 2024-02-22T19:40:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:00.495854+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.53:0/1920165501' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "1.4", "id": [3, 5, 6]}]: dispatch 2024-02-22T19:40:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:00.870256+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? 172.21.15.53:0/1920165501' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.4", "id": [3, 5, 6]}]': finished 2024-02-22T19:40:01.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:00.870323+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e286: 8 total, 8 up, 6 in 2024-02-22T19:40:01.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:40:01.861 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:01.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:01.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:01.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:01.993 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:02.242 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:02.245 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:02.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:02.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:02.914 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:01.553322+0000 mgr.y (mgr.14535) 423 : cluster [DBG] pgmap v543: 148 pgs: 16 unknown, 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:02.914 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:01.871475+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e287: 8 total, 8 up, 6 in 2024-02-22T19:40:02.914 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:02.236684+0000 mon.a (mon.0) 1269 : cluster [DBG] osdmap e288: 8 total, 8 up, 6 in 2024-02-22T19:40:02.914 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:01.553322+0000 mgr.y (mgr.14535) 423 : cluster [DBG] pgmap v543: 148 pgs: 16 unknown, 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:02.915 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:01.871475+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e287: 8 total, 8 up, 6 in 2024-02-22T19:40:02.915 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:02.236684+0000 mon.a (mon.0) 1269 : cluster [DBG] osdmap e288: 8 total, 8 up, 6 in 2024-02-22T19:40:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:03.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:03.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:03.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:03.168 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:01.553322+0000 mgr.y (mgr.14535) 423 : cluster [DBG] pgmap v543: 148 pgs: 16 unknown, 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:01.871475+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e287: 8 total, 8 up, 6 in 2024-02-22T19:40:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:02.236684+0000 mon.a (mon.0) 1269 : cluster [DBG] osdmap e288: 8 total, 8 up, 6 in 2024-02-22T19:40:03.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-02-22T19:40:03.652 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:40:03.653 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:40:05.066 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:05.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:40:05.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:40:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:40:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:03.553681+0000 mgr.y (mgr.14535) 424 : cluster [DBG] pgmap v546: 148 pgs: 16 unknown, 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:04.840477+0000 mon.a (mon.0) 1270 : audit [DBG] from='client.? 172.21.15.53:0/307324167' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:40:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:40:05.155 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:03.553681+0000 mgr.y (mgr.14535) 424 : cluster [DBG] pgmap v546: 148 pgs: 16 unknown, 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:05.155 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:04.840477+0000 mon.a (mon.0) 1270 : audit [DBG] from='client.? 172.21.15.53:0/307324167' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:05.156 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:03.553681+0000 mgr.y (mgr.14535) 424 : cluster [DBG] pgmap v546: 148 pgs: 16 unknown, 1 active+remapped+backfill_toofull, 131 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:05.156 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:04.840477+0000 mon.a (mon.0) 1270 : audit [DBG] from='client.? 172.21.15.53:0/307324167' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.163 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.164 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.164 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.164 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.164 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:40:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:05.824 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:05.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:05.842 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:05.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:05.954 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:06.069 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:04.872032+0000 mgr.y (mgr.14535) 425 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:06.069 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:04.872032+0000 mgr.y (mgr.14535) 425 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:06.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:06.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:06.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:06.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:04.872032+0000 mgr.y (mgr.14535) 425 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:06.223 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:06.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:06.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:06.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:06.402 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:06.404 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:06.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:06.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:06.536 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:06.558 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:06.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:06.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:06.863 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:40:06.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:40:07.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:05.554417+0000 mgr.y (mgr.14535) 426 : cluster [DBG] pgmap v547: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 900 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:07.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:05.554417+0000 mgr.y (mgr.14535) 426 : cluster [DBG] pgmap v547: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 900 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:07.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:07.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:07.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:07.251 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:05.554417+0000 mgr.y (mgr.14535) 426 : cluster [DBG] pgmap v547: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 900 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:07.415 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:07.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:40:08.416 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:07.554893+0000 mgr.y (mgr.14535) 427 : cluster [DBG] pgmap v548: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 765 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:08.416 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:07.554893+0000 mgr.y (mgr.14535) 427 : cluster [DBG] pgmap v548: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 765 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:08.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:07.554893+0000 mgr.y (mgr.14535) 427 : cluster [DBG] pgmap v548: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 765 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:08.705 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:08.709 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:08.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:08.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:08.840 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:09.118 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:09.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:09.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:09.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:09.251 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-02-22T19:40:09.396 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:08.361221+0000 mon.a (mon.0) 1271 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:09.396 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:08.718280+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:09.396 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:08.718365+0000 osd.6 (osd.6) 131 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:08.899694+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.53:0/1807360596' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:08.900053+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:08.361221+0000 mon.a (mon.0) 1271 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:08.718280+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:08.718365+0000 osd.6 (osd.6) 131 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:08.899694+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.53:0/1807360596' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:08.900053+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:40:09.397 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:40:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:09] "GET /metrics HTTP/1.1" 200 176020 "" "Prometheus/2.33.4" 2024-02-22T19:40:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:08.361221+0000 mon.a (mon.0) 1271 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:08.718280+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:08.718365+0000 osd.6 (osd.6) 131 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:08.899694+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.53:0/1807360596' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:40:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:08.900053+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:40:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:10.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:10.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:10.305 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:10.436 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:10.438 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:09.367321+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:40:10.438 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:09.367408+0000 mon.a (mon.0) 1274 : cluster [DBG] osdmap e289: 8 total, 8 up, 6 in 2024-02-22T19:40:10.438 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:09.555383+0000 mgr.y (mgr.14535) 428 : cluster [DBG] pgmap v550: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:10.438 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:09.367321+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:40:10.438 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:09.367408+0000 mon.a (mon.0) 1274 : cluster [DBG] osdmap e289: 8 total, 8 up, 6 in 2024-02-22T19:40:10.438 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:09.555383+0000 mgr.y (mgr.14535) 428 : cluster [DBG] pgmap v550: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:10.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:10.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:10.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:10.573 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:10.597 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:10.600 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:10.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:10.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:09.367321+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:40:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:09.367408+0000 mon.a (mon.0) 1274 : cluster [DBG] osdmap e289: 8 total, 8 up, 6 in 2024-02-22T19:40:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:09.555383+0000 mgr.y (mgr.14535) 428 : cluster [DBG] pgmap v550: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:10.980 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:10.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:10.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:11.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:11.146 DEBUG:teuthology.orchestra.run.smithi053:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-02-22T19:40:11.146 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.log 2024-02-22T19:40:11.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:40:11.149 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-02-22T19:40:11.149 INFO:teuthology.orchestra.run:waiting for 3000 2024-02-22T19:40:11.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:hints = 1 2024-02-22T19:40:11.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-02-22T19:40:11.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object prefix: benchmark_data_smithi053_157429 2024-02-22T19:40:11.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:40:11.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 0 0 0 0 0 0 - 0 2024-02-22T19:40:11.213 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:11.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:11.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:11.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:11.388 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:11.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:09.586784+0000 osd.5 (osd.5) 78 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:11.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:09.586863+0000 osd.5 (osd.5) 79 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:11.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:10.373608+0000 mon.a (mon.0) 1275 : cluster [DBG] osdmap e290: 8 total, 8 up, 6 in 2024-02-22T19:40:11.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:09.586784+0000 osd.5 (osd.5) 78 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:11.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:09.586863+0000 osd.5 (osd.5) 79 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:11.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:10.373608+0000 mon.a (mon.0) 1275 : cluster [DBG] osdmap e290: 8 total, 8 up, 6 in 2024-02-22T19:40:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:11.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:11.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:11.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:11.743 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:11.778 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:11.783 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:11.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:11.793 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:40:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:40:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:40:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:40:11.794 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:40:11.828 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:09.586784+0000 osd.5 (osd.5) 78 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:09.586863+0000 osd.5 (osd.5) 79 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:10.373608+0000 mon.a (mon.0) 1275 : cluster [DBG] osdmap e290: 8 total, 8 up, 6 in 2024-02-22T19:40:11.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:11.929 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:12.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:12.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:12.172 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:12.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 1 16 3019 3003 187.675 187.688 0.00261198 0.00531511 2024-02-22T19:40:12.218 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:12.411 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:40:12.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:11.556962+0000 mgr.y (mgr.14535) 429 : cluster [DBG] pgmap v552: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:12.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:11.847596+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.53:0/2141587293' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:12.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:11.849178+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:12.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:11.556962+0000 mgr.y (mgr.14535) 429 : cluster [DBG] pgmap v552: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:12.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:11.847596+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.53:0/2141587293' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:12.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:11.849178+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:11.556962+0000 mgr.y (mgr.14535) 429 : cluster [DBG] pgmap v552: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:12.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:11.847596+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.53:0/2141587293' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:12.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:11.849178+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:13.092 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:13.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:13.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:13.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:40:13.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:13.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 2 16 6335 6319 197.454 207.25 0.00516994 0.00505268 2024-02-22T19:40:13.250 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:13.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:12.381612+0000 mon.a (mon.0) 1277 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:13.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:12.396923+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:40:13.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:12.396966+0000 mon.a (mon.0) 1279 : cluster [DBG] osdmap e291: 8 total, 8 up, 6 in 2024-02-22T19:40:13.398 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:12.381612+0000 mon.a (mon.0) 1277 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:13.398 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:12.396923+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:40:13.398 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:12.396966+0000 mon.a (mon.0) 1279 : cluster [DBG] osdmap e291: 8 total, 8 up, 6 in 2024-02-22T19:40:13.730 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:13.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:13.736 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:13.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:12.381612+0000 mon.a (mon.0) 1277 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:12.396923+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:40:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:12.396966+0000 mon.a (mon.0) 1279 : cluster [DBG] osdmap e291: 8 total, 8 up, 6 in 2024-02-22T19:40:13.877 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:13.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:13.908 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:14.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:14.059 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:14.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 3 15 9365 9350 194.775 189.438 0.00605749 0.00512795 2024-02-22T19:40:14.268 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:14.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:14.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:14.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:14.428 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:14.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:13.557435+0000 mgr.y (mgr.14535) 430 : cluster [DBG] pgmap v554: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:14.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:13.557435+0000 mgr.y (mgr.14535) 430 : cluster [DBG] pgmap v554: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:14.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:13.557435+0000 mgr.y (mgr.14535) 430 : cluster [DBG] pgmap v554: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:40:15.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:40:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:40:15.103 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:40:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:40:15.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 4 15 12325 12310 192.328 185 0.00335126 0.0051929 2024-02-22T19:40:15.327 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:15.331 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:15.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:15.408 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:40:15.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:15.440 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:40:15.471 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:40:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:15.501 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:40:15.505 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:15.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:15.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:15.534 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:40:15.570 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:40:15.603 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:40:15.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:15.637 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:40:15.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:14.882459+0000 mgr.y (mgr.14535) 431 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:15.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:15.373028+0000 mon.b (mon.1) 69 : audit [DBG] from='client.? 172.21.15.53:0/537710240' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:15.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:14.882459+0000 mgr.y (mgr.14535) 431 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:15.648 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:15.373028+0000 mon.b (mon.1) 69 : audit [DBG] from='client.? 172.21.15.53:0/537710240' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:40:15.649 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:14.882459+0000 mgr.y (mgr.14535) 431 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:15.373028+0000 mon.b (mon.1) 69 : audit [DBG] from='client.? 172.21.15.53:0/537710240' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:15.873 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:15.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:15.880 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:15.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:16.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:16.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:16.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:16.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 5 16 14972 14956 186.934 165.375 0.00898135 0.00534344 2024-02-22T19:40:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:16.230 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:16.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:16.234 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:16.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:16.372 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:16.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:16.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:16.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:16.557 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:16.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:15.558822+0000 mgr.y (mgr.14535) 432 : cluster [DBG] pgmap v555: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 623 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 826 B/s rd, 101 MiB/s wr, 1.61k op/s; 1/30468 objects misplaced (0.003%) 2024-02-22T19:40:16.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:15.558822+0000 mgr.y (mgr.14535) 432 : cluster [DBG] pgmap v555: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 623 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 826 B/s rd, 101 MiB/s wr, 1.61k op/s; 1/30468 objects misplaced (0.003%) 2024-02-22T19:40:16.819 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:16.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:16.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:15.558822+0000 mgr.y (mgr.14535) 432 : cluster [DBG] pgmap v555: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 623 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 826 B/s rd, 101 MiB/s wr, 1.61k op/s; 1/30468 objects misplaced (0.003%) 2024-02-22T19:40:16.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:16.973 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:17.007 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:17.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:17.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:17.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:17.199 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 6 16 17518 17502 182.297 159.125 0.00486486 0.00547726 2024-02-22T19:40:17.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:40:17.744 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:17.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:17.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:17.827 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:17.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:40:18.066 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:40:18.201 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 7 16 19789 19773 176.51 141.938 0.00749322 0.00565991 2024-02-22T19:40:18.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:17.247697+0000 mon.a (mon.0) 1280 : cluster [DBG] osdmap e292: 8 total, 8 up, 6 in 2024-02-22T19:40:18.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:17.559345+0000 mgr.y (mgr.14535) 433 : cluster [DBG] pgmap v557: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 623 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 712 B/s rd, 87 MiB/s wr, 1.39k op/s; 1/30468 objects misplaced (0.003%) 2024-02-22T19:40:18.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:18.035333+0000 mgr.y (mgr.14535) 434 : audit [DBG] from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:40:18.464 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:17.247697+0000 mon.a (mon.0) 1280 : cluster [DBG] osdmap e292: 8 total, 8 up, 6 in 2024-02-22T19:40:18.464 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:17.559345+0000 mgr.y (mgr.14535) 433 : cluster [DBG] pgmap v557: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 623 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 712 B/s rd, 87 MiB/s wr, 1.39k op/s; 1/30468 objects misplaced (0.003%) 2024-02-22T19:40:18.464 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:18.035333+0000 mgr.y (mgr.14535) 434 : audit [DBG] from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:40:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:17.247697+0000 mon.a (mon.0) 1280 : cluster [DBG] osdmap e292: 8 total, 8 up, 6 in 2024-02-22T19:40:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:17.559345+0000 mgr.y (mgr.14535) 433 : cluster [DBG] pgmap v557: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 623 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 712 B/s rd, 87 MiB/s wr, 1.39k op/s; 1/30468 objects misplaced (0.003%) 2024-02-22T19:40:18.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:18.035333+0000 mgr.y (mgr.14535) 434 : audit [DBG] from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:40:18.759 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:18.763 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:18.763 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:18.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:40:18.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:18.910 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:18.941 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:18.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:18.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:19.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:19.140 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:19.200 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 8 15 22105 22090 172.543 144.812 0.00699111 0.00578913 2024-02-22T19:40:19.345 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:19.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:19.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:19.460 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:19.490 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:40:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:19] "GET /metrics HTTP/1.1" 200 176020 "" "Prometheus/2.33.4" 2024-02-22T19:40:19.507 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:19.842 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-02-22T19:40:17.559146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653006500,"num_objects":10156,"num_object_clones":0,"num_object_copies":30468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":1782,"num_read_kb":1534,"num_write":10678,"num_write_kb":637737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13666,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10770,"ondisk_log_size":10770,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":3425204,"kb_used_data":1580356,"kb_used_omap":0,"kb_used_meta":1844736,"kb_avail":559045708,"statfs":{"total":575970213888,"available":572462804992,"internally_reserved":0,"allocated":1618284544,"data_stored":1613469832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1889009664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":653000704,"num_objects":9964,"num_object_clones":0,"num_object_copies":29892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9964,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9964,"num_write_kb":637696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.189069"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":243,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747019+0000","last_change":"2024-02-22T19:38:47.158144+0000","last_active":"2024-02-22T19:40:10.747019+0000","last_peered":"2024-02-22T19:40:10.747019+0000","last_clean":"2024-02-22T19:40:10.747019+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:40:10.747019+0000","last_undegraded":"2024-02-22T19:40:10.747019+0000","last_fullsized":"2024-02-22T19:40:10.747019+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:14.806069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:14.806069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":269,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747016+0000","last_change":"2024-02-22T19:39:22.268923+0000","last_active":"2024-02-22T19:40:10.747016+0000","last_peered":"2024-02-22T19:40:10.747016+0000","last_clean":"2024-02-22T19:40:10.747016+0000","last_became_active":"2024-02-22T19:39:22.268587+0000","last_became_peered":"2024-02-22T19:39:22.268587+0000","last_unstale":"2024-02-22T19:40:10.747016+0000","last_undegraded":"2024-02-22T19:40:10.747016+0000","last_fullsized":"2024-02-22T19:40:10.747016+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":106,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745612+0000","last_change":"2024-02-22T19:39:36.528260+0000","last_active":"2024-02-22T19:40:10.745612+0000","last_peered":"2024-02-22T19:40:10.745612+0000","last_clean":"2024-02-22T19:40:10.745612+0000","last_became_active":"2024-02-22T19:38:47.164599+0000","last_became_peered":"2024-02-22T19:38:47.164599+0000","last_unstale":"2024-02-22T19:40:10.745612+0000","last_undegraded":"2024-02-22T19:40:10.745612+0000","last_fullsized":"2024-02-22T19:40:10.745612+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:39:36.528175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.528175+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":249,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651774+0000","last_change":"2024-02-22T19:36:59.298132+0000","last_active":"2024-02-22T19:40:10.651774+0000","last_peered":"2024-02-22T19:40:10.651774+0000","last_clean":"2024-02-22T19:40:10.651774+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:40:10.651774+0000","last_undegraded":"2024-02-22T19:40:10.651774+0000","last_fullsized":"2024-02-22T19:40:10.651774+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:59.298021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:36:59.298021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":250,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651878+0000","last_change":"2024-02-22T19:35:33.275524+0000","last_active":"2024-02-22T19:40:10.651878+0000","last_peered":"2024-02-22T19:40:10.651878+0000","last_clean":"2024-02-22T19:40:10.651878+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:40:10.651878+0000","last_undegraded":"2024-02-22T19:40:10.651878+0000","last_fullsized":"2024-02-22T19:40:10.651878+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.275417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.275417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":371,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651558+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:40:10.651558+0000","last_peered":"2024-02-22T19:40:10.651558+0000","last_clean":"2024-02-22T19:40:10.651558+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:40:10.651558+0000","last_undegraded":"2024-02-22T19:40:10.651558+0000","last_fullsized":"2024-02-22T19:40:10.651558+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":87,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.371436+0000","last_change":"2024-02-22T19:39:36.296064+0000","last_active":"2024-02-22T19:40:10.371436+0000","last_peered":"2024-02-22T19:40:10.371436+0000","last_clean":"2024-02-22T19:40:10.371436+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:40:10.371436+0000","last_undegraded":"2024-02-22T19:40:10.371436+0000","last_fullsized":"2024-02-22T19:40:10.371436+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:36.295988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.295988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":92,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747754+0000","last_change":"2024-02-22T19:38:47.169310+0000","last_active":"2024-02-22T19:40:10.747754+0000","last_peered":"2024-02-22T19:40:10.747754+0000","last_clean":"2024-02-22T19:40:10.747754+0000","last_became_active":"2024-02-22T19:38:47.168764+0000","last_became_peered":"2024-02-22T19:38:47.168764+0000","last_unstale":"2024-02-22T19:40:10.747754+0000","last_undegraded":"2024-02-22T19:40:10.747754+0000","last_fullsized":"2024-02-22T19:40:10.747754+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"264'15","reported_seq":140,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651507+0000","last_change":"2024-02-22T19:39:22.646901+0000","last_active":"2024-02-22T19:40:10.651507+0000","last_peered":"2024-02-22T19:40:10.651507+0000","last_clean":"2024-02-22T19:40:10.651507+0000","last_became_active":"2024-02-22T19:39:22.269400+0000","last_became_peered":"2024-02-22T19:39:22.269400+0000","last_unstale":"2024-02-22T19:40:10.651507+0000","last_undegraded":"2024-02-22T19:40:10.651507+0000","last_fullsized":"2024-02-22T19:40:10.651507+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:36:21.735513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:21.735513+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":83,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746075+0000","last_change":"2024-02-22T19:37:57.186178+0000","last_active":"2024-02-22T19:40:10.746075+0000","last_peered":"2024-02-22T19:40:10.746075+0000","last_clean":"2024-02-22T19:40:10.746075+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:40:10.746075+0000","last_undegraded":"2024-02-22T19:40:10.746075+0000","last_fullsized":"2024-02-22T19:40:10.746075+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:08.908571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:36:08.908571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":105,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746339+0000","last_change":"2024-02-22T19:39:53.438381+0000","last_active":"2024-02-22T19:40:10.746339+0000","last_peered":"2024-02-22T19:40:10.746339+0000","last_clean":"2024-02-22T19:40:10.746339+0000","last_became_active":"2024-02-22T19:38:47.179485+0000","last_became_peered":"2024-02-22T19:38:47.179485+0000","last_unstale":"2024-02-22T19:40:10.746339+0000","last_undegraded":"2024-02-22T19:40:10.746339+0000","last_fullsized":"2024-02-22T19:40:10.746339+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":91,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651627+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:40:10.651627+0000","last_peered":"2024-02-22T19:40:10.651627+0000","last_clean":"2024-02-22T19:40:10.651627+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:40:10.651627+0000","last_undegraded":"2024-02-22T19:40:10.651627+0000","last_fullsized":"2024-02-22T19:40:10.651627+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":244,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746678+0000","last_change":"2024-02-22T19:39:34.555380+0000","last_active":"2024-02-22T19:40:10.746678+0000","last_peered":"2024-02-22T19:40:10.746678+0000","last_clean":"2024-02-22T19:40:10.746678+0000","last_became_active":"2024-02-22T19:38:47.168230+0000","last_became_peered":"2024-02-22T19:38:47.168230+0000","last_unstale":"2024-02-22T19:40:10.746678+0000","last_undegraded":"2024-02-22T19:40:10.746678+0000","last_fullsized":"2024-02-22T19:40:10.746678+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:34.555298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.555298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":60,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593048+0000","last_change":"2024-02-22T19:39:22.245523+0000","last_active":"2024-02-22T19:40:10.593048+0000","last_peered":"2024-02-22T19:40:10.593048+0000","last_clean":"2024-02-22T19:40:10.593048+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:40:10.593048+0000","last_undegraded":"2024-02-22T19:40:10.593048+0000","last_fullsized":"2024-02-22T19:40:10.593048+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.732856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.732856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":313,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.375094+0000","last_change":"2024-02-22T19:39:36.387949+0000","last_active":"2024-02-22T19:40:10.375094+0000","last_peered":"2024-02-22T19:40:10.375094+0000","last_clean":"2024-02-22T19:40:10.375094+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:40:10.375094+0000","last_undegraded":"2024-02-22T19:40:10.375094+0000","last_fullsized":"2024-02-22T19:40:10.375094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:39:36.387857+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.387857+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":267,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.652225+0000","last_change":"2024-02-22T19:37:57.180410+0000","last_active":"2024-02-22T19:40:10.652225+0000","last_peered":"2024-02-22T19:40:10.652225+0000","last_clean":"2024-02-22T19:40:10.652225+0000","last_became_active":"2024-02-22T19:37:57.180089+0000","last_became_peered":"2024-02-22T19:37:57.180089+0000","last_unstale":"2024-02-22T19:40:10.652225+0000","last_undegraded":"2024-02-22T19:40:10.652225+0000","last_fullsized":"2024-02-22T19:40:10.652225+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":73,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372044+0000","last_change":"2024-02-22T19:39:22.252423+0000","last_active":"2024-02-22T19:40:10.372044+0000","last_peered":"2024-02-22T19:40:10.372044+0000","last_clean":"2024-02-22T19:40:10.372044+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:40:10.372044+0000","last_undegraded":"2024-02-22T19:40:10.372044+0000","last_fullsized":"2024-02-22T19:40:10.372044+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.583926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.583926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":306,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651959+0000","last_change":"2024-02-22T19:38:48.219376+0000","last_active":"2024-02-22T19:40:10.651959+0000","last_peered":"2024-02-22T19:40:10.651959+0000","last_clean":"2024-02-22T19:40:10.651959+0000","last_became_active":"2024-02-22T19:38:47.180453+0000","last_became_peered":"2024-02-22T19:38:47.180453+0000","last_unstale":"2024-02-22T19:40:10.651959+0000","last_undegraded":"2024-02-22T19:40:10.651959+0000","last_fullsized":"2024-02-22T19:40:10.651959+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'17","last_scrub_stamp":"2024-02-22T19:35:36.338717+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.338717+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":297,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593262+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:40:10.593262+0000","last_peered":"2024-02-22T19:40:10.593262+0000","last_clean":"2024-02-22T19:40:10.593262+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:40:10.593262+0000","last_undegraded":"2024-02-22T19:40:10.593262+0000","last_fullsized":"2024-02-22T19:40:10.593262+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":272,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.371830+0000","last_change":"2024-02-22T19:38:47.168587+0000","last_active":"2024-02-22T19:40:10.371830+0000","last_peered":"2024-02-22T19:40:10.371830+0000","last_clean":"2024-02-22T19:40:10.371830+0000","last_became_active":"2024-02-22T19:38:47.168149+0000","last_became_peered":"2024-02-22T19:38:47.168149+0000","last_unstale":"2024-02-22T19:40:10.371830+0000","last_undegraded":"2024-02-22T19:40:10.371830+0000","last_fullsized":"2024-02-22T19:40:10.371830+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:50.144179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:50.144179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":98,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593546+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:40:10.593546+0000","last_peered":"2024-02-22T19:40:10.593546+0000","last_clean":"2024-02-22T19:40:10.593546+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:40:10.593546+0000","last_undegraded":"2024-02-22T19:40:10.593546+0000","last_fullsized":"2024-02-22T19:40:10.593546+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":281,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.373265+0000","last_change":"2024-02-22T19:38:47.150417+0000","last_active":"2024-02-22T19:40:10.373265+0000","last_peered":"2024-02-22T19:40:10.373265+0000","last_clean":"2024-02-22T19:40:10.373265+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:40:10.373265+0000","last_undegraded":"2024-02-22T19:40:10.373265+0000","last_fullsized":"2024-02-22T19:40:10.373265+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.995002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.995002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":284,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651178+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:40:10.651178+0000","last_peered":"2024-02-22T19:40:10.651178+0000","last_clean":"2024-02-22T19:40:10.651178+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:40:10.651178+0000","last_undegraded":"2024-02-22T19:40:10.651178+0000","last_fullsized":"2024-02-22T19:40:10.651178+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":362,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746149+0000","last_change":"2024-02-22T19:37:57.212060+0000","last_active":"2024-02-22T19:40:10.746149+0000","last_peered":"2024-02-22T19:40:10.746149+0000","last_clean":"2024-02-22T19:40:10.746149+0000","last_became_active":"2024-02-22T19:37:57.190476+0000","last_became_peered":"2024-02-22T19:37:57.190476+0000","last_unstale":"2024-02-22T19:40:10.746149+0000","last_undegraded":"2024-02-22T19:40:10.746149+0000","last_fullsized":"2024-02-22T19:40:10.746149+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:52.432675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:52.432675+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":266,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592768+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:40:10.592768+0000","last_peered":"2024-02-22T19:40:10.592768+0000","last_clean":"2024-02-22T19:40:10.592768+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:40:10.592768+0000","last_undegraded":"2024-02-22T19:40:10.592768+0000","last_fullsized":"2024-02-22T19:40:10.592768+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":282,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746443+0000","last_change":"2024-02-22T19:39:22.256927+0000","last_active":"2024-02-22T19:40:10.746443+0000","last_peered":"2024-02-22T19:40:10.746443+0000","last_clean":"2024-02-22T19:40:10.746443+0000","last_became_active":"2024-02-22T19:39:22.256576+0000","last_became_peered":"2024-02-22T19:39:22.256576+0000","last_unstale":"2024-02-22T19:40:10.746443+0000","last_undegraded":"2024-02-22T19:40:10.746443+0000","last_fullsized":"2024-02-22T19:40:10.746443+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.400534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.400534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":269,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746926+0000","last_change":"2024-02-22T19:39:22.246647+0000","last_active":"2024-02-22T19:40:10.746926+0000","last_peered":"2024-02-22T19:40:10.746926+0000","last_clean":"2024-02-22T19:40:10.746926+0000","last_became_active":"2024-02-22T19:39:22.246323+0000","last_became_peered":"2024-02-22T19:39:22.246323+0000","last_unstale":"2024-02-22T19:40:10.746926+0000","last_undegraded":"2024-02-22T19:40:10.746926+0000","last_fullsized":"2024-02-22T19:40:10.746926+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":265,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651271+0000","last_change":"2024-02-22T19:36:58.263929+0000","last_active":"2024-02-22T19:40:10.651271+0000","last_peered":"2024-02-22T19:40:10.651271+0000","last_clean":"2024-02-22T19:40:10.651271+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:40:10.651271+0000","last_undegraded":"2024-02-22T19:40:10.651271+0000","last_fullsized":"2024-02-22T19:40:10.651271+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:36:58.263834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.263834+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":278,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651866+0000","last_change":"2024-02-22T19:39:22.250184+0000","last_active":"2024-02-22T19:40:10.651866+0000","last_peered":"2024-02-22T19:40:10.651866+0000","last_clean":"2024-02-22T19:40:10.651866+0000","last_became_active":"2024-02-22T19:39:22.249890+0000","last_became_peered":"2024-02-22T19:39:22.249890+0000","last_unstale":"2024-02-22T19:40:10.651866+0000","last_undegraded":"2024-02-22T19:40:10.651866+0000","last_fullsized":"2024-02-22T19:40:10.651866+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":284,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592873+0000","last_change":"2024-02-22T19:39:22.253288+0000","last_active":"2024-02-22T19:40:10.592873+0000","last_peered":"2024-02-22T19:40:10.592873+0000","last_clean":"2024-02-22T19:40:10.592873+0000","last_became_active":"2024-02-22T19:39:22.252705+0000","last_became_peered":"2024-02-22T19:39:22.252705+0000","last_unstale":"2024-02-22T19:40:10.592873+0000","last_undegraded":"2024-02-22T19:40:10.592873+0000","last_fullsized":"2024-02-22T19:40:10.592873+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:54.418958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:37:54.418958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":83,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746494+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:40:10.746494+0000","last_peered":"2024-02-22T19:40:10.746494+0000","last_clean":"2024-02-22T19:40:10.746494+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:40:10.746494+0000","last_undegraded":"2024-02-22T19:40:10.746494+0000","last_fullsized":"2024-02-22T19:40:10.746494+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":284,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651461+0000","last_change":"2024-02-22T19:39:37.816765+0000","last_active":"2024-02-22T19:40:10.651461+0000","last_peered":"2024-02-22T19:40:10.651461+0000","last_clean":"2024-02-22T19:40:10.651461+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:40:10.651461+0000","last_undegraded":"2024-02-22T19:40:10.651461+0000","last_fullsized":"2024-02-22T19:40:10.651461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":260,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745614+0000","last_change":"2024-02-22T19:39:22.247453+0000","last_active":"2024-02-22T19:40:10.745614+0000","last_peered":"2024-02-22T19:40:10.745614+0000","last_clean":"2024-02-22T19:40:10.745614+0000","last_became_active":"2024-02-22T19:39:22.247266+0000","last_became_peered":"2024-02-22T19:39:22.247266+0000","last_unstale":"2024-02-22T19:40:10.745614+0000","last_undegraded":"2024-02-22T19:40:10.745614+0000","last_fullsized":"2024-02-22T19:40:10.745614+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":278,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651725+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:40:10.651725+0000","last_peered":"2024-02-22T19:40:10.651725+0000","last_clean":"2024-02-22T19:40:10.651725+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:40:10.651725+0000","last_undegraded":"2024-02-22T19:40:10.651725+0000","last_fullsized":"2024-02-22T19:40:10.651725+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":292,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593672+0000","last_change":"2024-02-22T19:39:22.254738+0000","last_active":"2024-02-22T19:40:10.593672+0000","last_peered":"2024-02-22T19:40:10.593672+0000","last_clean":"2024-02-22T19:40:10.593672+0000","last_became_active":"2024-02-22T19:39:22.254411+0000","last_became_peered":"2024-02-22T19:39:22.254411+0000","last_unstale":"2024-02-22T19:40:10.593672+0000","last_undegraded":"2024-02-22T19:40:10.593672+0000","last_fullsized":"2024-02-22T19:40:10.593672+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:52.463774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.463774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":286,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372382+0000","last_change":"2024-02-22T19:39:44.297110+0000","last_active":"2024-02-22T19:40:10.372382+0000","last_peered":"2024-02-22T19:40:10.372382+0000","last_clean":"2024-02-22T19:40:10.372382+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:40:10.372382+0000","last_undegraded":"2024-02-22T19:40:10.372382+0000","last_fullsized":"2024-02-22T19:40:10.372382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:39:44.296975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:44.296975+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":294,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651259+0000","last_change":"2024-02-22T19:39:22.270145+0000","last_active":"2024-02-22T19:40:10.651259+0000","last_peered":"2024-02-22T19:40:10.651259+0000","last_clean":"2024-02-22T19:40:10.651259+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:40:10.651259+0000","last_undegraded":"2024-02-22T19:40:10.651259+0000","last_fullsized":"2024-02-22T19:40:10.651259+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:19.229816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:19.229816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":272,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372451+0000","last_change":"2024-02-22T19:37:57.187682+0000","last_active":"2024-02-22T19:40:10.372451+0000","last_peered":"2024-02-22T19:40:10.372451+0000","last_clean":"2024-02-22T19:40:10.372451+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:40:10.372451+0000","last_undegraded":"2024-02-22T19:40:10.372451+0000","last_fullsized":"2024-02-22T19:40:10.372451+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:47.993328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:47.993328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":55,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592850+0000","last_change":"2024-02-22T19:38:47.162281+0000","last_active":"2024-02-22T19:40:10.592850+0000","last_peered":"2024-02-22T19:40:10.592850+0000","last_clean":"2024-02-22T19:40:10.592850+0000","last_became_active":"2024-02-22T19:38:47.161791+0000","last_became_peered":"2024-02-22T19:38:47.161791+0000","last_unstale":"2024-02-22T19:40:10.592850+0000","last_undegraded":"2024-02-22T19:40:10.592850+0000","last_fullsized":"2024-02-22T19:40:10.592850+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.685418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.685418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":109,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747059+0000","last_change":"2024-02-22T19:38:47.783561+0000","last_active":"2024-02-22T19:40:10.747059+0000","last_peered":"2024-02-22T19:40:10.747059+0000","last_clean":"2024-02-22T19:40:10.747059+0000","last_became_active":"2024-02-22T19:38:47.168328+0000","last_became_peered":"2024-02-22T19:38:47.168328+0000","last_unstale":"2024-02-22T19:40:10.747059+0000","last_undegraded":"2024-02-22T19:40:10.747059+0000","last_fullsized":"2024-02-22T19:40:10.747059+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:53.461286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.461286+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":86,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593197+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:40:10.593197+0000","last_peered":"2024-02-22T19:40:10.593197+0000","last_clean":"2024-02-22T19:40:10.593197+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:40:10.593197+0000","last_undegraded":"2024-02-22T19:40:10.593197+0000","last_fullsized":"2024-02-22T19:40:10.593197+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":157,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.403998+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:40:12.403998+0000","last_peered":"2024-02-22T19:40:12.403998+0000","last_clean":"2024-02-22T19:40:12.403998+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:40:12.403998+0000","last_undegraded":"2024-02-22T19:40:12.403998+0000","last_fullsized":"2024-02-22T19:40:12.403998+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":265,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593353+0000","last_change":"2024-02-22T19:38:47.162564+0000","last_active":"2024-02-22T19:40:10.593353+0000","last_peered":"2024-02-22T19:40:10.593353+0000","last_clean":"2024-02-22T19:40:10.593353+0000","last_became_active":"2024-02-22T19:38:47.162182+0000","last_became_peered":"2024-02-22T19:38:47.162182+0000","last_unstale":"2024-02-22T19:40:10.593353+0000","last_undegraded":"2024-02-22T19:40:10.593353+0000","last_fullsized":"2024-02-22T19:40:10.593353+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:56.463502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:36:56.463502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":288,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651760+0000","last_change":"2024-02-22T19:37:09.345416+0000","last_active":"2024-02-22T19:40:10.651760+0000","last_peered":"2024-02-22T19:40:10.651760+0000","last_clean":"2024-02-22T19:40:10.651760+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:40:10.651760+0000","last_undegraded":"2024-02-22T19:40:10.651760+0000","last_fullsized":"2024-02-22T19:40:10.651760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:37:09.345319+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.345319+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":266,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.375035+0000","last_change":"2024-02-22T19:39:41.365968+0000","last_active":"2024-02-22T19:40:10.375035+0000","last_peered":"2024-02-22T19:40:10.375035+0000","last_clean":"2024-02-22T19:40:10.375035+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:40:10.375035+0000","last_undegraded":"2024-02-22T19:40:10.375035+0000","last_fullsized":"2024-02-22T19:40:10.375035+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:41.365877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.365877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":389,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.405197+0000","last_change":"2024-02-22T19:35:38.331498+0000","last_active":"2024-02-22T19:40:12.405197+0000","last_peered":"2024-02-22T19:40:12.405197+0000","last_clean":"2024-02-22T19:40:12.405197+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:40:12.405197+0000","last_undegraded":"2024-02-22T19:40:12.405197+0000","last_fullsized":"2024-02-22T19:40:12.405197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:38.331392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:38.331392+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":72,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592496+0000","last_change":"2024-02-22T19:39:22.246092+0000","last_active":"2024-02-22T19:40:10.592496+0000","last_peered":"2024-02-22T19:40:10.592496+0000","last_clean":"2024-02-22T19:40:10.592496+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:40:10.592496+0000","last_undegraded":"2024-02-22T19:40:10.592496+0000","last_fullsized":"2024-02-22T19:40:10.592496+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":102,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747823+0000","last_change":"2024-02-22T19:37:57.198915+0000","last_active":"2024-02-22T19:40:10.747823+0000","last_peered":"2024-02-22T19:40:10.747823+0000","last_clean":"2024-02-22T19:40:10.747823+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:40:10.747823+0000","last_undegraded":"2024-02-22T19:40:10.747823+0000","last_fullsized":"2024-02-22T19:40:10.747823+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":289,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.375012+0000","last_change":"2024-02-22T19:39:53.126573+0000","last_active":"2024-02-22T19:40:10.375012+0000","last_peered":"2024-02-22T19:40:10.375012+0000","last_clean":"2024-02-22T19:40:10.375012+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:40:10.375012+0000","last_undegraded":"2024-02-22T19:40:10.375012+0000","last_fullsized":"2024-02-22T19:40:10.375012+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:53.126490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.126490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":288,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592719+0000","last_change":"2024-02-22T19:39:22.255233+0000","last_active":"2024-02-22T19:40:10.592719+0000","last_peered":"2024-02-22T19:40:10.592719+0000","last_clean":"2024-02-22T19:40:10.592719+0000","last_became_active":"2024-02-22T19:39:22.254888+0000","last_became_peered":"2024-02-22T19:39:22.254888+0000","last_unstale":"2024-02-22T19:40:10.592719+0000","last_undegraded":"2024-02-22T19:40:10.592719+0000","last_fullsized":"2024-02-22T19:40:10.592719+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.470149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.470149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":74,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.652025+0000","last_change":"2024-02-22T19:39:23.200612+0000","last_active":"2024-02-22T19:40:10.652025+0000","last_peered":"2024-02-22T19:40:10.652025+0000","last_clean":"2024-02-22T19:40:10.652025+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:40:10.652025+0000","last_undegraded":"2024-02-22T19:40:10.652025+0000","last_fullsized":"2024-02-22T19:40:10.652025+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-22T19:35:27.254984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:35:27.254984+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":264,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592977+0000","last_change":"2024-02-22T19:38:47.162050+0000","last_active":"2024-02-22T19:40:10.592977+0000","last_peered":"2024-02-22T19:40:10.592977+0000","last_clean":"2024-02-22T19:40:10.592977+0000","last_became_active":"2024-02-22T19:38:47.161672+0000","last_became_peered":"2024-02-22T19:38:47.161672+0000","last_unstale":"2024-02-22T19:40:10.592977+0000","last_undegraded":"2024-02-22T19:40:10.592977+0000","last_fullsized":"2024-02-22T19:40:10.592977+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":311,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745328+0000","last_change":"2024-02-22T19:39:54.559858+0000","last_active":"2024-02-22T19:40:10.745328+0000","last_peered":"2024-02-22T19:40:10.745328+0000","last_clean":"2024-02-22T19:40:10.745328+0000","last_became_active":"2024-02-22T19:39:22.253387+0000","last_became_peered":"2024-02-22T19:39:22.253387+0000","last_unstale":"2024-02-22T19:40:10.745328+0000","last_undegraded":"2024-02-22T19:40:10.745328+0000","last_fullsized":"2024-02-22T19:40:10.745328+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":288,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.650864+0000","last_change":"2024-02-22T19:36:26.647888+0000","last_active":"2024-02-22T19:40:10.650864+0000","last_peered":"2024-02-22T19:40:10.650864+0000","last_clean":"2024-02-22T19:40:10.650864+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:40:10.650864+0000","last_undegraded":"2024-02-22T19:40:10.650864+0000","last_fullsized":"2024-02-22T19:40:10.650864+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-02-22T19:36:25.296484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.296484+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"291'607","reported_seq":621,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.348335+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:40:14.348335+0000","last_peered":"2024-02-22T19:40:14.348335+0000","last_clean":"2024-02-22T19:40:14.348335+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:40:14.348335+0000","last_undegraded":"2024-02-22T19:40:14.348335+0000","last_fullsized":"2024-02-22T19:40:14.348335+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":607,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39780352,"num_objects":607,"num_object_clones":0,"num_object_copies":1821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":38848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":110,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746958+0000","last_change":"2024-02-22T19:39:22.272688+0000","last_active":"2024-02-22T19:40:10.746958+0000","last_peered":"2024-02-22T19:40:10.746958+0000","last_clean":"2024-02-22T19:40:10.746958+0000","last_became_active":"2024-02-22T19:39:22.272214+0000","last_became_peered":"2024-02-22T19:39:22.272214+0000","last_unstale":"2024-02-22T19:40:10.746958+0000","last_undegraded":"2024-02-22T19:40:10.746958+0000","last_fullsized":"2024-02-22T19:40:10.746958+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.151294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.151294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":336,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593465+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:40:10.593465+0000","last_peered":"2024-02-22T19:40:10.593465+0000","last_clean":"2024-02-22T19:40:10.593465+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:40:10.593465+0000","last_undegraded":"2024-02-22T19:40:10.593465+0000","last_fullsized":"2024-02-22T19:40:10.593465+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":117,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745343+0000","last_change":"2024-02-22T19:37:58.397461+0000","last_active":"2024-02-22T19:40:10.745343+0000","last_peered":"2024-02-22T19:40:10.745343+0000","last_clean":"2024-02-22T19:40:10.745343+0000","last_became_active":"2024-02-22T19:37:57.181199+0000","last_became_peered":"2024-02-22T19:37:57.181199+0000","last_unstale":"2024-02-22T19:40:10.745343+0000","last_undegraded":"2024-02-22T19:40:10.745343+0000","last_fullsized":"2024-02-22T19:40:10.745343+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-22T19:36:09.623521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:09.623521+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"251'13","reported_seq":441,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.399765+0000","last_change":"2024-02-22T19:38:50.149948+0000","last_active":"2024-02-22T19:40:12.399765+0000","last_peered":"2024-02-22T19:40:12.399765+0000","last_clean":"2024-02-22T19:40:12.399765+0000","last_became_active":"2024-02-22T19:38:50.148285+0000","last_became_peered":"2024-02-22T19:38:50.148285+0000","last_unstale":"2024-02-22T19:40:12.399765+0000","last_undegraded":"2024-02-22T19:40:12.399765+0000","last_fullsized":"2024-02-22T19:40:12.399765+0000","mapping_epoch":250,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:36:53.503128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.503128+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"291'654","reported_seq":668,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.440423+0000","last_change":"2024-02-22T19:40:00.882365+0000","last_active":"2024-02-22T19:40:14.440423+0000","last_peered":"2024-02-22T19:40:14.440423+0000","last_clean":"2024-02-22T19:40:14.440423+0000","last_became_active":"2024-02-22T19:40:00.882035+0000","last_became_peered":"2024-02-22T19:40:00.882035+0000","last_unstale":"2024-02-22T19:40:14.440423+0000","last_undegraded":"2024-02-22T19:40:14.440423+0000","last_fullsized":"2024-02-22T19:40:14.440423+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":654,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42860544,"num_objects":654,"num_object_clones":0,"num_object_copies":1962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":654,"num_write_kb":41856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":244,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651830+0000","last_change":"2024-02-22T19:36:11.224751+0000","last_active":"2024-02-22T19:40:10.651830+0000","last_peered":"2024-02-22T19:40:10.651830+0000","last_clean":"2024-02-22T19:40:10.651830+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:40:10.651830+0000","last_undegraded":"2024-02-22T19:40:10.651830+0000","last_fullsized":"2024-02-22T19:40:10.651830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.224646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.224646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":300,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593448+0000","last_change":"2024-02-22T19:39:22.248740+0000","last_active":"2024-02-22T19:40:10.593448+0000","last_peered":"2024-02-22T19:40:10.593448+0000","last_clean":"2024-02-22T19:40:10.593448+0000","last_became_active":"2024-02-22T19:39:22.248120+0000","last_became_peered":"2024-02-22T19:39:22.248120+0000","last_unstale":"2024-02-22T19:40:10.593448+0000","last_undegraded":"2024-02-22T19:40:10.593448+0000","last_fullsized":"2024-02-22T19:40:10.593448+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.431527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.431527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":111,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.371518+0000","last_change":"2024-02-22T19:38:47.208139+0000","last_active":"2024-02-22T19:40:10.371518+0000","last_peered":"2024-02-22T19:40:10.371518+0000","last_clean":"2024-02-22T19:40:10.371518+0000","last_became_active":"2024-02-22T19:38:47.167662+0000","last_became_peered":"2024-02-22T19:38:47.167662+0000","last_unstale":"2024-02-22T19:40:10.371518+0000","last_undegraded":"2024-02-22T19:40:10.371518+0000","last_fullsized":"2024-02-22T19:40:10.371518+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:05.645703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.645703+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":304,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374146+0000","last_change":"2024-02-22T19:39:48.227541+0000","last_active":"2024-02-22T19:40:10.374146+0000","last_peered":"2024-02-22T19:40:10.374146+0000","last_clean":"2024-02-22T19:40:10.374146+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:40:10.374146+0000","last_undegraded":"2024-02-22T19:40:10.374146+0000","last_fullsized":"2024-02-22T19:40:10.374146+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.227439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.227439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"291'622","reported_seq":636,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.348360+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:40:14.348360+0000","last_peered":"2024-02-22T19:40:14.348360+0000","last_clean":"2024-02-22T19:40:14.348360+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:40:14.348360+0000","last_undegraded":"2024-02-22T19:40:14.348360+0000","last_fullsized":"2024-02-22T19:40:14.348360+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":622,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40763392,"num_objects":622,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":622,"num_write_kb":39808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":250,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592206+0000","last_change":"2024-02-22T19:36:54.489612+0000","last_active":"2024-02-22T19:40:10.592206+0000","last_peered":"2024-02-22T19:40:10.592206+0000","last_clean":"2024-02-22T19:40:10.592206+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:40:10.592206+0000","last_undegraded":"2024-02-22T19:40:10.592206+0000","last_fullsized":"2024-02-22T19:40:10.592206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.489509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.489509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":330,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374648+0000","last_change":"2024-02-22T19:39:39.403851+0000","last_active":"2024-02-22T19:40:10.374648+0000","last_peered":"2024-02-22T19:40:10.374648+0000","last_clean":"2024-02-22T19:40:10.374648+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:40:10.374648+0000","last_undegraded":"2024-02-22T19:40:10.374648+0000","last_fullsized":"2024-02-22T19:40:10.374648+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:39.403775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:39:39.403775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":279,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593352+0000","last_change":"2024-02-22T19:37:57.185054+0000","last_active":"2024-02-22T19:40:10.593352+0000","last_peered":"2024-02-22T19:40:10.593352+0000","last_clean":"2024-02-22T19:40:10.593352+0000","last_became_active":"2024-02-22T19:37:57.184627+0000","last_became_peered":"2024-02-22T19:37:57.184627+0000","last_unstale":"2024-02-22T19:40:10.593352+0000","last_undegraded":"2024-02-22T19:40:10.593352+0000","last_fullsized":"2024-02-22T19:40:10.593352+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.438112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.438112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":312,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746122+0000","last_change":"2024-02-22T19:37:18.374920+0000","last_active":"2024-02-22T19:40:10.746122+0000","last_peered":"2024-02-22T19:40:10.746122+0000","last_clean":"2024-02-22T19:40:10.746122+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:40:10.746122+0000","last_undegraded":"2024-02-22T19:40:10.746122+0000","last_fullsized":"2024-02-22T19:40:10.746122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-22T19:37:18.374807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.374807+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.6","version":"291'621","reported_seq":635,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.330574+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:40:14.330574+0000","last_peered":"2024-02-22T19:40:14.330574+0000","last_clean":"2024-02-22T19:40:14.330574+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:40:14.330574+0000","last_undegraded":"2024-02-22T19:40:14.330574+0000","last_fullsized":"2024-02-22T19:40:14.330574+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":621,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40697856,"num_objects":621,"num_object_clones":0,"num_object_copies":1863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":621,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":72,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592552+0000","last_change":"2024-02-22T19:39:22.254050+0000","last_active":"2024-02-22T19:40:10.592552+0000","last_peered":"2024-02-22T19:40:10.592552+0000","last_clean":"2024-02-22T19:40:10.592552+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:40:10.592552+0000","last_undegraded":"2024-02-22T19:40:10.592552+0000","last_fullsized":"2024-02-22T19:40:10.592552+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:17.690298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:17.690298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":309,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374299+0000","last_change":"2024-02-22T19:39:54.127947+0000","last_active":"2024-02-22T19:40:10.374299+0000","last_peered":"2024-02-22T19:40:10.374299+0000","last_clean":"2024-02-22T19:40:10.374299+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:40:10.374299+0000","last_undegraded":"2024-02-22T19:40:10.374299+0000","last_fullsized":"2024-02-22T19:40:10.374299+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.127876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.127876+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":83,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746233+0000","last_change":"2024-02-22T19:37:57.182314+0000","last_active":"2024-02-22T19:40:10.746233+0000","last_peered":"2024-02-22T19:40:10.746233+0000","last_clean":"2024-02-22T19:40:10.746233+0000","last_became_active":"2024-02-22T19:37:57.181963+0000","last_became_peered":"2024-02-22T19:37:57.181963+0000","last_unstale":"2024-02-22T19:40:10.746233+0000","last_undegraded":"2024-02-22T19:40:10.746233+0000","last_fullsized":"2024-02-22T19:40:10.746233+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.297621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.297621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"264'15","reported_seq":80,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374777+0000","last_change":"2024-02-22T19:39:46.254971+0000","last_active":"2024-02-22T19:40:10.374777+0000","last_peered":"2024-02-22T19:40:10.374777+0000","last_clean":"2024-02-22T19:40:10.374777+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:40:10.374777+0000","last_undegraded":"2024-02-22T19:40:10.374777+0000","last_fullsized":"2024-02-22T19:40:10.374777+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:39:46.254899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.254899+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"291'595","reported_seq":609,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.297674+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:40:14.297674+0000","last_peered":"2024-02-22T19:40:14.297674+0000","last_clean":"2024-02-22T19:40:14.297674+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:40:14.297674+0000","last_undegraded":"2024-02-22T19:40:14.297674+0000","last_fullsized":"2024-02-22T19:40:14.297674+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":595,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38993920,"num_objects":595,"num_object_clones":0,"num_object_copies":1785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":38080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746017+0000","last_change":"2024-02-22T19:39:22.262082+0000","last_active":"2024-02-22T19:40:10.746017+0000","last_peered":"2024-02-22T19:40:10.746017+0000","last_clean":"2024-02-22T19:40:10.746017+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:40:10.746017+0000","last_undegraded":"2024-02-22T19:40:10.746017+0000","last_fullsized":"2024-02-22T19:40:10.746017+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":258,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.652031+0000","last_change":"2024-02-22T19:38:47.171603+0000","last_active":"2024-02-22T19:40:10.652031+0000","last_peered":"2024-02-22T19:40:10.652031+0000","last_clean":"2024-02-22T19:40:10.652031+0000","last_became_active":"2024-02-22T19:38:47.171002+0000","last_became_peered":"2024-02-22T19:38:47.171002+0000","last_unstale":"2024-02-22T19:40:10.652031+0000","last_undegraded":"2024-02-22T19:40:10.652031+0000","last_fullsized":"2024-02-22T19:40:10.652031+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:45.317440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:45.317440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746839+0000","last_change":"2024-02-22T19:37:57.185637+0000","last_active":"2024-02-22T19:40:10.746839+0000","last_peered":"2024-02-22T19:40:10.746839+0000","last_clean":"2024-02-22T19:40:10.746839+0000","last_became_active":"2024-02-22T19:37:57.185312+0000","last_became_peered":"2024-02-22T19:37:57.185312+0000","last_unstale":"2024-02-22T19:40:10.746839+0000","last_undegraded":"2024-02-22T19:40:10.746839+0000","last_fullsized":"2024-02-22T19:40:10.746839+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":263,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746918+0000","last_change":"2024-02-22T19:39:51.569245+0000","last_active":"2024-02-22T19:40:10.746918+0000","last_peered":"2024-02-22T19:40:10.746918+0000","last_clean":"2024-02-22T19:40:10.746918+0000","last_became_active":"2024-02-22T19:39:22.251742+0000","last_became_peered":"2024-02-22T19:39:22.251742+0000","last_unstale":"2024-02-22T19:40:10.746918+0000","last_undegraded":"2024-02-22T19:40:10.746918+0000","last_fullsized":"2024-02-22T19:40:10.746918+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":470,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.396301+0000","last_change":"2024-02-22T19:39:37.302965+0000","last_active":"2024-02-22T19:40:12.396301+0000","last_peered":"2024-02-22T19:40:12.396301+0000","last_clean":"2024-02-22T19:40:12.396301+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:40:12.396301+0000","last_undegraded":"2024-02-22T19:40:12.396301+0000","last_fullsized":"2024-02-22T19:40:12.396301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:39:37.302890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.302890+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":86,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.373235+0000","last_change":"2024-02-22T19:39:46.299764+0000","last_active":"2024-02-22T19:40:10.373235+0000","last_peered":"2024-02-22T19:40:10.373235+0000","last_clean":"2024-02-22T19:40:10.373235+0000","last_became_active":"2024-02-22T19:39:22.251614+0000","last_became_peered":"2024-02-22T19:39:22.251614+0000","last_unstale":"2024-02-22T19:40:10.373235+0000","last_undegraded":"2024-02-22T19:40:10.373235+0000","last_fullsized":"2024-02-22T19:40:10.373235+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"291'591","reported_seq":605,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.341995+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:40:14.341995+0000","last_peered":"2024-02-22T19:40:14.341995+0000","last_clean":"2024-02-22T19:40:14.341995+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:40:14.341995+0000","last_undegraded":"2024-02-22T19:40:14.341995+0000","last_fullsized":"2024-02-22T19:40:14.341995+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":591,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38731776,"num_objects":591,"num_object_clones":0,"num_object_copies":1773,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":591,"num_write_kb":37824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":309,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592637+0000","last_change":"2024-02-22T19:39:22.253396+0000","last_active":"2024-02-22T19:40:10.592637+0000","last_peered":"2024-02-22T19:40:10.592637+0000","last_clean":"2024-02-22T19:40:10.592637+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:40:10.592637+0000","last_undegraded":"2024-02-22T19:40:10.592637+0000","last_fullsized":"2024-02-22T19:40:10.592637+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:24.596531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:24.596531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":255,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593597+0000","last_change":"2024-02-22T19:39:47.645026+0000","last_active":"2024-02-22T19:40:10.593597+0000","last_peered":"2024-02-22T19:40:10.593597+0000","last_clean":"2024-02-22T19:40:10.593597+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:40:10.593597+0000","last_undegraded":"2024-02-22T19:40:10.593597+0000","last_fullsized":"2024-02-22T19:40:10.593597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":263,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.371616+0000","last_change":"2024-02-22T19:39:45.253946+0000","last_active":"2024-02-22T19:40:10.371616+0000","last_peered":"2024-02-22T19:40:10.371616+0000","last_clean":"2024-02-22T19:40:10.371616+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:40:10.371616+0000","last_undegraded":"2024-02-22T19:40:10.371616+0000","last_fullsized":"2024-02-22T19:40:10.371616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.253853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.253853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374512+0000","last_change":"2024-02-22T19:39:52.175249+0000","last_active":"2024-02-22T19:40:10.374512+0000","last_peered":"2024-02-22T19:40:10.374512+0000","last_clean":"2024-02-22T19:40:10.374512+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:40:10.374512+0000","last_undegraded":"2024-02-22T19:40:10.374512+0000","last_fullsized":"2024-02-22T19:40:10.374512+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:39:52.175144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":360,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.375119+0000","last_change":"2024-02-22T19:39:34.410627+0000","last_active":"2024-02-22T19:40:10.375119+0000","last_peered":"2024-02-22T19:40:10.375119+0000","last_clean":"2024-02-22T19:40:10.375119+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:40:10.375119+0000","last_undegraded":"2024-02-22T19:40:10.375119+0000","last_fullsized":"2024-02-22T19:40:10.375119+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:39:34.410547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.410547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":303,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.375135+0000","last_change":"2024-02-22T19:39:51.129196+0000","last_active":"2024-02-22T19:40:10.375135+0000","last_peered":"2024-02-22T19:40:10.375135+0000","last_clean":"2024-02-22T19:40:10.375135+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:40:10.375135+0000","last_undegraded":"2024-02-22T19:40:10.375135+0000","last_fullsized":"2024-02-22T19:40:10.375135+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.129010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.129010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"291'648","reported_seq":662,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.601139+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:40:14.601139+0000","last_peered":"2024-02-22T19:40:14.601139+0000","last_clean":"2024-02-22T19:40:14.601139+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:40:14.601139+0000","last_undegraded":"2024-02-22T19:40:14.601139+0000","last_fullsized":"2024-02-22T19:40:14.601139+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":648,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42467328,"num_objects":648,"num_object_clones":0,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":41472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":97,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747410+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:40:10.747410+0000","last_peered":"2024-02-22T19:40:10.747410+0000","last_clean":"2024-02-22T19:40:10.747410+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:40:10.747410+0000","last_undegraded":"2024-02-22T19:40:10.747410+0000","last_fullsized":"2024-02-22T19:40:10.747410+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592458+0000","last_change":"2024-02-22T19:39:22.252856+0000","last_active":"2024-02-22T19:40:10.592458+0000","last_peered":"2024-02-22T19:40:10.592458+0000","last_clean":"2024-02-22T19:40:10.592458+0000","last_became_active":"2024-02-22T19:39:22.252166+0000","last_became_peered":"2024-02-22T19:39:22.252166+0000","last_unstale":"2024-02-22T19:40:10.592458+0000","last_undegraded":"2024-02-22T19:40:10.592458+0000","last_fullsized":"2024-02-22T19:40:10.592458+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:23.598758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:23.598758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":273,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651384+0000","last_change":"2024-02-22T19:39:47.705812+0000","last_active":"2024-02-22T19:40:10.651384+0000","last_peered":"2024-02-22T19:40:10.651384+0000","last_clean":"2024-02-22T19:40:10.651384+0000","last_became_active":"2024-02-22T19:37:57.180638+0000","last_became_peered":"2024-02-22T19:37:57.180638+0000","last_unstale":"2024-02-22T19:40:10.651384+0000","last_undegraded":"2024-02-22T19:40:10.651384+0000","last_fullsized":"2024-02-22T19:40:10.651384+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":379,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747475+0000","last_change":"2024-02-22T19:39:22.439994+0000","last_active":"2024-02-22T19:40:10.747475+0000","last_peered":"2024-02-22T19:40:10.747475+0000","last_clean":"2024-02-22T19:40:10.747475+0000","last_became_active":"2024-02-22T19:39:22.256569+0000","last_became_peered":"2024-02-22T19:39:22.256569+0000","last_unstale":"2024-02-22T19:40:10.747475+0000","last_undegraded":"2024-02-22T19:40:10.747475+0000","last_fullsized":"2024-02-22T19:40:10.747475+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:37:47.421622+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:47.421622+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":270,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593170+0000","last_change":"2024-02-22T19:37:57.175982+0000","last_active":"2024-02-22T19:40:10.593170+0000","last_peered":"2024-02-22T19:40:10.593170+0000","last_clean":"2024-02-22T19:40:10.593170+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:40:10.593170+0000","last_undegraded":"2024-02-22T19:40:10.593170+0000","last_fullsized":"2024-02-22T19:40:10.593170+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:12.513648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:12.513648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"291'630","reported_seq":644,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.310237+0000","last_change":"2024-02-22T19:40:01.098193+0000","last_active":"2024-02-22T19:40:14.310237+0000","last_peered":"2024-02-22T19:40:14.310237+0000","last_clean":"2024-02-22T19:40:14.310237+0000","last_became_active":"2024-02-22T19:40:01.097792+0000","last_became_peered":"2024-02-22T19:40:01.097792+0000","last_unstale":"2024-02-22T19:40:14.310237+0000","last_undegraded":"2024-02-22T19:40:14.310237+0000","last_fullsized":"2024-02-22T19:40:14.310237+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":630,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41287680,"num_objects":630,"num_object_clones":0,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":40320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":154,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372229+0000","last_change":"2024-02-22T19:38:47.168318+0000","last_active":"2024-02-22T19:40:10.372229+0000","last_peered":"2024-02-22T19:40:10.372229+0000","last_clean":"2024-02-22T19:40:10.372229+0000","last_became_active":"2024-02-22T19:38:47.167994+0000","last_became_peered":"2024-02-22T19:38:47.167994+0000","last_unstale":"2024-02-22T19:40:10.372229+0000","last_undegraded":"2024-02-22T19:40:10.372229+0000","last_fullsized":"2024-02-22T19:40:10.372229+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:01.957854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:01.957854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":275,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372478+0000","last_change":"2024-02-22T19:39:50.386984+0000","last_active":"2024-02-22T19:40:10.372478+0000","last_peered":"2024-02-22T19:40:10.372478+0000","last_clean":"2024-02-22T19:40:10.372478+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:40:10.372478+0000","last_undegraded":"2024-02-22T19:40:10.372478+0000","last_fullsized":"2024-02-22T19:40:10.372478+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.386916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.386916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":65,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651336+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:40:10.651336+0000","last_peered":"2024-02-22T19:40:10.651336+0000","last_clean":"2024-02-22T19:40:10.651336+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:40:10.651336+0000","last_undegraded":"2024-02-22T19:40:10.651336+0000","last_fullsized":"2024-02-22T19:40:10.651336+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":974,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.401809+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:40:12.401809+0000","last_peered":"2024-02-22T19:40:12.401809+0000","last_clean":"2024-02-22T19:40:12.401809+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:40:12.401809+0000","last_undegraded":"2024-02-22T19:40:12.401809+0000","last_fullsized":"2024-02-22T19:40:12.401809+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":656,"num_read_kb":651,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":92,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.373410+0000","last_change":"2024-02-22T19:39:22.301413+0000","last_active":"2024-02-22T19:40:10.373410+0000","last_peered":"2024-02-22T19:40:10.373410+0000","last_clean":"2024-02-22T19:40:10.373410+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:40:10.373410+0000","last_undegraded":"2024-02-22T19:40:10.373410+0000","last_fullsized":"2024-02-22T19:40:10.373410+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-22T19:37:08.878452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.878452+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":137,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747509+0000","last_change":"2024-02-22T19:39:22.248529+0000","last_active":"2024-02-22T19:40:10.747509+0000","last_peered":"2024-02-22T19:40:10.747509+0000","last_clean":"2024-02-22T19:40:10.747509+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:40:10.747509+0000","last_undegraded":"2024-02-22T19:40:10.747509+0000","last_fullsized":"2024-02-22T19:40:10.747509+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:15.908714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:15.908714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"291'612","reported_seq":626,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.291608+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:40:14.291608+0000","last_peered":"2024-02-22T19:40:14.291608+0000","last_clean":"2024-02-22T19:40:14.291608+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:40:14.291608+0000","last_undegraded":"2024-02-22T19:40:14.291608+0000","last_fullsized":"2024-02-22T19:40:14.291608+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":612,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40108032,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":612,"num_write_kb":39168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":200,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746532+0000","last_change":"2024-02-22T19:39:38.562584+0000","last_active":"2024-02-22T19:40:10.746532+0000","last_peered":"2024-02-22T19:40:10.746532+0000","last_clean":"2024-02-22T19:40:10.746532+0000","last_became_active":"2024-02-22T19:39:22.256349+0000","last_became_peered":"2024-02-22T19:39:22.256349+0000","last_unstale":"2024-02-22T19:40:10.746532+0000","last_undegraded":"2024-02-22T19:40:10.746532+0000","last_fullsized":"2024-02-22T19:40:10.746532+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:38.562500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:39:38.562500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":252,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.371527+0000","last_change":"2024-02-22T19:37:03.977945+0000","last_active":"2024-02-22T19:40:10.371527+0000","last_peered":"2024-02-22T19:40:10.371527+0000","last_clean":"2024-02-22T19:40:10.371527+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:40:10.371527+0000","last_undegraded":"2024-02-22T19:40:10.371527+0000","last_fullsized":"2024-02-22T19:40:10.371527+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:03.977850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:37:03.977850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":281,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372647+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:40:10.372647+0000","last_peered":"2024-02-22T19:40:10.372647+0000","last_clean":"2024-02-22T19:40:10.372647+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:40:10.372647+0000","last_undegraded":"2024-02-22T19:40:10.372647+0000","last_fullsized":"2024-02-22T19:40:10.372647+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":102,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745850+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:40:10.745850+0000","last_peered":"2024-02-22T19:40:10.745850+0000","last_clean":"2024-02-22T19:40:10.745850+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:40:10.745850+0000","last_undegraded":"2024-02-22T19:40:10.745850+0000","last_fullsized":"2024-02-22T19:40:10.745850+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":307,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746207+0000","last_change":"2024-02-22T19:39:49.629194+0000","last_active":"2024-02-22T19:40:10.746207+0000","last_peered":"2024-02-22T19:40:10.746207+0000","last_clean":"2024-02-22T19:40:10.746207+0000","last_became_active":"2024-02-22T19:39:22.263999+0000","last_became_peered":"2024-02-22T19:39:22.263999+0000","last_unstale":"2024-02-22T19:40:10.746207+0000","last_undegraded":"2024-02-22T19:40:10.746207+0000","last_fullsized":"2024-02-22T19:40:10.746207+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"291'667","reported_seq":681,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.350091+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:40:14.350091+0000","last_peered":"2024-02-22T19:40:14.350091+0000","last_clean":"2024-02-22T19:40:14.350091+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:40:14.350091+0000","last_undegraded":"2024-02-22T19:40:14.350091+0000","last_fullsized":"2024-02-22T19:40:14.350091+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":667,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43712512,"num_objects":667,"num_object_clones":0,"num_object_copies":2001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":667,"num_write_kb":42688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":160,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746054+0000","last_change":"2024-02-22T19:39:42.557957+0000","last_active":"2024-02-22T19:40:10.746054+0000","last_peered":"2024-02-22T19:40:10.746054+0000","last_clean":"2024-02-22T19:40:10.746054+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:40:10.746054+0000","last_undegraded":"2024-02-22T19:40:10.746054+0000","last_fullsized":"2024-02-22T19:40:10.746054+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:42.557792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.557792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":72,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651145+0000","last_change":"2024-02-22T19:39:22.271772+0000","last_active":"2024-02-22T19:40:10.651145+0000","last_peered":"2024-02-22T19:40:10.651145+0000","last_clean":"2024-02-22T19:40:10.651145+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:40:10.651145+0000","last_undegraded":"2024-02-22T19:40:10.651145+0000","last_fullsized":"2024-02-22T19:40:10.651145+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:33.417727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:35:33.417727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":83,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747810+0000","last_change":"2024-02-22T19:37:57.190664+0000","last_active":"2024-02-22T19:40:10.747810+0000","last_peered":"2024-02-22T19:40:10.747810+0000","last_clean":"2024-02-22T19:40:10.747810+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:40:10.747810+0000","last_undegraded":"2024-02-22T19:40:10.747810+0000","last_fullsized":"2024-02-22T19:40:10.747810+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.261537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.261537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":396,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372298+0000","last_change":"2024-02-22T19:39:40.321965+0000","last_active":"2024-02-22T19:40:10.372298+0000","last_peered":"2024-02-22T19:40:10.372298+0000","last_clean":"2024-02-22T19:40:10.372298+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:40:10.372298+0000","last_undegraded":"2024-02-22T19:40:10.372298+0000","last_fullsized":"2024-02-22T19:40:10.372298+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:40.321851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:40.321851+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"268'13","reported_seq":126,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.405592+0000","last_change":"2024-02-22T19:39:41.566614+0000","last_active":"2024-02-22T19:40:12.405592+0000","last_peered":"2024-02-22T19:40:12.405592+0000","last_clean":"2024-02-22T19:40:12.405592+0000","last_became_active":"2024-02-22T19:39:22.252945+0000","last_became_peered":"2024-02-22T19:39:22.252945+0000","last_unstale":"2024-02-22T19:40:12.405592+0000","last_undegraded":"2024-02-22T19:40:12.405592+0000","last_fullsized":"2024-02-22T19:40:12.405592+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:39:41.566527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.566527+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"291'613","reported_seq":627,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.610019+0000","last_change":"2024-02-22T19:40:00.882269+0000","last_active":"2024-02-22T19:40:14.610019+0000","last_peered":"2024-02-22T19:40:14.610019+0000","last_clean":"2024-02-22T19:40:14.610019+0000","last_became_active":"2024-02-22T19:40:00.881943+0000","last_became_peered":"2024-02-22T19:40:00.881943+0000","last_unstale":"2024-02-22T19:40:14.610019+0000","last_undegraded":"2024-02-22T19:40:14.610019+0000","last_fullsized":"2024-02-22T19:40:14.610019+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":613,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40173568,"num_objects":613,"num_object_clones":0,"num_object_copies":1839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":613,"num_write_kb":39232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":159,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651649+0000","last_change":"2024-02-22T19:40:02.507342+0000","last_active":"2024-02-22T19:40:10.651649+0000","last_peered":"2024-02-22T19:40:10.651649+0000","last_clean":"2024-02-22T19:40:10.651649+0000","last_became_active":"2024-02-22T19:40:02.507127+0000","last_became_peered":"2024-02-22T19:40:02.507127+0000","last_unstale":"2024-02-22T19:40:10.651649+0000","last_undegraded":"2024-02-22T19:40:10.651649+0000","last_fullsized":"2024-02-22T19:40:10.651649+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":253,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.593232+0000","last_change":"2024-02-22T19:39:55.666653+0000","last_active":"2024-02-22T19:40:10.593232+0000","last_peered":"2024-02-22T19:40:10.593232+0000","last_clean":"2024-02-22T19:40:10.593232+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:40:10.593232+0000","last_undegraded":"2024-02-22T19:40:10.593232+0000","last_fullsized":"2024-02-22T19:40:10.593232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:11.554382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:36:11.554382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":81,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.373303+0000","last_change":"2024-02-22T19:39:22.245754+0000","last_active":"2024-02-22T19:40:10.373303+0000","last_peered":"2024-02-22T19:40:10.373303+0000","last_clean":"2024-02-22T19:40:10.373303+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:40:10.373303+0000","last_undegraded":"2024-02-22T19:40:10.373303+0000","last_fullsized":"2024-02-22T19:40:10.373303+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:48.680723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.680723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":378,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746168+0000","last_change":"2024-02-22T19:39:46.430095+0000","last_active":"2024-02-22T19:40:10.746168+0000","last_peered":"2024-02-22T19:40:10.746168+0000","last_clean":"2024-02-22T19:40:10.746168+0000","last_became_active":"2024-02-22T19:39:22.248416+0000","last_became_peered":"2024-02-22T19:39:22.248416+0000","last_unstale":"2024-02-22T19:40:10.746168+0000","last_undegraded":"2024-02-22T19:40:10.746168+0000","last_fullsized":"2024-02-22T19:40:10.746168+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":430,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.401970+0000","last_change":"2024-02-22T19:39:23.137035+0000","last_active":"2024-02-22T19:40:12.401970+0000","last_peered":"2024-02-22T19:40:12.401970+0000","last_clean":"2024-02-22T19:40:12.401970+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:40:12.401970+0000","last_undegraded":"2024-02-22T19:40:12.401970+0000","last_fullsized":"2024-02-22T19:40:12.401970+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:35:25.316835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:25.316835+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"291'588","reported_seq":602,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.291570+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:40:14.291570+0000","last_peered":"2024-02-22T19:40:14.291570+0000","last_clean":"2024-02-22T19:40:14.291570+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:40:14.291570+0000","last_undegraded":"2024-02-22T19:40:14.291570+0000","last_fullsized":"2024-02-22T19:40:14.291570+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":588,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38535168,"num_objects":588,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":588,"num_write_kb":37632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374907+0000","last_change":"2024-02-22T19:38:47.167886+0000","last_active":"2024-02-22T19:40:10.374907+0000","last_peered":"2024-02-22T19:40:10.374907+0000","last_clean":"2024-02-22T19:40:10.374907+0000","last_became_active":"2024-02-22T19:38:47.167514+0000","last_became_peered":"2024-02-22T19:38:47.167514+0000","last_unstale":"2024-02-22T19:40:10.374907+0000","last_undegraded":"2024-02-22T19:40:10.374907+0000","last_fullsized":"2024-02-22T19:40:10.374907+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.583038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.583038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":55,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651621+0000","last_change":"2024-02-22T19:38:47.170822+0000","last_active":"2024-02-22T19:40:10.651621+0000","last_peered":"2024-02-22T19:40:10.651621+0000","last_clean":"2024-02-22T19:40:10.651621+0000","last_became_active":"2024-02-22T19:38:47.170472+0000","last_became_peered":"2024-02-22T19:38:47.170472+0000","last_unstale":"2024-02-22T19:40:10.651621+0000","last_undegraded":"2024-02-22T19:40:10.651621+0000","last_fullsized":"2024-02-22T19:40:10.651621+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":282,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.652158+0000","last_change":"2024-02-22T19:39:22.243142+0000","last_active":"2024-02-22T19:40:10.652158+0000","last_peered":"2024-02-22T19:40:10.652158+0000","last_clean":"2024-02-22T19:40:10.652158+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:40:10.652158+0000","last_undegraded":"2024-02-22T19:40:10.652158+0000","last_fullsized":"2024-02-22T19:40:10.652158+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:37:06.317125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":386,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651398+0000","last_change":"2024-02-22T19:39:23.568647+0000","last_active":"2024-02-22T19:40:10.651398+0000","last_peered":"2024-02-22T19:40:10.651398+0000","last_clean":"2024-02-22T19:40:10.651398+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:40:10.651398+0000","last_undegraded":"2024-02-22T19:40:10.651398+0000","last_fullsized":"2024-02-22T19:40:10.651398+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":73,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651450+0000","last_change":"2024-02-22T19:39:22.248060+0000","last_active":"2024-02-22T19:40:10.651450+0000","last_peered":"2024-02-22T19:40:10.651450+0000","last_clean":"2024-02-22T19:40:10.651450+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:40:10.651450+0000","last_undegraded":"2024-02-22T19:40:10.651450+0000","last_fullsized":"2024-02-22T19:40:10.651450+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:36.900547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:35:36.900547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"291'661","reported_seq":675,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.587946+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:40:14.587946+0000","last_peered":"2024-02-22T19:40:14.587946+0000","last_clean":"2024-02-22T19:40:14.587946+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:40:14.587946+0000","last_undegraded":"2024-02-22T19:40:14.587946+0000","last_fullsized":"2024-02-22T19:40:14.587946+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":661,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43319296,"num_objects":661,"num_object_clones":0,"num_object_copies":1983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":86,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747046+0000","last_change":"2024-02-22T19:39:55.457507+0000","last_active":"2024-02-22T19:40:10.747046+0000","last_peered":"2024-02-22T19:40:10.747046+0000","last_clean":"2024-02-22T19:40:10.747046+0000","last_became_active":"2024-02-22T19:37:57.189767+0000","last_became_peered":"2024-02-22T19:37:57.189767+0000","last_unstale":"2024-02-22T19:40:10.747046+0000","last_undegraded":"2024-02-22T19:40:10.747046+0000","last_fullsized":"2024-02-22T19:40:10.747046+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.071500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.071500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":110,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.747642+0000","last_change":"2024-02-22T19:39:22.269372+0000","last_active":"2024-02-22T19:40:10.747642+0000","last_peered":"2024-02-22T19:40:10.747642+0000","last_clean":"2024-02-22T19:40:10.747642+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:40:10.747642+0000","last_undegraded":"2024-02-22T19:40:10.747642+0000","last_fullsized":"2024-02-22T19:40:10.747642+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.979469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.979469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"269'25","reported_seq":528,"reported_epoch":291,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:40:12.404384+0000","last_change":"2024-02-22T19:39:53.251052+0000","last_active":"2024-02-22T19:40:12.404384+0000","last_peered":"2024-02-22T19:40:12.404384+0000","last_clean":"2024-02-22T19:39:19.281649+0000","last_became_active":"2024-02-22T19:39:23.246947+0000","last_became_peered":"2024-02-22T19:39:23.246947+0000","last_unstale":"2024-02-22T19:40:12.404384+0000","last_undegraded":"2024-02-22T19:40:12.404384+0000","last_fullsized":"2024-02-22T19:40:12.404384+0000","mapping_epoch":268,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'9","last_scrub_stamp":"2024-02-22T19:34:43.126157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:34:43.126157+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,0],"avail_no_missing":["5","0","2"],"object_location_counts":[{"shards":"0,2,5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":260,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651364+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:40:10.651364+0000","last_peered":"2024-02-22T19:40:10.651364+0000","last_clean":"2024-02-22T19:40:10.651364+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:40:10.651364+0000","last_undegraded":"2024-02-22T19:40:10.651364+0000","last_fullsized":"2024-02-22T19:40:10.651364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":196,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.373491+0000","last_change":"2024-02-22T19:39:35.331000+0000","last_active":"2024-02-22T19:40:10.373491+0000","last_peered":"2024-02-22T19:40:10.373491+0000","last_clean":"2024-02-22T19:40:10.373491+0000","last_became_active":"2024-02-22T19:38:47.093284+0000","last_became_peered":"2024-02-22T19:38:47.093284+0000","last_unstale":"2024-02-22T19:40:10.373491+0000","last_undegraded":"2024-02-22T19:40:10.373491+0000","last_fullsized":"2024-02-22T19:40:10.373491+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:35.330913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:35.330913+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"291'610","reported_seq":624,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.291572+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:40:14.291572+0000","last_peered":"2024-02-22T19:40:14.291572+0000","last_clean":"2024-02-22T19:40:14.291572+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:40:14.291572+0000","last_undegraded":"2024-02-22T19:40:14.291572+0000","last_fullsized":"2024-02-22T19:40:14.291572+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":610,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39976960,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":610,"num_write_kb":39040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":262,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651752+0000","last_change":"2024-02-22T19:37:57.180115+0000","last_active":"2024-02-22T19:40:10.651752+0000","last_peered":"2024-02-22T19:40:10.651752+0000","last_clean":"2024-02-22T19:40:10.651752+0000","last_became_active":"2024-02-22T19:37:57.179488+0000","last_became_peered":"2024-02-22T19:37:57.179488+0000","last_unstale":"2024-02-22T19:40:10.651752+0000","last_undegraded":"2024-02-22T19:40:10.651752+0000","last_fullsized":"2024-02-22T19:40:10.651752+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:35:37.337251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:35:37.337251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"251'17","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.404041+0000","last_change":"2024-02-22T19:38:50.146296+0000","last_active":"2024-02-22T19:40:12.404041+0000","last_peered":"2024-02-22T19:40:12.404041+0000","last_clean":"2024-02-22T19:40:12.404041+0000","last_became_active":"2024-02-22T19:38:50.145632+0000","last_became_peered":"2024-02-22T19:38:50.145632+0000","last_unstale":"2024-02-22T19:40:12.404041+0000","last_undegraded":"2024-02-22T19:40:12.404041+0000","last_fullsized":"2024-02-22T19:40:12.404041+0000","mapping_epoch":250,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:36:53.427951+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.427951+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":60,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374651+0000","last_change":"2024-02-22T19:39:32.447723+0000","last_active":"2024-02-22T19:40:10.374651+0000","last_peered":"2024-02-22T19:40:10.374651+0000","last_clean":"2024-02-22T19:40:10.374651+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:40:10.374651+0000","last_undegraded":"2024-02-22T19:40:10.374651+0000","last_fullsized":"2024-02-22T19:40:10.374651+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:32.447647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.447647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":114,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.746793+0000","last_change":"2024-02-22T19:39:43.589553+0000","last_active":"2024-02-22T19:40:10.746793+0000","last_peered":"2024-02-22T19:40:10.746793+0000","last_clean":"2024-02-22T19:40:10.746793+0000","last_became_active":"2024-02-22T19:38:47.183168+0000","last_became_peered":"2024-02-22T19:38:47.183168+0000","last_unstale":"2024-02-22T19:40:10.746793+0000","last_undegraded":"2024-02-22T19:40:10.746793+0000","last_fullsized":"2024-02-22T19:40:10.746793+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"291'600","reported_seq":614,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.430869+0000","last_change":"2024-02-22T19:40:01.097839+0000","last_active":"2024-02-22T19:40:14.430869+0000","last_peered":"2024-02-22T19:40:14.430869+0000","last_clean":"2024-02-22T19:40:14.430869+0000","last_became_active":"2024-02-22T19:40:01.097523+0000","last_became_peered":"2024-02-22T19:40:01.097523+0000","last_unstale":"2024-02-22T19:40:14.430869+0000","last_undegraded":"2024-02-22T19:40:14.430869+0000","last_fullsized":"2024-02-22T19:40:14.430869+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":600,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39321600,"num_objects":600,"num_object_clones":0,"num_object_copies":1800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":38400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":88,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745510+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:40:10.745510+0000","last_peered":"2024-02-22T19:40:10.745510+0000","last_clean":"2024-02-22T19:40:10.745510+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:40:10.745510+0000","last_undegraded":"2024-02-22T19:40:10.745510+0000","last_fullsized":"2024-02-22T19:40:10.745510+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":266,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374990+0000","last_change":"2024-02-22T19:38:47.172343+0000","last_active":"2024-02-22T19:40:10.374990+0000","last_peered":"2024-02-22T19:40:10.374990+0000","last_clean":"2024-02-22T19:40:10.374990+0000","last_became_active":"2024-02-22T19:38:47.171941+0000","last_became_peered":"2024-02-22T19:38:47.171941+0000","last_unstale":"2024-02-22T19:40:10.374990+0000","last_undegraded":"2024-02-22T19:40:10.374990+0000","last_fullsized":"2024-02-22T19:40:10.374990+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.312692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.312692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":313,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.372831+0000","last_change":"2024-02-22T19:39:49.403011+0000","last_active":"2024-02-22T19:40:10.372831+0000","last_peered":"2024-02-22T19:40:10.372831+0000","last_clean":"2024-02-22T19:40:10.372831+0000","last_became_active":"2024-02-22T19:39:22.244710+0000","last_became_peered":"2024-02-22T19:39:22.244710+0000","last_unstale":"2024-02-22T19:40:10.372831+0000","last_undegraded":"2024-02-22T19:40:10.372831+0000","last_fullsized":"2024-02-22T19:40:10.372831+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":273,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.745980+0000","last_change":"2024-02-22T19:39:45.615429+0000","last_active":"2024-02-22T19:40:10.745980+0000","last_peered":"2024-02-22T19:40:10.745980+0000","last_clean":"2024-02-22T19:40:10.745980+0000","last_became_active":"2024-02-22T19:38:47.164172+0000","last_became_peered":"2024-02-22T19:38:47.164172+0000","last_unstale":"2024-02-22T19:40:10.745980+0000","last_undegraded":"2024-02-22T19:40:10.745980+0000","last_fullsized":"2024-02-22T19:40:10.745980+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":134,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.374982+0000","last_change":"2024-02-22T19:38:48.843443+0000","last_active":"2024-02-22T19:40:10.374982+0000","last_peered":"2024-02-22T19:40:10.374982+0000","last_clean":"2024-02-22T19:40:10.374982+0000","last_became_active":"2024-02-22T19:38:47.181392+0000","last_became_peered":"2024-02-22T19:38:47.181392+0000","last_unstale":"2024-02-22T19:40:10.374982+0000","last_undegraded":"2024-02-22T19:40:10.374982+0000","last_fullsized":"2024-02-22T19:40:10.374982+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-22T19:37:48.379789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:48.379789+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"291'645","reported_seq":659,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:14.603433+0000","last_change":"2024-02-22T19:40:00.881912+0000","last_active":"2024-02-22T19:40:14.603433+0000","last_peered":"2024-02-22T19:40:14.603433+0000","last_clean":"2024-02-22T19:40:14.603433+0000","last_became_active":"2024-02-22T19:40:00.881492+0000","last_became_peered":"2024-02-22T19:40:00.881492+0000","last_unstale":"2024-02-22T19:40:14.603433+0000","last_undegraded":"2024-02-22T19:40:14.603433+0000","last_fullsized":"2024-02-22T19:40:14.603433+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":645,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42270720,"num_objects":645,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":645,"num_write_kb":41280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":273,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.592462+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:40:10.592462+0000","last_peered":"2024-02-22T19:40:10.592462+0000","last_clean":"2024-02-22T19:40:10.592462+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:40:10.592462+0000","last_undegraded":"2024-02-22T19:40:10.592462+0000","last_fullsized":"2024-02-22T19:40:10.592462+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":99,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.373252+0000","last_change":"2024-02-22T19:39:32.393413+0000","last_active":"2024-02-22T19:40:10.373252+0000","last_peered":"2024-02-22T19:40:10.373252+0000","last_clean":"2024-02-22T19:40:10.373252+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:40:10.373252+0000","last_undegraded":"2024-02-22T19:40:10.373252+0000","last_fullsized":"2024-02-22T19:40:10.373252+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:39:32.393320+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.393320+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":469,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-22T19:40:12.406280+0000","last_change":"2024-02-22T19:39:55.160226+0000","last_active":"2024-02-22T19:40:12.406280+0000","last_peered":"2024-02-22T19:40:12.406280+0000","last_clean":"2024-02-22T19:40:12.406280+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:40:12.406280+0000","last_undegraded":"2024-02-22T19:40:12.406280+0000","last_fullsized":"2024-02-22T19:40:12.406280+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:39:55.160157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:55.160157+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":316,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651678+0000","last_change":"2024-02-22T19:39:22.400760+0000","last_active":"2024-02-22T19:40:10.651678+0000","last_peered":"2024-02-22T19:40:10.651678+0000","last_clean":"2024-02-22T19:40:10.651678+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:40:10.651678+0000","last_undegraded":"2024-02-22T19:40:10.651678+0000","last_fullsized":"2024-02-22T19:40:10.651678+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:37:05.357880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.357880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":114,"reported_epoch":290,"state":"active+clean","last_fresh":"2024-02-22T19:40:10.651864+0000","last_change":"2024-02-22T19:39:53.646743+0000","last_active":"2024-02-22T19:40:10.651864+0000","last_peered":"2024-02-22T19:40:10.651864+0000","last_clean":"2024-02-22T19:40:10.651864+0000","last_became_active":"2024-02-22T19:38:47.169571+0000","last_became_peered":"2024-02-22T19:38:47.169571+0000","last_unstale":"2024-02-22T19:40:10.651864+0000","last_undegraded":"2024-02-22T19:40:10.651864+0000","last_fullsized":"2024-02-22T19:40:10.651864+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":653000704,"num_objects":9964,"num_object_clones":0,"num_object_copies":29892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9964,"num_write_kb":637696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1965555712,"data_stored":1965555712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9964,"ondisk_log_size":9964,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":656,"num_read_kb":651,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":6257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1109,"num_read_kb":866,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":7172,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266826,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":625788,"kb_used_data":324468,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93119364,"statfs":{"total":95995035648,"available":95354228736,"internally_reserved":0,"allocated":332255232,"data_stored":331455347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.456,"15min":1.244},"min":{"1min":0.27400000000000002,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":5.4539999999999997,"5min":7.0700000000000003,"15min":42.597999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":2.387,"5min":1.8779999999999999,"15min":1.2609999999999999},"min":{"1min":0.223,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":8.0489999999999995,"5min":9.8870000000000005,"15min":42.155999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":3.9180000000000001,"5min":2.0470000000000002,"15min":1.337},"min":{"1min":0.245,"5min":0.223,"15min":0.223},"max":{"1min":54.951000000000001,"5min":54.951000000000001,"15min":54.951000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":7.2450000000000001,"5min":3.0049999999999999,"15min":1.714},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.219},"max":{"1min":57.012,"5min":57.012,"15min":57.012},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":3.8820000000000001,"5min":1.8200000000000001,"15min":1.236},"min":{"1min":0.28199999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":40.606000000000002,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":5.0039999999999996,"5min":2.1480000000000001,"15min":1.4319999999999999},"min":{"1min":0.33700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":38.411000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":4.5229999999999997,"5min":2.1059999999999999,"15min":1.4410000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":38.692999999999998,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":4.2539999999999996,"5min":2.2280000000000002,"15min":1.4279999999999999},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":38.219000000000001,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":0.80500000000000005,"15min":0.57799999999999996},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.23999999999999999},"max":{"1min":5.5199999999999996,"5min":5.5199999999999996,"15min":5.5199999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.179,"5min":0.68400000000000005,"15min":0.53800000000000003},"min":{"1min":0.27500000000000002,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":7.6219999999999999,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.75600000000000001,"15min":0.57799999999999996},"min":{"1min":0.28399999999999997,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":3.254,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.2909999999999999,"5min":0.83399999999999996,"15min":0.626},"min":{"1min":0.317,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:39:12 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":0.96899999999999997,"15min":0.71499999999999997},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":12.199999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.444,"5min":0.75900000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.6390000000000002,"5min":7.6390000000000002,"15min":12.067},"last":0.51000000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528463,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":452844,"kb_used_data":147428,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93292308,"statfs":{"total":95995035648,"available":95531323392,"internally_reserved":0,"allocated":150966272,"data_stored":150181406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.3819999999999999,"15min":1.0600000000000001},"min":{"1min":0.22500000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":4.1280000000000001,"5min":6.6660000000000004,"15min":17.885000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.458,"5min":1.3480000000000001,"15min":1.026},"min":{"1min":0.29099999999999998,"5min":0.218,"15min":0.21199999999999999},"max":{"1min":4.835,"5min":8.1120000000000001,"15min":8.2699999999999996},"last":0.85999999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":2.5939999999999999,"5min":2.0259999999999998,"15min":1.1819999999999999},"min":{"1min":0.499,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":6.5209999999999999,"5min":12.808999999999999,"15min":12.808999999999999},"last":1.012},{"interface":"front","average":{"1min":2.9609999999999999,"5min":1.7949999999999999,"15min":1.087},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":13.590999999999999,"5min":13.590999999999999,"15min":13.590999999999999},"last":2.9169999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.9830000000000001,"15min":1.2689999999999999},"min":{"1min":0.308,"5min":0.253,"15min":0.249},"max":{"1min":6.5919999999999996,"5min":11.198,"15min":12.859999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.7529999999999999,"5min":1.8500000000000001,"15min":1.1439999999999999},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":8.5779999999999994,"5min":11.565,"15min":11.565},"last":0.315}]},{"osd":3,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.587,"15min":1.254},"min":{"1min":0.38,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":8.1240000000000006,"5min":10.135,"15min":38.762},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.984,"5min":1.4950000000000001,"15min":1.381},"min":{"1min":0.40000000000000002,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":10.315,"5min":10.315,"15min":35.612000000000002},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.57099999999999995,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.73899999999999999,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.861,"5min":9.3949999999999996,"15min":9.3949999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.754,"15min":0.55000000000000004},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29299999999999998},"max":{"1min":3.5129999999999999,"5min":7.3780000000000001,"15min":7.3780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.77700000000000002,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.177,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.39200000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":0.73799999999999999,"15min":0.58699999999999997},"min":{"1min":0.32200000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":3.2709999999999999,"5min":3.488,"15min":3.488},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.294,"5min":0.86399999999999999,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":5.7939999999999996,"5min":5.7939999999999996,"15min":5.7939999999999996},"last":0.46400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":711520,"kb_used_data":397912,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93033632,"statfs":{"total":95995035648,"available":95266439168,"internally_reserved":0,"allocated":407461888,"data_stored":406625895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:56 2024","interfaces":[{"interface":"back","average":{"1min":1.8620000000000001,"5min":1.746,"15min":1.363},"min":{"1min":0.246,"5min":0.185,"15min":0.185},"max":{"1min":9.1319999999999997,"5min":10.6,"15min":15.613},"last":0.874},{"interface":"front","average":{"1min":1.393,"5min":1.948,"15min":1.446},"min":{"1min":0.20999999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":6.0019999999999998,"5min":9.1259999999999994,"15min":13.515000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:40:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.673,"15min":1.4650000000000001},"min":{"1min":0.24199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.0700000000000003,"5min":11.682,"15min":16.503},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":2.1560000000000001,"15min":1.556},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.192},"max":{"1min":3.3439999999999999,"5min":14.994,"15min":14.994},"last":1.502}]},{"osd":3,"last update":"Thu Feb 22 19:39:44 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":2.0619999999999998,"15min":1.6180000000000001},"min":{"1min":0.27000000000000002,"5min":0.23100000000000001,"15min":0.214},"max":{"1min":9.7650000000000006,"5min":10.734999999999999,"15min":25.145},"last":0.98199999999999998},{"interface":"front","average":{"1min":2.8039999999999998,"5min":2.1779999999999999,"15min":1.6839999999999999},"min":{"1min":0.32000000000000001,"5min":0.251,"15min":0.251},"max":{"1min":9.0939999999999994,"5min":9.0939999999999994,"15min":22.25},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:40:03 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.446,"15min":1.198},"min":{"1min":0.438,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9390000000000001,"5min":7.9390000000000001,"15min":15.542999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.3180000000000001,"15min":1.04},"min":{"1min":0.41799999999999998,"5min":0.30499999999999999,"15min":0.245},"max":{"1min":3.6680000000000001,"5min":10.706,"15min":10.706},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:39:21 2024","interfaces":[{"interface":"back","average":{"1min":2.6699999999999999,"5min":1.6990000000000001,"15min":1.2370000000000001},"min":{"1min":0.442,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.976,"5min":8.2189999999999994,"15min":15.587999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.514,"15min":1.115},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":4.4059999999999997,"5min":6.6790000000000003,"15min":9.4719999999999995},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:39:51 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.5029999999999999,"15min":1.0760000000000001},"min":{"1min":0.371,"5min":0.35199999999999998,"15min":0.28599999999999998},"max":{"1min":3.6400000000000001,"5min":10.631,"15min":10.631},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.494,"15min":1.202},"min":{"1min":0.49199999999999999,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":8.0259999999999998,"5min":8.0259999999999998,"15min":15.651999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:40:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":1.7190000000000001,"15min":1.2250000000000001},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":2.0030000000000001,"5min":13.169,"15min":13.169},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.4179999999999999,"15min":1.093},"min":{"1min":0.47099999999999997,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":10.673,"15min":10.673},"last":1.0289999999999999}]}]},{"osd":0,"up_from":0,"seq":34359738541,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411490,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":445552,"kb_used_data":140136,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93299600,"statfs":{"total":95995035648,"available":95538790400,"internally_reserved":0,"allocated":143499264,"data_stored":142710200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:11 2024","interfaces":[{"interface":"back","average":{"1min":2.8220000000000001,"5min":2.5899999999999999,"15min":1.5489999999999999},"min":{"1min":0.19900000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":16.137,"5min":22.338999999999999,"15min":22.338999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.9220000000000002,"15min":1.804},"min":{"1min":0.23899999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":12.438000000000001,"5min":30.427,"15min":30.427},"last":1.4399999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:40:11 2024","interfaces":[{"interface":"back","average":{"1min":1.865,"5min":2.0790000000000002,"15min":1.446},"min":{"1min":0.21299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":12.349,"5min":12.349,"15min":12.349},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.7849999999999999,"5min":2.1539999999999999,"15min":1.3500000000000001},"min":{"1min":0.25700000000000001,"5min":0.22900000000000001,"15min":0.193},"max":{"1min":10.396000000000001,"5min":13.782,"15min":13.782},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:39:48 2024","interfaces":[{"interface":"back","average":{"1min":3.589,"5min":2.6269999999999998,"15min":1.722},"min":{"1min":0.307,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":16.076000000000001,"5min":22.302,"15min":30.071000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":3.79,"5min":2.7829999999999999,"15min":1.865},"min":{"1min":0.29399999999999998,"5min":0.184,"15min":0.184},"max":{"1min":11.768000000000001,"5min":30.282,"15min":32.119},"last":1.8460000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:40:01 2024","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5009999999999999,"15min":1.196},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":12.43,"5min":12.43,"15min":12.43},"last":1.0029999999999999},{"interface":"front","average":{"1min":2.5070000000000001,"5min":1.776,"15min":1.0980000000000001},"min":{"1min":0.42399999999999999,"5min":0.28000000000000003,"15min":0.253},"max":{"1min":10.34,"5min":10.34,"15min":10.34},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:39:18 2024","interfaces":[{"interface":"back","average":{"1min":3.3149999999999999,"5min":1.7709999999999999,"15min":1.0940000000000001},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":10.300000000000001,"5min":10.300000000000001,"15min":10.300000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":2.774,"5min":1.7629999999999999,"15min":1.23},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":12.391999999999999,"5min":12.391999999999999,"15min":12.391999999999999},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:39:56 2024","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":1.76,"15min":1.294},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":21.640000000000001,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":2.1600000000000001,"5min":1.6519999999999999,"15min":1.103},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":10.369999999999999,"5min":10.369999999999999,"15min":10.369999999999999},"last":0.88300000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:40:11 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.635,"15min":1.29},"min":{"1min":0.41799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":12.484,"5min":12.484,"15min":12.484},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.625,"5min":1.913,"15min":1.198},"min":{"1min":0.496,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":10.539,"5min":10.539,"15min":10.539},"last":0.95499999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149852,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":586252,"kb_used_data":276676,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93158900,"statfs":{"total":95995035648,"available":95394713600,"internally_reserved":0,"allocated":283316224,"data_stored":282500182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:39 2024","interfaces":[{"interface":"back","average":{"1min":3.0339999999999998,"5min":2.3109999999999999,"15min":1.573},"min":{"1min":0.21199999999999999,"5min":0.19400000000000001,"15min":0.193},"max":{"1min":8.4339999999999993,"5min":23.48,"15min":23.48},"last":1.0700000000000001},{"interface":"front","average":{"1min":2.919,"5min":2.6640000000000001,"15min":1.8069999999999999},"min":{"1min":0.23599999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.352,"5min":23.411999999999999,"15min":23.411999999999999},"last":1.7010000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:39:39 2024","interfaces":[{"interface":"back","average":{"1min":4.7839999999999998,"5min":3.2869999999999999,"15min":2.0499999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":23.077999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":1.7849999999999999},{"interface":"front","average":{"1min":4.2789999999999999,"5min":2.8260000000000001,"15min":1.6970000000000001},"min":{"1min":0.23499999999999999,"5min":0.214,"15min":0.214},"max":{"1min":22.187000000000001,"5min":22.187000000000001,"15min":22.187000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:39:39 2024","interfaces":[{"interface":"back","average":{"1min":3.8690000000000002,"5min":2.4100000000000001,"15min":1.7430000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.27400000000000002},"max":{"1min":13.532,"5min":13.532,"15min":16.271000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":5.0819999999999999,"5min":2.5720000000000001,"15min":1.7230000000000001},"min":{"1min":0.29999999999999999,"5min":0.245,"15min":0.217},"max":{"1min":35.939999999999998,"5min":35.939999999999998,"15min":35.939999999999998},"last":1.038}]},{"osd":4,"last update":"Thu Feb 22 19:39:58 2024","interfaces":[{"interface":"back","average":{"1min":2.827,"5min":2.165,"15min":1.5109999999999999},"min":{"1min":0.41799999999999998,"5min":0.26300000000000001,"15min":0.249},"max":{"1min":12.416,"5min":12.416,"15min":13.680999999999999},"last":1.621},{"interface":"front","average":{"1min":2.133,"5min":2.1389999999999998,"15min":1.4219999999999999},"min":{"1min":0.38200000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.273,"5min":13.548,"15min":13.548},"last":0.39600000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:39:27 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":1.6919999999999999,"15min":1.2549999999999999},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":7.9779999999999998,"5min":8.0289999999999999,"15min":12.879},"last":1.0860000000000001},{"interface":"front","average":{"1min":3.4079999999999999,"5min":2.1120000000000001,"15min":1.431},"min":{"1min":0.39700000000000002,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":10.446999999999999,"5min":11.167999999999999,"15min":13.707000000000001},"last":1.6779999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:39:50 2024","interfaces":[{"interface":"back","average":{"1min":1.9610000000000001,"5min":1.5289999999999999,"15min":1.234},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":5.452,"5min":7.9640000000000004,"15min":12.898999999999999},"last":7.0449999999999999},{"interface":"front","average":{"1min":3.0329999999999999,"5min":2.0859999999999999,"15min":1.532},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.30199999999999999},"max":{"1min":12.396000000000001,"5min":12.396000000000001,"15min":13.803000000000001},"last":1.655}]},{"osd":7,"last update":"Thu Feb 22 19:40:04 2024","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":1.409},"min":{"1min":0.435,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":12.327,"5min":13.483000000000001,"15min":13.483000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.716,"5min":1.6339999999999999,"15min":1.2889999999999999},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":5.7300000000000004,"5min":7.9909999999999997,"15min":12.846},"last":7.093}]}]},{"osd":4,"up_from":0,"seq":137438953626,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724692,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":603248,"kb_used_data":293736,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93141904,"statfs":{"total":95995035648,"available":95377309696,"internally_reserved":0,"allocated":300785664,"data_stored":299996802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":2.3490000000000002,"5min":1.6879999999999999,"15min":1.147},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.222},"max":{"1min":17.684000000000001,"5min":17.684000000000001,"15min":17.684000000000001},"last":1.216},{"interface":"front","average":{"1min":2.1789999999999998,"5min":1.7829999999999999,"15min":1.2130000000000001},"min":{"1min":0.252,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":17.402999999999999,"5min":17.402999999999999,"15min":17.402999999999999},"last":1.8180000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.827,"15min":1.202},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.22500000000000001},"max":{"1min":11.202,"5min":29.390999999999998,"15min":29.390999999999998},"last":1.762},{"interface":"front","average":{"1min":1.7549999999999999,"5min":1.6200000000000001,"15min":1.173},"min":{"1min":0.30599999999999999,"5min":0.26400000000000001,"15min":0.218},"max":{"1min":8.7490000000000006,"5min":29.367000000000001,"15min":29.367000000000001},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":3.3069999999999999,"5min":1.8069999999999999,"15min":1.218},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":11.367000000000001,"5min":11.367000000000001,"15min":14.613},"last":1.855},{"interface":"front","average":{"1min":3.2989999999999999,"5min":1.847,"15min":1.284},"min":{"1min":0.315,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":16.099,"5min":16.099,"15min":16.099},"last":1.74}]},{"osd":3,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":1.766,"5min":1.79,"15min":1.361},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.5780000000000003,"5min":9.6029999999999998,"15min":13.366},"last":1.901},{"interface":"front","average":{"1min":2.0409999999999999,"5min":1.73,"15min":1.3440000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.28299999999999997},"max":{"1min":8.6069999999999993,"5min":9.4649999999999999,"15min":10.16},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Feb 22 19:39:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65400000000000003,"15min":0.501},"min":{"1min":0.223,"5min":0.223,"15min":0.219},"max":{"1min":2.403,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.67300000000000004,"15min":0.51100000000000001},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":1.998}]},{"osd":6,"last update":"Thu Feb 22 19:39:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.58399999999999996,"15min":0.51700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.76600000000000001,"5min":3.246,"15min":5.6040000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.72899999999999998,"15min":0.58699999999999997},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.309},"max":{"1min":3.0590000000000002,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":2.113}]},{"osd":7,"last update":"Thu Feb 22 19:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61299999999999999,"15min":0.63100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.6420000000000003,"5min":4.6420000000000003,"15min":5.5789999999999997},"last":0.496},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.59599999999999997},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":3.2829999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":414973952,"data_stored":414973952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167641088,"data_stored":167641088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":391380992,"data_stored":391380992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":404750336,"data_stored":404750336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196608000,"data_stored":196608000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":390201344,"data_stored":390201344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:40:19.848 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set default.rgw.log pgp_num 32 2024-02-22T19:40:20.074 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:20.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:20.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:20.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:20.203 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 9 16 24241 24225 168.198 133.438 0.00535625 0.00593912 2024-02-22T19:40:20.221 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:20.256 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:20.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:20.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:20.396 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:20.398 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:20.400 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:20.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:20.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:20.459 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:40:20.459 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:40:20.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:20.584 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:20.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:19.560048+0000 mgr.y (mgr.14535) 435 : cluster [DBG] pgmap v558: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.43k op/s; 1/58812 objects misplaced (0.002%) 2024-02-22T19:40:20.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:19.560048+0000 mgr.y (mgr.14535) 435 : cluster [DBG] pgmap v558: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.43k op/s; 1/58812 objects misplaced (0.002%) 2024-02-22T19:40:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:19.560048+0000 mgr.y (mgr.14535) 435 : cluster [DBG] pgmap v558: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.43k op/s; 1/58812 objects misplaced (0.002%) 2024-02-22T19:40:21.202 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 10 16 26270 26254 164.028 126.812 0.00849108 0.00609015 2024-02-22T19:40:22.203 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 11 16 28479 28463 161.654 138.062 0.00657109 0.00618012 2024-02-22T19:40:22.569 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:22.571 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:22.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:22.588 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:22.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:40:22.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:22.755 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:23.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:21.560570+0000 mgr.y (mgr.14535) 436 : cluster [DBG] pgmap v559: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 154 MiB/s wr, 2.46k op/s; 1/68274 objects misplaced (0.001%) 2024-02-22T19:40:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:21.560570+0000 mgr.y (mgr.14535) 436 : cluster [DBG] pgmap v559: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 154 MiB/s wr, 2.46k op/s; 1/68274 objects misplaced (0.001%) 2024-02-22T19:40:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:21.560570+0000 mgr.y (mgr.14535) 436 : cluster [DBG] pgmap v559: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 154 MiB/s wr, 2.46k op/s; 1/68274 objects misplaced (0.001%) 2024-02-22T19:40:23.204 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 12 16 30296 30280 157.647 113.562 0.00791842 0.00633874 2024-02-22T19:40:24.204 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 13 15 32332 32317 155.313 127.312 0.0108265 0.006433 2024-02-22T19:40:24.861 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:24.893 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:40:24.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:23.090274+0000 osd.4 (osd.4) 149 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:24.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:23.090363+0000 osd.4 (osd.4) 150 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:24.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:23.560969+0000 mgr.y (mgr.14535) 437 : cluster [DBG] pgmap v560: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 1023 B/s rd, 141 MiB/s wr, 2.26k op/s; 1/68274 objects misplaced (0.001%) 2024-02-22T19:40:24.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:23.641447+0000 mon.a (mon.0) 1281 : cluster [DBG] osdmap e293: 8 total, 8 up, 6 in 2024-02-22T19:40:24.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:24.198584+0000 osd.7 (osd.7) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:24.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:24.198634+0000 osd.7 (osd.7) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:24.934 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:24.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:23.090274+0000 osd.4 (osd.4) 149 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:24.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:23.090363+0000 osd.4 (osd.4) 150 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:23.560969+0000 mgr.y (mgr.14535) 437 : cluster [DBG] pgmap v560: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 1023 B/s rd, 141 MiB/s wr, 2.26k op/s; 1/68274 objects misplaced (0.001%) 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:23.641447+0000 mon.a (mon.0) 1281 : cluster [DBG] osdmap e293: 8 total, 8 up, 6 in 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:24.198584+0000 osd.7 (osd.7) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:24.198634+0000 osd.7 (osd.7) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:23.090274+0000 osd.4 (osd.4) 149 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:23.090363+0000 osd.4 (osd.4) 150 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:23.560969+0000 mgr.y (mgr.14535) 437 : cluster [DBG] pgmap v560: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 1023 B/s rd, 141 MiB/s wr, 2.26k op/s; 1/68274 objects misplaced (0.001%) 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:23.641447+0000 mon.a (mon.0) 1281 : cluster [DBG] osdmap e293: 8 total, 8 up, 6 in 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:24.198584+0000 osd.7 (osd.7) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:24.935 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:24.198634+0000 osd.7 (osd.7) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:24.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:25.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:25.079 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:25.108 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:25.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:25.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:25.204 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 14 16 34176 34160 152.447 115.188 0.0062476 0.00655471 2024-02-22T19:40:25.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:25.269 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:25.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:40:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:40:26.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:24.654953+0000 mon.a (mon.0) 1282 : cluster [DBG] osdmap e294: 8 total, 8 up, 6 in 2024-02-22T19:40:26.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:24.893190+0000 mgr.y (mgr.14535) 438 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:26.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:24.654953+0000 mon.a (mon.0) 1282 : cluster [DBG] osdmap e294: 8 total, 8 up, 6 in 2024-02-22T19:40:26.009 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:24.893190+0000 mgr.y (mgr.14535) 438 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:24.654953+0000 mon.a (mon.0) 1282 : cluster [DBG] osdmap e294: 8 total, 8 up, 6 in 2024-02-22T19:40:26.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:24.893190+0000 mgr.y (mgr.14535) 438 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:26.204 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 15 16 35865 35849 149.322 105.562 0.00476886 0.00669149 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:25.561373+0000 mgr.y (mgr.14535) 439 : cluster [DBG] pgmap v563: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 177 MiB/s wr, 2.84k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:25.651924+0000 mon.a (mon.0) 1283 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:26.001366+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:26.433003+0000 mon.a (mon.0) 1285 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:26.434330+0000 mon.a (mon.0) 1286 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:26.435564+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:26.440833+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:26.450365+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:27.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:25.561373+0000 mgr.y (mgr.14535) 439 : cluster [DBG] pgmap v563: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 177 MiB/s wr, 2.84k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:25.651924+0000 mon.a (mon.0) 1283 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:26.001366+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:26.433003+0000 mon.a (mon.0) 1285 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:26.434330+0000 mon.a (mon.0) 1286 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:26.435564+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:26.440833+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:26.450365+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:25.561373+0000 mgr.y (mgr.14535) 439 : cluster [DBG] pgmap v563: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 177 MiB/s wr, 2.84k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:25.651924+0000 mon.a (mon.0) 1283 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:26.001366+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:26.433003+0000 mon.a (mon.0) 1285 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:26.434330+0000 mon.a (mon.0) 1286 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:26.435564+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:26.440833+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-22T19:40:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:26.450365+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:27.207 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 16 16 37630 37614 146.86 110.312 0.0053891 0.00680352 2024-02-22T19:40:28.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cephadm 2024-02-22T19:40:26.441908+0000 mgr.y (mgr.14535) 440 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 3256M 2024-02-22T19:40:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cephadm 2024-02-22T19:40:26.441908+0000 mgr.y (mgr.14535) 440 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 3256M 2024-02-22T19:40:28.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cephadm 2024-02-22T19:40:26.441908+0000 mgr.y (mgr.14535) 440 : cephadm [INF] Adjusting osd_memory_target on smithi064 to 3256M 2024-02-22T19:40:28.206 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 17 16 39367 39351 144.607 108.562 0.00315253 0.00689464 2024-02-22T19:40:28.334 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:28.338 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:28.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:28.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:28.496 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:28.527 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:28.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:28.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:28.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:28.829 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:28.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:28.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:27.561773+0000 mgr.y (mgr.14535) 441 : cluster [DBG] pgmap v564: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 104 MiB/s wr, 1.66k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:27.913093+0000 osd.3 (osd.3) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:27.913152+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:28.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:28.985 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:27.561773+0000 mgr.y (mgr.14535) 441 : cluster [DBG] pgmap v564: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 104 MiB/s wr, 1.66k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:28.985 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:27.913093+0000 osd.3 (osd.3) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:28.985 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:27.913152+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:28.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:27.561773+0000 mgr.y (mgr.14535) 441 : cluster [DBG] pgmap v564: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 104 MiB/s wr, 1.66k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:28.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:27.913093+0000 osd.3 (osd.3) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:28.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:27.913152+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:28.991 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:29.207 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 18 16 40991 40975 142.212 101.5 0.00814789 0.00702698 2024-02-22T19:40:29.218 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:29.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:29.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:29.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:29.341 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:40:29.341 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:40:29.341 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:40:29.341 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:40:29.341 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:40:29.374 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:29.374 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:40:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:29.409 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:40:29.411 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:29.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:29.446 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:40:29.459 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.459 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.459 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.459 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.460 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.460 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.460 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.460 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:40:29.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:29] "GET /metrics HTTP/1.1" 200 176099 "" "Prometheus/2.33.4" 2024-02-22T19:40:29.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:29.593 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:29.599 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:29.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:29.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:29.753 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:29.781 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:29.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:29.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:29.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:28.675277+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e295: 8 total, 8 up, 6 in 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:28.991329+0000 mon.a (mon.0) 1291 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:28.995011+0000 mon.a (mon.0) 1292 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:29.001513+0000 mon.a (mon.0) 1293 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:29.141395+0000 mon.b (mon.1) 70 : audit [DBG] from='client.? 172.21.15.53:0/1992808013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:28.675277+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e295: 8 total, 8 up, 6 in 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:28.991329+0000 mon.a (mon.0) 1291 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:28.995011+0000 mon.a (mon.0) 1292 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:29.001513+0000 mon.a (mon.0) 1293 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:29.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:29.141395+0000 mon.b (mon.1) 70 : audit [DBG] from='client.? 172.21.15.53:0/1992808013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:29.936 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:30.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:30.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:28.675277+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e295: 8 total, 8 up, 6 in 2024-02-22T19:40:30.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:28.991329+0000 mon.a (mon.0) 1291 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:40:30.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:28.995011+0000 mon.a (mon.0) 1292 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:40:30.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:29.001513+0000 mon.a (mon.0) 1293 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:30.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:29.141395+0000 mon.b (mon.1) 70 : audit [DBG] from='client.? 172.21.15.53:0/1992808013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:30.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:30.210 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 19 16 42521 42505 139.737 95.625 0.0118091 0.00715194 2024-02-22T19:40:30.243 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:31.046 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:40:31.210 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:40:31.211070+0000 min lat: 0.00135644 max lat: 0.0825774 avg lat: 0.00724597 2024-02-22T19:40:31.210 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:40:31.210 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 20 16 44170 44154 137.904 103.062 0.00534556 0.00724597 2024-02-22T19:40:31.262 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:29.562718+0000 mgr.y (mgr.14535) 442 : cluster [DBG] pgmap v566: 148 pgs: 148 active+clean; 2.4 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 1.7 KiB/s rd, 173 MiB/s wr, 2.77k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:31.262 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:30.280560+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? 172.21.15.53:0/2979012609' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:40:31.262 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:29.562718+0000 mgr.y (mgr.14535) 442 : cluster [DBG] pgmap v566: 148 pgs: 148 active+clean; 2.4 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 1.7 KiB/s rd, 173 MiB/s wr, 2.77k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:31.262 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:30.280560+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? 172.21.15.53:0/2979012609' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:40:31.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:29.562718+0000 mgr.y (mgr.14535) 442 : cluster [DBG] pgmap v566: 148 pgs: 148 active+clean; 2.4 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 1.7 KiB/s rd, 173 MiB/s wr, 2.77k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:40:31.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:30.280560+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? 172.21.15.53:0/2979012609' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:40:31.609 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:31.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:31.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:31.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:31.785 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:32.040 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 4 pgp_num to 32 2024-02-22T19:40:32.212 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 21 16 45706 45690 135.9 96 0.00705931 0.00735359 2024-02-22T19:40:32.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.003558+0000 mon.a (mon.0) 1295 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:32.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:31.018804+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.53:0/2979012609' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:40:32.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.018849+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e296: 8 total, 8 up, 6 in 2024-02-22T19:40:32.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:31.963711+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? 172.21.15.53:0/3035733489' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.003558+0000 mon.a (mon.0) 1295 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:31.018804+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.53:0/2979012609' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.018849+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e296: 8 total, 8 up, 6 in 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:31.963711+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? 172.21.15.53:0/3035733489' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.003558+0000 mon.a (mon.0) 1295 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:31.018804+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.53:0/2979012609' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:40:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.018849+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e296: 8 total, 8 up, 6 in 2024-02-22T19:40:32.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:31.963711+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? 172.21.15.53:0/3035733489' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-22T19:40:33.052 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.499465+0000 osd.6 (osd.6) 132 : cluster [DBG] 4.a scrub starts 2024-02-22T19:40:33.052 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.504357+0000 osd.6 (osd.6) 133 : cluster [DBG] 4.a scrub ok 2024-02-22T19:40:33.052 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.563173+0000 mgr.y (mgr.14535) 443 : cluster [DBG] pgmap v568: 148 pgs: 148 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 740 B/s rd, 78 MiB/s wr, 1.24k op/s 2024-02-22T19:40:33.052 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.630322+0000 osd.7 (osd.7) 183 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:40:33.052 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:31.663748+0000 osd.7 (osd.7) 184 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:40:33.052 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:32.028175+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? 172.21.15.53:0/3035733489' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]': finished 2024-02-22T19:40:33.213 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 22 16 47342 47326 134.368 102.25 0.00685563 0.00743758 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.499465+0000 osd.6 (osd.6) 132 : cluster [DBG] 4.a scrub starts 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.504357+0000 osd.6 (osd.6) 133 : cluster [DBG] 4.a scrub ok 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.563173+0000 mgr.y (mgr.14535) 443 : cluster [DBG] pgmap v568: 148 pgs: 148 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 740 B/s rd, 78 MiB/s wr, 1.24k op/s 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.630322+0000 osd.7 (osd.7) 183 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:31.663748+0000 osd.7 (osd.7) 184 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:32.028175+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? 172.21.15.53:0/3035733489' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]': finished 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:32.028245+0000 mon.a (mon.0) 1300 : cluster [DBG] osdmap e297: 8 total, 8 up, 6 in 2024-02-22T19:40:33.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:32.254763+0000 mon.a (mon.0) 1301 : cluster [DBG] osdmap e298: 8 total, 8 up, 6 in 2024-02-22T19:40:33.279 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:32.028245+0000 mon.a (mon.0) 1300 : cluster [DBG] osdmap e297: 8 total, 8 up, 6 in 2024-02-22T19:40:33.279 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:32.254763+0000 mon.a (mon.0) 1301 : cluster [DBG] osdmap e298: 8 total, 8 up, 6 in 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.499465+0000 osd.6 (osd.6) 132 : cluster [DBG] 4.a scrub starts 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.504357+0000 osd.6 (osd.6) 133 : cluster [DBG] 4.a scrub ok 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.563173+0000 mgr.y (mgr.14535) 443 : cluster [DBG] pgmap v568: 148 pgs: 148 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 740 B/s rd, 78 MiB/s wr, 1.24k op/s 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.630322+0000 osd.7 (osd.7) 183 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:31.663748+0000 osd.7 (osd.7) 184 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:32.028175+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? 172.21.15.53:0/3035733489' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]': finished 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:32.028245+0000 mon.a (mon.0) 1300 : cluster [DBG] osdmap e297: 8 total, 8 up, 6 in 2024-02-22T19:40:33.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:32.254763+0000 mon.a (mon.0) 1301 : cluster [DBG] osdmap e298: 8 total, 8 up, 6 in 2024-02-22T19:40:33.881 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:33.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:33.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:33.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:34.103 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:34.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:34.110 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:34.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:34.216 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 23 16 48885 48869 132.719 96.4375 0.00946613 0.00753097 2024-02-22T19:40:34.220 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:32.662980+0000 osd.3 (osd.3) 181 : cluster [DBG] 6.c scrub starts 2024-02-22T19:40:34.220 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:32.668400+0000 osd.3 (osd.3) 182 : cluster [DBG] 6.c scrub ok 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:33.041957+0000 osd.2 (osd.2) 154 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:33.048426+0000 osd.2 (osd.2) 155 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:33.995231+0000 osd.2 (osd.2) 156 : cluster [DBG] 6.f scrub starts 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:32.662980+0000 osd.3 (osd.3) 181 : cluster [DBG] 6.c scrub starts 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:32.668400+0000 osd.3 (osd.3) 182 : cluster [DBG] 6.c scrub ok 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:33.041957+0000 osd.2 (osd.2) 154 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:33.048426+0000 osd.2 (osd.2) 155 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:40:34.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:33.995231+0000 osd.2 (osd.2) 156 : cluster [DBG] 6.f scrub starts 2024-02-22T19:40:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:34.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:40:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:32.662980+0000 osd.3 (osd.3) 181 : cluster [DBG] 6.c scrub starts 2024-02-22T19:40:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:32.668400+0000 osd.3 (osd.3) 182 : cluster [DBG] 6.c scrub ok 2024-02-22T19:40:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:33.041957+0000 osd.2 (osd.2) 154 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:40:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:33.048426+0000 osd.2 (osd.2) 155 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:40:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:33.995231+0000 osd.2 (osd.2) 156 : cluster [DBG] 6.f scrub starts 2024-02-22T19:40:34.728 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:34.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:34.752 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:34.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:34.906 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:34.932 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:34.935 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:40:34.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:40:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:40:34.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:34.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:35.004 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:35.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:35.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:40:35.112 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:35.212 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 24 15 50270 50255 130.799 86.625 0.00728878 0.00764077 2024-02-22T19:40:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:33.563904+0000 mgr.y (mgr.14535) 444 : cluster [DBG] pgmap v571: 148 pgs: 148 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 1.0 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-02-22T19:40:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:33.636342+0000 osd.3 (osd.3) 183 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:40:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:33.643331+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:40:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.092012+0000 osd.2 (osd.2) 157 : cluster [DBG] 6.f scrub ok 2024-02-22T19:40:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.368386+0000 osd.5 (osd.5) 80 : cluster [DBG] 5.e scrub starts 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.372224+0000 osd.5 (osd.5) 81 : cluster [DBG] 5.e scrub ok 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:33.563904+0000 mgr.y (mgr.14535) 444 : cluster [DBG] pgmap v571: 148 pgs: 148 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 1.0 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:33.636342+0000 osd.3 (osd.3) 183 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:33.643331+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.092012+0000 osd.2 (osd.2) 157 : cluster [DBG] 6.f scrub ok 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.368386+0000 osd.5 (osd.5) 80 : cluster [DBG] 5.e scrub starts 2024-02-22T19:40:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.372224+0000 osd.5 (osd.5) 81 : cluster [DBG] 5.e scrub ok 2024-02-22T19:40:35.285 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:35.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:35.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:35.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:33.563904+0000 mgr.y (mgr.14535) 444 : cluster [DBG] pgmap v571: 148 pgs: 148 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 1.0 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-02-22T19:40:35.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:33.636342+0000 osd.3 (osd.3) 183 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:40:35.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:33.643331+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:40:35.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.092012+0000 osd.2 (osd.2) 157 : cluster [DBG] 6.f scrub ok 2024-02-22T19:40:35.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.368386+0000 osd.5 (osd.5) 80 : cluster [DBG] 5.e scrub starts 2024-02-22T19:40:35.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.372224+0000 osd.5 (osd.5) 81 : cluster [DBG] 5.e scrub ok 2024-02-22T19:40:35.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:35.455 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:35.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:35.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15324.0:50609 9.c 9:33423209:::benchmark_data_smithi053_157429_object50608:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e298)", 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:40:35.483444+0000", 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0059541899999999998, 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059910550000000003, 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15324", 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:40:35.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50609 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483444+0000", 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483444+0000", 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483443+0000", 2024-02-22T19:40:35.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4388000000000001e-05 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483467+0000", 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.45e-05 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483482+0000", 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1339999999999998e-06 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483489+0000", 2024-02-22T19:40:35.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101828 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483591+0000", 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012960599999999999 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483720+0000", 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015621999999999999 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6,7", 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.483876+0000", 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042523029999999998 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488129+0000", 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046843880000000003 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15324.0:50607 9.f e298/285 9:f0510a3c:::benchmark_data_smithi053_157429_object50606:head v 298'3267, mlcod=298'3264)", 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:40:35.485764+0000", 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0036340629999999999, 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037643949999999998, 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:35.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485764+0000", 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485764+0000", 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485763+0000", 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7292000000000001e-05 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485801+0000", 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0356e-05 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485811+0000", 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485817+0000", 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137682 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485955+0000", 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9848e-05 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.485975+0000", 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021052000000000001 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15324.0:50621 9.8 9:1c4aeb92:::benchmark_data_smithi053_157429_object50620:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e298)", 2024-02-22T19:40:35.497 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:40:35.487982+0000", 2024-02-22T19:40:35.499 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.001416077, 2024-02-22T19:40:35.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00161568, 2024-02-22T19:40:35.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:35.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:40:35.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15324", 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50621 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.487982+0000", 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.487982+0000", 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.487981+0000", 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5502e-05 2024-02-22T19:40:35.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:35.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488016+0000", 2024-02-22T19:40:35.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1867e-05 2024-02-22T19:40:35.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:35.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488028+0000", 2024-02-22T19:40:35.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5950000000000004e-06 2024-02-22T19:40:35.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488035+0000", 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1768999999999998e-05 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488097+0000", 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136699 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488233+0000", 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015599200000000001 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5,7", 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488389+0000", 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040677999999999998 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15324.0:50622 9.8 9.8a72d488 (undecoded) ondisk+write+known_if_redirected e298)", 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:40:35.488051+0000", 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0013471889999999999, 2024-02-22T19:40:35.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016219610000000001, 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15324", 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50622 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488051+0000", 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488051+0000", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488051+0000", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1107999999999998e-05 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488072+0000", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.861e-06 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488079+0000", 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8859999999999999e-06 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488088+0000", 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15324.0:50619 9.4 e298/285 9:2601cbf8:::benchmark_data_smithi053_157429_object50618:head v 298'3220, mlcod=298'3218)", 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:40:35.488197+0000", 2024-02-22T19:40:35.508 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.001201854, 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001536649, 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488197+0000", 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488197+0000", 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:35.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488196+0000", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6875000000000002e-05 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488233+0000", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661e-05 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488243+0000", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3149999999999998e-06 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488253+0000", 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9826000000000001e-05 2024-02-22T19:40:35.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488293+0000", 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0111000000000001e-05 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:40:35.488313+0000", 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000116011 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 5 2024-02-22T19:40:35.511 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:40:35.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:36.212 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 25 16 51682 51666 129.095 88.1875 0.00632723 0.0077422 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.601383+0000 osd.7 (osd.7) 185 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.605230+0000 osd.7 (osd.7) 186 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.656254+0000 osd.3 (osd.3) 185 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:34.661037+0000 osd.3 (osd.3) 186 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:34.904339+0000 mgr.y (mgr.14535) 445 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:35.032954+0000 osd.2 (osd.2) 158 : cluster [DBG] 4.d scrub starts 2024-02-22T19:40:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:35.038158+0000 osd.2 (osd.2) 159 : cluster [DBG] 4.d scrub ok 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.601383+0000 osd.7 (osd.7) 185 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.605230+0000 osd.7 (osd.7) 186 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.656254+0000 osd.3 (osd.3) 185 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:34.661037+0000 osd.3 (osd.3) 186 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:34.904339+0000 mgr.y (mgr.14535) 445 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:35.032954+0000 osd.2 (osd.2) 158 : cluster [DBG] 4.d scrub starts 2024-02-22T19:40:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:35.038158+0000 osd.2 (osd.2) 159 : cluster [DBG] 4.d scrub ok 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.601383+0000 osd.7 (osd.7) 185 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.605230+0000 osd.7 (osd.7) 186 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.656254+0000 osd.3 (osd.3) 185 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:34.661037+0000 osd.3 (osd.3) 186 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:34.904339+0000 mgr.y (mgr.14535) 445 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:35.032954+0000 osd.2 (osd.2) 158 : cluster [DBG] 4.d scrub starts 2024-02-22T19:40:36.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:35.038158+0000 osd.2 (osd.2) 159 : cluster [DBG] 4.d scrub ok 2024-02-22T19:40:37.189 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:37.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:37.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:37.213 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 26 16 53190 53174 127.753 94.25 0.00673214 0.00782252 2024-02-22T19:40:37.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:40:37.294 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:37.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:35.564724+0000 mgr.y (mgr.14535) 446 : cluster [DBG] pgmap v572: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 853 B/s rd, 104 MiB/s wr, 1.66k op/s 2024-02-22T19:40:37.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:35.680623+0000 osd.3 (osd.3) 187 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:40:37.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:35.695647+0000 osd.3 (osd.3) 188 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:40:37.341 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:35.564724+0000 mgr.y (mgr.14535) 446 : cluster [DBG] pgmap v572: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 853 B/s rd, 104 MiB/s wr, 1.66k op/s 2024-02-22T19:40:37.341 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:35.680623+0000 osd.3 (osd.3) 187 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:40:37.341 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:35.695647+0000 osd.3 (osd.3) 188 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:40:37.346 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:35.564724+0000 mgr.y (mgr.14535) 446 : cluster [DBG] pgmap v572: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 853 B/s rd, 104 MiB/s wr, 1.66k op/s 2024-02-22T19:40:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:35.680623+0000 osd.3 (osd.3) 187 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:40:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:35.695647+0000 osd.3 (osd.3) 188 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:40:37.461 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:37.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:37.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:37.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:37.634 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:37.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:37.673 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:37.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:37.844 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:37.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:37.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:37.871 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:37.882 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:40:37.928 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:40:37.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:37.966 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:40:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:40:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:38.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:38.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:38.023 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:40:38.060 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:40:38.090 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:36.427913+0000 osd.5 (osd.5) 82 : cluster [DBG] 1.a scrub starts 2024-02-22T19:40:38.090 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:36.433277+0000 osd.5 (osd.5) 83 : cluster [DBG] 1.a scrub ok 2024-02-22T19:40:38.090 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:36.490051+0000 osd.1 (osd.1) 168 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:40:38.091 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:40:38.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:38.120 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.132 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:40:38.168 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:38.213 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 27 16 54350 54334 125.707 72.5 0.00270329 0.00794824 2024-02-22T19:40:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:36.491180+0000 osd.1 (osd.1) 169 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:40:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:36.672218+0000 osd.3 (osd.3) 189 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:40:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:36.677469+0000 osd.3 (osd.3) 190 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:40:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:37.829319+0000 mon.c (mon.2) 57 : audit [DBG] from='client.? 172.21.15.53:0/2739593152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:38.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:36.427913+0000 osd.5 (osd.5) 82 : cluster [DBG] 1.a scrub starts 2024-02-22T19:40:38.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:36.433277+0000 osd.5 (osd.5) 83 : cluster [DBG] 1.a scrub ok 2024-02-22T19:40:38.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:36.490051+0000 osd.1 (osd.1) 168 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:40:38.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:36.491180+0000 osd.1 (osd.1) 169 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:36.672218+0000 osd.3 (osd.3) 189 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:36.677469+0000 osd.3 (osd.3) 190 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:37.829319+0000 mon.c (mon.2) 57 : audit [DBG] from='client.? 172.21.15.53:0/2739593152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:36.427913+0000 osd.5 (osd.5) 82 : cluster [DBG] 1.a scrub starts 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:36.433277+0000 osd.5 (osd.5) 83 : cluster [DBG] 1.a scrub ok 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:36.490051+0000 osd.1 (osd.1) 168 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:36.491180+0000 osd.1 (osd.1) 169 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:36.672218+0000 osd.3 (osd.3) 189 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:36.677469+0000 osd.3 (osd.3) 190 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:40:38.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:37.829319+0000 mon.c (mon.2) 57 : audit [DBG] from='client.? 172.21.15.53:0/2739593152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:38.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:38.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:38.955 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:39.004 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:39.039 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:40:39.040 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:40:39.040 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-02-22T19:40:39.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-22T19:40:39.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:39.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:39.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:39.213 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 28 16 55784 55768 124.419 89.625 0.00582277 0.00803224 2024-02-22T19:40:39.232 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:39.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:39.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:37.568028+0000 mgr.y (mgr.14535) 447 : cluster [DBG] pgmap v573: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 781 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:37.711069+0000 osd.3 (osd.3) 191 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:37.721401+0000 osd.3 (osd.3) 192 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:38.367376+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:38.911308+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:37.568028+0000 mgr.y (mgr.14535) 447 : cluster [DBG] pgmap v573: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 781 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:37.711069+0000 osd.3 (osd.3) 191 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:37.721401+0000 osd.3 (osd.3) 192 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:38.367376+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:39.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:38.911308+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:39.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:37.568028+0000 mgr.y (mgr.14535) 447 : cluster [DBG] pgmap v573: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 781 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:40:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:37.711069+0000 osd.3 (osd.3) 191 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:40:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:37.721401+0000 osd.3 (osd.3) 192 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:40:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:38.367376+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:38.911308+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:39.384 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:39.413 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:39.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:39.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:39.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:39.562 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:39.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:39.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:39.648 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:40:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:39] "GET /metrics HTTP/1.1" 200 176108 "" "Prometheus/2.33.4" 2024-02-22T19:40:39.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:40.217 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 29 16 57120 57104 122.992 83.5 0.00857974 0.00812594 2024-02-22T19:40:40.239 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:40.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:40.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:40.244 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:38.665022+0000 osd.3 (osd.3) 193 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:40:40.244 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:38.671685+0000 osd.3 (osd.3) 194 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:40:40.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:38.665022+0000 osd.3 (osd.3) 193 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:40:40.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:38.671685+0000 osd.3 (osd.3) 194 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:40:40.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:40.413 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:38.665022+0000 osd.3 (osd.3) 193 : cluster [DBG] 5.14 scrub starts 2024-02-22T19:40:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:38.671685+0000 osd.3 (osd.3) 194 : cluster [DBG] 5.14 scrub ok 2024-02-22T19:40:40.680 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:40.686 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:40.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:40.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:40.871 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:40.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:40.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:40.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:41.024 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:41.060 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:41.063 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:41.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:41.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:41.217 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 30 16 58472 58456 121.709 84.5 0.0112798 0.00821238 2024-02-22T19:40:41.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:39.381220+0000 osd.1 (osd.1) 170 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:40:41.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:39.441470+0000 osd.1 (osd.1) 171 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:40:41.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:39.568721+0000 mgr.y (mgr.14535) 448 : cluster [DBG] pgmap v574: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-02-22T19:40:41.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:39.677554+0000 osd.3 (osd.3) 195 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:40:41.221 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:39.681779+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:40:41.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:39.381220+0000 osd.1 (osd.1) 170 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:40:41.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:39.441470+0000 osd.1 (osd.1) 171 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:40:41.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:39.568721+0000 mgr.y (mgr.14535) 448 : cluster [DBG] pgmap v574: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-02-22T19:40:41.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:39.677554+0000 osd.3 (osd.3) 195 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:40:41.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:39.681779+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:40:41.227 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:41.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:39.381220+0000 osd.1 (osd.1) 170 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:40:41.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:39.441470+0000 osd.1 (osd.1) 171 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:40:41.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:39.568721+0000 mgr.y (mgr.14535) 448 : cluster [DBG] pgmap v574: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-02-22T19:40:41.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:39.677554+0000 osd.3 (osd.3) 195 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:40:41.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:39.681779+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:40:41.650 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:41.916 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:41.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:41.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:42.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:42.065 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:42.098 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:42.107 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:42.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:42.141 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:40:42.186 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:40.418045+0000 osd.1 (osd.1) 172 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:40:42.186 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:40.450452+0000 osd.1 (osd.1) 173 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:40:42.186 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:41.574416+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.53:0/3581578162' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:42.186 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:41.574921+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:42.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:40.418045+0000 osd.1 (osd.1) 172 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:40:42.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:40.450452+0000 osd.1 (osd.1) 173 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:40:42.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:41.574416+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.53:0/3581578162' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:42.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:41.574921+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:42.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:42.217 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 31 16 59771 59755 120.402 81.1875 0.0278866 0.00830078 2024-02-22T19:40:42.287 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:40.418045+0000 osd.1 (osd.1) 172 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:40:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:40.450452+0000 osd.1 (osd.1) 173 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:40:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:41.574416+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.53:0/3581578162' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:41.574921+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:40:42.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:40:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:40:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:40:43.217 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 32 16 61022 61006 119.084 78.1875 0.0108086 0.00839203 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:41.403780+0000 osd.1 (osd.1) 174 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:41.405402+0000 osd.1 (osd.1) 175 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:41.575609+0000 mgr.y (mgr.14535) 449 : cluster [DBG] pgmap v575: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 1.0 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:42.115468+0000 mon.a (mon.0) 1305 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:42.115497+0000 mon.a (mon.0) 1306 : cluster [INF] Cluster is now healthy 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:42.129292+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:42.129369+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e299: 8 total, 8 up, 6 in 2024-02-22T19:40:43.411 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:42.262414+0000 mon.a (mon.0) 1309 : cluster [DBG] osdmap e300: 8 total, 8 up, 6 in 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:41.403780+0000 osd.1 (osd.1) 174 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:41.405402+0000 osd.1 (osd.1) 175 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:41.575609+0000 mgr.y (mgr.14535) 449 : cluster [DBG] pgmap v575: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 1.0 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:42.115468+0000 mon.a (mon.0) 1305 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:42.115497+0000 mon.a (mon.0) 1306 : cluster [INF] Cluster is now healthy 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:42.129292+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:42.129369+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e299: 8 total, 8 up, 6 in 2024-02-22T19:40:43.412 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:42.262414+0000 mon.a (mon.0) 1309 : cluster [DBG] osdmap e300: 8 total, 8 up, 6 in 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:41.403780+0000 osd.1 (osd.1) 174 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:41.405402+0000 osd.1 (osd.1) 175 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:41.575609+0000 mgr.y (mgr.14535) 449 : cluster [DBG] pgmap v575: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 1.0 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:42.115468+0000 mon.a (mon.0) 1305 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:42.115497+0000 mon.a (mon.0) 1306 : cluster [INF] Cluster is now healthy 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:42.129292+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:42.129369+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e299: 8 total, 8 up, 6 in 2024-02-22T19:40:43.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:42.262414+0000 mon.a (mon.0) 1309 : cluster [DBG] osdmap e300: 8 total, 8 up, 6 in 2024-02-22T19:40:43.873 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:43.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:43.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:44.136 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:42.664537+0000 osd.7 (osd.7) 187 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:40:44.136 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:42.672662+0000 osd.7 (osd.7) 188 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:40:44.136 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:42.664537+0000 osd.7 (osd.7) 187 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:40:44.136 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:42.672662+0000 osd.7 (osd.7) 188 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:40:44.218 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 33 16 62386 62370 118.058 85.25 0.0131694 0.00846588 2024-02-22T19:40:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:42.664537+0000 osd.7 (osd.7) 187 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:40:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:42.672662+0000 osd.7 (osd.7) 188 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:40:44.921 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:40:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:40:45.106 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:40:45.106 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:45.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:45.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:45.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:45.220 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 34 16 63730 63714 117.05 84 0.0102012 0.00853808 2024-02-22T19:40:45.303 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:43.357685+0000 osd.1 (osd.1) 176 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:40:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:43.362462+0000 osd.1 (osd.1) 177 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:40:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:43.576014+0000 mgr.y (mgr.14535) 450 : cluster [DBG] pgmap v578: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 54 MiB/s wr, 858 op/s 2024-02-22T19:40:45.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:43.625461+0000 osd.3 (osd.3) 197 : cluster [DBG] 6.b scrub starts 2024-02-22T19:40:45.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:43.701750+0000 osd.3 (osd.3) 198 : cluster [DBG] 6.b scrub ok 2024-02-22T19:40:45.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:40:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:40:45.382 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:45.396 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:45.396 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:43.357685+0000 osd.1 (osd.1) 176 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:40:45.396 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:43.362462+0000 osd.1 (osd.1) 177 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:43.576014+0000 mgr.y (mgr.14535) 450 : cluster [DBG] pgmap v578: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 54 MiB/s wr, 858 op/s 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:43.625461+0000 osd.3 (osd.3) 197 : cluster [DBG] 6.b scrub starts 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:43.701750+0000 osd.3 (osd.3) 198 : cluster [DBG] 6.b scrub ok 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:43.357685+0000 osd.1 (osd.1) 176 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:43.362462+0000 osd.1 (osd.1) 177 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:43.576014+0000 mgr.y (mgr.14535) 450 : cluster [DBG] pgmap v578: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 54 MiB/s wr, 858 op/s 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:43.625461+0000 osd.3 (osd.3) 197 : cluster [DBG] 6.b scrub starts 2024-02-22T19:40:45.397 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:43.701750+0000 osd.3 (osd.3) 198 : cluster [DBG] 6.b scrub ok 2024-02-22T19:40:45.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:45.400 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:45.499 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:45.544 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:45.578 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:45.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:45.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:45.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:45.745 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 300, 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-02-22T19:22:59.746063+0000", 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-02-22T19:40:42.248429+0000", 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-02-22T19:28:53.274045+0000", 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-02-22T19:39:21.706990+0000", 2024-02-22T19:40:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5799936, 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 18, 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-02-22T19:40:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 9, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "pacific", 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "device_health_metrics", 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:26:54.342689+0000", 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 11, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 11, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 11, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 11, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 11, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:40:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "164", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "163", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:40:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_devicehealth": {} 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "iscsi", 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.555543+0000", 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 3, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 3, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 3, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 3, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 3, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:40:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "242", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".rgw.root", 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.986190+0000", 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "58", 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 4, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.log", 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:01.494051+0000", 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:40:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "297", 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 5, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.control", 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:03.518927+0000", 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 22, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 22, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 22, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 22, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 22, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "5.16", 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 229, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 228, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 228, 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "229", 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "229", 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "227", 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 6, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.meta", 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:05.596341+0000", 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "64", 2024-02-22T19:40:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_bias": 4 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:40:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 9, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "unique_pool_2", 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:39:59.813801+0000", 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:40:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 16, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 16, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 16, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 16, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 16, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:40:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "289", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:40:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:40:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:40:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "rados": {} 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab", 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 8, 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 276, 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/162271081", 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6802/162271081", 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6804/162271081", 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6803/162271081", 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2", 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 285, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-02-22T19:40:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.802 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6805/2973403044", 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6806/2973403044", 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6808/2973403044", 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6807/2973403044", 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83", 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.803 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 18, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 272, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.804 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:40:45.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/936980935", 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6810/936980935", 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6812/936980935", 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6811/936980935", 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.806 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993", 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 26, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 286, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.807 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:40:45.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.809 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6813/3878043918", 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6814/3878043918", 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6816/3878043918", 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6815/3878043918", 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:40:45.810 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211", 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 32, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 260, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6800", 2024-02-22T19:40:45.811 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6801", 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6803", 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.813 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6802", 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6800/2373040976", 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6801/2373040976", 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6803/2373040976", 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6802/2373040976", 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:40:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07", 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 39, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 293, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6804", 2024-02-22T19:40:45.815 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6805", 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6807", 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6806", 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6804/2413553331", 2024-02-22T19:40:45.817 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6805/2413553331", 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6807/2413553331", 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6806/2413553331", 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e", 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.818 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 45, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 285, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6808", 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.819 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6809", 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6811", 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:40:45.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6810", 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6808/3043274732", 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6809/3043274732", 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6811/3043274732", 2024-02-22T19:40:45.821 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6810/3043274732", 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a", 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 53, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 285, 2024-02-22T19:40:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6812", 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6813", 2024-02-22T19:40:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6815", 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:40:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6814", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6812/2605082260", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6813/2605082260", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6815/2605082260", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6814/2605082260", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:40:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:39.864921+0000", 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:37.853088+0000", 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:39:31.314509+0000", 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:40:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:40:27.913155+0000", 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:40:23.090368+0000", 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:40:09.586868+0000", 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:40:08.718368+0000", 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:40:24.198637+0000", 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:40:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [ 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "1.4", 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: 6 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.1", 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:40:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.7", 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.b", 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:40:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.17", 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-02-22T19:40:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": { 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4161439976": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2578736337": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/762516580": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3676085495": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3516945073": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/828421682": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4127851809": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/1906929179": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2640420245": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/2854906057": "2024-02-23T19:23:32.485117+0000" 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-02-22T19:40:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "ruleset-failure-domain": "osd", 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-02-22T19:40:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-02-22T19:40:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-02-22T19:40:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-02-22T19:40:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:45.837 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:40:45.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:45.838 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:45.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:45.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:45.997 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:46.222 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 35 16 64887 64871 115.764 72.3125 0.00637412 0.00863345 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:44.400746+0000 osd.5 (osd.5) 84 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:44.421707+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:44.918811+0000 mgr.y (mgr.14535) 451 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:45.397581+0000 osd.1 (osd.1) 178 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:45.398698+0000 osd.1 (osd.1) 179 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:45.741853+0000 mon.c (mon.2) 58 : audit [DBG] from='client.? 172.21.15.53:0/1584142783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:40:46.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:44.400746+0000 osd.5 (osd.5) 84 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:40:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:44.421707+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:40:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:44.918811+0000 mgr.y (mgr.14535) 451 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:45.397581+0000 osd.1 (osd.1) 178 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:40:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:45.398698+0000 osd.1 (osd.1) 179 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:40:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:45.741853+0000 mon.c (mon.2) 58 : audit [DBG] from='client.? 172.21.15.53:0/1584142783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:40:46.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:40:46.439 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:46.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:46.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:46.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:46.593 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:44.400746+0000 osd.5 (osd.5) 84 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:40:46.593 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:44.421707+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:40:46.593 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:44.918811+0000 mgr.y (mgr.14535) 451 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:46.593 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:45.397581+0000 osd.1 (osd.1) 178 : cluster [DBG] 5.12 scrub starts 2024-02-22T19:40:46.594 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:45.398698+0000 osd.1 (osd.1) 179 : cluster [DBG] 5.12 scrub ok 2024-02-22T19:40:46.594 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:45.741853+0000 mon.c (mon.2) 58 : audit [DBG] from='client.? 172.21.15.53:0/1584142783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:40:46.599 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:46.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:46.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:46.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:46.910 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:46.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:46.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:46.959 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:47.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:47.066 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:47.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 36 16 66103 66087 114.656 76 0.0149266 0.00871703 2024-02-22T19:40:47.230 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:47.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:47.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:47.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:47.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:45.576790+0000 mgr.y (mgr.14535) 452 : cluster [DBG] pgmap v579: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-02-22T19:40:47.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:46.431758+0000 osd.1 (osd.1) 180 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:40:47.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:46.441560+0000 osd.1 (osd.1) 181 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:40:47.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:45.576790+0000 mgr.y (mgr.14535) 452 : cluster [DBG] pgmap v579: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-02-22T19:40:47.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:46.431758+0000 osd.1 (osd.1) 180 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:40:47.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:46.441560+0000 osd.1 (osd.1) 181 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:40:47.379 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:47.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:47.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:47.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:47.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:45.576790+0000 mgr.y (mgr.14535) 452 : cluster [DBG] pgmap v579: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-02-22T19:40:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:46.431758+0000 osd.1 (osd.1) 180 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:40:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:46.441560+0000 osd.1 (osd.1) 181 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:40:47.690 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:40:47.721 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:40:47.751 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:40:47.790 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:40:47.829 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:40:47.854 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:40:47.883 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:40:47.919 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:40:47.948 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:47.982 INFO:tasks.thrashosds.thrasher:j is {'epoch': 300, 'fsid': 'b6783888-d1b7-11ee-95bf-87774f69a715', 'created': '2024-02-22T19:22:59.746063+0000', 'modified': '2024-02-22T19:40:42.248429+0000', 'last_up_change': '2024-02-22T19:28:53.274045+0000', 'last_in_change': '2024-02-22T19:39:21.706990+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '163', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '297', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '5.16', 'ready_epoch': 229, 'last_epoch_started': 228, 'last_epoch_clean': 228, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '229', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '229', 'last_force_op_resend_preluminous': '227', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-02-22T19:39:59.813801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '289', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '4326a76a-6144-4859-9b40-5ff69edbbcab', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 276, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6801', 'nonce': 162271081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 162271081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 162271081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6803', 'nonce': 162271081}]}, 'public_addr': '172.21.15.53:6801/162271081', 'cluster_addr': '172.21.15.53:6802/162271081', 'heartbeat_back_addr': '172.21.15.53:6804/162271081', 'heartbeat_front_addr': '172.21.15.53:6803/162271081', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 285, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6805', 'nonce': 2973403044}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2973403044}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2973403044}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6807', 'nonce': 2973403044}]}, 'public_addr': '172.21.15.53:6805/2973403044', 'cluster_addr': '172.21.15.53:6806/2973403044', 'heartbeat_back_addr': '172.21.15.53:6808/2973403044', 'heartbeat_front_addr': '172.21.15.53:6807/2973403044', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 272, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6809', 'nonce': 936980935}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 936980935}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 936980935}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6811', 'nonce': 936980935}]}, 'public_addr': '172.21.15.53:6809/936980935', 'cluster_addr': '172.21.15.53:6810/936980935', 'heartbeat_back_addr': '172.21.15.53:6812/936980935', 'heartbeat_front_addr': '172.21.15.53:6811/936980935', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3f285320-aa28-4fc1-b1c7-456c9aecd993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 286, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6813', 'nonce': 3878043918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 3878043918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 3878043918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6815', 'nonce': 3878043918}]}, 'public_addr': '172.21.15.53:6813/3878043918', 'cluster_addr': '172.21.15.53:6814/3878043918', 'heartbeat_back_addr': '172.21.15.53:6816/3878043918', 'heartbeat_front_addr': '172.21.15.53:6815/3878043918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '773e8080-1028-48fe-8f6e-ed6beca60211', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 260, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2373040976}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6801', 'nonce': 2373040976}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 2373040976}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2373040976}]}, 'public_addr': '172.21.15.64:6800/2373040976', 'cluster_addr': '172.21.15.64:6801/2373040976', 'heartbeat_back_addr': '172.21.15.64:6803/2373040976', 'heartbeat_front_addr': '172.21.15.64:6802/2373040976', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6804', 'nonce': 2413553331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6805', 'nonce': 2413553331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 2413553331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2413553331}]}, 'public_addr': '172.21.15.64:6804/2413553331', 'cluster_addr': '172.21.15.64:6805/2413553331', 'heartbeat_back_addr': '172.21.15.64:6807/2413553331', 'heartbeat_front_addr': '172.21.15.64:6806/2413553331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 285, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6808', 'nonce': 3043274732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6809', 'nonce': 3043274732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 3043274732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 3043274732}]}, 'public_addr': '172.21.15.64:6808/3043274732', 'cluster_addr': '172.21.15.64:6809/3043274732', 'heartbeat_back_addr': '172.21.15.64:6811/3043274732', 'heartbeat_front_addr': '172.21.15.64:6810/3043274732', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'cde36201-cb77-4967-9402-6fdbe633b40a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 285, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6812', 'nonce': 2605082260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6813', 'nonce': 2605082260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6815', 'nonce': 2605082260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 2605082260}]}, 'public_addr': '172.21.15.64:6812/2605082260', 'cluster_addr': '172.21.15.64:6813/2605082260', 'heartbeat_back_addr': '172.21.15.64:6815/2605082260', 'heartbeat_front_addr': '172.21.15.64:6814/2605082260', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:39:39.864921+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:39:37.853088+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:39:31.314509+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:40:27.913155+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:40:23.090368+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:40:09.586868+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:40:08.718368+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:40:24.198637+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.4', 'osds': [3, 5, 6]}, {'pgid': '2.1', 'osds': [5, 3, 4]}], 'pg_upmap_items': [{'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.53:0/4161439976': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/2578736337': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/762516580': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:6800/3676085495': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:6800/3516945073': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/828421682': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/4127851809': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/1906929179': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:0/2640420245': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:6800/2854906057': '2024-02-23T19:23:32.485117+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-22T19:40:47.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:40:48.202 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:48.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:48.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:46.411375+0000 osd.5 (osd.5) 86 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:40:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:46.420343+0000 osd.5 (osd.5) 87 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:40:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.058932+0000 osd.0 (osd.0) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.058995+0000 osd.0 (osd.0) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.158752+0000 mon.a (mon.0) 1310 : cluster [DBG] osdmap e301: 8 total, 8 up, 6 in 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.264113+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e302: 8 total, 8 up, 6 in 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.584698+0000 osd.7 (osd.7) 189 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.591620+0000 osd.7 (osd.7) 190 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:47.656647+0000 mon.a (mon.0) 1312 : audit [DBG] from='client.? 172.21.15.53:0/3707014525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:46.411375+0000 osd.5 (osd.5) 86 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:46.420343+0000 osd.5 (osd.5) 87 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.058932+0000 osd.0 (osd.0) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:48.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.058995+0000 osd.0 (osd.0) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:48.207 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.158752+0000 mon.a (mon.0) 1310 : cluster [DBG] osdmap e301: 8 total, 8 up, 6 in 2024-02-22T19:40:48.207 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.264113+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e302: 8 total, 8 up, 6 in 2024-02-22T19:40:48.207 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.584698+0000 osd.7 (osd.7) 189 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:40:48.207 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.591620+0000 osd.7 (osd.7) 190 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:40:48.207 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:47.656647+0000 mon.a (mon.0) 1312 : audit [DBG] from='client.? 172.21.15.53:0/3707014525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:48.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 37 16 67265 67249 113.521 72.625 0.0111817 0.00880308 2024-02-22T19:40:48.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:48.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:48.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:48.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:46.411375+0000 osd.5 (osd.5) 86 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:46.420343+0000 osd.5 (osd.5) 87 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.058932+0000 osd.0 (osd.0) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.058995+0000 osd.0 (osd.0) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.158752+0000 mon.a (mon.0) 1310 : cluster [DBG] osdmap e301: 8 total, 8 up, 6 in 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.264113+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e302: 8 total, 8 up, 6 in 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.584698+0000 osd.7 (osd.7) 189 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:40:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.591620+0000 osd.7 (osd.7) 190 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:40:48.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:47.656647+0000 mon.a (mon.0) 1312 : audit [DBG] from='client.? 172.21.15.53:0/3707014525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:40:48.629 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:48.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:48.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:48.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:48.796 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:48.965 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:48.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:48.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:49.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:40:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:49.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 38 16 68440 68424 112.466 73.4375 0.0101781 0.00888675 2024-02-22T19:40:49.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:49] "GET /metrics HTTP/1.1" 200 176108 "" "Prometheus/2.33.4" 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.580413+0000 mgr.y (mgr.14535) 453 : cluster [DBG] pgmap v582: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 938 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.650595+0000 osd.3 (osd.3) 199 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:47.668059+0000 osd.3 (osd.3) 200 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.111249+0000 osd.2 (osd.2) 160 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.125078+0000 osd.2 (osd.2) 161 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.453350+0000 osd.1 (osd.1) 182 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.455220+0000 osd.1 (osd.1) 183 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.580413+0000 mgr.y (mgr.14535) 453 : cluster [DBG] pgmap v582: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 938 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.650595+0000 osd.3 (osd.3) 199 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:47.668059+0000 osd.3 (osd.3) 200 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:40:49.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.111249+0000 osd.2 (osd.2) 160 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:40:49.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.125078+0000 osd.2 (osd.2) 161 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:40:49.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.453350+0000 osd.1 (osd.1) 182 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:40:49.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.455220+0000 osd.1 (osd.1) 183 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.580413+0000 mgr.y (mgr.14535) 453 : cluster [DBG] pgmap v582: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 938 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.650595+0000 osd.3 (osd.3) 199 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:47.668059+0000 osd.3 (osd.3) 200 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.111249+0000 osd.2 (osd.2) 160 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.125078+0000 osd.2 (osd.2) 161 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.453350+0000 osd.1 (osd.1) 182 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:40:49.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.455220+0000 osd.1 (osd.1) 183 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:40:50.091 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:40:50.094 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:50.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:40:50.096 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:50.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:50.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 39 16 69772 69756 111.717 83.25 0.0101977 0.00894635 2024-02-22T19:40:50.252 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:50.284 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:50.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:50.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:50.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.465164+0000 osd.5 (osd.5) 88 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:40:50.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.494655+0000 osd.5 (osd.5) 89 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:40:50.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.673178+0000 osd.3 (osd.3) 201 : cluster [DBG] 4.1a deep-scrub starts 2024-02-22T19:40:50.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:48.683105+0000 osd.3 (osd.3) 202 : cluster [DBG] 4.1a deep-scrub ok 2024-02-22T19:40:50.323 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:49.581193+0000 mgr.y (mgr.14535) 454 : cluster [DBG] pgmap v583: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-02-22T19:40:50.323 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.465164+0000 osd.5 (osd.5) 88 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:40:50.323 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.494655+0000 osd.5 (osd.5) 89 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:40:50.323 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.673178+0000 osd.3 (osd.3) 201 : cluster [DBG] 4.1a deep-scrub starts 2024-02-22T19:40:50.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:48.683105+0000 osd.3 (osd.3) 202 : cluster [DBG] 4.1a deep-scrub ok 2024-02-22T19:40:50.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:49.581193+0000 mgr.y (mgr.14535) 454 : cluster [DBG] pgmap v583: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-02-22T19:40:50.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:50.439 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:50.472 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.465164+0000 osd.5 (osd.5) 88 : cluster [DBG] 6.19 scrub starts 2024-02-22T19:40:50.472 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.494655+0000 osd.5 (osd.5) 89 : cluster [DBG] 6.19 scrub ok 2024-02-22T19:40:50.472 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.673178+0000 osd.3 (osd.3) 201 : cluster [DBG] 4.1a deep-scrub starts 2024-02-22T19:40:50.472 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:48.683105+0000 osd.3 (osd.3) 202 : cluster [DBG] 4.1a deep-scrub ok 2024-02-22T19:40:50.472 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:49.581193+0000 mgr.y (mgr.14535) 454 : cluster [DBG] pgmap v583: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-02-22T19:40:50.473 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:50.473 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:50.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:50.657 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:50.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:50.663 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:50.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:40:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:40:50.844 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:50.845 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:40:50.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:50.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:40:51.004 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:51.154 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:51.159 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:51.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:51.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:40:51.224803+0000 min lat: 0.00135644 max lat: 0.088657 avg lat: 0.00901802 2024-02-22T19:40:51.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:40:51.224 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 40 16 70993 70977 110.832 76.3125 0.0197139 0.00901802 2024-02-22T19:40:51.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:51.337 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:51.340 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:51.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:51.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:40:51.484 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:49.496433+0000 osd.5 (osd.5) 90 : cluster [DBG] 6.a scrub starts 2024-02-22T19:40:51.484 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:49.502335+0000 osd.5 (osd.5) 91 : cluster [DBG] 6.a scrub ok 2024-02-22T19:40:51.484 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:49.578213+0000 osd.6 (osd.6) 134 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:40:51.484 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:49.582854+0000 osd.6 (osd.6) 135 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:40:51.484 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:50.383549+0000 osd.1 (osd.1) 184 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:40:51.484 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:50.388451+0000 osd.1 (osd.1) 185 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:40:51.492 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:49.496433+0000 osd.5 (osd.5) 90 : cluster [DBG] 6.a scrub starts 2024-02-22T19:40:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:49.502335+0000 osd.5 (osd.5) 91 : cluster [DBG] 6.a scrub ok 2024-02-22T19:40:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:49.578213+0000 osd.6 (osd.6) 134 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:40:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:49.582854+0000 osd.6 (osd.6) 135 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:40:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:50.383549+0000 osd.1 (osd.1) 184 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:40:51.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:50.388451+0000 osd.1 (osd.1) 185 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:40:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:49.496433+0000 osd.5 (osd.5) 90 : cluster [DBG] 6.a scrub starts 2024-02-22T19:40:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:49.502335+0000 osd.5 (osd.5) 91 : cluster [DBG] 6.a scrub ok 2024-02-22T19:40:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:49.578213+0000 osd.6 (osd.6) 134 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:40:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:49.582854+0000 osd.6 (osd.6) 135 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:40:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:50.383549+0000 osd.1 (osd.1) 184 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:40:51.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:50.388451+0000 osd.1 (osd.1) 185 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:40:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:40:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:51.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:40:51.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:51.674 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:51.825 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:51.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:51.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:51.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:52.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:50.675367+0000 osd.3 (osd.3) 203 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:40:52.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:50.680428+0000 osd.3 (osd.3) 204 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:40:52.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:51.569520+0000 osd.7 (osd.7) 191 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:40:52.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:51.581653+0000 mgr.y (mgr.14535) 455 : cluster [DBG] pgmap v584: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-02-22T19:40:52.215 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:51.593972+0000 osd.7 (osd.7) 192 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:40:52.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 41 16 72192 72176 109.953 74.9375 0.0134771 0.00908894 2024-02-22T19:40:52.275 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:40:52.275 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:40:52.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:20875 7.a 7:5368db82:::benchmark_data_smithi053_107660_object20874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 543.24175356600006, 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001, 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20875 2024-02-22T19:40:52.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030656+0000", 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030654+0000", 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5442000000000001e-05 2024-02-22T19:40:52.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030680+0000", 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1599e-05 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030691+0000", 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8590000000000001e-06 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030697+0000", 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5996999999999998e-05 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030763+0000", 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1030000000000002e-06 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:49.030771+0000", 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96976369299999998 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000535+0000", 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3981999999999999e-05 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:50.000549+0000", 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449743229999999 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045523+0000", 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015328299999999999 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045676+0000", 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167651 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.045844+0000", 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003101946 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.048946+0000", 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029887199999999998 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.051935+0000", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057814199999999998 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057716+0000", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5281000000000001e-05 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.057771+0000", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092475900000000004 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:31:51.058696+0000", 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0280405720000001 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111852 7.a 7:5162ddfb:::benchmark_data_smithi053_107660_object22645:head [delete] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.16359196299999, 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.030160867, 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111852 2024-02-22T19:40:52.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108817+0000", 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108815+0000", 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8550000000000008e-06 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108824+0000", 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3647e-05 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108837+0000", 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9809999999999997e-06 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108844+0000", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9802000000000001e-05 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.108904+0000", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013143000000000001 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.109036+0000", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016598399999999999 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.109202+0000", 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012752025 2024-02-22T19:40:52.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121954+0000", 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1360000000000002e-06 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121960+0000", 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016965211 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.138925+0000", 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3065000000000001e-05 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.138978+0000", 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.030160867 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111854 7.a 7:5a0cbb42:::benchmark_data_smithi053_107660_object22647:head [delete] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.15861634599997, 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531219629999999, 2024-02-22T19:40:52.288 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111854 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113793+0000", 2024-02-22T19:40:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113791+0000", 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3090000000000003e-06 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113797+0000", 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8690000000000004e-06 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113805+0000", 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6439999999999997e-06 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113810+0000", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109718 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.113920+0000", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125108 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.114045+0000", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016840599999999999 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.114214+0000", 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077607329999999997 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121974+0000", 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6069999999999999e-06 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.121978+0000", 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170384509999999 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139016+0000", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100129 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139116+0000", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027745477000000001 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166862+0000", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2849000000000002e-05 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166915+0000", 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531219629999999 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111874 7.a 7:51c8a40d:::benchmark_data_smithi053_107660_object22667:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.13927878300001, 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0339878849999999, 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111874 2024-02-22T19:40:52.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133130+0000", 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133128+0000", 2024-02-22T19:40:52.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7869999999999996e-06 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133138+0000", 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2543e-05 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133150+0000", 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8499999999999994e-06 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.133158+0000", 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019007130000000001 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.135059+0000", 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2880000000000004e-06 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.135066+0000", 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0040841840000001 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139150+0000", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011695400000000001 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139267+0000", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014209699999999999 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.139409+0000", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011988443999999999 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151398+0000", 2024-02-22T19:40:52.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015560497 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166958+0000", 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1764000000000002e-05 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167000+0000", 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4150000000000003e-05 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167034+0000", 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4145000000000002e-05 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167118+0000", 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0339878849999999 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111911 7.a 7:53f6a651:::benchmark_data_smithi053_107660_object22704:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.08344834399998, 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97843876299999999, 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111911 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188961+0000", 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:40:52.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188959+0000", 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188967+0000", 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1484000000000001e-05 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188978+0000", 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7379999999999997e-06 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.188985+0000", 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00214072 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.191126+0000", 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1008e-05 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.191137+0000", 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95036631800000004 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141503+0000", 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001082 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141611+0000", 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015199899999999999 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.141763+0000", 2024-02-22T19:40:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096414059999999999 2024-02-22T19:40:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151405+0000", 2024-02-22T19:40:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015767331999999998 2024-02-22T19:40:52.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167172+0000", 2024-02-22T19:40:52.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9851000000000003e-05 2024-02-22T19:40:52.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167272+0000", 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2466999999999999e-05 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167304+0000", 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5302000000000003e-05 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167400+0000", 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97843876299999999 2024-02-22T19:40:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111925 7.a 7:55d29d8b:::benchmark_data_smithi053_107660_object22718:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.07517145600002, 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97039658799999995, 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111925 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197238+0000", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197237+0000", 2024-02-22T19:40:52.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.563e-06 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197241+0000", 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3770000000000004e-06 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197246+0000", 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197250+0000", 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2380999999999997e-05 2024-02-22T19:40:52.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197322+0000", 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.268e-06 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197332+0000", 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94598222200000004 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143314+0000", 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013365499999999999 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143448+0000", 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014185900000000001 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.143589+0000", 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078197079999999999 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151409+0000", 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015815124 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167224+0000", 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027372900000000001 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167498+0000", 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5688000000000001e-05 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167534+0000", 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010068699999999999 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167634+0000", 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97039658799999995 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111926 7.a 7:5f8a638d:::benchmark_data_smithi053_107660_object22719:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.07514799900002, 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97061990799999998, 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111926 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197261+0000", 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5790000000000002e-06 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197263+0000", 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197268+0000", 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6339999999999999e-06 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197272+0000", 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0308000000000004e-05 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197352+0000", 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.28e-06 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.197358+0000", 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94796157400000003 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145320+0000", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012371600000000001 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145444+0000", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158498 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.145602+0000", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095829260000000003 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155185+0000", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012273795000000001 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167459+0000", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028113900000000002 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167740+0000", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4212999999999998e-05 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167784+0000", 2024-02-22T19:40:52.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6879000000000005e-05 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167881+0000", 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97061990799999998 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:111975 7.a 7:5962aaaf:::benchmark_data_smithi053_107660_object22768:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 431.01563657899999, 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91135001299999996, 2024-02-22T19:40:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 111975 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256773+0000", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256772+0000", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7850000000000002e-06 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256776+0000", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0140000000000001e-06 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256783+0000", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9890000000000003e-06 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256787+0000", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6700999999999998e-05 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256843+0000", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8880000000000003e-06 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.256852+0000", 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89037246800000003 2024-02-22T19:40:52.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147225+0000", 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011598499999999999 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147341+0000", 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013154 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.147472+0000", 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077203439999999996 2024-02-22T19:40:52.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155193+0000", 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012737145 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167930+0000", 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0123999999999997e-05 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.167970+0000", 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3705000000000003e-05 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168014+0000", 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108915 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168123+0000", 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91135001299999996 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112025 7.a 7:5114742b:::benchmark_data_smithi053_107660_object22818:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:40:52.326 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.95751313800002, 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85347167700000004, 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112025 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314896+0000", 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314895+0000", 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5490000000000001e-06 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314902+0000", 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0531e-05 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314913+0000", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1970000000000004e-06 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314918+0000", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1729999999999993e-05 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314980+0000", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1537e-05 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.314991+0000", 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.83431657000000004 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149308+0000", 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011637900000000001 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149424+0000", 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000147207 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.149571+0000", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011278806000000001 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160850+0000", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073124599999999998 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168162+0000", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2563000000000004e-05 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168225+0000", 2024-02-22T19:40:52.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5136999999999998e-05 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168270+0000", 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7520000000000001e-05 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168368+0000", 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85347167700000004 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112063 7.a 7:5ff85e4d:::benchmark_data_smithi053_107660_object22856:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.91545159899999, 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.811635777, 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112063 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356958+0000", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356956+0000", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356962+0000", 2024-02-22T19:40:52.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8179999999999993e-06 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356971+0000", 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4229999999999996e-06 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.356977+0000", 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1233000000000001e-05 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.357038+0000", 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.357048+0000", 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.79440474900000002 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151452+0000", 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012225300000000001 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151575+0000", 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8306999999999993e-05 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.151663+0000", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091947409999999993 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160858+0000", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075599589999999998 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168418+0000", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2265999999999998e-05 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168460+0000", 2024-02-22T19:40:52.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0454999999999998e-05 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168500+0000", 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2958e-05 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168593+0000", 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.811635777 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112096 7.a 7:5442534a:::benchmark_data_smithi053_107660_object22889:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.881106875, 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.777506645, 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112096 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391302+0000", 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.313e-06 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391306+0000", 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0849999999999999e-06 2024-02-22T19:40:52.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391313+0000", 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0760000000000002e-06 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391318+0000", 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1353999999999994e-05 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391399+0000", 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0243e-05 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.391410+0000", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.76178757200000002 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153197+0000", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109708 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153307+0000", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013573900000000001 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.153443+0000", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011010321999999999 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164453+0000", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041854379999999997 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168638+0000", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0509999999999997e-05 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168679+0000", 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8803000000000001e-05 2024-02-22T19:40:52.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168718+0000", 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1186000000000004e-05 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168809+0000", 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.777506645 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112098 7.a 7:519628cb:::benchmark_data_smithi053_107660_object22891:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.87916776899999, 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.77578508499999999, 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112098 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393241+0000", 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393240+0000", 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1999999999999997e-06 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393247+0000", 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5688999999999998e-05 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393263+0000", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-06 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393270+0000", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4311000000000001e-05 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393334+0000", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3839999999999997e-06 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.393342+0000", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.76188229600000001 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155225+0000", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5840999999999993e-05 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155310+0000", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5882000000000002e-05 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.155396+0000", 2024-02-22T19:40:52.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0090645329999999996 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164461+0000", 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043933849999999997 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168854+0000", 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0268000000000001e-05 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168895+0000", 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9428000000000003e-05 2024-02-22T19:40:52.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.168934+0000", 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2540000000000005e-05 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169026+0000", 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.77578508499999999 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112130 7.a 7:5e909bd4:::benchmark_data_smithi053_107660_object22923:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.84061684199997, 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73752859400000004, 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112130 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431792+0000", 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431791+0000", 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.251e-06 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431797+0000", 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0682999999999999e-05 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431808+0000", 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3590000000000002e-06 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431816+0000", 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5444999999999994e-05 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431881+0000", 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0912e-05 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.431892+0000", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.72501750099999995 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.156909+0000", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111295 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.157021+0000", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7911999999999994e-05 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.157119+0000", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073499309999999997 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.164468+0000", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046038349999999997 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169072+0000", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105296 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169178+0000", 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4647999999999997e-05 2024-02-22T19:40:52.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169222+0000", 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8640000000000004e-05 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169321+0000", 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73752859400000004 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.356 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112203 7.a 7:5c4dbf81:::benchmark_data_smithi053_107660_object22996:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.75616289099997, 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65325745000000002, 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112203 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516246+0000", 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516245+0000", 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516253+0000", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7668000000000001e-05 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516270+0000", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6799999999999993e-06 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516278+0000", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8060000000000003e-05 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516336+0000", 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6299999999999993e-06 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.516346+0000", 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64245662999999997 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.158802+0000", 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103798 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.158906+0000", 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129794 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.159036+0000", 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071273559999999996 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166163+0000", 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029783499999999998 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169142+0000", 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022555399999999999 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169367+0000", 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2079000000000001e-05 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169409+0000", 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4319000000000003e-05 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169504+0000", 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65325745000000002 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24670.0:112210 7.a 7:56d9e136:::benchmark_data_smithi053_107660_object23003:head [delete] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 430.74595782300003, 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64327634499999997, 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24670", 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/296927568", 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112210 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526451+0000", 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526450+0000", 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2809999999999997e-06 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526456+0000", 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0599e-05 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526467+0000", 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4539999999999998e-06 2024-02-22T19:40:52.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526472+0000", 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6777000000000003e-05 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526519+0000", 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3770000000000001e-06 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:41.526526+0000", 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63437577199999995 2024-02-22T19:40:52.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.160901+0000", 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106143 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.161007+0000", 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109844 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.161117+0000", 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050523790000000001 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.166170+0000", 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003381743 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169551+0000", 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1298999999999999e-05 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169593+0000", 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0349000000000003e-05 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169633+0000", 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4598000000000005e-05 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:33:42.169728+0000", 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64327634499999997 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197527 8.b 8:d5caae7e:::benchmark_data_smithi053_130774_object85793:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.099502+0000", 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 126.172907205, 2024-02-22T19:40:52.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119527755, 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197527 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099502+0000", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099502+0000", 2024-02-22T19:40:52.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999952 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099497+0000", 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6509999999999996e-06 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099507+0000", 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099517+0000", 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5379e-05 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099532+0000", 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000157899 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099690+0000", 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0009e-05 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099700+0000", 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98788994600000002 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087590+0000", 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0902e-05 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087601+0000", 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1114758579999999 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.199077+0000", 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013723000000000001 2024-02-22T19:40:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.199214+0000", 2024-02-22T19:40:52.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041806900000000003 2024-02-22T19:40:52.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.199632+0000", 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017562853999999999 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217195+0000", 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010780099999999999 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217303+0000", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023422699999999999 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217537+0000", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013003050000000001 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218837+0000", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8440999999999998e-05 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218896+0000", 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134117 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.219030+0000", 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119527755 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197529 8.b 8:d1af294b:::benchmark_data_smithi053_130774_object85795:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 126.172868687, 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1239954820000002, 2024-02-22T19:40:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197529 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:40:52.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5370000000000002e-06 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099543+0000", 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.905e-06 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099550+0000", 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4659999999999998e-06 2024-02-22T19:40:52.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099556+0000", 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175448 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099731+0000", 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0879999999999997e-06 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099738+0000", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98789419499999997 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087632+0000", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8289999999999997e-06 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087641+0000", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.121857726 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.209499+0000", 2024-02-22T19:40:52.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133626 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.209633+0000", 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016687000000000001 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.209800+0000", 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074392160000000002 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217239+0000", 2024-02-22T19:40:52.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030646299999999998 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217545+0000", 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001521435 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.219067+0000", 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042951250000000003 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223362+0000", 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3224999999999999e-05 2024-02-22T19:40:52.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223415+0000", 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000120957 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223536+0000", 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1239954820000002 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197563 8.b 8:d21bad24:::benchmark_data_smithi053_130774_object85829:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:40:52.384 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.165645+0000", 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 126.106764578, 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0580848550000002, 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197563 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165645+0000", 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165645+0000", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165643+0000", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0870000000000003e-06 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165650+0000", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0628999999999999e-05 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165660+0000", 2024-02-22T19:40:52.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9530000000000002e-06 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165667+0000", 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8199999999999999e-05 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165715+0000", 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0727000000000001e-05 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165726+0000", 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92193606900000002 2024-02-22T19:40:52.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087662+0000", 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087669+0000", 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.123577021 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.211246+0000", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112305 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.211358+0000", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001133886 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.212492+0000", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050974360000000003 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217590+0000", 2024-02-22T19:40:52.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011748539999999999 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218765+0000", 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015207249999999999 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.220285+0000", 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032889820000000002 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223574+0000", 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2823e-05 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223617+0000", 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011241 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223729+0000", 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0580848550000002 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197570 8.b 8:ddea507a:::benchmark_data_smithi053_130774_object85836:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:40:52.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.187988+0000", 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 126.084421615, 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0375089649999998, 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197570 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187988+0000", 2024-02-22T19:40:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187988+0000", 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187986+0000", 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1079999999999998e-06 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187993+0000", 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8490000000000001e-06 2024-02-22T19:40:52.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188002+0000", 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8790000000000002e-06 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188007+0000", 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7867000000000001e-05 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188055+0000", 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.37e-06 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188062+0000", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89962829 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087690+0000", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7170000000000008e-06 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087698+0000", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.126183054 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.213881+0000", 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011198599999999999 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.213993+0000", 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016729099999999999 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.214160+0000", 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034367669999999999 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217597+0000", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012083549999999999 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218805+0000", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001519357 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.220324+0000", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050053090000000003 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.225330+0000", 2024-02-22T19:40:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2014e-05 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.225382+0000", 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114812 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.225497+0000", 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0375089649999998 2024-02-22T19:40:52.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197603 8.b 8:d4577564:::benchmark_data_smithi053_130774_object85869:head [delete] snapc 0=[] ondisk+write+known_if_redirected e248)", 2024-02-22T19:40:52.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:47.359648+0000", 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 124.912761316, 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87032219899999996, 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197603 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359648+0000", 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359648+0000", 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359646+0000", 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.126e-06 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359653+0000", 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2813000000000001e-05 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359666+0000", 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3090000000000001e-06 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359673+0000", 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5130999999999997e-05 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359738+0000", 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0944999999999999e-05 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359749+0000", 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85562269999999996 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.215371+0000", 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102002 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.215473+0000", 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000138956 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.215612+0000", 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019917979999999999 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217604+0000", 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039613310000000002 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.221565+0000", 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037830200000000001 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.221944+0000", 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078577630000000002 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.229801+0000", 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7669e-05 2024-02-22T19:40:52.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.229849+0000", 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012089 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.229970+0000", 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87032219899999996 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:40:52.406 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:40:52.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:50.675367+0000 osd.3 (osd.3) 203 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:40:52.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:50.680428+0000 osd.3 (osd.3) 204 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:40:52.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:51.569520+0000 osd.7 (osd.7) 191 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:40:52.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:51.581653+0000 mgr.y (mgr.14535) 455 : cluster [DBG] pgmap v584: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-02-22T19:40:52.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:51.593972+0000 osd.7 (osd.7) 192 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:40:52.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:50.675367+0000 osd.3 (osd.3) 203 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:40:52.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:50.680428+0000 osd.3 (osd.3) 204 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:40:52.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:51.569520+0000 osd.7 (osd.7) 191 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:40:52.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:51.581653+0000 mgr.y (mgr.14535) 455 : cluster [DBG] pgmap v584: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-02-22T19:40:52.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:51.593972+0000 osd.7 (osd.7) 192 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:40:52.748 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:52.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:52.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:52.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:40:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:52.924 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:40:52.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:52.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:40:53.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:53.085 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:53.225 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 42 16 73499 73483 109.281 81.6875 0.0162833 0.0091458 2024-02-22T19:40:53.359 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:53.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:52.121912+0000 osd.2 (osd.2) 162 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:40:53.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:52.130011+0000 osd.2 (osd.2) 163 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:40:53.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:52.554423+0000 osd.7 (osd.7) 193 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:40:53.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:52.559978+0000 osd.7 (osd.7) 194 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:40:53.362 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:53.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:53.362 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:52.121912+0000 osd.2 (osd.2) 162 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:40:53.363 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:52.130011+0000 osd.2 (osd.2) 163 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:40:53.363 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:52.554423+0000 osd.7 (osd.7) 193 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:40:53.363 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:52.559978+0000 osd.7 (osd.7) 194 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:40:53.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:40:53.515 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:52.121912+0000 osd.2 (osd.2) 162 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:40:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:52.130011+0000 osd.2 (osd.2) 163 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:40:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:52.554423+0000 osd.7 (osd.7) 193 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:40:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:52.559978+0000 osd.7 (osd.7) 194 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:40:53.603 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:40:53.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:53.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:40:53.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:53.762 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:40:53.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:53.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:40:53.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:40:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:54.037 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:40:54.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:54.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:40:54.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:40:54.208 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:54.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 43 16 74721 74705 108.515 76.375 0.0114928 0.00921023 2024-02-22T19:40:54.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:52.538328+0000 osd.5 (osd.5) 92 : cluster [DBG] 5.7 deep-scrub starts 2024-02-22T19:40:54.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:52.541452+0000 osd.5 (osd.5) 93 : cluster [DBG] 5.7 deep-scrub ok 2024-02-22T19:40:54.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:53.368098+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:54.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:53.372169+0000 osd.1 (osd.1) 186 : cluster [DBG] 4.f scrub starts 2024-02-22T19:40:54.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:53.383906+0000 osd.1 (osd.1) 187 : cluster [DBG] 4.f scrub ok 2024-02-22T19:40:54.494 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:53.582108+0000 mgr.y (mgr.14535) 456 : cluster [DBG] pgmap v585: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 639 B/s rd, 47 MiB/s wr, 756 op/s 2024-02-22T19:40:54.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:52.538328+0000 osd.5 (osd.5) 92 : cluster [DBG] 5.7 deep-scrub starts 2024-02-22T19:40:54.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:52.541452+0000 osd.5 (osd.5) 93 : cluster [DBG] 5.7 deep-scrub ok 2024-02-22T19:40:54.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:53.368098+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:54.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:53.372169+0000 osd.1 (osd.1) 186 : cluster [DBG] 4.f scrub starts 2024-02-22T19:40:54.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:53.383906+0000 osd.1 (osd.1) 187 : cluster [DBG] 4.f scrub ok 2024-02-22T19:40:54.495 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:53.582108+0000 mgr.y (mgr.14535) 456 : cluster [DBG] pgmap v585: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 639 B/s rd, 47 MiB/s wr, 756 op/s 2024-02-22T19:40:54.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:52.538328+0000 osd.5 (osd.5) 92 : cluster [DBG] 5.7 deep-scrub starts 2024-02-22T19:40:54.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:52.541452+0000 osd.5 (osd.5) 93 : cluster [DBG] 5.7 deep-scrub ok 2024-02-22T19:40:54.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:53.368098+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:40:54.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:53.372169+0000 osd.1 (osd.1) 186 : cluster [DBG] 4.f scrub starts 2024-02-22T19:40:54.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:53.383906+0000 osd.1 (osd.1) 187 : cluster [DBG] 4.f scrub ok 2024-02-22T19:40:54.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:53.582108+0000 mgr.y (mgr.14535) 456 : cluster [DBG] pgmap v585: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 639 B/s rd, 47 MiB/s wr, 756 op/s 2024-02-22T19:40:54.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:40:54.935 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:40:55.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 44 16 75924 75908 107.757 75.1875 0.0195366 0.00927603 2024-02-22T19:40:55.229 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:40:55.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:40:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:40:55.430 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:40:55.667 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.179016+0000 osd.2 (osd.2) 164 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:40:55.667 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.192633+0000 osd.2 (osd.2) 165 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:54.549001+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.53:0/2344067356' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:54.549436+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.784955+0000 osd.3 (osd.3) 205 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.790673+0000 osd.3 (osd.3) 206 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.907109+0000 osd.1 (osd.1) 188 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.907156+0000 osd.1 (osd.1) 189 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:55.668 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:54.932469+0000 mgr.y (mgr.14535) 457 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.179016+0000 osd.2 (osd.2) 164 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.192633+0000 osd.2 (osd.2) 165 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:54.549001+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.53:0/2344067356' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:54.549436+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.784955+0000 osd.3 (osd.3) 205 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.790673+0000 osd.3 (osd.3) 206 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.907109+0000 osd.1 (osd.1) 188 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.907156+0000 osd.1 (osd.1) 189 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:55.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:54.932469+0000 mgr.y (mgr.14535) 457 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.179016+0000 osd.2 (osd.2) 164 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.192633+0000 osd.2 (osd.2) 165 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:54.549001+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.53:0/2344067356' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:54.549436+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.784955+0000 osd.3 (osd.3) 205 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.790673+0000 osd.3 (osd.3) 206 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.907109+0000 osd.1 (osd.1) 188 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.907156+0000 osd.1 (osd.1) 189 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:40:55.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:54.932469+0000 mgr.y (mgr.14535) 457 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:40:56.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 45 16 77137 77121 107.048 75.8125 0.00327532 0.00933539 2024-02-22T19:40:56.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.549708+0000 osd.5 (osd.5) 94 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:40:56.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:54.553505+0000 osd.5 (osd.5) 95 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:40:56.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:55.384726+0000 mon.a (mon.0) 1315 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:56.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:55.403318+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:55.403396+0000 mon.a (mon.0) 1317 : cluster [DBG] osdmap e303: 8 total, 8 up, 6 in 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:55.584864+0000 mgr.y (mgr.14535) 458 : cluster [DBG] pgmap v587: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.549708+0000 osd.5 (osd.5) 94 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:54.553505+0000 osd.5 (osd.5) 95 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:55.384726+0000 mon.a (mon.0) 1315 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:55.403318+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:55.403396+0000 mon.a (mon.0) 1317 : cluster [DBG] osdmap e303: 8 total, 8 up, 6 in 2024-02-22T19:40:56.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:55.584864+0000 mgr.y (mgr.14535) 458 : cluster [DBG] pgmap v587: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:40:56.800 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.549708+0000 osd.5 (osd.5) 94 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:40:56.800 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:54.553505+0000 osd.5 (osd.5) 95 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:40:56.800 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:55.384726+0000 mon.a (mon.0) 1315 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:40:56.800 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:55.403318+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:40:56.800 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:55.403396+0000 mon.a (mon.0) 1317 : cluster [DBG] osdmap e303: 8 total, 8 up, 6 in 2024-02-22T19:40:56.800 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:55.584864+0000 mgr.y (mgr.14535) 458 : cluster [DBG] pgmap v587: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:40:57.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 46 16 78409 78393 106.449 79.5 0.0172349 0.00939017 2024-02-22T19:40:57.417 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:40:56.307069+0000 mgr.y (mgr.14535) 459 : audit [DBG] from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:40:57.417 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:57.265671+0000 mon.a (mon.0) 1318 : cluster [DBG] osdmap e304: 8 total, 8 up, 6 in 2024-02-22T19:40:57.468 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:40:57.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:40:57.468 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:40:57.468 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:40:57.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:40:56.307069+0000 mgr.y (mgr.14535) 459 : audit [DBG] from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:40:57.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:57.265671+0000 mon.a (mon.0) 1318 : cluster [DBG] osdmap e304: 8 total, 8 up, 6 in 2024-02-22T19:40:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:40:56.307069+0000 mgr.y (mgr.14535) 459 : audit [DBG] from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:40:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:57.265671+0000 mon.a (mon.0) 1318 : cluster [DBG] osdmap e304: 8 total, 8 up, 6 in 2024-02-22T19:40:57.802 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:40:57.807 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:40:57.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:40:57.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:40:57.963 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:40:58.227 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 47 16 79589 79573 105.754 73.75 0.0100925 0.00945161 2024-02-22T19:40:58.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:40:58.728 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:40:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:40:58.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:40:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:57.585237+0000 mgr.y (mgr.14535) 460 : cluster [DBG] pgmap v589: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 639 B/s rd, 60 MiB/s wr, 962 op/s 2024-02-22T19:40:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:40:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:57.585237+0000 mgr.y (mgr.14535) 460 : cluster [DBG] pgmap v589: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 639 B/s rd, 60 MiB/s wr, 962 op/s 2024-02-22T19:40:58.801 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:40:58.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:40:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:57.585237+0000 mgr.y (mgr.14535) 460 : cluster [DBG] pgmap v589: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 639 B/s rd, 60 MiB/s wr, 962 op/s 2024-02-22T19:40:58.902 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:40:59.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 48 16 80754 80738 105.068 72.8125 0.00624274 0.00951367 2024-02-22T19:40:59.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:40:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:40:59] "GET /metrics HTTP/1.1" 200 176117 "" "Prometheus/2.33.4" 2024-02-22T19:41:00.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 49 16 82015 81999 104.532 78.8125 0.0101915 0.00956191 2024-02-22T19:41:00.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:59.454501+0000 osd.2 (osd.2) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:00.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:59.454557+0000 osd.2 (osd.2) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:00.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:59.511972+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e305: 8 total, 8 up, 6 in 2024-02-22T19:41:00.524 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:40:59.585697+0000 mgr.y (mgr.14535) 461 : cluster [DBG] pgmap v591: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-02-22T19:41:00.909 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:41:00.909 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:41:00.909 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:41:00.910 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:41:00.910 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:41:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:59.454501+0000 osd.2 (osd.2) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:59.454557+0000 osd.2 (osd.2) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:59.511972+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e305: 8 total, 8 up, 6 in 2024-02-22T19:41:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:40:59.585697+0000 mgr.y (mgr.14535) 461 : cluster [DBG] pgmap v591: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-02-22T19:41:01.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:59.454501+0000 osd.2 (osd.2) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:01.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:59.454557+0000 osd.2 (osd.2) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:01.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:59.511972+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e305: 8 total, 8 up, 6 in 2024-02-22T19:41:01.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:40:59.585697+0000 mgr.y (mgr.14535) 461 : cluster [DBG] pgmap v591: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-02-22T19:41:01.226 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 50 16 83209 83193 103.934 74.625 0.0099043 0.00961695 2024-02-22T19:41:01.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:41:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:01.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:01.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:02.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:02.093 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:02.176 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:02.227 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 51 16 84465 84449 103.435 78.5 0.0106233 0.00966299 2024-02-22T19:41:02.720 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":587,"stamp":"2024-02-22T19:40:55.584621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4912977572,"num_objects":75158,"num_object_clones":0,"num_object_copies":225474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75158,"num_whiteouts":0,"num_read":1822,"num_read_kb":1574,"num_write":75680,"num_write_kb":4797865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":13666,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63281,"ondisk_log_size":63281,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":16081100,"kb_used_data":14264988,"kb_used_omap":0,"kb_used_meta":1816064,"kb_avail":546389812,"statfs":{"total":575970213888,"available":559503167488,"internally_reserved":0,"allocated":14607347712,"data_stored":14602172196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1859649536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,6,14,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":806813696,"num_objects":12311,"num_object_clones":0,"num_object_copies":36933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12311,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12311,"num_write_kb":787904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.320014"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":259,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170356+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:40:47.170356+0000","last_peered":"2024-02-22T19:40:47.170356+0000","last_clean":"2024-02-22T19:40:47.170356+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:40:47.170356+0000","last_undegraded":"2024-02-22T19:40:47.170356+0000","last_fullsized":"2024-02-22T19:40:47.170356+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":280,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173615+0000","last_change":"2024-02-22T19:39:22.268923+0000","last_active":"2024-02-22T19:40:47.173615+0000","last_peered":"2024-02-22T19:40:47.173615+0000","last_clean":"2024-02-22T19:40:47.173615+0000","last_became_active":"2024-02-22T19:39:22.268587+0000","last_became_peered":"2024-02-22T19:39:22.268587+0000","last_unstale":"2024-02-22T19:40:47.173615+0000","last_undegraded":"2024-02-22T19:40:47.173615+0000","last_fullsized":"2024-02-22T19:40:47.173615+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":117,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170090+0000","last_change":"2024-02-22T19:39:36.528260+0000","last_active":"2024-02-22T19:40:47.170090+0000","last_peered":"2024-02-22T19:40:47.170090+0000","last_clean":"2024-02-22T19:40:47.170090+0000","last_became_active":"2024-02-22T19:38:47.164599+0000","last_became_peered":"2024-02-22T19:38:47.164599+0000","last_unstale":"2024-02-22T19:40:47.170090+0000","last_undegraded":"2024-02-22T19:40:47.170090+0000","last_fullsized":"2024-02-22T19:40:47.170090+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:39:36.528175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.528175+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":260,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171361+0000","last_change":"2024-02-22T19:36:59.298132+0000","last_active":"2024-02-22T19:40:47.171361+0000","last_peered":"2024-02-22T19:40:47.171361+0000","last_clean":"2024-02-22T19:40:47.171361+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:40:47.171361+0000","last_undegraded":"2024-02-22T19:40:47.171361+0000","last_fullsized":"2024-02-22T19:40:47.171361+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:59.298021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:36:59.298021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":266,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171379+0000","last_change":"2024-02-22T19:40:34.661155+0000","last_active":"2024-02-22T19:40:47.171379+0000","last_peered":"2024-02-22T19:40:47.171379+0000","last_clean":"2024-02-22T19:40:47.171379+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:40:47.171379+0000","last_undegraded":"2024-02-22T19:40:47.171379+0000","last_fullsized":"2024-02-22T19:40:47.171379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":382,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.174957+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:40:47.174957+0000","last_peered":"2024-02-22T19:40:47.174957+0000","last_clean":"2024-02-22T19:40:47.174957+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:40:47.174957+0000","last_undegraded":"2024-02-22T19:40:47.174957+0000","last_fullsized":"2024-02-22T19:40:47.174957+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":98,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.166478+0000","last_change":"2024-02-22T19:39:36.296064+0000","last_active":"2024-02-22T19:40:47.166478+0000","last_peered":"2024-02-22T19:40:47.166478+0000","last_clean":"2024-02-22T19:40:47.166478+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:40:47.166478+0000","last_undegraded":"2024-02-22T19:40:47.166478+0000","last_fullsized":"2024-02-22T19:40:47.166478+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:36.295988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.295988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":103,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173679+0000","last_change":"2024-02-22T19:38:47.169310+0000","last_active":"2024-02-22T19:40:47.173679+0000","last_peered":"2024-02-22T19:40:47.173679+0000","last_clean":"2024-02-22T19:40:47.173679+0000","last_became_active":"2024-02-22T19:38:47.168764+0000","last_became_peered":"2024-02-22T19:38:47.168764+0000","last_unstale":"2024-02-22T19:40:47.173679+0000","last_undegraded":"2024-02-22T19:40:47.173679+0000","last_fullsized":"2024-02-22T19:40:47.173679+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":160,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:48.683218+0000","last_change":"2024-02-22T19:40:48.683218+0000","last_active":"2024-02-22T19:40:48.683218+0000","last_peered":"2024-02-22T19:40:48.683218+0000","last_clean":"2024-02-22T19:40:48.683218+0000","last_became_active":"2024-02-22T19:39:22.269400+0000","last_became_peered":"2024-02-22T19:39:22.269400+0000","last_unstale":"2024-02-22T19:40:48.683218+0000","last_undegraded":"2024-02-22T19:40:48.683218+0000","last_fullsized":"2024-02-22T19:40:48.683218+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":99,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:48.494721+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:40:48.494721+0000","last_peered":"2024-02-22T19:40:48.494721+0000","last_clean":"2024-02-22T19:40:48.494721+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:40:48.494721+0000","last_undegraded":"2024-02-22T19:40:48.494721+0000","last_fullsized":"2024-02-22T19:40:48.494721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":116,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171585+0000","last_change":"2024-02-22T19:39:53.438381+0000","last_active":"2024-02-22T19:40:47.171585+0000","last_peered":"2024-02-22T19:40:47.171585+0000","last_clean":"2024-02-22T19:40:47.171585+0000","last_became_active":"2024-02-22T19:38:47.179485+0000","last_became_peered":"2024-02-22T19:38:47.179485+0000","last_unstale":"2024-02-22T19:40:47.171585+0000","last_undegraded":"2024-02-22T19:40:47.171585+0000","last_fullsized":"2024-02-22T19:40:47.171585+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":102,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171183+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:40:47.171183+0000","last_peered":"2024-02-22T19:40:47.171183+0000","last_clean":"2024-02-22T19:40:47.171183+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:40:47.171183+0000","last_undegraded":"2024-02-22T19:40:47.171183+0000","last_fullsized":"2024-02-22T19:40:47.171183+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":255,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173359+0000","last_change":"2024-02-22T19:39:34.555380+0000","last_active":"2024-02-22T19:40:47.173359+0000","last_peered":"2024-02-22T19:40:47.173359+0000","last_clean":"2024-02-22T19:40:47.173359+0000","last_became_active":"2024-02-22T19:38:47.168230+0000","last_became_peered":"2024-02-22T19:38:47.168230+0000","last_unstale":"2024-02-22T19:40:47.173359+0000","last_undegraded":"2024-02-22T19:40:47.173359+0000","last_fullsized":"2024-02-22T19:40:47.173359+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:34.555298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.555298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":71,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169360+0000","last_change":"2024-02-22T19:39:22.245523+0000","last_active":"2024-02-22T19:40:47.169360+0000","last_peered":"2024-02-22T19:40:47.169360+0000","last_clean":"2024-02-22T19:40:47.169360+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:40:47.169360+0000","last_undegraded":"2024-02-22T19:40:47.169360+0000","last_fullsized":"2024-02-22T19:40:47.169360+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.732856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.732856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":329,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.192769+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:40:54.192769+0000","last_peered":"2024-02-22T19:40:54.192769+0000","last_clean":"2024-02-22T19:40:54.192769+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:40:54.192769+0000","last_undegraded":"2024-02-22T19:40:54.192769+0000","last_fullsized":"2024-02-22T19:40:54.192769+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":278,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171971+0000","last_change":"2024-02-22T19:37:57.180410+0000","last_active":"2024-02-22T19:40:47.171971+0000","last_peered":"2024-02-22T19:40:47.171971+0000","last_clean":"2024-02-22T19:40:47.171971+0000","last_became_active":"2024-02-22T19:37:57.180089+0000","last_became_peered":"2024-02-22T19:37:57.180089+0000","last_unstale":"2024-02-22T19:40:47.171971+0000","last_undegraded":"2024-02-22T19:40:47.171971+0000","last_fullsized":"2024-02-22T19:40:47.171971+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":89,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169652+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:40:47.169652+0000","last_peered":"2024-02-22T19:40:47.169652+0000","last_clean":"2024-02-22T19:40:47.169652+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:40:47.169652+0000","last_undegraded":"2024-02-22T19:40:47.169652+0000","last_fullsized":"2024-02-22T19:40:47.169652+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":322,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170949+0000","last_change":"2024-02-22T19:40:39.682028+0000","last_active":"2024-02-22T19:40:47.170949+0000","last_peered":"2024-02-22T19:40:47.170949+0000","last_clean":"2024-02-22T19:40:47.170949+0000","last_became_active":"2024-02-22T19:38:47.180453+0000","last_became_peered":"2024-02-22T19:38:47.180453+0000","last_unstale":"2024-02-22T19:40:47.170949+0000","last_undegraded":"2024-02-22T19:40:47.170949+0000","last_fullsized":"2024-02-22T19:40:47.170949+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":308,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169301+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:40:47.169301+0000","last_peered":"2024-02-22T19:40:47.169301+0000","last_clean":"2024-02-22T19:40:47.169301+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:40:47.169301+0000","last_undegraded":"2024-02-22T19:40:47.169301+0000","last_fullsized":"2024-02-22T19:40:47.169301+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":288,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:50.388588+0000","last_change":"2024-02-22T19:40:50.388588+0000","last_active":"2024-02-22T19:40:50.388588+0000","last_peered":"2024-02-22T19:40:50.388588+0000","last_clean":"2024-02-22T19:40:50.388588+0000","last_became_active":"2024-02-22T19:38:47.168149+0000","last_became_peered":"2024-02-22T19:38:47.168149+0000","last_unstale":"2024-02-22T19:40:50.388588+0000","last_undegraded":"2024-02-22T19:40:50.388588+0000","last_fullsized":"2024-02-22T19:40:50.388588+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":109,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169981+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:40:47.169981+0000","last_peered":"2024-02-22T19:40:47.169981+0000","last_clean":"2024-02-22T19:40:47.169981+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:40:47.169981+0000","last_undegraded":"2024-02-22T19:40:47.169981+0000","last_fullsized":"2024-02-22T19:40:47.169981+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":297,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168024+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:40:47.168024+0000","last_peered":"2024-02-22T19:40:47.168024+0000","last_clean":"2024-02-22T19:40:47.168024+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:40:47.168024+0000","last_undegraded":"2024-02-22T19:40:47.168024+0000","last_fullsized":"2024-02-22T19:40:47.168024+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":295,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170571+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:40:47.170571+0000","last_peered":"2024-02-22T19:40:47.170571+0000","last_clean":"2024-02-22T19:40:47.170571+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:40:47.170571+0000","last_undegraded":"2024-02-22T19:40:47.170571+0000","last_fullsized":"2024-02-22T19:40:47.170571+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":378,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.582930+0000","last_change":"2024-02-22T19:40:49.582930+0000","last_active":"2024-02-22T19:40:49.582930+0000","last_peered":"2024-02-22T19:40:49.582930+0000","last_clean":"2024-02-22T19:40:49.582930+0000","last_became_active":"2024-02-22T19:37:57.190476+0000","last_became_peered":"2024-02-22T19:37:57.190476+0000","last_unstale":"2024-02-22T19:40:49.582930+0000","last_undegraded":"2024-02-22T19:40:49.582930+0000","last_fullsized":"2024-02-22T19:40:49.582930+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":277,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168359+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:40:47.168359+0000","last_peered":"2024-02-22T19:40:47.168359+0000","last_clean":"2024-02-22T19:40:47.168359+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:40:47.168359+0000","last_undegraded":"2024-02-22T19:40:47.168359+0000","last_fullsized":"2024-02-22T19:40:47.168359+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":293,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173672+0000","last_change":"2024-02-22T19:39:22.256927+0000","last_active":"2024-02-22T19:40:47.173672+0000","last_peered":"2024-02-22T19:40:47.173672+0000","last_clean":"2024-02-22T19:40:47.173672+0000","last_became_active":"2024-02-22T19:39:22.256576+0000","last_became_peered":"2024-02-22T19:39:22.256576+0000","last_unstale":"2024-02-22T19:40:47.173672+0000","last_undegraded":"2024-02-22T19:40:47.173672+0000","last_fullsized":"2024-02-22T19:40:47.173672+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.400534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.400534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":280,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173534+0000","last_change":"2024-02-22T19:39:22.246647+0000","last_active":"2024-02-22T19:40:47.173534+0000","last_peered":"2024-02-22T19:40:47.173534+0000","last_clean":"2024-02-22T19:40:47.173534+0000","last_became_active":"2024-02-22T19:39:22.246323+0000","last_became_peered":"2024-02-22T19:39:22.246323+0000","last_unstale":"2024-02-22T19:40:47.173534+0000","last_undegraded":"2024-02-22T19:40:47.173534+0000","last_fullsized":"2024-02-22T19:40:47.173534+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":276,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170633+0000","last_change":"2024-02-22T19:36:58.263929+0000","last_active":"2024-02-22T19:40:47.170633+0000","last_peered":"2024-02-22T19:40:47.170633+0000","last_clean":"2024-02-22T19:40:47.170633+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:40:47.170633+0000","last_undegraded":"2024-02-22T19:40:47.170633+0000","last_fullsized":"2024-02-22T19:40:47.170633+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:36:58.263834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.263834+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":289,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171532+0000","last_change":"2024-02-22T19:39:22.250184+0000","last_active":"2024-02-22T19:40:47.171532+0000","last_peered":"2024-02-22T19:40:47.171532+0000","last_clean":"2024-02-22T19:40:47.171532+0000","last_became_active":"2024-02-22T19:39:22.249890+0000","last_became_peered":"2024-02-22T19:39:22.249890+0000","last_unstale":"2024-02-22T19:40:47.171532+0000","last_undegraded":"2024-02-22T19:40:47.171532+0000","last_fullsized":"2024-02-22T19:40:47.171532+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":295,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168795+0000","last_change":"2024-02-22T19:39:22.253288+0000","last_active":"2024-02-22T19:40:47.168795+0000","last_peered":"2024-02-22T19:40:47.168795+0000","last_clean":"2024-02-22T19:40:47.168795+0000","last_became_active":"2024-02-22T19:39:22.252705+0000","last_became_peered":"2024-02-22T19:39:22.252705+0000","last_unstale":"2024-02-22T19:40:47.168795+0000","last_undegraded":"2024-02-22T19:40:47.168795+0000","last_fullsized":"2024-02-22T19:40:47.168795+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:54.418958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:37:54.418958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":94,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173978+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:40:47.173978+0000","last_peered":"2024-02-22T19:40:47.173978+0000","last_clean":"2024-02-22T19:40:47.173978+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:40:47.173978+0000","last_undegraded":"2024-02-22T19:40:47.173978+0000","last_fullsized":"2024-02-22T19:40:47.173978+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":295,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.174911+0000","last_change":"2024-02-22T19:39:37.816765+0000","last_active":"2024-02-22T19:40:47.174911+0000","last_peered":"2024-02-22T19:40:47.174911+0000","last_clean":"2024-02-22T19:40:47.174911+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:40:47.174911+0000","last_undegraded":"2024-02-22T19:40:47.174911+0000","last_fullsized":"2024-02-22T19:40:47.174911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":271,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168861+0000","last_change":"2024-02-22T19:39:22.247453+0000","last_active":"2024-02-22T19:40:47.168861+0000","last_peered":"2024-02-22T19:40:47.168861+0000","last_clean":"2024-02-22T19:40:47.168861+0000","last_became_active":"2024-02-22T19:39:22.247266+0000","last_became_peered":"2024-02-22T19:39:22.247266+0000","last_unstale":"2024-02-22T19:40:47.168861+0000","last_undegraded":"2024-02-22T19:40:47.168861+0000","last_fullsized":"2024-02-22T19:40:47.168861+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":289,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171273+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:40:47.171273+0000","last_peered":"2024-02-22T19:40:47.171273+0000","last_clean":"2024-02-22T19:40:47.171273+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:40:47.171273+0000","last_undegraded":"2024-02-22T19:40:47.171273+0000","last_fullsized":"2024-02-22T19:40:47.171273+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":303,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170052+0000","last_change":"2024-02-22T19:39:22.254738+0000","last_active":"2024-02-22T19:40:47.170052+0000","last_peered":"2024-02-22T19:40:47.170052+0000","last_clean":"2024-02-22T19:40:47.170052+0000","last_became_active":"2024-02-22T19:39:22.254411+0000","last_became_peered":"2024-02-22T19:39:22.254411+0000","last_unstale":"2024-02-22T19:40:47.170052+0000","last_undegraded":"2024-02-22T19:40:47.170052+0000","last_fullsized":"2024-02-22T19:40:47.170052+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:52.463774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.463774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":297,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169868+0000","last_change":"2024-02-22T19:39:44.297110+0000","last_active":"2024-02-22T19:40:47.169868+0000","last_peered":"2024-02-22T19:40:47.169868+0000","last_clean":"2024-02-22T19:40:47.169868+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:40:47.169868+0000","last_undegraded":"2024-02-22T19:40:47.169868+0000","last_fullsized":"2024-02-22T19:40:47.169868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:39:44.296975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:44.296975+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":310,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:50.680522+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:40:50.680522+0000","last_peered":"2024-02-22T19:40:50.680522+0000","last_clean":"2024-02-22T19:40:50.680522+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:40:50.680522+0000","last_undegraded":"2024-02-22T19:40:50.680522+0000","last_fullsized":"2024-02-22T19:40:50.680522+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":288,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169791+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:40:47.169791+0000","last_peered":"2024-02-22T19:40:47.169791+0000","last_clean":"2024-02-22T19:40:47.169791+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:40:47.169791+0000","last_undegraded":"2024-02-22T19:40:47.169791+0000","last_fullsized":"2024-02-22T19:40:47.169791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":71,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:52.560079+0000","last_change":"2024-02-22T19:40:52.560079+0000","last_active":"2024-02-22T19:40:52.560079+0000","last_peered":"2024-02-22T19:40:52.560079+0000","last_clean":"2024-02-22T19:40:52.560079+0000","last_became_active":"2024-02-22T19:38:47.161791+0000","last_became_peered":"2024-02-22T19:38:47.161791+0000","last_unstale":"2024-02-22T19:40:52.560079+0000","last_undegraded":"2024-02-22T19:40:52.560079+0000","last_fullsized":"2024-02-22T19:40:52.560079+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":120,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172676+0000","last_change":"2024-02-22T19:38:47.783561+0000","last_active":"2024-02-22T19:40:47.172676+0000","last_peered":"2024-02-22T19:40:47.172676+0000","last_clean":"2024-02-22T19:40:47.172676+0000","last_became_active":"2024-02-22T19:38:47.168328+0000","last_became_peered":"2024-02-22T19:38:47.168328+0000","last_unstale":"2024-02-22T19:40:47.172676+0000","last_undegraded":"2024-02-22T19:40:47.172676+0000","last_fullsized":"2024-02-22T19:40:47.172676+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:53.461286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.461286+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":97,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169676+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:40:47.169676+0000","last_peered":"2024-02-22T19:40:47.169676+0000","last_clean":"2024-02-22T19:40:47.169676+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:40:47.169676+0000","last_undegraded":"2024-02-22T19:40:47.169676+0000","last_fullsized":"2024-02-22T19:40:47.169676+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":176,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.301561+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:40:49.301561+0000","last_peered":"2024-02-22T19:40:49.301561+0000","last_clean":"2024-02-22T19:40:49.301561+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:40:49.301561+0000","last_undegraded":"2024-02-22T19:40:49.301561+0000","last_fullsized":"2024-02-22T19:40:49.301561+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":276,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168832+0000","last_change":"2024-02-22T19:38:47.162564+0000","last_active":"2024-02-22T19:40:47.168832+0000","last_peered":"2024-02-22T19:40:47.168832+0000","last_clean":"2024-02-22T19:40:47.168832+0000","last_became_active":"2024-02-22T19:38:47.162182+0000","last_became_peered":"2024-02-22T19:38:47.162182+0000","last_unstale":"2024-02-22T19:40:47.168832+0000","last_undegraded":"2024-02-22T19:40:47.168832+0000","last_fullsized":"2024-02-22T19:40:47.168832+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:56.463502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:36:56.463502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":299,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171414+0000","last_change":"2024-02-22T19:37:09.345416+0000","last_active":"2024-02-22T19:40:47.171414+0000","last_peered":"2024-02-22T19:40:47.171414+0000","last_clean":"2024-02-22T19:40:47.171414+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:40:47.171414+0000","last_undegraded":"2024-02-22T19:40:47.171414+0000","last_fullsized":"2024-02-22T19:40:47.171414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:37:09.345319+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.345319+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":277,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160176+0000","last_change":"2024-02-22T19:39:41.365968+0000","last_active":"2024-02-22T19:40:47.160176+0000","last_peered":"2024-02-22T19:40:47.160176+0000","last_clean":"2024-02-22T19:40:47.160176+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:40:47.160176+0000","last_undegraded":"2024-02-22T19:40:47.160176+0000","last_fullsized":"2024-02-22T19:40:47.160176+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:41.365877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.365877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":413,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.299748+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:40:49.299748+0000","last_peered":"2024-02-22T19:40:49.299748+0000","last_clean":"2024-02-22T19:40:49.299748+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:40:49.299748+0000","last_undegraded":"2024-02-22T19:40:49.299748+0000","last_fullsized":"2024-02-22T19:40:49.299748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":83,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169071+0000","last_change":"2024-02-22T19:39:22.246092+0000","last_active":"2024-02-22T19:40:47.169071+0000","last_peered":"2024-02-22T19:40:47.169071+0000","last_clean":"2024-02-22T19:40:47.169071+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:40:47.169071+0000","last_undegraded":"2024-02-22T19:40:47.169071+0000","last_fullsized":"2024-02-22T19:40:47.169071+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":113,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173861+0000","last_change":"2024-02-22T19:37:57.198915+0000","last_active":"2024-02-22T19:40:47.173861+0000","last_peered":"2024-02-22T19:40:47.173861+0000","last_clean":"2024-02-22T19:40:47.173861+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:40:47.173861+0000","last_undegraded":"2024-02-22T19:40:47.173861+0000","last_fullsized":"2024-02-22T19:40:47.173861+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":300,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160427+0000","last_change":"2024-02-22T19:39:53.126573+0000","last_active":"2024-02-22T19:40:47.160427+0000","last_peered":"2024-02-22T19:40:47.160427+0000","last_clean":"2024-02-22T19:40:47.160427+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:40:47.160427+0000","last_undegraded":"2024-02-22T19:40:47.160427+0000","last_fullsized":"2024-02-22T19:40:47.160427+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:53.126490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.126490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":299,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170127+0000","last_change":"2024-02-22T19:39:22.255233+0000","last_active":"2024-02-22T19:40:47.170127+0000","last_peered":"2024-02-22T19:40:47.170127+0000","last_clean":"2024-02-22T19:40:47.170127+0000","last_became_active":"2024-02-22T19:39:22.254888+0000","last_became_peered":"2024-02-22T19:39:22.254888+0000","last_unstale":"2024-02-22T19:40:47.170127+0000","last_undegraded":"2024-02-22T19:40:47.170127+0000","last_fullsized":"2024-02-22T19:40:47.170127+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.470149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.470149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":90,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171531+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:40:47.171531+0000","last_peered":"2024-02-22T19:40:47.171531+0000","last_clean":"2024-02-22T19:40:47.171531+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:40:47.171531+0000","last_undegraded":"2024-02-22T19:40:47.171531+0000","last_fullsized":"2024-02-22T19:40:47.171531+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":275,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169980+0000","last_change":"2024-02-22T19:38:47.162050+0000","last_active":"2024-02-22T19:40:47.169980+0000","last_peered":"2024-02-22T19:40:47.169980+0000","last_clean":"2024-02-22T19:40:47.169980+0000","last_became_active":"2024-02-22T19:38:47.161672+0000","last_became_peered":"2024-02-22T19:38:47.161672+0000","last_unstale":"2024-02-22T19:40:47.169980+0000","last_undegraded":"2024-02-22T19:40:47.169980+0000","last_fullsized":"2024-02-22T19:40:47.169980+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":322,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172070+0000","last_change":"2024-02-22T19:39:54.559858+0000","last_active":"2024-02-22T19:40:47.172070+0000","last_peered":"2024-02-22T19:40:47.172070+0000","last_clean":"2024-02-22T19:40:47.172070+0000","last_became_active":"2024-02-22T19:39:22.253387+0000","last_became_peered":"2024-02-22T19:39:22.253387+0000","last_unstale":"2024-02-22T19:40:47.172070+0000","last_undegraded":"2024-02-22T19:40:47.172070+0000","last_fullsized":"2024-02-22T19:40:47.172070+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":304,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.668144+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:40:47.668144+0000","last_peered":"2024-02-22T19:40:47.668144+0000","last_clean":"2024-02-22T19:40:47.668144+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:40:47.668144+0000","last_undegraded":"2024-02-22T19:40:47.668144+0000","last_fullsized":"2024-02-22T19:40:47.668144+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"302'4688","reported_seq":4713,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.335617+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:40:54.335617+0000","last_peered":"2024-02-22T19:40:54.335617+0000","last_clean":"2024-02-22T19:40:54.335617+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:40:54.335617+0000","last_undegraded":"2024-02-22T19:40:54.335617+0000","last_fullsized":"2024-02-22T19:40:54.335617+0000","mapping_epoch":285,"log_start":"291'800","ondisk_log_start":"291'800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3888,"ondisk_log_size":3888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":307232768,"num_objects":4688,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4688,"num_write_kb":300032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":126,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.502459+0000","last_change":"2024-02-22T19:40:49.502459+0000","last_active":"2024-02-22T19:40:49.502459+0000","last_peered":"2024-02-22T19:40:49.502459+0000","last_clean":"2024-02-22T19:40:49.502459+0000","last_became_active":"2024-02-22T19:39:22.272214+0000","last_became_peered":"2024-02-22T19:39:22.272214+0000","last_unstale":"2024-02-22T19:40:49.502459+0000","last_undegraded":"2024-02-22T19:40:49.502459+0000","last_fullsized":"2024-02-22T19:40:49.502459+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":347,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169728+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:40:47.169728+0000","last_peered":"2024-02-22T19:40:47.169728+0000","last_clean":"2024-02-22T19:40:47.169728+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:40:47.169728+0000","last_undegraded":"2024-02-22T19:40:47.169728+0000","last_fullsized":"2024-02-22T19:40:47.169728+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":133,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168683+0000","last_change":"2024-02-22T19:40:46.420455+0000","last_active":"2024-02-22T19:40:47.168683+0000","last_peered":"2024-02-22T19:40:47.168683+0000","last_clean":"2024-02-22T19:40:47.168683+0000","last_became_active":"2024-02-22T19:37:57.181199+0000","last_became_peered":"2024-02-22T19:37:57.181199+0000","last_unstale":"2024-02-22T19:40:47.168683+0000","last_undegraded":"2024-02-22T19:40:47.168683+0000","last_fullsized":"2024-02-22T19:40:47.168683+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"251'13","reported_seq":460,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.302922+0000","last_change":"2024-02-22T19:38:50.149948+0000","last_active":"2024-02-22T19:40:49.302922+0000","last_peered":"2024-02-22T19:40:49.302922+0000","last_clean":"2024-02-22T19:40:49.302922+0000","last_became_active":"2024-02-22T19:38:50.148285+0000","last_became_peered":"2024-02-22T19:38:50.148285+0000","last_unstale":"2024-02-22T19:40:49.302922+0000","last_undegraded":"2024-02-22T19:40:49.302922+0000","last_fullsized":"2024-02-22T19:40:49.302922+0000","mapping_epoch":250,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:36:53.503128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.503128+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"302'4781","reported_seq":4806,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.455465+0000","last_change":"2024-02-22T19:40:00.882365+0000","last_active":"2024-02-22T19:40:54.455465+0000","last_peered":"2024-02-22T19:40:54.455465+0000","last_clean":"2024-02-22T19:40:54.455465+0000","last_became_active":"2024-02-22T19:40:00.882035+0000","last_became_peered":"2024-02-22T19:40:00.882035+0000","last_unstale":"2024-02-22T19:40:54.455465+0000","last_undegraded":"2024-02-22T19:40:54.455465+0000","last_fullsized":"2024-02-22T19:40:54.455465+0000","mapping_epoch":285,"log_start":"291'800","ondisk_log_start":"291'800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3981,"ondisk_log_size":3981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":313327616,"num_objects":4781,"num_object_clones":0,"num_object_copies":14343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4781,"num_write_kb":305984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":260,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171444+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:40:47.171444+0000","last_peered":"2024-02-22T19:40:47.171444+0000","last_clean":"2024-02-22T19:40:47.171444+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:40:47.171444+0000","last_undegraded":"2024-02-22T19:40:47.171444+0000","last_fullsized":"2024-02-22T19:40:47.171444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":311,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169873+0000","last_change":"2024-02-22T19:39:22.248740+0000","last_active":"2024-02-22T19:40:47.169873+0000","last_peered":"2024-02-22T19:40:47.169873+0000","last_clean":"2024-02-22T19:40:47.169873+0000","last_became_active":"2024-02-22T19:39:22.248120+0000","last_became_peered":"2024-02-22T19:39:22.248120+0000","last_unstale":"2024-02-22T19:40:47.169873+0000","last_undegraded":"2024-02-22T19:40:47.169873+0000","last_fullsized":"2024-02-22T19:40:47.169873+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.431527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.431527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":127,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:48.455295+0000","last_change":"2024-02-22T19:40:48.455295+0000","last_active":"2024-02-22T19:40:48.455295+0000","last_peered":"2024-02-22T19:40:48.455295+0000","last_clean":"2024-02-22T19:40:48.455295+0000","last_became_active":"2024-02-22T19:38:47.167662+0000","last_became_peered":"2024-02-22T19:38:47.167662+0000","last_unstale":"2024-02-22T19:40:48.455295+0000","last_undegraded":"2024-02-22T19:40:48.455295+0000","last_fullsized":"2024-02-22T19:40:48.455295+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":315,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160016+0000","last_change":"2024-02-22T19:39:48.227541+0000","last_active":"2024-02-22T19:40:47.160016+0000","last_peered":"2024-02-22T19:40:47.160016+0000","last_clean":"2024-02-22T19:40:47.160016+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:40:47.160016+0000","last_undegraded":"2024-02-22T19:40:47.160016+0000","last_fullsized":"2024-02-22T19:40:47.160016+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.227439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.227439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"302'4639","reported_seq":4662,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.355710+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:40:54.355710+0000","last_peered":"2024-02-22T19:40:54.355710+0000","last_clean":"2024-02-22T19:40:54.355710+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:40:54.355710+0000","last_undegraded":"2024-02-22T19:40:54.355710+0000","last_fullsized":"2024-02-22T19:40:54.355710+0000","mapping_epoch":285,"log_start":"291'800","ondisk_log_start":"291'800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3839,"ondisk_log_size":3839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":304021504,"num_objects":4639,"num_object_clones":0,"num_object_copies":13917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":296896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":261,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168500+0000","last_change":"2024-02-22T19:36:54.489612+0000","last_active":"2024-02-22T19:40:47.168500+0000","last_peered":"2024-02-22T19:40:47.168500+0000","last_clean":"2024-02-22T19:40:47.168500+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:40:47.168500+0000","last_undegraded":"2024-02-22T19:40:47.168500+0000","last_fullsized":"2024-02-22T19:40:47.168500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.489509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.489509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":341,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.165012+0000","last_change":"2024-02-22T19:39:39.403851+0000","last_active":"2024-02-22T19:40:47.165012+0000","last_peered":"2024-02-22T19:40:47.165012+0000","last_clean":"2024-02-22T19:40:47.165012+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:40:47.165012+0000","last_undegraded":"2024-02-22T19:40:47.165012+0000","last_fullsized":"2024-02-22T19:40:47.165012+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:39.403775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:39:39.403775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":290,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170168+0000","last_change":"2024-02-22T19:37:57.185054+0000","last_active":"2024-02-22T19:40:47.170168+0000","last_peered":"2024-02-22T19:40:47.170168+0000","last_clean":"2024-02-22T19:40:47.170168+0000","last_became_active":"2024-02-22T19:37:57.184627+0000","last_became_peered":"2024-02-22T19:37:57.184627+0000","last_unstale":"2024-02-22T19:40:47.170168+0000","last_undegraded":"2024-02-22T19:40:47.170168+0000","last_fullsized":"2024-02-22T19:40:47.170168+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.438112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.438112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":328,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172846+0000","last_change":"2024-02-22T19:40:31.504524+0000","last_active":"2024-02-22T19:40:47.172846+0000","last_peered":"2024-02-22T19:40:47.172846+0000","last_clean":"2024-02-22T19:40:47.172846+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:40:47.172846+0000","last_undegraded":"2024-02-22T19:40:47.172846+0000","last_fullsized":"2024-02-22T19:40:47.172846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.6","version":"302'4691","reported_seq":4716,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.347966+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:40:54.347966+0000","last_peered":"2024-02-22T19:40:54.347966+0000","last_clean":"2024-02-22T19:40:54.347966+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:40:54.347966+0000","last_undegraded":"2024-02-22T19:40:54.347966+0000","last_fullsized":"2024-02-22T19:40:54.347966+0000","mapping_epoch":285,"log_start":"291'800","ondisk_log_start":"291'800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3891,"ondisk_log_size":3891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":307429376,"num_objects":4691,"num_object_clones":0,"num_object_copies":14073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":300224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":88,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:51.594081+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:40:51.594081+0000","last_peered":"2024-02-22T19:40:51.594081+0000","last_clean":"2024-02-22T19:40:51.594081+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:40:51.594081+0000","last_undegraded":"2024-02-22T19:40:51.594081+0000","last_fullsized":"2024-02-22T19:40:51.594081+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":320,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160904+0000","last_change":"2024-02-22T19:39:54.127947+0000","last_active":"2024-02-22T19:40:47.160904+0000","last_peered":"2024-02-22T19:40:47.160904+0000","last_clean":"2024-02-22T19:40:47.160904+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:40:47.160904+0000","last_undegraded":"2024-02-22T19:40:47.160904+0000","last_fullsized":"2024-02-22T19:40:47.160904+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.127876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.127876+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":94,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171508+0000","last_change":"2024-02-22T19:37:57.182314+0000","last_active":"2024-02-22T19:40:47.171508+0000","last_peered":"2024-02-22T19:40:47.171508+0000","last_clean":"2024-02-22T19:40:47.171508+0000","last_became_active":"2024-02-22T19:37:57.181963+0000","last_became_peered":"2024-02-22T19:37:57.181963+0000","last_unstale":"2024-02-22T19:40:47.171508+0000","last_undegraded":"2024-02-22T19:40:47.171508+0000","last_fullsized":"2024-02-22T19:40:47.171508+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.297621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.297621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":100,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:48.125201+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:40:48.125201+0000","last_peered":"2024-02-22T19:40:48.125201+0000","last_clean":"2024-02-22T19:40:48.125201+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:40:48.125201+0000","last_undegraded":"2024-02-22T19:40:48.125201+0000","last_fullsized":"2024-02-22T19:40:48.125201+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"302'4715","reported_seq":4740,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.291405+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:40:54.291405+0000","last_peered":"2024-02-22T19:40:54.291405+0000","last_clean":"2024-02-22T19:40:54.291405+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:40:54.291405+0000","last_undegraded":"2024-02-22T19:40:54.291405+0000","last_fullsized":"2024-02-22T19:40:54.291405+0000","mapping_epoch":285,"log_start":"291'700","ondisk_log_start":"291'700","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4015,"ondisk_log_size":4015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":309002240,"num_objects":4715,"num_object_clones":0,"num_object_copies":14145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4715,"num_write_kb":301760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170668+0000","last_change":"2024-02-22T19:39:22.262082+0000","last_active":"2024-02-22T19:40:47.170668+0000","last_peered":"2024-02-22T19:40:47.170668+0000","last_clean":"2024-02-22T19:40:47.170668+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:40:47.170668+0000","last_undegraded":"2024-02-22T19:40:47.170668+0000","last_fullsized":"2024-02-22T19:40:47.170668+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:25.576862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:25.576862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":269,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171625+0000","last_change":"2024-02-22T19:38:47.171603+0000","last_active":"2024-02-22T19:40:47.171625+0000","last_peered":"2024-02-22T19:40:47.171625+0000","last_clean":"2024-02-22T19:40:47.171625+0000","last_became_active":"2024-02-22T19:38:47.171002+0000","last_became_peered":"2024-02-22T19:38:47.171002+0000","last_unstale":"2024-02-22T19:40:47.171625+0000","last_undegraded":"2024-02-22T19:40:47.171625+0000","last_fullsized":"2024-02-22T19:40:47.171625+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:45.317440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:45.317440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171682+0000","last_change":"2024-02-22T19:37:57.185637+0000","last_active":"2024-02-22T19:40:47.171682+0000","last_peered":"2024-02-22T19:40:47.171682+0000","last_clean":"2024-02-22T19:40:47.171682+0000","last_became_active":"2024-02-22T19:37:57.185312+0000","last_became_peered":"2024-02-22T19:37:57.185312+0000","last_unstale":"2024-02-22T19:40:47.171682+0000","last_undegraded":"2024-02-22T19:40:47.171682+0000","last_fullsized":"2024-02-22T19:40:47.171682+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":274,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172540+0000","last_change":"2024-02-22T19:39:51.569245+0000","last_active":"2024-02-22T19:40:47.172540+0000","last_peered":"2024-02-22T19:40:47.172540+0000","last_clean":"2024-02-22T19:40:47.172540+0000","last_became_active":"2024-02-22T19:39:22.251742+0000","last_became_peered":"2024-02-22T19:39:22.251742+0000","last_unstale":"2024-02-22T19:40:47.172540+0000","last_undegraded":"2024-02-22T19:40:47.172540+0000","last_fullsized":"2024-02-22T19:40:47.172540+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":489,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.300115+0000","last_change":"2024-02-22T19:39:37.302965+0000","last_active":"2024-02-22T19:40:49.300115+0000","last_peered":"2024-02-22T19:40:49.300115+0000","last_clean":"2024-02-22T19:40:49.300115+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:40:49.300115+0000","last_undegraded":"2024-02-22T19:40:49.300115+0000","last_fullsized":"2024-02-22T19:40:49.300115+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:39:37.302890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.302890+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":97,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168097+0000","last_change":"2024-02-22T19:39:46.299764+0000","last_active":"2024-02-22T19:40:47.168097+0000","last_peered":"2024-02-22T19:40:47.168097+0000","last_clean":"2024-02-22T19:40:47.168097+0000","last_became_active":"2024-02-22T19:39:22.251614+0000","last_became_peered":"2024-02-22T19:39:22.251614+0000","last_unstale":"2024-02-22T19:40:47.168097+0000","last_undegraded":"2024-02-22T19:40:47.168097+0000","last_fullsized":"2024-02-22T19:40:47.168097+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"302'4648","reported_seq":4673,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.338603+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:40:54.338603+0000","last_peered":"2024-02-22T19:40:54.338603+0000","last_clean":"2024-02-22T19:40:54.338603+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:40:54.338603+0000","last_undegraded":"2024-02-22T19:40:54.338603+0000","last_fullsized":"2024-02-22T19:40:54.338603+0000","mapping_epoch":285,"log_start":"291'800","ondisk_log_start":"291'800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3848,"ondisk_log_size":3848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":304611328,"num_objects":4648,"num_object_clones":0,"num_object_copies":13944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":297472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":325,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170212+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:40:47.170212+0000","last_peered":"2024-02-22T19:40:47.170212+0000","last_clean":"2024-02-22T19:40:47.170212+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:40:47.170212+0000","last_undegraded":"2024-02-22T19:40:47.170212+0000","last_fullsized":"2024-02-22T19:40:47.170212+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":266,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170325+0000","last_change":"2024-02-22T19:39:47.645026+0000","last_active":"2024-02-22T19:40:47.170325+0000","last_peered":"2024-02-22T19:40:47.170325+0000","last_clean":"2024-02-22T19:40:47.170325+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:40:47.170325+0000","last_undegraded":"2024-02-22T19:40:47.170325+0000","last_fullsized":"2024-02-22T19:40:47.170325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":274,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.166726+0000","last_change":"2024-02-22T19:39:45.253946+0000","last_active":"2024-02-22T19:40:47.166726+0000","last_peered":"2024-02-22T19:40:47.166726+0000","last_clean":"2024-02-22T19:40:47.166726+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:40:47.166726+0000","last_undegraded":"2024-02-22T19:40:47.166726+0000","last_fullsized":"2024-02-22T19:40:47.166726+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.253853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.253853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.163532+0000","last_change":"2024-02-22T19:39:52.175249+0000","last_active":"2024-02-22T19:40:47.163532+0000","last_peered":"2024-02-22T19:40:47.163532+0000","last_clean":"2024-02-22T19:40:47.163532+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:40:47.163532+0000","last_undegraded":"2024-02-22T19:40:47.163532+0000","last_fullsized":"2024-02-22T19:40:47.163532+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:39:52.175144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":371,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160225+0000","last_change":"2024-02-22T19:39:34.410627+0000","last_active":"2024-02-22T19:40:47.160225+0000","last_peered":"2024-02-22T19:40:47.160225+0000","last_clean":"2024-02-22T19:40:47.160225+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:40:47.160225+0000","last_undegraded":"2024-02-22T19:40:47.160225+0000","last_fullsized":"2024-02-22T19:40:47.160225+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:39:34.410547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.410547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":314,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.164520+0000","last_change":"2024-02-22T19:39:51.129196+0000","last_active":"2024-02-22T19:40:47.164520+0000","last_peered":"2024-02-22T19:40:47.164520+0000","last_clean":"2024-02-22T19:40:47.164520+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:40:47.164520+0000","last_undegraded":"2024-02-22T19:40:47.164520+0000","last_fullsized":"2024-02-22T19:40:47.164520+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.129010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.129010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"302'4681","reported_seq":4706,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.613275+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:40:54.613275+0000","last_peered":"2024-02-22T19:40:54.613275+0000","last_clean":"2024-02-22T19:40:54.613275+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:40:54.613275+0000","last_undegraded":"2024-02-22T19:40:54.613275+0000","last_fullsized":"2024-02-22T19:40:54.613275+0000","mapping_epoch":285,"log_start":"291'1000","ondisk_log_start":"291'1000","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3681,"ondisk_log_size":3681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":306774016,"num_objects":4681,"num_object_clones":0,"num_object_copies":14043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4681,"num_write_kb":299584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":108,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169516+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:40:47.169516+0000","last_peered":"2024-02-22T19:40:47.169516+0000","last_clean":"2024-02-22T19:40:47.169516+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:40:47.169516+0000","last_undegraded":"2024-02-22T19:40:47.169516+0000","last_fullsized":"2024-02-22T19:40:47.169516+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":316,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.591722+0000","last_change":"2024-02-22T19:40:47.591722+0000","last_active":"2024-02-22T19:40:47.591722+0000","last_peered":"2024-02-22T19:40:47.591722+0000","last_clean":"2024-02-22T19:40:47.591722+0000","last_became_active":"2024-02-22T19:39:22.252166+0000","last_became_peered":"2024-02-22T19:39:22.252166+0000","last_unstale":"2024-02-22T19:40:47.591722+0000","last_undegraded":"2024-02-22T19:40:47.591722+0000","last_fullsized":"2024-02-22T19:40:47.591722+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":284,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170679+0000","last_change":"2024-02-22T19:39:47.705812+0000","last_active":"2024-02-22T19:40:47.170679+0000","last_peered":"2024-02-22T19:40:47.170679+0000","last_clean":"2024-02-22T19:40:47.170679+0000","last_became_active":"2024-02-22T19:37:57.180638+0000","last_became_peered":"2024-02-22T19:37:57.180638+0000","last_unstale":"2024-02-22T19:40:47.170679+0000","last_undegraded":"2024-02-22T19:40:47.170679+0000","last_fullsized":"2024-02-22T19:40:47.170679+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":390,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172951+0000","last_change":"2024-02-22T19:39:22.439994+0000","last_active":"2024-02-22T19:40:47.172951+0000","last_peered":"2024-02-22T19:40:47.172951+0000","last_clean":"2024-02-22T19:40:47.172951+0000","last_became_active":"2024-02-22T19:39:22.256569+0000","last_became_peered":"2024-02-22T19:39:22.256569+0000","last_unstale":"2024-02-22T19:40:47.172951+0000","last_undegraded":"2024-02-22T19:40:47.172951+0000","last_fullsized":"2024-02-22T19:40:47.172951+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:37:47.421622+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:47.421622+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":286,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169409+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:40:47.169409+0000","last_peered":"2024-02-22T19:40:47.169409+0000","last_clean":"2024-02-22T19:40:47.169409+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:40:47.169409+0000","last_undegraded":"2024-02-22T19:40:47.169409+0000","last_fullsized":"2024-02-22T19:40:47.169409+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"302'4675","reported_seq":4700,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.291152+0000","last_change":"2024-02-22T19:40:01.098193+0000","last_active":"2024-02-22T19:40:54.291152+0000","last_peered":"2024-02-22T19:40:54.291152+0000","last_clean":"2024-02-22T19:40:54.291152+0000","last_became_active":"2024-02-22T19:40:01.097792+0000","last_became_peered":"2024-02-22T19:40:01.097792+0000","last_unstale":"2024-02-22T19:40:54.291152+0000","last_undegraded":"2024-02-22T19:40:54.291152+0000","last_fullsized":"2024-02-22T19:40:54.291152+0000","mapping_epoch":285,"log_start":"290'200","ondisk_log_start":"290'200","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4475,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":306380800,"num_objects":4675,"num_object_clones":0,"num_object_copies":14025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4675,"num_write_kb":299200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":170,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.167125+0000","last_change":"2024-02-22T19:40:41.405503+0000","last_active":"2024-02-22T19:40:47.167125+0000","last_peered":"2024-02-22T19:40:47.167125+0000","last_clean":"2024-02-22T19:40:47.167125+0000","last_became_active":"2024-02-22T19:38:47.167994+0000","last_became_peered":"2024-02-22T19:38:47.167994+0000","last_unstale":"2024-02-22T19:40:47.167125+0000","last_undegraded":"2024-02-22T19:40:47.167125+0000","last_fullsized":"2024-02-22T19:40:47.167125+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":286,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.166964+0000","last_change":"2024-02-22T19:39:50.386984+0000","last_active":"2024-02-22T19:40:47.166964+0000","last_peered":"2024-02-22T19:40:47.166964+0000","last_clean":"2024-02-22T19:40:47.166964+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:40:47.166964+0000","last_undegraded":"2024-02-22T19:40:47.166964+0000","last_fullsized":"2024-02-22T19:40:47.166964+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.386916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.386916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":76,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.174689+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:40:47.174689+0000","last_peered":"2024-02-22T19:40:47.174689+0000","last_clean":"2024-02-22T19:40:47.174689+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:40:47.174689+0000","last_undegraded":"2024-02-22T19:40:47.174689+0000","last_fullsized":"2024-02-22T19:40:47.174689+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1025,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:52.485561+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:40:52.485561+0000","last_peered":"2024-02-22T19:40:52.485561+0000","last_clean":"2024-02-22T19:40:52.485561+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:40:52.485561+0000","last_undegraded":"2024-02-22T19:40:52.485561+0000","last_fullsized":"2024-02-22T19:40:52.485561+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":696,"num_read_kb":691,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":108,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168374+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:40:47.168374+0000","last_peered":"2024-02-22T19:40:47.168374+0000","last_clean":"2024-02-22T19:40:47.168374+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:40:47.168374+0000","last_undegraded":"2024-02-22T19:40:47.168374+0000","last_fullsized":"2024-02-22T19:40:47.168374+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":153,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:52.541524+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:40:52.541524+0000","last_peered":"2024-02-22T19:40:52.541524+0000","last_clean":"2024-02-22T19:40:52.541524+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:40:52.541524+0000","last_undegraded":"2024-02-22T19:40:52.541524+0000","last_fullsized":"2024-02-22T19:40:52.541524+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"302'4682","reported_seq":4707,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.303720+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:40:54.303720+0000","last_peered":"2024-02-22T19:40:54.303720+0000","last_clean":"2024-02-22T19:40:54.303720+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:40:54.303720+0000","last_undegraded":"2024-02-22T19:40:54.303720+0000","last_fullsized":"2024-02-22T19:40:54.303720+0000","mapping_epoch":285,"log_start":"291'700","ondisk_log_start":"291'700","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3982,"ondisk_log_size":3982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":306839552,"num_objects":4682,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4682,"num_write_kb":299648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":211,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173216+0000","last_change":"2024-02-22T19:39:38.562584+0000","last_active":"2024-02-22T19:40:47.173216+0000","last_peered":"2024-02-22T19:40:47.173216+0000","last_clean":"2024-02-22T19:40:47.173216+0000","last_became_active":"2024-02-22T19:39:22.256349+0000","last_became_peered":"2024-02-22T19:39:22.256349+0000","last_unstale":"2024-02-22T19:40:47.173216+0000","last_undegraded":"2024-02-22T19:40:47.173216+0000","last_fullsized":"2024-02-22T19:40:47.173216+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:38.562500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:39:38.562500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":268,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.166440+0000","last_change":"2024-02-22T19:40:40.450558+0000","last_active":"2024-02-22T19:40:47.166440+0000","last_peered":"2024-02-22T19:40:47.166440+0000","last_clean":"2024-02-22T19:40:47.166440+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:40:47.166440+0000","last_undegraded":"2024-02-22T19:40:47.166440+0000","last_fullsized":"2024-02-22T19:40:47.166440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":292,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.167182+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:40:47.167182+0000","last_peered":"2024-02-22T19:40:47.167182+0000","last_clean":"2024-02-22T19:40:47.167182+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:40:47.167182+0000","last_undegraded":"2024-02-22T19:40:47.167182+0000","last_fullsized":"2024-02-22T19:40:47.167182+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":113,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171090+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:40:47.171090+0000","last_peered":"2024-02-22T19:40:47.171090+0000","last_clean":"2024-02-22T19:40:47.171090+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:40:47.171090+0000","last_undegraded":"2024-02-22T19:40:47.171090+0000","last_fullsized":"2024-02-22T19:40:47.171090+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":318,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172928+0000","last_change":"2024-02-22T19:39:49.629194+0000","last_active":"2024-02-22T19:40:47.172928+0000","last_peered":"2024-02-22T19:40:47.172928+0000","last_clean":"2024-02-22T19:40:47.172928+0000","last_became_active":"2024-02-22T19:39:22.263999+0000","last_became_peered":"2024-02-22T19:39:22.263999+0000","last_unstale":"2024-02-22T19:40:47.172928+0000","last_undegraded":"2024-02-22T19:40:47.172928+0000","last_fullsized":"2024-02-22T19:40:47.172928+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"302'4800","reported_seq":4825,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.359339+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:40:54.359339+0000","last_peered":"2024-02-22T19:40:54.359339+0000","last_clean":"2024-02-22T19:40:54.359339+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:40:54.359339+0000","last_undegraded":"2024-02-22T19:40:54.359339+0000","last_fullsized":"2024-02-22T19:40:54.359339+0000","mapping_epoch":285,"log_start":"291'1000","ondisk_log_start":"291'1000","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3800,"ondisk_log_size":3800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":314572800,"num_objects":4800,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4800,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":171,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173741+0000","last_change":"2024-02-22T19:39:42.557957+0000","last_active":"2024-02-22T19:40:47.173741+0000","last_peered":"2024-02-22T19:40:47.173741+0000","last_clean":"2024-02-22T19:40:47.173741+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:40:47.173741+0000","last_undegraded":"2024-02-22T19:40:47.173741+0000","last_fullsized":"2024-02-22T19:40:47.173741+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:42.557792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.557792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":88,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170521+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:40:47.170521+0000","last_peered":"2024-02-22T19:40:47.170521+0000","last_clean":"2024-02-22T19:40:47.170521+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:40:47.170521+0000","last_undegraded":"2024-02-22T19:40:47.170521+0000","last_fullsized":"2024-02-22T19:40:47.170521+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":94,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173775+0000","last_change":"2024-02-22T19:37:57.190664+0000","last_active":"2024-02-22T19:40:47.173775+0000","last_peered":"2024-02-22T19:40:47.173775+0000","last_clean":"2024-02-22T19:40:47.173775+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:40:47.173775+0000","last_undegraded":"2024-02-22T19:40:47.173775+0000","last_fullsized":"2024-02-22T19:40:47.173775+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.261537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.261537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":407,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169852+0000","last_change":"2024-02-22T19:39:40.321965+0000","last_active":"2024-02-22T19:40:47.169852+0000","last_peered":"2024-02-22T19:40:47.169852+0000","last_clean":"2024-02-22T19:40:47.169852+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:40:47.169852+0000","last_undegraded":"2024-02-22T19:40:47.169852+0000","last_fullsized":"2024-02-22T19:40:47.169852+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:40.321851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:40.321851+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"268'13","reported_seq":145,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.299606+0000","last_change":"2024-02-22T19:39:41.566614+0000","last_active":"2024-02-22T19:40:49.299606+0000","last_peered":"2024-02-22T19:40:49.299606+0000","last_clean":"2024-02-22T19:40:49.299606+0000","last_became_active":"2024-02-22T19:39:22.252945+0000","last_became_peered":"2024-02-22T19:39:22.252945+0000","last_unstale":"2024-02-22T19:40:49.299606+0000","last_undegraded":"2024-02-22T19:40:49.299606+0000","last_fullsized":"2024-02-22T19:40:49.299606+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:39:41.566527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.566527+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"302'4717","reported_seq":4742,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.611443+0000","last_change":"2024-02-22T19:40:00.882269+0000","last_active":"2024-02-22T19:40:54.611443+0000","last_peered":"2024-02-22T19:40:54.611443+0000","last_clean":"2024-02-22T19:40:54.611443+0000","last_became_active":"2024-02-22T19:40:00.881943+0000","last_became_peered":"2024-02-22T19:40:00.881943+0000","last_unstale":"2024-02-22T19:40:54.611443+0000","last_undegraded":"2024-02-22T19:40:54.611443+0000","last_fullsized":"2024-02-22T19:40:54.611443+0000","mapping_epoch":285,"log_start":"291'1000","ondisk_log_start":"291'1000","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3717,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":309133312,"num_objects":4717,"num_object_clones":0,"num_object_copies":14151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4717,"num_write_kb":301888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":170,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171073+0000","last_change":"2024-02-22T19:40:02.507342+0000","last_active":"2024-02-22T19:40:47.171073+0000","last_peered":"2024-02-22T19:40:47.171073+0000","last_clean":"2024-02-22T19:40:47.171073+0000","last_became_active":"2024-02-22T19:40:02.507127+0000","last_became_peered":"2024-02-22T19:40:02.507127+0000","last_unstale":"2024-02-22T19:40:47.171073+0000","last_undegraded":"2024-02-22T19:40:47.171073+0000","last_fullsized":"2024-02-22T19:40:47.171073+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":269,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170391+0000","last_change":"2024-02-22T19:40:31.663872+0000","last_active":"2024-02-22T19:40:47.170391+0000","last_peered":"2024-02-22T19:40:47.170391+0000","last_clean":"2024-02-22T19:40:47.170391+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:40:47.170391+0000","last_undegraded":"2024-02-22T19:40:47.170391+0000","last_fullsized":"2024-02-22T19:40:47.170391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":97,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168287+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:40:47.168287+0000","last_peered":"2024-02-22T19:40:47.168287+0000","last_clean":"2024-02-22T19:40:47.168287+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:40:47.168287+0000","last_undegraded":"2024-02-22T19:40:47.168287+0000","last_fullsized":"2024-02-22T19:40:47.168287+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":389,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170852+0000","last_change":"2024-02-22T19:39:46.430095+0000","last_active":"2024-02-22T19:40:47.170852+0000","last_peered":"2024-02-22T19:40:47.170852+0000","last_clean":"2024-02-22T19:40:47.170852+0000","last_became_active":"2024-02-22T19:39:22.248416+0000","last_became_peered":"2024-02-22T19:39:22.248416+0000","last_unstale":"2024-02-22T19:40:47.170852+0000","last_undegraded":"2024-02-22T19:40:47.170852+0000","last_fullsized":"2024-02-22T19:40:47.170852+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":454,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.299724+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:40:49.299724+0000","last_peered":"2024-02-22T19:40:49.299724+0000","last_clean":"2024-02-22T19:40:49.299724+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:40:49.299724+0000","last_undegraded":"2024-02-22T19:40:49.299724+0000","last_fullsized":"2024-02-22T19:40:49.299724+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"302'4597","reported_seq":4622,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.304229+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:40:54.304229+0000","last_peered":"2024-02-22T19:40:54.304229+0000","last_clean":"2024-02-22T19:40:54.304229+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:40:54.304229+0000","last_undegraded":"2024-02-22T19:40:54.304229+0000","last_fullsized":"2024-02-22T19:40:54.304229+0000","mapping_epoch":285,"log_start":"291'600","ondisk_log_start":"291'600","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3997,"ondisk_log_size":3997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":301268992,"num_objects":4597,"num_object_clones":0,"num_object_copies":13791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4597,"num_write_kb":294208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160704+0000","last_change":"2024-02-22T19:40:33.048538+0000","last_active":"2024-02-22T19:40:47.160704+0000","last_peered":"2024-02-22T19:40:47.160704+0000","last_clean":"2024-02-22T19:40:47.160704+0000","last_became_active":"2024-02-22T19:38:47.167514+0000","last_became_peered":"2024-02-22T19:38:47.167514+0000","last_unstale":"2024-02-22T19:40:47.160704+0000","last_undegraded":"2024-02-22T19:40:47.160704+0000","last_fullsized":"2024-02-22T19:40:47.160704+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:33.048434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.048434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":66,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170994+0000","last_change":"2024-02-22T19:38:47.170822+0000","last_active":"2024-02-22T19:40:47.170994+0000","last_peered":"2024-02-22T19:40:47.170994+0000","last_clean":"2024-02-22T19:40:47.170994+0000","last_became_active":"2024-02-22T19:38:47.170472+0000","last_became_peered":"2024-02-22T19:38:47.170472+0000","last_unstale":"2024-02-22T19:40:47.170994+0000","last_undegraded":"2024-02-22T19:40:47.170994+0000","last_fullsized":"2024-02-22T19:40:47.170994+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":293,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.172129+0000","last_change":"2024-02-22T19:39:22.243142+0000","last_active":"2024-02-22T19:40:47.172129+0000","last_peered":"2024-02-22T19:40:47.172129+0000","last_clean":"2024-02-22T19:40:47.172129+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:40:47.172129+0000","last_undegraded":"2024-02-22T19:40:47.172129+0000","last_fullsized":"2024-02-22T19:40:47.172129+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:37:06.317125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":397,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170831+0000","last_change":"2024-02-22T19:39:23.568647+0000","last_active":"2024-02-22T19:40:47.170831+0000","last_peered":"2024-02-22T19:40:47.170831+0000","last_clean":"2024-02-22T19:40:47.170831+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:40:47.170831+0000","last_undegraded":"2024-02-22T19:40:47.170831+0000","last_fullsized":"2024-02-22T19:40:47.170831+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-22T19:36:26.284462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:26.284462+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":89,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.174820+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:40:47.174820+0000","last_peered":"2024-02-22T19:40:47.174820+0000","last_clean":"2024-02-22T19:40:47.174820+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:40:47.174820+0000","last_undegraded":"2024-02-22T19:40:47.174820+0000","last_fullsized":"2024-02-22T19:40:47.174820+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"302'4713","reported_seq":4738,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.609368+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:40:54.609368+0000","last_peered":"2024-02-22T19:40:54.609368+0000","last_clean":"2024-02-22T19:40:54.609368+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:40:54.609368+0000","last_undegraded":"2024-02-22T19:40:54.609368+0000","last_fullsized":"2024-02-22T19:40:54.609368+0000","mapping_epoch":285,"log_start":"291'1000","ondisk_log_start":"291'1000","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3713,"ondisk_log_size":3713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":308871168,"num_objects":4713,"num_object_clones":0,"num_object_copies":14139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":301632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":102,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171852+0000","last_change":"2024-02-22T19:40:36.433351+0000","last_active":"2024-02-22T19:40:47.171852+0000","last_peered":"2024-02-22T19:40:47.171852+0000","last_clean":"2024-02-22T19:40:47.171852+0000","last_became_active":"2024-02-22T19:37:57.189767+0000","last_became_peered":"2024-02-22T19:37:57.189767+0000","last_unstale":"2024-02-22T19:40:47.171852+0000","last_undegraded":"2024-02-22T19:40:47.171852+0000","last_fullsized":"2024-02-22T19:40:47.171852+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":121,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173504+0000","last_change":"2024-02-22T19:39:22.269372+0000","last_active":"2024-02-22T19:40:47.173504+0000","last_peered":"2024-02-22T19:40:47.173504+0000","last_clean":"2024-02-22T19:40:47.173504+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:40:47.173504+0000","last_undegraded":"2024-02-22T19:40:47.173504+0000","last_fullsized":"2024-02-22T19:40:47.173504+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.979469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.979469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":569,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.301805+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:40:49.301805+0000","last_peered":"2024-02-22T19:40:49.301805+0000","last_clean":"2024-02-22T19:40:49.301805+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:40:49.301805+0000","last_undegraded":"2024-02-22T19:40:49.301805+0000","last_fullsized":"2024-02-22T19:40:49.301805+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":271,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170734+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:40:47.170734+0000","last_peered":"2024-02-22T19:40:47.170734+0000","last_clean":"2024-02-22T19:40:47.170734+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:40:47.170734+0000","last_undegraded":"2024-02-22T19:40:47.170734+0000","last_fullsized":"2024-02-22T19:40:47.170734+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":212,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:53.384009+0000","last_change":"2024-02-22T19:40:53.384009+0000","last_active":"2024-02-22T19:40:53.384009+0000","last_peered":"2024-02-22T19:40:53.384009+0000","last_clean":"2024-02-22T19:40:53.384009+0000","last_became_active":"2024-02-22T19:38:47.093284+0000","last_became_peered":"2024-02-22T19:38:47.093284+0000","last_unstale":"2024-02-22T19:40:53.384009+0000","last_undegraded":"2024-02-22T19:40:53.384009+0000","last_fullsized":"2024-02-22T19:40:53.384009+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"302'4665","reported_seq":4690,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.266416+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:40:54.266416+0000","last_peered":"2024-02-22T19:40:54.266416+0000","last_clean":"2024-02-22T19:40:54.266416+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:40:54.266416+0000","last_undegraded":"2024-02-22T19:40:54.266416+0000","last_fullsized":"2024-02-22T19:40:54.266416+0000","mapping_epoch":285,"log_start":"291'700","ondisk_log_start":"291'700","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3965,"ondisk_log_size":3965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":305725440,"num_objects":4665,"num_object_clones":0,"num_object_copies":13995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":298560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":278,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171326+0000","last_change":"2024-02-22T19:40:32.668512+0000","last_active":"2024-02-22T19:40:47.171326+0000","last_peered":"2024-02-22T19:40:47.171326+0000","last_clean":"2024-02-22T19:40:47.171326+0000","last_became_active":"2024-02-22T19:37:57.179488+0000","last_became_peered":"2024-02-22T19:37:57.179488+0000","last_unstale":"2024-02-22T19:40:47.171326+0000","last_undegraded":"2024-02-22T19:40:47.171326+0000","last_fullsized":"2024-02-22T19:40:47.171326+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"251'17","reported_seq":283,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.299857+0000","last_change":"2024-02-22T19:38:50.146296+0000","last_active":"2024-02-22T19:40:49.299857+0000","last_peered":"2024-02-22T19:40:49.299857+0000","last_clean":"2024-02-22T19:40:49.299857+0000","last_became_active":"2024-02-22T19:38:50.145632+0000","last_became_peered":"2024-02-22T19:38:50.145632+0000","last_unstale":"2024-02-22T19:40:49.299857+0000","last_undegraded":"2024-02-22T19:40:49.299857+0000","last_fullsized":"2024-02-22T19:40:49.299857+0000","mapping_epoch":250,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:36:53.427951+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.427951+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":76,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:52.130146+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:40:52.130146+0000","last_peered":"2024-02-22T19:40:52.130146+0000","last_clean":"2024-02-22T19:40:52.130146+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:40:52.130146+0000","last_undegraded":"2024-02-22T19:40:52.130146+0000","last_fullsized":"2024-02-22T19:40:52.130146+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":125,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173452+0000","last_change":"2024-02-22T19:39:43.589553+0000","last_active":"2024-02-22T19:40:47.173452+0000","last_peered":"2024-02-22T19:40:47.173452+0000","last_clean":"2024-02-22T19:40:47.173452+0000","last_became_active":"2024-02-22T19:38:47.183168+0000","last_became_peered":"2024-02-22T19:38:47.183168+0000","last_unstale":"2024-02-22T19:40:47.173452+0000","last_undegraded":"2024-02-22T19:40:47.173452+0000","last_fullsized":"2024-02-22T19:40:47.173452+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"302'4657","reported_seq":4682,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.458541+0000","last_change":"2024-02-22T19:40:01.097839+0000","last_active":"2024-02-22T19:40:54.458541+0000","last_peered":"2024-02-22T19:40:54.458541+0000","last_clean":"2024-02-22T19:40:54.458541+0000","last_became_active":"2024-02-22T19:40:01.097523+0000","last_became_peered":"2024-02-22T19:40:01.097523+0000","last_unstale":"2024-02-22T19:40:54.458541+0000","last_undegraded":"2024-02-22T19:40:54.458541+0000","last_fullsized":"2024-02-22T19:40:54.458541+0000","mapping_epoch":285,"log_start":"291'700","ondisk_log_start":"291'700","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3957,"ondisk_log_size":3957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":305201152,"num_objects":4657,"num_object_clones":0,"num_object_copies":13971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":298048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":99,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168802+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:40:47.168802+0000","last_peered":"2024-02-22T19:40:47.168802+0000","last_clean":"2024-02-22T19:40:47.168802+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:40:47.168802+0000","last_undegraded":"2024-02-22T19:40:47.168802+0000","last_fullsized":"2024-02-22T19:40:47.168802+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":282,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160127+0000","last_change":"2024-02-22T19:40:34.092106+0000","last_active":"2024-02-22T19:40:47.160127+0000","last_peered":"2024-02-22T19:40:47.160127+0000","last_clean":"2024-02-22T19:40:47.160127+0000","last_became_active":"2024-02-22T19:38:47.171941+0000","last_became_peered":"2024-02-22T19:38:47.171941+0000","last_unstale":"2024-02-22T19:40:47.160127+0000","last_undegraded":"2024-02-22T19:40:47.160127+0000","last_fullsized":"2024-02-22T19:40:47.160127+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":324,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.167759+0000","last_change":"2024-02-22T19:39:49.403011+0000","last_active":"2024-02-22T19:40:47.167759+0000","last_peered":"2024-02-22T19:40:47.167759+0000","last_clean":"2024-02-22T19:40:47.167759+0000","last_became_active":"2024-02-22T19:39:22.244710+0000","last_became_peered":"2024-02-22T19:39:22.244710+0000","last_unstale":"2024-02-22T19:40:47.167759+0000","last_undegraded":"2024-02-22T19:40:47.167759+0000","last_fullsized":"2024-02-22T19:40:47.167759+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":284,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.170606+0000","last_change":"2024-02-22T19:39:45.615429+0000","last_active":"2024-02-22T19:40:47.170606+0000","last_peered":"2024-02-22T19:40:47.170606+0000","last_clean":"2024-02-22T19:40:47.170606+0000","last_became_active":"2024-02-22T19:38:47.164172+0000","last_became_peered":"2024-02-22T19:38:47.164172+0000","last_unstale":"2024-02-22T19:40:47.170606+0000","last_undegraded":"2024-02-22T19:40:47.170606+0000","last_fullsized":"2024-02-22T19:40:47.170606+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":150,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.160790+0000","last_change":"2024-02-22T19:40:35.038302+0000","last_active":"2024-02-22T19:40:47.160790+0000","last_peered":"2024-02-22T19:40:47.160790+0000","last_clean":"2024-02-22T19:40:47.160790+0000","last_became_active":"2024-02-22T19:38:47.181392+0000","last_became_peered":"2024-02-22T19:38:47.181392+0000","last_unstale":"2024-02-22T19:40:47.160790+0000","last_undegraded":"2024-02-22T19:40:47.160790+0000","last_fullsized":"2024-02-22T19:40:47.160790+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"302'4617","reported_seq":4642,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:54.618216+0000","last_change":"2024-02-22T19:40:00.881912+0000","last_active":"2024-02-22T19:40:54.618216+0000","last_peered":"2024-02-22T19:40:54.618216+0000","last_clean":"2024-02-22T19:40:54.618216+0000","last_became_active":"2024-02-22T19:40:00.881492+0000","last_became_peered":"2024-02-22T19:40:00.881492+0000","last_unstale":"2024-02-22T19:40:54.618216+0000","last_undegraded":"2024-02-22T19:40:54.618216+0000","last_fullsized":"2024-02-22T19:40:54.618216+0000","mapping_epoch":285,"log_start":"291'900","ondisk_log_start":"291'900","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3717,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":302579712,"num_objects":4617,"num_object_clones":0,"num_object_copies":13851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":295488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":284,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.169802+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:40:47.169802+0000","last_peered":"2024-02-22T19:40:47.169802+0000","last_clean":"2024-02-22T19:40:47.169802+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:40:47.169802+0000","last_undegraded":"2024-02-22T19:40:47.169802+0000","last_fullsized":"2024-02-22T19:40:47.169802+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":110,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.168200+0000","last_change":"2024-02-22T19:39:32.393413+0000","last_active":"2024-02-22T19:40:47.168200+0000","last_peered":"2024-02-22T19:40:47.168200+0000","last_clean":"2024-02-22T19:40:47.168200+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:40:47.168200+0000","last_undegraded":"2024-02-22T19:40:47.168200+0000","last_fullsized":"2024-02-22T19:40:47.168200+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:39:32.393320+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.393320+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":488,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-02-22T19:40:49.299638+0000","last_change":"2024-02-22T19:39:55.160226+0000","last_active":"2024-02-22T19:40:49.299638+0000","last_peered":"2024-02-22T19:40:49.299638+0000","last_clean":"2024-02-22T19:40:49.299638+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:40:49.299638+0000","last_undegraded":"2024-02-22T19:40:49.299638+0000","last_fullsized":"2024-02-22T19:40:49.299638+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:39:55.160157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:55.160157+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":327,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.173840+0000","last_change":"2024-02-22T19:39:22.400760+0000","last_active":"2024-02-22T19:40:47.173840+0000","last_peered":"2024-02-22T19:40:47.173840+0000","last_clean":"2024-02-22T19:40:47.173840+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:40:47.173840+0000","last_undegraded":"2024-02-22T19:40:47.173840+0000","last_fullsized":"2024-02-22T19:40:47.173840+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:37:05.357880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.357880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":125,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-02-22T19:40:47.171566+0000","last_change":"2024-02-22T19:39:53.646743+0000","last_active":"2024-02-22T19:40:47.171566+0000","last_peered":"2024-02-22T19:40:47.171566+0000","last_clean":"2024-02-22T19:40:47.171566+0000","last_became_active":"2024-02-22T19:38:47.169571+0000","last_became_peered":"2024-02-22T19:38:47.169571+0000","last_unstale":"2024-02-22T19:40:47.171566+0000","last_undegraded":"2024-02-22T19:40:47.171566+0000","last_fullsized":"2024-02-22T19:40:47.171566+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":4912971776,"num_objects":74966,"num_object_clones":0,"num_object_copies":224898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74966,"num_write_kb":4797824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14742192128,"data_stored":14742192128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62466,"ondisk_log_size":62466,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":696,"num_read_kb":691,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":6257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1109,"num_read_kb":866,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":7172,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266834,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3257408,"kb_used_data":2947896,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":90487744,"statfs":{"total":95995035648,"available":92659449856,"internally_reserved":0,"allocated":3018645504,"data_stored":3017785485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":1.323,"15min":1.258},"min":{"1min":0.27800000000000002,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":3.4340000000000002,"5min":6.8040000000000003,"15min":42.597999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.653,"15min":1.3080000000000001},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":6.7670000000000003,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.32800000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.8220000000000001,"15min":1.345},"min":{"1min":0.29199999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.4820000000000002,"5min":54.951000000000001,"15min":54.951000000000001},"last":3.5089999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":2.7200000000000002,"15min":1.7210000000000001},"min":{"1min":0.37,"5min":0.28000000000000003,"15min":0.219},"max":{"1min":2.0249999999999999,"5min":57.012,"15min":57.012},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.829,"15min":1.2589999999999999},"min":{"1min":0.32200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.5129999999999999,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":2.02,"15min":1.4359999999999999},"min":{"1min":0.35499999999999998,"5min":0.25700000000000001,"15min":0.23599999999999999},"max":{"1min":2.4550000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":2.0720000000000001,"15min":1.462},"min":{"1min":0.374,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.823,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":2.145,"15min":1.4410000000000001},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":3.0670000000000002,"5min":38.219000000000001,"15min":41.311999999999998},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.80300000000000005,"15min":0.59199999999999997},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":5.5199999999999996,"15min":5.5199999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.55800000000000005},"min":{"1min":0.377,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":3.8650000000000002,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80600000000000005,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":2.9510000000000001,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.81999999999999995,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":2.1560000000000001,"5min":5.556,"15min":5.556},"last":0.29399999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.93300000000000005,"15min":0.72499999999999998},"min":{"1min":0.371,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":2.3530000000000002,"5min":5.5789999999999997,"15min":12.199999999999999},"last":0.313},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79600000000000004,"15min":0.65600000000000003},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.8439999999999999,"5min":7.6390000000000002,"15min":12.067},"last":0.60899999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528471,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1780400,"kb_used_data":1483176,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":91964752,"statfs":{"total":95995035648,"available":94171906048,"internally_reserved":0,"allocated":1518772224,"data_stored":1517927224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.3819999999999999,"15min":1.0600000000000001},"min":{"1min":0.22500000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":4.1280000000000001,"5min":6.6660000000000004,"15min":17.885000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.3480000000000001,"15min":1.026},"min":{"1min":0.29099999999999998,"5min":0.218,"15min":0.21199999999999999},"max":{"1min":4.835,"5min":8.1120000000000001,"15min":8.2699999999999996},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":2.5939999999999999,"5min":2.0259999999999998,"15min":1.1819999999999999},"min":{"1min":0.499,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":6.5209999999999999,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":2.9609999999999999,"5min":1.7949999999999999,"15min":1.087},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":13.590999999999999,"5min":13.590999999999999,"15min":13.590999999999999},"last":1.597}]},{"osd":2,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.9830000000000001,"15min":1.2689999999999999},"min":{"1min":0.308,"5min":0.253,"15min":0.249},"max":{"1min":6.5919999999999996,"5min":11.198,"15min":12.859999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.7529999999999999,"5min":1.8500000000000001,"15min":1.1439999999999999},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":8.5779999999999994,"5min":11.565,"15min":11.565},"last":1.823}]},{"osd":3,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.587,"15min":1.254},"min":{"1min":0.38,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":8.1240000000000006,"5min":10.135,"15min":38.762},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.984,"5min":1.4950000000000001,"15min":1.381},"min":{"1min":0.40000000000000002,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":10.315,"5min":10.315,"15min":35.612000000000002},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.57099999999999995,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.73899999999999999,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.861,"5min":9.3949999999999996,"15min":9.3949999999999996},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:39:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.754,"15min":0.55000000000000004},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29299999999999998},"max":{"1min":3.5129999999999999,"5min":7.3780000000000001,"15min":7.3780000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.77700000000000002,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.177,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.74099999999999999,"15min":0.59799999999999998},"min":{"1min":0.376,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.4430000000000001,"5min":3.2709999999999999,"15min":3.488},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88400000000000001,"15min":0.68999999999999995},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":2.5110000000000001,"5min":5.7939999999999996,"15min":5.7939999999999996},"last":1.268}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3301284,"kb_used_data":3004060,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":90443868,"statfs":{"total":95995035648,"available":92614520832,"internally_reserved":0,"allocated":3076157440,"data_stored":3075261313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:39:56 2024","interfaces":[{"interface":"back","average":{"1min":1.8620000000000001,"5min":1.746,"15min":1.363},"min":{"1min":0.246,"5min":0.185,"15min":0.185},"max":{"1min":9.1319999999999997,"5min":10.6,"15min":15.613},"last":3.8490000000000002},{"interface":"front","average":{"1min":1.393,"5min":1.948,"15min":1.446},"min":{"1min":0.20999999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":6.0019999999999998,"5min":9.1259999999999994,"15min":13.515000000000001},"last":0.246}]},{"osd":2,"last update":"Thu Feb 22 19:40:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.673,"15min":1.4650000000000001},"min":{"1min":0.24199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.0700000000000003,"5min":11.682,"15min":16.503},"last":1.9630000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":2.1560000000000001,"15min":1.556},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.192},"max":{"1min":3.3439999999999999,"5min":14.994,"15min":14.994},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Feb 22 19:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":1.9410000000000001,"15min":1.6459999999999999},"min":{"1min":0.29499999999999998,"5min":0.253,"15min":0.214},"max":{"1min":2.5920000000000001,"5min":10.734999999999999,"15min":25.145},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.226,"5min":2.1850000000000001,"15min":1.7310000000000001},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":9.5820000000000007,"5min":9.5820000000000007,"15min":22.25},"last":2.0070000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:40:03 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.446,"15min":1.198},"min":{"1min":0.438,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9390000000000001,"5min":7.9390000000000001,"15min":15.542999999999999},"last":7.6100000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.3180000000000001,"15min":1.04},"min":{"1min":0.41799999999999998,"5min":0.30499999999999999,"15min":0.245},"max":{"1min":3.6680000000000001,"5min":10.706,"15min":10.706},"last":0.378}]},{"osd":5,"last update":"Thu Feb 22 19:40:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":1.569,"15min":1.246},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0110000000000001,"5min":8.0579999999999998,"15min":15.587999999999999},"last":4.1799999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.4179999999999999,"15min":1.137},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":4.3730000000000002,"5min":5.4359999999999999,"15min":9.4719999999999995},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:40:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.46,"15min":1.1180000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":6.8970000000000002,"5min":10.631,"15min":10.631},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.018,"5min":1.4950000000000001,"15min":1.23},"min":{"1min":0.38600000000000001,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":3.508,"5min":8.0259999999999998,"15min":15.651999999999999},"last":3.508}]},{"osd":7,"last update":"Thu Feb 22 19:40:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":1.7190000000000001,"15min":1.2250000000000001},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":2.0030000000000001,"5min":13.169,"15min":13.169},"last":3.4740000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.4179999999999999,"15min":1.093},"min":{"1min":0.47099999999999997,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":10.673,"15min":10.673},"last":3.444}]}]},{"osd":0,"up_from":0,"seq":34359738549,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411498,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1497268,"kb_used_data":1195948,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":92247884,"statfs":{"total":95995035648,"available":94461833216,"internally_reserved":0,"allocated":1224650752,"data_stored":1223801554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:11 2024","interfaces":[{"interface":"back","average":{"1min":2.8220000000000001,"5min":2.5899999999999999,"15min":1.5489999999999999},"min":{"1min":0.19900000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":16.137,"5min":22.338999999999999,"15min":22.338999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.9220000000000002,"15min":1.804},"min":{"1min":0.23899999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":12.438000000000001,"5min":30.427,"15min":30.427},"last":0.309}]},{"osd":1,"last update":"Thu Feb 22 19:40:11 2024","interfaces":[{"interface":"back","average":{"1min":1.865,"5min":2.0790000000000002,"15min":1.446},"min":{"1min":0.21299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":12.349,"5min":12.349,"15min":12.349},"last":3.2559999999999998},{"interface":"front","average":{"1min":1.7849999999999999,"5min":2.1539999999999999,"15min":1.3500000000000001},"min":{"1min":0.25700000000000001,"5min":0.22900000000000001,"15min":0.193},"max":{"1min":10.396000000000001,"5min":13.782,"15min":13.782},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:40:49 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":2.7389999999999999,"15min":1.7809999999999999},"min":{"1min":0.29099999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.8889999999999998,"5min":22.302,"15min":30.071000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.294,"5min":2.827,"15min":1.9219999999999999},"min":{"1min":0.29899999999999999,"5min":0.184,"15min":0.184},"max":{"1min":7.3600000000000003,"5min":30.282,"15min":32.119},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:40:01 2024","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5009999999999999,"15min":1.196},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":12.43,"5min":12.43,"15min":12.43},"last":2.4950000000000001},{"interface":"front","average":{"1min":2.5070000000000001,"5min":1.776,"15min":1.0980000000000001},"min":{"1min":0.42399999999999999,"5min":0.28000000000000003,"15min":0.253},"max":{"1min":10.34,"5min":10.34,"15min":10.34},"last":1.117}]},{"osd":5,"last update":"Thu Feb 22 19:40:20 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.766,"15min":1.121},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.7199999999999998,"5min":10.300000000000001,"15min":10.300000000000001},"last":2.008},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.5609999999999999,"15min":1.242},"min":{"1min":0.33200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.992,"5min":12.391999999999999,"15min":12.391999999999999},"last":2.524}]},{"osd":6,"last update":"Thu Feb 22 19:39:56 2024","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":1.76,"15min":1.294},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":21.640000000000001,"5min":21.640000000000001,"15min":21.640000000000001},"last":3.927},{"interface":"front","average":{"1min":2.1600000000000001,"5min":1.6519999999999999,"15min":1.103},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":10.369999999999999,"5min":10.369999999999999,"15min":10.369999999999999},"last":1.6930000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:40:11 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.635,"15min":1.29},"min":{"1min":0.41799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":12.484,"5min":12.484,"15min":12.484},"last":2.714},{"interface":"front","average":{"1min":1.625,"5min":1.913,"15min":1.198},"min":{"1min":0.496,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":10.539,"5min":10.539,"15min":10.539},"last":2.1320000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149860,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2987600,"kb_used_data":2678088,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":90757552,"statfs":{"total":95995035648,"available":92935733248,"internally_reserved":0,"allocated":2742362112,"data_stored":2741485936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":2.1789999999999998,"15min":1.6080000000000001},"min":{"1min":0.23000000000000001,"5min":0.21199999999999999,"15min":0.193},"max":{"1min":5.4960000000000004,"5min":23.48,"15min":23.48},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.96,"5min":2.681,"15min":1.9059999999999999},"min":{"1min":0.315,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":18.172000000000001,"5min":23.411999999999999,"15min":23.411999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":3.3809999999999998,"15min":2.1280000000000001},"min":{"1min":0.27200000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":18.213999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":2.5590000000000002},{"interface":"front","average":{"1min":1.3779999999999999,"5min":2.8940000000000001,"15min":1.758},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":12.234,"5min":22.187000000000001,"15min":22.187000000000001},"last":2.911}]},{"osd":2,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":2.569,"15min":1.8220000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.105,"5min":20.105,"15min":20.105},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":2.52,"15min":1.7629999999999999},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.217},"max":{"1min":7.617,"5min":35.939999999999998,"15min":35.939999999999998},"last":1.3280000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:39:58 2024","interfaces":[{"interface":"back","average":{"1min":2.827,"5min":2.165,"15min":1.5109999999999999},"min":{"1min":0.41799999999999998,"5min":0.26300000000000001,"15min":0.249},"max":{"1min":12.416,"5min":12.416,"15min":13.680999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":2.133,"5min":2.1389999999999998,"15min":1.4219999999999999},"min":{"1min":0.38200000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.273,"5min":13.548,"15min":13.548},"last":1.734}]},{"osd":5,"last update":"Thu Feb 22 19:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.653,"15min":1.2749999999999999},"min":{"1min":0.434,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":3.6789999999999998,"5min":8.0289999999999999,"15min":12.879},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":2.0659999999999998,"15min":1.48},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":12.372999999999999,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.435}]},{"osd":6,"last update":"Thu Feb 22 19:40:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.6559999999999999,"15min":1.2949999999999999},"min":{"1min":0.39900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.0449999999999999,"5min":7.9640000000000004,"15min":12.898999999999999},"last":1.276},{"interface":"front","average":{"1min":1.0449999999999999,"5min":2.0920000000000001,"15min":1.5589999999999999},"min":{"1min":0.45200000000000001,"5min":0.34799999999999998,"15min":0.30199999999999999},"max":{"1min":3.9910000000000001,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.45200000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:40:04 2024","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":1.409},"min":{"1min":0.435,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":12.327,"5min":13.483000000000001,"15min":13.483000000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.716,"5min":1.6339999999999999,"15min":1.2889999999999999},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":5.7300000000000004,"5min":7.9909999999999997,"15min":12.846},"last":1.2969999999999999}]}]},{"osd":4,"up_from":0,"seq":137438953634,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724700,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3257140,"kb_used_data":2955820,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":90488012,"statfs":{"total":95995035648,"available":92659724288,"internally_reserved":0,"allocated":3026759680,"data_stored":3025910684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.3560000000000001,"15min":1.151},"min":{"1min":0.27800000000000002,"5min":0.23999999999999999,"15min":0.222},"max":{"1min":1.9139999999999999,"5min":17.684000000000001,"15min":17.684000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.605,"15min":1.2250000000000001},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":3.919,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.702,"15min":1.212},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.22500000000000001},"max":{"1min":2.0089999999999999,"5min":29.390999999999998,"15min":29.390999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.194},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.218},"max":{"1min":2.7770000000000001,"5min":29.367000000000001,"15min":29.367000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.7230000000000001,"15min":1.2470000000000001},"min":{"1min":0.36299999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.1779999999999999,"5min":11.367000000000001,"15min":14.613},"last":0.436},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.722,"15min":1.3029999999999999},"min":{"1min":0.33400000000000002,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":3.8130000000000002,"5min":16.099,"15min":16.099},"last":0.38}]},{"osd":3,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.593,"15min":1.395},"min":{"1min":0.41099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.1399999999999997,"5min":7.0309999999999997,"15min":13.366},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.5349999999999999,"15min":1.3520000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.8839999999999999,"5min":8.6069999999999993,"15min":10.16},"last":1.6870000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68600000000000005,"15min":0.51900000000000002},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.219},"max":{"1min":1.6879999999999999,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67800000000000005,"15min":0.52500000000000002},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":1.998,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.623,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":1.853,"5min":3.246,"15min":5.6040000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.60899999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.698,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.315}]},{"osd":7,"last update":"Thu Feb 22 19:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.35799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":4.6420000000000003,"15min":5.5789999999999997},"last":1.649},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.61099999999999999},"min":{"1min":0.42399999999999999,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":1.538,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3078488064,"data_stored":3078488064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1229651968,"data_stored":1229651968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2780889088,"data_stored":2780889088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3068264448,"data_stored":3068264448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1532297216,"data_stored":1532297216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3052601344,"data_stored":3052601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:41:02.726 INFO:tasks.thrashosds.thrasher:Setting 5.8 to [1, 2, 6, 5, 7, 3] 2024-02-22T19:41:02.726 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '5.8', '1', '2', '6', '5', '7', '3'] 2024-02-22T19:41:02.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pg-upmap-items 5.8 1 2 6 5 7 3 2024-02-22T19:41:02.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:01.586256+0000 mgr.y (mgr.14535) 462 : cluster [DBG] pgmap v592: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 828 B/s rd, 61 MiB/s wr, 975 op/s 2024-02-22T19:41:02.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:02.266027+0000 mon.a (mon.0) 1320 : cluster [DBG] osdmap e306: 8 total, 8 up, 6 in 2024-02-22T19:41:03.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:01.586256+0000 mgr.y (mgr.14535) 462 : cluster [DBG] pgmap v592: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 828 B/s rd, 61 MiB/s wr, 975 op/s 2024-02-22T19:41:03.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:02.266027+0000 mon.a (mon.0) 1320 : cluster [DBG] osdmap e306: 8 total, 8 up, 6 in 2024-02-22T19:41:03.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:01.586256+0000 mgr.y (mgr.14535) 462 : cluster [DBG] pgmap v592: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 828 B/s rd, 61 MiB/s wr, 975 op/s 2024-02-22T19:41:03.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:02.266027+0000 mon.a (mon.0) 1320 : cluster [DBG] osdmap e306: 8 total, 8 up, 6 in 2024-02-22T19:41:03.228 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 52 16 85675 85659 102.897 75.625 0.0182616 0.00971356 2024-02-22T19:41:03.504 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:03.516 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:03.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:03.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:41:03.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:03.725 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:03.769 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:03.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:03.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:03.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:03.936 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:03.971 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:03.980 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:03.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:04.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:694 5.e 5:7759931f:::notify.3:head [watch ping cookie 94129597099392] snapc 0=[] ondisk+write+known_if_redirected e98)", 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:49.154376+0000", 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 554.98679620300004, 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85628850199999995, 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 694 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154376+0000", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154376+0000", 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154374+0000", 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0577000000000001e-05 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154385+0000", 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9412999999999999e-05 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154404+0000", 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6293000000000001e-05 2024-02-22T19:41:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154421+0000", 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000222499 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154643+0000", 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4482e-05 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154678+0000", 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85175950099999997 2024-02-22T19:41:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.006437+0000", 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00037694099999999999 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for degraded object", 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.006814+0000", 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036508669999999999 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010465+0000", 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2737000000000003e-05 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010558+0000", 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000106976 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010665+0000", 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85628850199999995 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:700 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e99)", 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:31:49.154699+0000", 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 554.98647347500003, 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86230325500000005, 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 700 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154699+0000", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154699+0000", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154698+0000", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2769999999999996e-06 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154703+0000", 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8890000000000008e-06 2024-02-22T19:41:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154712+0000", 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1720000000000004e-06 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154718+0000", 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8034e-05 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154776+0000", 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2228999999999999e-05 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:49.154789+0000", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85235517100000002 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.007144+0000", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1841000000000006e-05 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for degraded object", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.007206+0000", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035078069999999999 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010714+0000", 2024-02-22T19:41:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5166000000000005e-05 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.010789+0000", 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062134900000000003 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:31:50.017002+0000", 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86230325500000005 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71273 8.2 8:4d5bc216:::benchmark_data_smithi053_130774_object71272:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.014979+0000", 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 283.12619373899997, 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078888333000000005, 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71273 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.014979+0000", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.014979+0000", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.014977+0000", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4988000000000003e-05 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015032+0000", 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2476e-05 2024-02-22T19:41:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015045+0000", 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.516e-06 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015051+0000", 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2370000000000003e-05 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015094+0000", 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014868999999999999 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015242+0000", 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000250423 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015493+0000", 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084955440000000007 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.023988+0000", 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00087941699999999996 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.024868+0000", 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068938761000000001 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093806+0000", 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3099000000000003e-05 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093839+0000", 2024-02-22T19:41:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7535999999999999e-05 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093867+0000", 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078888333000000005 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71277 8.2 8:41883358:::benchmark_data_smithi053_130774_object71276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.015237+0000", 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 283.125935664, 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078755951000000005, 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71277 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015237+0000", 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015237+0000", 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015236+0000", 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2608999999999999e-05 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015269+0000", 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1059999999999996e-06 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015275+0000", 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9690000000000002e-06 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015280+0000", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001987417 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.017267+0000", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012988 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.017397+0000", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102525 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.017500+0000", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064942439999999997 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.023994+0000", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00095847700000000001 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.024952+0000", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068964619000000005 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093917+0000", 2024-02-22T19:41:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7161999999999998e-05 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093944+0000", 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8488000000000001e-05 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.093993+0000", 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078755951000000005 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71278 8.2 8:451cdea5:::benchmark_data_smithi053_130774_object71277:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:41:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.015291+0000", 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 283.12588153399997, 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078881751, 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71278 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015291+0000", 2024-02-22T19:41:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015291+0000", 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015290+0000", 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8708999999999995e-05 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015369+0000", 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5310000000000004e-06 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015376+0000", 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6179999999999999e-06 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.015380+0000", 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038524800000000001 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.019232+0000", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116681 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.019349+0000", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146007 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.019495+0000", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045034209999999996 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.023999+0000", 2024-02-22T19:41:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032699700000000001 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.027269+0000", 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.066793575999999993 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094062+0000", 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3528000000000006e-05 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094126+0000", 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6968999999999997e-05 2024-02-22T19:41:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094173+0000", 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.078881751 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:71284 8.2 8:482c922c:::benchmark_data_smithi053_130774_object71283:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e171)", 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:21.018930+0000", 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 283.12224276500001, 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075350869000000001, 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71284 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018930+0000", 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018930+0000", 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018928+0000", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5718999999999999e-05 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018954+0000", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5325999999999999e-05 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018969+0000", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.182e-06 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.018976+0000", 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026448090000000001 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.021621+0000", 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014431000000000001 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.021765+0000", 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000181514 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.021947+0000", 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002211073 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.024158+0000", 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057578457999999999 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.081736+0000", 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012481897 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094218+0000", 2024-02-22T19:41:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1065000000000002e-05 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094249+0000", 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1066999999999999e-05 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:21.094281+0000", 2024-02-22T19:41:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075350869000000001 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73406 8.2 8:445e69a0:::benchmark_data_smithi053_130774_object73405:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.189313+0000", 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 280.95185936199999, 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.076342253999999998, 2024-02-22T19:41:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73406 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189313+0000", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189313+0000", 2024-02-22T19:41:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189312+0000", 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0809000000000002e-05 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189353+0000", 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8460000000000002e-06 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189360+0000", 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.941e-06 2024-02-22T19:41:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189364+0000", 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5580000000000006e-05 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189429+0000", 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1738999999999997e-05 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189521+0000", 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107824 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189629+0000", 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003956846 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.193586+0000", 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071767559999999994 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265353+0000", 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019441099999999999 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265548+0000", 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1397999999999997e-05 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265609+0000", 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6378000000000002e-05 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265655+0000", 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.076342253999999998 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:73408 8.2 8:4a6b68d5:::benchmark_data_smithi053_130774_object73407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e172)", 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 280.951673067, 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.079867652999999997, 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73408 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189499+0000", 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100202 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189599+0000", 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1030000000000008e-06 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189608+0000", 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6629999999999998e-06 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.189614+0000", 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018859739999999999 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.191500+0000", 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000125933 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.191626+0000", 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017495499999999999 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.191800+0000", 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019171380000000001 2024-02-22T19:41:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.193718+0000", 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071980290000000002 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.265698+0000", 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035836459999999998 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.269282+0000", 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7651999999999997e-05 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.269329+0000", 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7818999999999998e-05 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:36:23.269367+0000", 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.079867652999999997 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:156980 8.2 8:446b8e74:::benchmark_data_smithi053_130774_object45246:head [delete] snapc 0=[] ondisk+write+known_if_redirected e208)", 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:41:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 195.92396780999999, 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064537973999999998, 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 156980 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217204+0000", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6099999999999999e-06 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217208+0000", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4339999999999997e-06 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217215+0000", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1739999999999999e-06 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217220+0000", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002322302 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219542+0000", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9874e-05 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219632+0000", 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9351000000000006e-05 2024-02-22T19:41:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219711+0000", 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012168019999999999 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.220928+0000", 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004561847 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.225490+0000", 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.056184824000000001 2024-02-22T19:41:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281675+0000", 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2453999999999998e-05 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281707+0000", 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5172000000000001e-05 2024-02-22T19:41:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281743+0000", 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064537973999999998 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:157456 8.2 8:47cb7be2:::benchmark_data_smithi053_130774_object45722:head [delete] snapc 0=[] ondisk+write+known_if_redirected e209)", 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 195.40173608399999, 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065090921999999996, 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 157456 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739435+0000", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4889999999999998e-06 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739443+0000", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6900000000000004e-06 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739453+0000", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5810000000000002e-06 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739459+0000", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5296000000000001e-05 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739505+0000", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7620999999999994e-05 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739582+0000", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6128e-05 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739658+0000", 2024-02-22T19:41:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022481889999999998 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.741906+0000", 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00070410199999999996 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.742611+0000", 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061844286999999998 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804455+0000", 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6128999999999998e-05 2024-02-22T19:41:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804501+0000", 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6259e-05 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804527+0000", 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065090921999999996 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:158760 8.2 8:4b5b1757:::benchmark_data_smithi053_130774_object47026:head [delete] snapc 0=[] ondisk+write+known_if_redirected e210)", 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:41:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 194.07257155900001, 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.070793354000000003, 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 158760 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068599+0000", 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3269999999999997e-06 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068605+0000", 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0729e-05 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068616+0000", 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9429999999999999e-06 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068621+0000", 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4260000000000001e-05 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068655+0000", 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110695 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068766+0000", 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104616 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068871+0000", 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019013000000000001 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.070772+0000", 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065132173000000002 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.135904+0000", 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033978099999999998 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139302+0000", 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1437000000000002e-05 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139353+0000", 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0763999999999998e-05 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139394+0000", 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.070793354000000003 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165007 8.2 8:4b3d7df9:::benchmark_data_smithi053_130774_object53273:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:41:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 187.50257122599999, 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.689438121, 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165007 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638600+0000", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8800000000000008e-06 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638608+0000", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0635999999999999e-05 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638618+0000", 2024-02-22T19:41:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9420000000000003e-06 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638623+0000", 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5618999999999998e-05 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638669+0000", 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012975000000000001 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638799+0000", 2024-02-22T19:41:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015391499999999999 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638953+0000", 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040805320000000004 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.643033+0000", 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012758660000000001 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.644309+0000", 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015138571999999999 2024-02-22T19:41:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659448+0000", 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0815999999999999e-05 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659458+0000", 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50944174600000003 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168900+0000", 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.302e-06 2024-02-22T19:41:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168909+0000", 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1590766800000001 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327986+0000", 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3182e-05 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328039+0000", 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.689438121 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165013 8.2 8:42c6cb88:::benchmark_data_smithi053_130774_object53279:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 187.50240753400001, 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6893490839999998, 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165013 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638764+0000", 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9380000000000003e-06 2024-02-22T19:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638771+0000", 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4669999999999993e-06 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638780+0000", 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9320000000000001e-06 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638786+0000", 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024245949999999999 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641211+0000", 2024-02-22T19:41:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011383399999999999 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641324+0000", 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000133826 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641458+0000", 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017005779999999999 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.643159+0000", 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005364382 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.648523+0000", 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010966531999999999 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659490+0000", 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3050000000000004e-06 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659497+0000", 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50944507900000002 2024-02-22T19:41:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168942+0000", 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4430000000000003e-06 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168951+0000", 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1591219230000001 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328073+0000", 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1409999999999998e-05 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328114+0000", 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6893490839999998 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1372 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 4] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 187.46888588600001, 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.542875494, 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1372 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672285+0000", 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6209999999999998e-06 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672291+0000", 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0467e-05 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672302+0000", 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0189999999999995e-06 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672311+0000", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3849999999999993e-05 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672375+0000", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3922e-05 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672389+0000", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.496154551 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168543+0000", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3271999999999999e-05 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168556+0000", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0231974429999999 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.191754+0000", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000199951 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.191954+0000", 2024-02-22T19:41:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023208168000000001 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.215162+0000", 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.542875494 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1374 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 5] snapc 0=[] ondisk+write+known_if_redirected e218)", 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 186.973436642, 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0476472020000001, 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1374 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167734+0000", 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7470000000000005e-06 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167742+0000", 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5179999999999993e-06 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167751+0000", 2024-02-22T19:41:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1430000000000001e-06 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167758+0000", 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00082432800000000004 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168583+0000", 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3889999999999994e-06 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168592+0000", 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.023928116 2024-02-22T19:41:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.192520+0000", 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8975999999999997e-05 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.192599+0000", 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022783739000000001 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.215383+0000", 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0476472020000001 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1382 5.e 5:7759931f:::notify.3:head [watch ping cookie 94129597099392 gen 5] snapc 0=[] ondisk+write+known_if_redirected e219)", 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 184.877347144, 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95001531500000003, 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1382 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0230000000000001e-06 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263828+0000", 2024-02-22T19:41:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3229999999999999e-06 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263832+0000", 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2860000000000001e-06 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263836+0000", 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4635000000000005e-05 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263921+0000", 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.167e-05 2024-02-22T19:41:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263933+0000", 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94966063199999995 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213593+0000", 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000170743 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213764+0000", 2024-02-22T19:41:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6550000000000004e-05 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213841+0000", 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95001531500000003 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1387 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 6] snapc 0=[] ondisk+write+known_if_redirected e220)", 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 184.87732896599999, 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96457482999999999, 2024-02-22T19:41:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1387 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5679999999999998e-06 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263846+0000", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1229999999999997e-06 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263852+0000", 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1789999999999999e-06 2024-02-22T19:41:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263857+0000", 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104499 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263961+0000", 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6989999999999997e-06 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263968+0000", 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95095029499999995 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.214918+0000", 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7822999999999998e-05 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.214976+0000", 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01344215 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.228418+0000", 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96457482999999999 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:197539 8.2 8:421bd531:::benchmark_data_smithi053_130774_object85805:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:46.112931+0000", 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 138.028241038, 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1586733920000001, 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 197539 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112931+0000", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112931+0000", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112930+0000", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1970000000000001e-06 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112936+0000", 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0519999999999999e-05 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112947+0000", 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5189999999999998e-06 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112953+0000", 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2479999999999998e-05 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.113016+0000", 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5166e-05 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.113031+0000", 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1526715759999999 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.265702+0000", 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0606000000000006e-05 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.265793+0000", 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011398199999999999 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5,6", 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.265907+0000", 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0049431880000000003 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270850+0000", 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8307999999999998e-05 2024-02-22T19:41:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270909+0000", 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7954000000000003e-05 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270946+0000", 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026440200000000001 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271211+0000", 2024-02-22T19:41:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000175093 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271386+0000", 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0007000000000003e-05 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271446+0000", 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000158803 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271605+0000", 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1586733920000001 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:197560 8.2 8:4ef66ec7:::benchmark_data_smithi053_130774_object85826:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:46.162279+0000", 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 137.978893363, 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1127937880000001, 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 197560 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162279+0000", 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162279+0000", 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162277+0000", 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0739999999999999e-06 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162286+0000", 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5593000000000001e-05 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162302+0000", 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3080000000000004e-06 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162308+0000", 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9870000000000002e-05 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162358+0000", 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6190000000000006e-06 2024-02-22T19:41:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162368+0000", 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.105709786 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.268078+0000", 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146847 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.268224+0000", 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000168157 2024-02-22T19:41:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5,6", 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.268393+0000", 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002563877 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270956+0000", 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5355999999999997e-05 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271002+0000", 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3988000000000001e-05 2024-02-22T19:41:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271036+0000", 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00093006199999999995 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271966+0000", 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028799379999999999 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.274846+0000", 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8474999999999997e-05 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.274914+0000", 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015857499999999999 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.275073+0000", 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1127937880000001 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:209454 8.2 8.b21c7012 (undecoded) ondisk+write+known_if_redirected e260)", 2024-02-22T19:41:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:39:07.009940+0000", 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 117.131232447, 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.21687268300000001, 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 209454 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009940+0000", 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009940+0000", 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009938+0000", 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.024e-06 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009945+0000", 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3556e-05 2024-02-22T19:41:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009959+0000", 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8280000000000003e-06 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009967+0000", 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8288000000000003e-05 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.010015+0000", 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9039999999999992e-06 2024-02-22T19:41:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.010023+0000", 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.216777631 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.226801+0000", 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2084e-05 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.226813+0000", 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.21687268300000001 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:04.260 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:41:04.260 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:41:04.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:41:04.261 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 53 16 86830 86814 102.318 72.1875 0.0196474 0.00976914 2024-02-22T19:41:04.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:41:04.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:41:04.325 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:41:04.359 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:41:04.368 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:04.377 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:04.378 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:04.405 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:41:04.426 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.434 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:41:04.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:04.529 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:04.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:03.588261+0000 mgr.y (mgr.14535) 463 : cluster [DBG] pgmap v594: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 809 B/s rd, 60 MiB/s wr, 953 op/s 2024-02-22T19:41:04.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:04.152812+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.53:0/2831405615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:04.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:03.588261+0000 mgr.y (mgr.14535) 463 : cluster [DBG] pgmap v594: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 809 B/s rd, 60 MiB/s wr, 953 op/s 2024-02-22T19:41:04.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:04.152812+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.53:0/2831405615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:04.856 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:41:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:41:04.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:03.588261+0000 mgr.y (mgr.14535) 463 : cluster [DBG] pgmap v594: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 809 B/s rd, 60 MiB/s wr, 953 op/s 2024-02-22T19:41:04.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:04.152812+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.53:0/2831405615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:05.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:05.005 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:05.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:41:05.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:05.153 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:05.229 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 54 16 88061 88045 101.846 76.9375 0.0151587 0.00981427 2024-02-22T19:41:05.267 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:05.273 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:05.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:05.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:41:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:41:05.374 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:05.425 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:05.463 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:05.464 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:05.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:05.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:05.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:04.944283+0000 mgr.y (mgr.14535) 464 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:05.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:04.944283+0000 mgr.y (mgr.14535) 464 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:05.764 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:05.768 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:05.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:05.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:05.922 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:06.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:04.944283+0000 mgr.y (mgr.14535) 464 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:06.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:06.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:06.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:06.166 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:06.216 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:06.219 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:06.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:06.229 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 55 16 89236 89220 101.33 73.4375 0.02112 0.00986458 2024-02-22T19:41:06.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:06.464 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:06.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:06.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:06.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:06.628 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:06.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:05.589413+0000 mgr.y (mgr.14535) 465 : cluster [DBG] pgmap v595: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:41:06.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:05.589413+0000 mgr.y (mgr.14535) 465 : cluster [DBG] pgmap v595: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:41:06.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:05.589413+0000 mgr.y (mgr.14535) 465 : cluster [DBG] pgmap v595: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:41:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:06.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:06.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:07.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:07.056 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:07.213 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:07.213 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:07.230 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 56 16 90429 90413 100.851 74.5625 0.0109144 0.00991106 2024-02-22T19:41:07.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:07.355 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:07.421 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:07.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:07.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:07.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:07.573 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:41:07.573 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:41:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:07.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:07.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:07.802 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:07.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:07.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:07.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:41:07.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:08.010 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:08.230 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 57 16 91594 91578 100.359 72.8125 0.00991251 0.00996016 2024-02-22T19:41:08.282 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:08.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:08.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:08.319 INFO:teuthology.orchestra.run.smithi053.stderr:set 5.8 pg_upmap_items mapping to [1->2,6->5,7->3] 2024-02-22T19:41:08.378 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:41:08.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:08.479 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:08.517 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:08.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:08.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:07.346447+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.53:0/1379570538' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]: dispatch 2024-02-22T19:41:08.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:07.348001+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]: dispatch 2024-02-22T19:41:08.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:07.589823+0000 mgr.y (mgr.14535) 466 : cluster [DBG] pgmap v596: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 633 B/s rd, 58 MiB/s wr, 932 op/s 2024-02-22T19:41:08.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:08.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:07.346447+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.53:0/1379570538' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]: dispatch 2024-02-22T19:41:08.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:07.348001+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]: dispatch 2024-02-22T19:41:08.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:07.589823+0000 mgr.y (mgr.14535) 466 : cluster [DBG] pgmap v596: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 633 B/s rd, 58 MiB/s wr, 932 op/s 2024-02-22T19:41:08.679 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:07.346447+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.53:0/1379570538' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]: dispatch 2024-02-22T19:41:08.679 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:07.348001+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]: dispatch 2024-02-22T19:41:08.680 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:07.589823+0000 mgr.y (mgr.14535) 466 : cluster [DBG] pgmap v596: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 633 B/s rd, 58 MiB/s wr, 932 op/s 2024-02-22T19:41:08.681 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:08.746 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:08.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:08.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:08.802 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:08.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:08.901 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:09.204 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:09.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:09.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:09.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 58 16 92726 92710 99.8482 70.75 0.0121971 0.0100111 2024-02-22T19:41:09.269 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:09.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:09.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:08.278722+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]': finished 2024-02-22T19:41:09.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:08.278805+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e307: 8 total, 8 up, 6 in 2024-02-22T19:41:09.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:08.376104+0000 mon.a (mon.0) 1324 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:09.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:08.896684+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? 172.21.15.53:0/1553951669' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:09.382 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:09.399 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:08.278722+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]': finished 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:08.278805+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e307: 8 total, 8 up, 6 in 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:08.376104+0000 mon.a (mon.0) 1324 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:08.896684+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? 172.21.15.53:0/1553951669' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:08.278722+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.8", "id": [1, 2, 6, 5, 7, 3]}]': finished 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:08.278805+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e307: 8 total, 8 up, 6 in 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:08.376104+0000 mon.a (mon.0) 1324 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:09.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:08.896684+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? 172.21.15.53:0/1553951669' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:09.505 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:41:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:09] "GET /metrics HTTP/1.1" 200 176129 "" "Prometheus/2.33.4" 2024-02-22T19:41:10.146 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:10.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:10.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:10.232 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 59 16 93991 93975 99.496 79.0625 0.0084116 0.010047 2024-02-22T19:41:10.258 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:10.299 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:10.437 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:10.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:10.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:10.451 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:09.373386+0000 mon.a (mon.0) 1326 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:10.451 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:09.385754+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.53:0/1553951669' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:10.451 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:09.385805+0000 mon.a (mon.0) 1328 : cluster [DBG] osdmap e308: 8 total, 8 up, 6 in 2024-02-22T19:41:10.451 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:09.590580+0000 mgr.y (mgr.14535) 467 : cluster [DBG] pgmap v599: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.4 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-02-22T19:41:10.542 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:10.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:09.373386+0000 mon.a (mon.0) 1326 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:10.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:09.385754+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.53:0/1553951669' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:10.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:09.385805+0000 mon.a (mon.0) 1328 : cluster [DBG] osdmap e308: 8 total, 8 up, 6 in 2024-02-22T19:41:10.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:09.590580+0000 mgr.y (mgr.14535) 467 : cluster [DBG] pgmap v599: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.4 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-02-22T19:41:10.586 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:09.373386+0000 mon.a (mon.0) 1326 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:09.385754+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.53:0/1553951669' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:09.385805+0000 mon.a (mon.0) 1328 : cluster [DBG] osdmap e308: 8 total, 8 up, 6 in 2024-02-22T19:41:10.587 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:09.590580+0000 mgr.y (mgr.14535) 467 : cluster [DBG] pgmap v599: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.4 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-02-22T19:41:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:10.749 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:10.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:10.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:10.800 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:41:10.800 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:41:10.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:11.066 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:11.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:11.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:11.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:11.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:41:11.231841+0000 min lat: 0.00135644 max lat: 0.088657 avg lat: 0.0100815 2024-02-22T19:41:11.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:41:11.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 60 16 95249 95233 99.1482 78.625 0.0227076 0.0100815 2024-02-22T19:41:11.237 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:11.462 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:11.467 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:11.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:11.568 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:11.614 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:11.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:41:11.713 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:11.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:11.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:11.798 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:41:11.816 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:11.826 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:41:11.868 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:41:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:11.904 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:41:11.915 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:11.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:11.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:11.931 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:41:11.966 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:41:11.998 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:41:12.022 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:12.032 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.041 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:41:12.052 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:12.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 61 16 96448 96432 98.7513 74.9375 0.00772242 0.0101224 2024-02-22T19:41:12.307 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:12.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:12.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:12.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:12.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:41:12.480 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:11.591134+0000 mgr.y (mgr.14535) 468 : cluster [DBG] pgmap v600: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.50k op/s 2024-02-22T19:41:12.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:11.768352+0000 mon.b (mon.1) 74 : audit [DBG] from='client.? 172.21.15.53:0/1549900713' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:12.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:11.591134+0000 mgr.y (mgr.14535) 468 : cluster [DBG] pgmap v600: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.50k op/s 2024-02-22T19:41:12.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:11.768352+0000 mon.b (mon.1) 74 : audit [DBG] from='client.? 172.21.15.53:0/1549900713' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:12.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:11.591134+0000 mgr.y (mgr.14535) 468 : cluster [DBG] pgmap v600: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.50k op/s 2024-02-22T19:41:12.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:11.768352+0000 mon.b (mon.1) 74 : audit [DBG] from='client.? 172.21.15.53:0/1549900713' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:12.872 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:12.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:12.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:12.977 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:13.037 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:13.085 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:13.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:13.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:13.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:13.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 62 16 97661 97645 98.3814 75.8125 0.00916071 0.0101604 2024-02-22T19:41:13.266 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:13.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:13.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:13.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:13.888 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:13.919 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:13.922 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:13.923 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:14.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:14.085 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:14.166 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:14.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:14.225 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:41:14.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:41:14.226 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-02-22T19:41:14.226 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:41:14.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 63 16 98898 98882 98.047 77.3125 0.0209299 0.0101952 2024-02-22T19:41:14.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:14.340 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:41:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:41:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:41:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:41:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165019 8.8 8.f9eb30f8 (undecoded) ondisk+retry+write+known_if_redirected e217)", 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.671025+0000", 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.66464139799999, 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49627743200000002, 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165019 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671025+0000", 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671025+0000", 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671023+0000", 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2129999999999997e-06 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671032+0000", 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3607e-05 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671046+0000", 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0129999999999999e-06 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671055+0000", 2024-02-22T19:41:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1451000000000002e-05 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671146+0000", 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4880000000000007e-06 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671155+0000", 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49613517299999998 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167290+0000", 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2617e-05 2024-02-22T19:41:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167302+0000", 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49627743200000002 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165021 8.8 8.8bb662f8 (undecoded) ondisk+retry+write+known_if_redirected e217)", 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.671071+0000", 2024-02-22T19:41:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.66459547900001, 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49624754100000001, 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165021 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671071+0000", 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671071+0000", 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671070+0000", 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1300000000000001e-06 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671074+0000", 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1552999999999999e-05 2024-02-22T19:41:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671085+0000", 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7370000000000003e-06 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671091+0000", 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9857999999999993e-05 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671171+0000", 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3150000000000002e-06 2024-02-22T19:41:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671175+0000", 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49614006300000002 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167315+0000", 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4120000000000001e-06 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167318+0000", 2024-02-22T19:41:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49624754100000001 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1369 5.15 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 2] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672397+0000", 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.66326920399999, 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.52036051100000003, 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:14.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1369 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672397+0000", 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672397+0000", 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672396+0000", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672402+0000", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3174e-05 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672416+0000", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.498e-06 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672423+0000", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7455999999999997e-05 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672511+0000", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.537e-06 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672518+0000", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.51161114399999996 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.184129+0000", 2024-02-22T19:41:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014382899999999999 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for missing object", 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.184273+0000", 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048510130000000004 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.189124+0000", 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0124000000000002e-05 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for rw locks", 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.189184+0000", 2024-02-22T19:41:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016101819999999999 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.190794+0000", 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1518000000000002e-05 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.190866+0000", 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001891799 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.192758+0000", 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.52036051100000003 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1371 5.f 5.4322fa9f (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.663230244, 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49085674200000001, 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1371 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0910000000000004e-06 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672443+0000", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7689999999999996e-06 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672450+0000", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5933000000000001e-05 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672466+0000", 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014442799999999999 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672611+0000", 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2309999999999996e-06 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672619+0000", 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49066390399999998 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.163283+0000", 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9820000000000003e-06 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.163293+0000", 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49085674200000001 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165028 8.8 8.bf9c1768 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.688837+0000", 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.64682971100001, 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47849444299999999, 2024-02-22T19:41:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165028 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688837+0000", 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688837+0000", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688835+0000", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7429999999999999e-06 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688841+0000", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2629999999999992e-06 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688850+0000", 2024-02-22T19:41:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6480000000000002e-06 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688855+0000", 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3702000000000002e-05 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688909+0000", 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0879999999999995e-06 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688917+0000", 2024-02-22T19:41:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478410891 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167328+0000", 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5750000000000001e-06 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167331+0000", 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47849444299999999 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165030 8.8 8.d08a68 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.646792191, 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478491316, 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165030 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4669999999999998e-06 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688881+0000", 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7190000000000004e-06 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688888+0000", 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3460000000000004e-06 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688892+0000", 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3278000000000001e-05 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688936+0000", 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-06 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688939+0000", 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478417957 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167357+0000", 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289999999999996e-06 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167366+0000", 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478491316 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165036 8.4 8.ff588524 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.692461+0000", 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.643205217, 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47865222200000002, 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165036 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692461+0000", 2024-02-22T19:41:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692461+0000", 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692460+0000", 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0480000000000004e-06 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692466+0000", 2024-02-22T19:41:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3577999999999999e-05 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692480+0000", 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.413e-06 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692487+0000", 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7818000000000002e-05 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692535+0000", 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692542+0000", 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47856282 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171105+0000", 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8910000000000001e-06 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171113+0000", 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47865222200000002 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165046 8.8 8.5f50e908 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.716885+0000", 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 197.61878129300001, 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.450508415, 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:41:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165046 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716885+0000", 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716885+0000", 2024-02-22T19:41:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716883+0000", 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.289e-06 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716891+0000", 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3481e-05 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716904+0000", 2024-02-22T19:41:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.827e-06 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716911+0000", 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6832000000000001e-05 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716968+0000", 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4290000000000001e-06 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716976+0000", 2024-02-22T19:41:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.45041040799999998 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167386+0000", 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0600000000000002e-06 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167394+0000", 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.450508415 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1385 5.f 5:f95f44c2:::notify.0:head [watch reconnect cookie 94129597095936 gen 6] snapc 0=[] ondisk+write+known_if_redirected e220)", 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.267450+0000", 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 195.06821686399999, 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94290151300000002, 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1385 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267450+0000", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267450+0000", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267448+0000", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3709999999999999e-06 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267453+0000", 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1401e-05 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267465+0000", 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0000000000000002e-06 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267471+0000", 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.774e-05 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267518+0000", 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7289999999999997e-06 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267524+0000", 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93230813000000001 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.199832+0000", 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000136332 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.199968+0000", 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010382590000000001 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.210351+0000", 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94290151300000002 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1562 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129603190784 gen 4] snapc 0=[] ondisk+write+known_if_redirected e263)", 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:39:09.377123+0000", 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.95854335600001, 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.079811426, 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1562 2024-02-22T19:41:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377123+0000", 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377123+0000", 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377121+0000", 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2539999999999998e-06 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377128+0000", 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2010000000000005e-06 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377137+0000", 2024-02-22T19:41:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-02-22T19:41:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377143+0000", 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0889999999999994e-05 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377214+0000", 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9229999999999999e-06 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377220+0000", 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0677836199999999 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:10.445004+0000", 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000121175 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:10.445125+0000", 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011809748 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:10.456934+0000", 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.079811426 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67443 9.7 9:e92be9d4:::benchmark_data_smithi053_157429_object67442:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.958459216000001, 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082204634999999998, 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67443 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3791999999999998e-05 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377240+0000", 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1e-06 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377247+0000", 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2690000000000001e-06 2024-02-22T19:41:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377251+0000", 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025018979999999998 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.379753+0000", 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000130087 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.379883+0000", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000156577 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.380039+0000", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004379371 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384419+0000", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017064900000000001 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384589+0000", 2024-02-22T19:41:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.074719061000000003 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459308+0000", 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4391000000000002e-05 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459363+0000", 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9033999999999998e-05 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459412+0000", 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082204634999999998 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67445 9.6 9:67a14049:::benchmark_data_smithi053_157429_object67444:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.958411189, 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085338631999999998, 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67445 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1210000000000001e-05 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377286+0000", 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4e-06 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377291+0000", 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8519999999999997e-06 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377295+0000", 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066316279999999997 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.383927+0000", 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118863 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384046+0000", 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014849399999999999 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384194+0000", 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075635162000000006 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459829+0000", 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5648000000000002e-05 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459865+0000", 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026404129999999999 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462505+0000", 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0389999999999997e-05 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462556+0000", 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8062000000000003e-05 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462594+0000", 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085338631999999998 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67446 9.f 9:fd8446b1:::benchmark_data_smithi053_157429_object67445:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.958360919, 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082320247999999999, 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67446 2024-02-22T19:41:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3516000000000002e-05 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377339+0000", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7749999999999998e-06 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377344+0000", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9770000000000002e-06 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377348+0000", 2024-02-22T19:41:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046235599999999996 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.381972+0000", 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000132559 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382104+0000", 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018340000000000001 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:41:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382288+0000", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002577415 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384865+0000", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.074599304000000005 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459465+0000", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3739999999999995e-05 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459538+0000", 2024-02-22T19:41:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8214999999999998e-05 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459587+0000", 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9246999999999999e-05 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459626+0000", 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082320247999999999 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67448 9.6 9:6112e6a4:::benchmark_data_smithi053_157429_object67447:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377353+0000", 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.958313643, 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085341474, 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67448 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377353+0000", 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377353+0000", 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377352+0000", 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051044819999999996 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382457+0000", 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0871e-05 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382468+0000", 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1190000000000002e-06 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382474+0000", 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034600680000000002 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.385934+0000", 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120294 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386054+0000", 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8580999999999997e-05 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386153+0000", 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.073719253999999998 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459872+0000", 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1220999999999997e-05 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459913+0000", 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00271478 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462628+0000", 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6109999999999998e-05 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462664+0000", 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0173000000000001e-05 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462694+0000", 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085341474 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67450 9.7 9:ee1c16d7:::benchmark_data_smithi053_157429_object67449:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.382481+0000", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.9531852, 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077561816000000006, 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67450 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382481+0000", 2024-02-22T19:41:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382481+0000", 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382480+0000", 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3385e-05 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382513+0000", 2024-02-22T19:41:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6250000000000004e-06 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382519+0000", 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1400000000000002e-06 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382523+0000", 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8408e-05 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382552+0000", 2024-02-22T19:41:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6744999999999998e-05 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382628+0000", 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014082299999999999 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382769+0000", 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021010009999999999 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384870+0000", 2024-02-22T19:41:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.074629951 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459500+0000", 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00044630100000000002 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459946+0000", 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8202999999999998e-05 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.460005+0000", 2024-02-22T19:41:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8498000000000003e-05 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.460043+0000", 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077561816000000006 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop_reply(client.15324.0:67443 9.7 e302/285 ondisk, result = 0)", 2024-02-22T19:41:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.386523+0000", 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.949143742, 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072901676999999998, 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386523+0000", 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386523+0000", 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386522+0000", 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9750000000000001e-06 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386526+0000", 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2989999999999999e-06 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386533+0000", 2024-02-22T19:41:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2849999999999996e-06 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386538+0000", 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072746221 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459284+0000", 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6127999999999999e-05 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459300+0000", 2024-02-22T19:41:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124368 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459424+0000", 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072901676999999998 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop_reply(client.15324.0:67450 9.7 e302/285 ondisk, result = 0)", 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.386688+0000", 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.948978245999999, 2024-02-22T19:41:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072813283000000006, 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386688+0000", 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386688+0000", 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386688+0000", 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7610000000000001e-06 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386691+0000", 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072566179999999994 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459258+0000", 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1659e-05 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459269+0000", 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000218882 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459488+0000", 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9449999999999998e-06 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459494+0000", 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459501+0000", 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072813283000000006 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop_reply(client.15324.0:67446 9.f e302/285 ondisk, result = 0)", 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.387274+0000", 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.948392439999999, 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072192585000000004, 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.387274+0000", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.387274+0000", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.387273+0000", 2024-02-22T19:41:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6779999999999999e-06 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.387278+0000", 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071988262999999997 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459266+0000", 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0104e-05 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459276+0000", 2024-02-22T19:41:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000174048 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459450+0000", 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2060000000000001e-06 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459458+0000", 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9619999999999999e-06 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459467+0000", 2024-02-22T19:41:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.072192585000000004 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:83018 9.f 9:f9322158:::benchmark_data_smithi053_157429_object83017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e305)", 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:01.032138+0000", 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.303528786999999, 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077364552000000003, 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 83018 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032138+0000", 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032138+0000", 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032137+0000", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0630000000000003e-05 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032168+0000", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.182e-06 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032175+0000", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5600000000000004e-06 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032179+0000", 2024-02-22T19:41:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025713250000000002 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.034751+0000", 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123466 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.034874+0000", 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000185787 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.035060+0000", 2024-02-22T19:41:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025580569999999999 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.037618+0000", 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064692199999999997 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038265+0000", 2024-02-22T19:41:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071143933000000006 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109409+0000", 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4432000000000003e-05 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109463+0000", 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8856999999999997e-05 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109502+0000", 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077364552000000003 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:83030 9.7 9:e485986d:::benchmark_data_smithi053_157429_object83029:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e305)", 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:01.038599+0000", 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.297067837, 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071112865999999997, 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 83030 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038599+0000", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038599+0000", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038597+0000", 2024-02-22T19:41:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8632999999999999e-05 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038656+0000", 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1514e-05 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038667+0000", 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1580000000000002e-06 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038673+0000", 2024-02-22T19:41:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1839999999999998e-05 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038725+0000", 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0462000000000002e-05 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038816+0000", 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011758500000000001 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:41:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038933+0000", 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030381359999999999 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.041971+0000", 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000426172 2024-02-22T19:41:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.042398+0000", 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.067245870999999999 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109643+0000", 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5899000000000003e-05 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109679+0000", 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2066000000000001e-05 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109711+0000", 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071112865999999997 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:14.444 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:41:14.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:14.445 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:14.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:14.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:14.591 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:14.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:14.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:14.839 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:13.591539+0000 mgr.y (mgr.14535) 469 : cluster [DBG] pgmap v601: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 46 MiB/s wr, 737 op/s 2024-02-22T19:41:14.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:13.591539+0000 mgr.y (mgr.14535) 469 : cluster [DBG] pgmap v601: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 46 MiB/s wr, 737 op/s 2024-02-22T19:41:14.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:14.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:41:14.912 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:13.591539+0000 mgr.y (mgr.14535) 469 : cluster [DBG] pgmap v601: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 46 MiB/s wr, 737 op/s 2024-02-22T19:41:14.913 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:41:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:41:14.915 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:15.150 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:15.160 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:15.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:15.168 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:41:15.170 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:41:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:41:15.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 64 16 99974 99958 97.5659 67.25 0.0143259 0.010244 2024-02-22T19:41:15.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:15.311 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:15.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:15.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:15.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:15.520 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:15.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:15.561 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:15.625 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:15.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:41:15.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:14.954572+0000 mgr.y (mgr.14535) 470 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:15.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:15.444111+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.53:0/12238147' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:41:15.720 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:15.444902+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:41:15.722 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:14.954572+0000 mgr.y (mgr.14535) 470 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:15.722 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:15.444111+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.53:0/12238147' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:41:15.722 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:15.444902+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:41:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:14.954572+0000 mgr.y (mgr.14535) 470 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:15.444111+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.53:0/12238147' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:41:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:15.444902+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:41:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:15.944 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:15.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:16.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:16.108 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:16.141 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:16.143 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:16.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:16.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 65 16 101128 101112 97.1744 72.125 0.0172678 0.0102866 2024-02-22T19:41:16.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:16.319 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:16.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:15.592328+0000 mgr.y (mgr.14535) 471 : cluster [DBG] pgmap v602: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:41:16.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:15.667668+0000 mon.a (mon.0) 1330 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:16.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:15.684836+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:41:16.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:15.684908+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e309: 8 total, 8 up, 6 in 2024-02-22T19:41:16.948 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:15.592328+0000 mgr.y (mgr.14535) 471 : cluster [DBG] pgmap v602: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:41:16.948 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:15.667668+0000 mon.a (mon.0) 1330 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:16.948 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:15.684836+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:41:16.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:15.684908+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e309: 8 total, 8 up, 6 in 2024-02-22T19:41:17.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:15.592328+0000 mgr.y (mgr.14535) 471 : cluster [DBG] pgmap v602: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.51k op/s 2024-02-22T19:41:17.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:15.667668+0000 mon.a (mon.0) 1330 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:17.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:15.684836+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:41:17.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:15.684908+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e309: 8 total, 8 up, 6 in 2024-02-22T19:41:17.231 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 66 16 102426 102410 96.9312 81.125 0.00794492 0.0103129 2024-02-22T19:41:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:17.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:17.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:17.295 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:17.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:17.449 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:17.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:17.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:17.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:17.654 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15324.0:102871 9.1 e310/285 9:8c7921ad:::benchmark_data_smithi053_157429_object102870:head v 310'6281, mlcod=310'6278)", 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:17.656916+0000", 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.005423336, 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054556960000000003, 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.656916+0000", 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.656916+0000", 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.656915+0000", 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8277e-05 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.656953+0000", 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5786000000000001e-05 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.656969+0000", 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.703e-06 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.656976+0000", 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4501999999999998e-05 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.657060+0000", 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3084000000000001e-05 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.657083+0000", 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000167286 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15324.0:102875 9.c e310/285 9:34e0f193:::benchmark_data_smithi053_157429_object102874:head v 310'6435, mlcod=310'6432)", 2024-02-22T19:41:17.666 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:17.659232+0000", 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.003107823, 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032028629999999998, 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659232+0000", 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659232+0000", 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659230+0000", 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7764999999999997e-05 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659328+0000", 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2496e-05 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659340+0000", 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.635e-06 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659347+0000", 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6050999999999995e-05 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659413+0000", 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4162999999999999e-05 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.659437+0000", 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000205811 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:17.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15324.0:102879 9.c e310/285 9:3ef6c040:::benchmark_data_smithi053_157429_object102878:head v 310'6436, mlcod=310'6433)", 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:17.660835+0000", 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0015038549999999999, 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00164448, 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.660835+0000", 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.660835+0000", 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.660834+0000", 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010516 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.660940+0000", 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.806e-05 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.660958+0000", 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8599999999999993e-06 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.660966+0000", 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00062117800000000001 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.661587+0000", 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8408999999999999e-05 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:17.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:17.661605+0000", 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000769686 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 3 2024-02-22T19:41:17.673 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:41:17.691 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:17.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:17.698 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:17.799 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:17.826 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.233742+0000 osd.2 (osd.2) 168 : cluster [DBG] 5.a scrub starts 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.236658+0000 osd.2 (osd.2) 169 : cluster [DBG] 5.a scrub ok 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.251711+0000 osd.1 (osd.1) 190 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.258264+0000 osd.1 (osd.1) 191 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.671298+0000 osd.3 (osd.3) 207 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.678311+0000 osd.3 (osd.3) 208 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.721044+0000 osd.7 (osd.7) 195 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.724616+0000 osd.7 (osd.7) 196 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.784187+0000 osd.6 (osd.6) 136 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:16.790217+0000 osd.6 (osd.6) 137 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:41:17.855 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:17.272323+0000 mon.a (mon.0) 1333 : cluster [DBG] osdmap e310: 8 total, 8 up, 6 in 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.233742+0000 osd.2 (osd.2) 168 : cluster [DBG] 5.a scrub starts 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.236658+0000 osd.2 (osd.2) 169 : cluster [DBG] 5.a scrub ok 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.251711+0000 osd.1 (osd.1) 190 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.258264+0000 osd.1 (osd.1) 191 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.671298+0000 osd.3 (osd.3) 207 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.678311+0000 osd.3 (osd.3) 208 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.721044+0000 osd.7 (osd.7) 195 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.724616+0000 osd.7 (osd.7) 196 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.784187+0000 osd.6 (osd.6) 136 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:41:17.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:16.790217+0000 osd.6 (osd.6) 137 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:41:17.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:17.272323+0000 mon.a (mon.0) 1333 : cluster [DBG] osdmap e310: 8 total, 8 up, 6 in 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.233742+0000 osd.2 (osd.2) 168 : cluster [DBG] 5.a scrub starts 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.236658+0000 osd.2 (osd.2) 169 : cluster [DBG] 5.a scrub ok 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.251711+0000 osd.1 (osd.1) 190 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.258264+0000 osd.1 (osd.1) 191 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.671298+0000 osd.3 (osd.3) 207 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.678311+0000 osd.3 (osd.3) 208 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.721044+0000 osd.7 (osd.7) 195 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.724616+0000 osd.7 (osd.7) 196 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.784187+0000 osd.6 (osd.6) 136 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:16.790217+0000 osd.6 (osd.6) 137 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:41:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:17.272323+0000 mon.a (mon.0) 1333 : cluster [DBG] osdmap e310: 8 total, 8 up, 6 in 2024-02-22T19:41:18.232 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 67 16 103464 103448 96.4528 64.875 0.0172598 0.0103635 2024-02-22T19:41:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:17.594925+0000 mgr.y (mgr.14535) 472 : cluster [DBG] pgmap v605: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 639 B/s rd, 60 MiB/s wr, 954 op/s 2024-02-22T19:41:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:17.699088+0000 osd.7 (osd.7) 197 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:41:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:17.712479+0000 osd.7 (osd.7) 198 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:41:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:17.857809+0000 osd.6 (osd.6) 138 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:41:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:17.864418+0000 osd.6 (osd.6) 139 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:41:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:17.594925+0000 mgr.y (mgr.14535) 472 : cluster [DBG] pgmap v605: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 639 B/s rd, 60 MiB/s wr, 954 op/s 2024-02-22T19:41:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:17.699088+0000 osd.7 (osd.7) 197 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:41:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:17.712479+0000 osd.7 (osd.7) 198 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:41:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:17.857809+0000 osd.6 (osd.6) 138 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:41:18.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:17.864418+0000 osd.6 (osd.6) 139 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:41:18.946 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:41:18.949 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:41:18.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:41:19.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:17.594925+0000 mgr.y (mgr.14535) 472 : cluster [DBG] pgmap v605: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 639 B/s rd, 60 MiB/s wr, 954 op/s 2024-02-22T19:41:19.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:17.699088+0000 osd.7 (osd.7) 197 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:41:19.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:17.712479+0000 osd.7 (osd.7) 198 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:41:19.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:17.857809+0000 osd.6 (osd.6) 138 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:41:19.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:17.864418+0000 osd.6 (osd.6) 139 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:41:19.232 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 68 16 104638 104622 96.1126 73.375 0.00994666 0.0104001 2024-02-22T19:41:19.560 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:19] "GET /metrics HTTP/1.1" 200 176129 "" "Prometheus/2.33.4" 2024-02-22T19:41:19.887 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:19.897 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:19.897 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:19.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:18.265882+0000 osd.1 (osd.1) 192 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:18.275058+0000 osd.1 (osd.1) 193 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:18.498499+0000 osd.5 (osd.5) 96 : cluster [DBG] 6.d scrub starts 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:18.520944+0000 mgr.y (mgr.14535) 473 : audit [DBG] from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:18.527700+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.d scrub ok 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:18.265882+0000 osd.1 (osd.1) 192 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:18.275058+0000 osd.1 (osd.1) 193 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:18.498499+0000 osd.5 (osd.5) 96 : cluster [DBG] 6.d scrub starts 2024-02-22T19:41:20.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:18.520944+0000 mgr.y (mgr.14535) 473 : audit [DBG] from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:41:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:18.527700+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.d scrub ok 2024-02-22T19:41:20.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:18.265882+0000 osd.1 (osd.1) 192 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:41:20.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:18.275058+0000 osd.1 (osd.1) 193 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:41:20.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:18.498499+0000 osd.5 (osd.5) 96 : cluster [DBG] 6.d scrub starts 2024-02-22T19:41:20.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:18.520944+0000 mgr.y (mgr.14535) 473 : audit [DBG] from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:41:20.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:18.527700+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.d scrub ok 2024-02-22T19:41:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:20.094 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":605,"stamp":"2024-02-22T19:41:17.594713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6495671972,"num_objects":99308,"num_object_clones":0,"num_object_copies":297924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99308,"num_whiteouts":0,"num_read":1842,"num_read_kb":1594,"num_write":99830,"num_write_kb":6343465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":13666,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63331,"ondisk_log_size":63331,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":21030460,"kb_used_data":18909580,"kb_used_omap":0,"kb_used_meta":2120832,"kb_avail":541440452,"statfs":{"total":575970213888,"available":554435022848,"internally_reserved":0,"allocated":19363409920,"data_stored":19358036734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2171731968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":500563968,"num_objects":7638,"num_object_clones":0,"num_object_copies":22914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7638,"num_write_kb":488832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004521"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":265,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278431+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:41:08.278431+0000","last_peered":"2024-02-22T19:41:08.278431+0000","last_clean":"2024-02-22T19:41:08.278431+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:41:08.278431+0000","last_undegraded":"2024-02-22T19:41:08.278431+0000","last_fullsized":"2024-02-22T19:41:08.278431+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":286,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.280409+0000","last_change":"2024-02-22T19:39:22.268923+0000","last_active":"2024-02-22T19:41:08.280409+0000","last_peered":"2024-02-22T19:41:08.280409+0000","last_clean":"2024-02-22T19:41:08.280409+0000","last_became_active":"2024-02-22T19:39:22.268587+0000","last_became_peered":"2024-02-22T19:39:22.268587+0000","last_unstale":"2024-02-22T19:41:08.280409+0000","last_undegraded":"2024-02-22T19:41:08.280409+0000","last_fullsized":"2024-02-22T19:41:08.280409+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":123,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.281515+0000","last_change":"2024-02-22T19:39:36.528260+0000","last_active":"2024-02-22T19:41:08.281515+0000","last_peered":"2024-02-22T19:41:08.281515+0000","last_clean":"2024-02-22T19:41:08.281515+0000","last_became_active":"2024-02-22T19:38:47.164599+0000","last_became_peered":"2024-02-22T19:38:47.164599+0000","last_unstale":"2024-02-22T19:41:08.281515+0000","last_undegraded":"2024-02-22T19:41:08.281515+0000","last_fullsized":"2024-02-22T19:41:08.281515+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:39:36.528175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.528175+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":266,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291659+0000","last_change":"2024-02-22T19:36:59.298132+0000","last_active":"2024-02-22T19:41:08.291659+0000","last_peered":"2024-02-22T19:41:08.291659+0000","last_clean":"2024-02-22T19:41:08.291659+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:41:08.291659+0000","last_undegraded":"2024-02-22T19:41:08.291659+0000","last_fullsized":"2024-02-22T19:41:08.291659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:59.298021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:36:59.298021+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":272,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290891+0000","last_change":"2024-02-22T19:40:34.661155+0000","last_active":"2024-02-22T19:41:08.290891+0000","last_peered":"2024-02-22T19:41:08.290891+0000","last_clean":"2024-02-22T19:41:08.290891+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:41:08.290891+0000","last_undegraded":"2024-02-22T19:41:08.290891+0000","last_fullsized":"2024-02-22T19:41:08.290891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":388,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290647+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:41:08.290647+0000","last_peered":"2024-02-22T19:41:08.290647+0000","last_clean":"2024-02-22T19:41:08.290647+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:41:08.290647+0000","last_undegraded":"2024-02-22T19:41:08.290647+0000","last_fullsized":"2024-02-22T19:41:08.290647+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":104,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291183+0000","last_change":"2024-02-22T19:39:36.296064+0000","last_active":"2024-02-22T19:41:08.291183+0000","last_peered":"2024-02-22T19:41:08.291183+0000","last_clean":"2024-02-22T19:41:08.291183+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:41:08.291183+0000","last_undegraded":"2024-02-22T19:41:08.291183+0000","last_fullsized":"2024-02-22T19:41:08.291183+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:36.295988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:36.295988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":109,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279223+0000","last_change":"2024-02-22T19:38:47.169310+0000","last_active":"2024-02-22T19:41:08.279223+0000","last_peered":"2024-02-22T19:41:08.279223+0000","last_clean":"2024-02-22T19:41:08.279223+0000","last_became_active":"2024-02-22T19:38:47.168764+0000","last_became_peered":"2024-02-22T19:38:47.168764+0000","last_unstale":"2024-02-22T19:41:08.279223+0000","last_undegraded":"2024-02-22T19:41:08.279223+0000","last_fullsized":"2024-02-22T19:41:08.279223+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":166,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.292323+0000","last_change":"2024-02-22T19:40:48.683218+0000","last_active":"2024-02-22T19:41:08.292323+0000","last_peered":"2024-02-22T19:41:08.292323+0000","last_clean":"2024-02-22T19:41:08.292323+0000","last_became_active":"2024-02-22T19:39:22.269400+0000","last_became_peered":"2024-02-22T19:39:22.269400+0000","last_unstale":"2024-02-22T19:41:08.292323+0000","last_undegraded":"2024-02-22T19:41:08.292323+0000","last_fullsized":"2024-02-22T19:41:08.292323+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":105,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278318+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:41:08.278318+0000","last_peered":"2024-02-22T19:41:08.278318+0000","last_clean":"2024-02-22T19:41:08.278318+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:41:08.278318+0000","last_undegraded":"2024-02-22T19:41:08.278318+0000","last_fullsized":"2024-02-22T19:41:08.278318+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":122,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278940+0000","last_change":"2024-02-22T19:39:53.438381+0000","last_active":"2024-02-22T19:41:08.278940+0000","last_peered":"2024-02-22T19:41:08.278940+0000","last_clean":"2024-02-22T19:41:08.278940+0000","last_became_active":"2024-02-22T19:38:47.179485+0000","last_became_peered":"2024-02-22T19:38:47.179485+0000","last_unstale":"2024-02-22T19:41:08.278940+0000","last_undegraded":"2024-02-22T19:41:08.278940+0000","last_fullsized":"2024-02-22T19:41:08.278940+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":108,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291518+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:41:08.291518+0000","last_peered":"2024-02-22T19:41:08.291518+0000","last_clean":"2024-02-22T19:41:08.291518+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:41:08.291518+0000","last_undegraded":"2024-02-22T19:41:08.291518+0000","last_fullsized":"2024-02-22T19:41:08.291518+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":261,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.282351+0000","last_change":"2024-02-22T19:39:34.555380+0000","last_active":"2024-02-22T19:41:08.282351+0000","last_peered":"2024-02-22T19:41:08.282351+0000","last_clean":"2024-02-22T19:41:08.282351+0000","last_became_active":"2024-02-22T19:38:47.168230+0000","last_became_peered":"2024-02-22T19:38:47.168230+0000","last_unstale":"2024-02-22T19:41:08.282351+0000","last_undegraded":"2024-02-22T19:41:08.282351+0000","last_fullsized":"2024-02-22T19:41:08.282351+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:34.555298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.555298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":77,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283781+0000","last_change":"2024-02-22T19:39:22.245523+0000","last_active":"2024-02-22T19:41:08.283781+0000","last_peered":"2024-02-22T19:41:08.283781+0000","last_clean":"2024-02-22T19:41:08.283781+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:41:08.283781+0000","last_undegraded":"2024-02-22T19:41:08.283781+0000","last_fullsized":"2024-02-22T19:41:08.283781+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.732856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.732856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":335,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.286681+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:41:08.286681+0000","last_peered":"2024-02-22T19:41:08.286681+0000","last_clean":"2024-02-22T19:41:08.286681+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:41:08.286681+0000","last_undegraded":"2024-02-22T19:41:08.286681+0000","last_fullsized":"2024-02-22T19:41:08.286681+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":284,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.295138+0000","last_change":"2024-02-22T19:37:57.180410+0000","last_active":"2024-02-22T19:41:08.295138+0000","last_peered":"2024-02-22T19:41:08.295138+0000","last_clean":"2024-02-22T19:41:08.295138+0000","last_became_active":"2024-02-22T19:37:57.180089+0000","last_became_peered":"2024-02-22T19:37:57.180089+0000","last_unstale":"2024-02-22T19:41:08.295138+0000","last_undegraded":"2024-02-22T19:41:08.295138+0000","last_fullsized":"2024-02-22T19:41:08.295138+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:36:28.335750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":95,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288866+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:41:08.288866+0000","last_peered":"2024-02-22T19:41:08.288866+0000","last_clean":"2024-02-22T19:41:08.288866+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:41:08.288866+0000","last_undegraded":"2024-02-22T19:41:08.288866+0000","last_fullsized":"2024-02-22T19:41:08.288866+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":328,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290418+0000","last_change":"2024-02-22T19:40:39.682028+0000","last_active":"2024-02-22T19:41:08.290418+0000","last_peered":"2024-02-22T19:41:08.290418+0000","last_clean":"2024-02-22T19:41:08.290418+0000","last_became_active":"2024-02-22T19:38:47.180453+0000","last_became_peered":"2024-02-22T19:38:47.180453+0000","last_unstale":"2024-02-22T19:41:08.290418+0000","last_undegraded":"2024-02-22T19:41:08.290418+0000","last_fullsized":"2024-02-22T19:41:08.290418+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":314,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283579+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:41:08.283579+0000","last_peered":"2024-02-22T19:41:08.283579+0000","last_clean":"2024-02-22T19:41:08.283579+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:41:08.283579+0000","last_undegraded":"2024-02-22T19:41:08.283579+0000","last_fullsized":"2024-02-22T19:41:08.283579+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":294,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.289566+0000","last_change":"2024-02-22T19:40:50.388588+0000","last_active":"2024-02-22T19:41:08.289566+0000","last_peered":"2024-02-22T19:41:08.289566+0000","last_clean":"2024-02-22T19:41:08.289566+0000","last_became_active":"2024-02-22T19:38:47.168149+0000","last_became_peered":"2024-02-22T19:38:47.168149+0000","last_unstale":"2024-02-22T19:41:08.289566+0000","last_undegraded":"2024-02-22T19:41:08.289566+0000","last_fullsized":"2024-02-22T19:41:08.289566+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":115,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.284123+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:41:08.284123+0000","last_peered":"2024-02-22T19:41:08.284123+0000","last_clean":"2024-02-22T19:41:08.284123+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:41:08.284123+0000","last_undegraded":"2024-02-22T19:41:08.284123+0000","last_fullsized":"2024-02-22T19:41:08.284123+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":303,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.294784+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:41:08.294784+0000","last_peered":"2024-02-22T19:41:08.294784+0000","last_clean":"2024-02-22T19:41:08.294784+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:41:08.294784+0000","last_undegraded":"2024-02-22T19:41:08.294784+0000","last_fullsized":"2024-02-22T19:41:08.294784+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":301,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291754+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:41:08.291754+0000","last_peered":"2024-02-22T19:41:08.291754+0000","last_clean":"2024-02-22T19:41:08.291754+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:41:08.291754+0000","last_undegraded":"2024-02-22T19:41:08.291754+0000","last_fullsized":"2024-02-22T19:41:08.291754+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":384,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.282109+0000","last_change":"2024-02-22T19:40:49.582930+0000","last_active":"2024-02-22T19:41:08.282109+0000","last_peered":"2024-02-22T19:41:08.282109+0000","last_clean":"2024-02-22T19:41:08.282109+0000","last_became_active":"2024-02-22T19:37:57.190476+0000","last_became_peered":"2024-02-22T19:37:57.190476+0000","last_unstale":"2024-02-22T19:41:08.282109+0000","last_undegraded":"2024-02-22T19:41:08.282109+0000","last_fullsized":"2024-02-22T19:41:08.282109+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":283,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283431+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:41:08.283431+0000","last_peered":"2024-02-22T19:41:08.283431+0000","last_clean":"2024-02-22T19:41:08.283431+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:41:08.283431+0000","last_undegraded":"2024-02-22T19:41:08.283431+0000","last_fullsized":"2024-02-22T19:41:08.283431+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":299,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279812+0000","last_change":"2024-02-22T19:39:22.256927+0000","last_active":"2024-02-22T19:41:08.279812+0000","last_peered":"2024-02-22T19:41:08.279812+0000","last_clean":"2024-02-22T19:41:08.279812+0000","last_became_active":"2024-02-22T19:39:22.256576+0000","last_became_peered":"2024-02-22T19:39:22.256576+0000","last_unstale":"2024-02-22T19:41:08.279812+0000","last_undegraded":"2024-02-22T19:41:08.279812+0000","last_fullsized":"2024-02-22T19:41:08.279812+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.400534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.400534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":286,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.280321+0000","last_change":"2024-02-22T19:39:22.246647+0000","last_active":"2024-02-22T19:41:08.280321+0000","last_peered":"2024-02-22T19:41:08.280321+0000","last_clean":"2024-02-22T19:41:08.280321+0000","last_became_active":"2024-02-22T19:39:22.246323+0000","last_became_peered":"2024-02-22T19:39:22.246323+0000","last_unstale":"2024-02-22T19:41:08.280321+0000","last_undegraded":"2024-02-22T19:41:08.280321+0000","last_fullsized":"2024-02-22T19:41:08.280321+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":282,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.292006+0000","last_change":"2024-02-22T19:36:58.263929+0000","last_active":"2024-02-22T19:41:08.292006+0000","last_peered":"2024-02-22T19:41:08.292006+0000","last_clean":"2024-02-22T19:41:08.292006+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:41:08.292006+0000","last_undegraded":"2024-02-22T19:41:08.292006+0000","last_fullsized":"2024-02-22T19:41:08.292006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-22T19:36:58.263834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.263834+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":295,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291749+0000","last_change":"2024-02-22T19:39:22.250184+0000","last_active":"2024-02-22T19:41:08.291749+0000","last_peered":"2024-02-22T19:41:08.291749+0000","last_clean":"2024-02-22T19:41:08.291749+0000","last_became_active":"2024-02-22T19:39:22.249890+0000","last_became_peered":"2024-02-22T19:39:22.249890+0000","last_unstale":"2024-02-22T19:41:08.291749+0000","last_undegraded":"2024-02-22T19:41:08.291749+0000","last_fullsized":"2024-02-22T19:41:08.291749+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":301,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283428+0000","last_change":"2024-02-22T19:39:22.253288+0000","last_active":"2024-02-22T19:41:08.283428+0000","last_peered":"2024-02-22T19:41:08.283428+0000","last_clean":"2024-02-22T19:41:08.283428+0000","last_became_active":"2024-02-22T19:39:22.252705+0000","last_became_peered":"2024-02-22T19:39:22.252705+0000","last_unstale":"2024-02-22T19:41:08.283428+0000","last_undegraded":"2024-02-22T19:41:08.283428+0000","last_fullsized":"2024-02-22T19:41:08.283428+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:54.418958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:37:54.418958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":100,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279359+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:41:08.279359+0000","last_peered":"2024-02-22T19:41:08.279359+0000","last_clean":"2024-02-22T19:41:08.279359+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:41:08.279359+0000","last_undegraded":"2024-02-22T19:41:08.279359+0000","last_fullsized":"2024-02-22T19:41:08.279359+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":301,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290544+0000","last_change":"2024-02-22T19:39:37.816765+0000","last_active":"2024-02-22T19:41:08.290544+0000","last_peered":"2024-02-22T19:41:08.290544+0000","last_clean":"2024-02-22T19:41:08.290544+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:41:08.290544+0000","last_undegraded":"2024-02-22T19:41:08.290544+0000","last_fullsized":"2024-02-22T19:41:08.290544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":277,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278156+0000","last_change":"2024-02-22T19:39:22.247453+0000","last_active":"2024-02-22T19:41:08.278156+0000","last_peered":"2024-02-22T19:41:08.278156+0000","last_clean":"2024-02-22T19:41:08.278156+0000","last_became_active":"2024-02-22T19:39:22.247266+0000","last_became_peered":"2024-02-22T19:39:22.247266+0000","last_unstale":"2024-02-22T19:41:08.278156+0000","last_undegraded":"2024-02-22T19:41:08.278156+0000","last_fullsized":"2024-02-22T19:41:08.278156+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":295,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291614+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:41:08.291614+0000","last_peered":"2024-02-22T19:41:08.291614+0000","last_clean":"2024-02-22T19:41:08.291614+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:41:08.291614+0000","last_undegraded":"2024-02-22T19:41:08.291614+0000","last_fullsized":"2024-02-22T19:41:08.291614+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":309,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283508+0000","last_change":"2024-02-22T19:39:22.254738+0000","last_active":"2024-02-22T19:41:08.283508+0000","last_peered":"2024-02-22T19:41:08.283508+0000","last_clean":"2024-02-22T19:41:08.283508+0000","last_became_active":"2024-02-22T19:39:22.254411+0000","last_became_peered":"2024-02-22T19:39:22.254411+0000","last_unstale":"2024-02-22T19:41:08.283508+0000","last_undegraded":"2024-02-22T19:41:08.283508+0000","last_fullsized":"2024-02-22T19:41:08.283508+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:52.463774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:36:52.463774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":303,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.289446+0000","last_change":"2024-02-22T19:39:44.297110+0000","last_active":"2024-02-22T19:41:08.289446+0000","last_peered":"2024-02-22T19:41:08.289446+0000","last_clean":"2024-02-22T19:41:08.289446+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:41:08.289446+0000","last_undegraded":"2024-02-22T19:41:08.289446+0000","last_fullsized":"2024-02-22T19:41:08.289446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:39:44.296975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:44.296975+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":316,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290359+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:41:08.290359+0000","last_peered":"2024-02-22T19:41:08.290359+0000","last_clean":"2024-02-22T19:41:08.290359+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:41:08.290359+0000","last_undegraded":"2024-02-22T19:41:08.290359+0000","last_fullsized":"2024-02-22T19:41:08.290359+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":294,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.289173+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:41:08.289173+0000","last_peered":"2024-02-22T19:41:08.289173+0000","last_clean":"2024-02-22T19:41:08.289173+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:41:08.289173+0000","last_undegraded":"2024-02-22T19:41:08.289173+0000","last_fullsized":"2024-02-22T19:41:08.289173+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":77,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283412+0000","last_change":"2024-02-22T19:40:52.560079+0000","last_active":"2024-02-22T19:41:08.283412+0000","last_peered":"2024-02-22T19:41:08.283412+0000","last_clean":"2024-02-22T19:41:08.283412+0000","last_became_active":"2024-02-22T19:38:47.161791+0000","last_became_peered":"2024-02-22T19:38:47.161791+0000","last_unstale":"2024-02-22T19:41:08.283412+0000","last_undegraded":"2024-02-22T19:41:08.283412+0000","last_fullsized":"2024-02-22T19:41:08.283412+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":126,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.281624+0000","last_change":"2024-02-22T19:38:47.783561+0000","last_active":"2024-02-22T19:41:08.281624+0000","last_peered":"2024-02-22T19:41:08.281624+0000","last_clean":"2024-02-22T19:41:08.281624+0000","last_became_active":"2024-02-22T19:38:47.168328+0000","last_became_peered":"2024-02-22T19:38:47.168328+0000","last_unstale":"2024-02-22T19:41:08.281624+0000","last_undegraded":"2024-02-22T19:41:08.281624+0000","last_fullsized":"2024-02-22T19:41:08.281624+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-22T19:37:53.461286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.461286+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":103,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.284046+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:41:08.284046+0000","last_peered":"2024-02-22T19:41:08.284046+0000","last_clean":"2024-02-22T19:41:08.284046+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:41:08.284046+0000","last_undegraded":"2024-02-22T19:41:08.284046+0000","last_fullsized":"2024-02-22T19:41:08.284046+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":186,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.404696+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:41:09.404696+0000","last_peered":"2024-02-22T19:41:09.404696+0000","last_clean":"2024-02-22T19:41:09.404696+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:41:09.404696+0000","last_undegraded":"2024-02-22T19:41:09.404696+0000","last_fullsized":"2024-02-22T19:41:09.404696+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":282,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283686+0000","last_change":"2024-02-22T19:38:47.162564+0000","last_active":"2024-02-22T19:41:08.283686+0000","last_peered":"2024-02-22T19:41:08.283686+0000","last_clean":"2024-02-22T19:41:08.283686+0000","last_became_active":"2024-02-22T19:38:47.162182+0000","last_became_peered":"2024-02-22T19:38:47.162182+0000","last_unstale":"2024-02-22T19:41:08.283686+0000","last_undegraded":"2024-02-22T19:41:08.283686+0000","last_fullsized":"2024-02-22T19:41:08.283686+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:56.463502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:36:56.463502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":305,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290922+0000","last_change":"2024-02-22T19:37:09.345416+0000","last_active":"2024-02-22T19:41:08.290922+0000","last_peered":"2024-02-22T19:41:08.290922+0000","last_clean":"2024-02-22T19:41:08.290922+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:41:08.290922+0000","last_undegraded":"2024-02-22T19:41:08.290922+0000","last_fullsized":"2024-02-22T19:41:08.290922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'15","last_scrub_stamp":"2024-02-22T19:37:09.345319+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.345319+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":283,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288454+0000","last_change":"2024-02-22T19:39:41.365968+0000","last_active":"2024-02-22T19:41:08.288454+0000","last_peered":"2024-02-22T19:41:08.288454+0000","last_clean":"2024-02-22T19:41:08.288454+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:41:08.288454+0000","last_undegraded":"2024-02-22T19:41:08.288454+0000","last_fullsized":"2024-02-22T19:41:08.288454+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:41.365877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.365877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":423,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.397925+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:41:09.397925+0000","last_peered":"2024-02-22T19:41:09.397925+0000","last_clean":"2024-02-22T19:41:09.397925+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:41:09.397925+0000","last_undegraded":"2024-02-22T19:41:09.397925+0000","last_fullsized":"2024-02-22T19:41:09.397925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":89,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283106+0000","last_change":"2024-02-22T19:39:22.246092+0000","last_active":"2024-02-22T19:41:08.283106+0000","last_peered":"2024-02-22T19:41:08.283106+0000","last_clean":"2024-02-22T19:41:08.283106+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:41:08.283106+0000","last_undegraded":"2024-02-22T19:41:08.283106+0000","last_fullsized":"2024-02-22T19:41:08.283106+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":119,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279315+0000","last_change":"2024-02-22T19:37:57.198915+0000","last_active":"2024-02-22T19:41:08.279315+0000","last_peered":"2024-02-22T19:41:08.279315+0000","last_clean":"2024-02-22T19:41:08.279315+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:41:08.279315+0000","last_undegraded":"2024-02-22T19:41:08.279315+0000","last_fullsized":"2024-02-22T19:41:08.279315+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'13","last_scrub_stamp":"2024-02-22T19:36:48.456659+0000","last_deep_scrub":"88'13","last_deep_scrub_stamp":"2024-02-22T19:31:15.685295+0000","last_clean_scrub_stamp":"2024-02-22T19:36:48.456659+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":306,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.286917+0000","last_change":"2024-02-22T19:39:53.126573+0000","last_active":"2024-02-22T19:41:08.286917+0000","last_peered":"2024-02-22T19:41:08.286917+0000","last_clean":"2024-02-22T19:41:08.286917+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:41:08.286917+0000","last_undegraded":"2024-02-22T19:41:08.286917+0000","last_fullsized":"2024-02-22T19:41:08.286917+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:53.126490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.126490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":305,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283271+0000","last_change":"2024-02-22T19:39:22.255233+0000","last_active":"2024-02-22T19:41:08.283271+0000","last_peered":"2024-02-22T19:41:08.283271+0000","last_clean":"2024-02-22T19:41:08.283271+0000","last_became_active":"2024-02-22T19:39:22.254888+0000","last_became_peered":"2024-02-22T19:39:22.254888+0000","last_unstale":"2024-02-22T19:41:08.283271+0000","last_undegraded":"2024-02-22T19:41:08.283271+0000","last_fullsized":"2024-02-22T19:41:08.283271+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.470149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.470149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":96,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291040+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:41:08.291040+0000","last_peered":"2024-02-22T19:41:08.291040+0000","last_clean":"2024-02-22T19:41:08.291040+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:41:08.291040+0000","last_undegraded":"2024-02-22T19:41:08.291040+0000","last_fullsized":"2024-02-22T19:41:08.291040+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":281,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283445+0000","last_change":"2024-02-22T19:38:47.162050+0000","last_active":"2024-02-22T19:41:08.283445+0000","last_peered":"2024-02-22T19:41:08.283445+0000","last_clean":"2024-02-22T19:41:08.283445+0000","last_became_active":"2024-02-22T19:38:47.161672+0000","last_became_peered":"2024-02-22T19:38:47.161672+0000","last_unstale":"2024-02-22T19:41:08.283445+0000","last_undegraded":"2024-02-22T19:41:08.283445+0000","last_fullsized":"2024-02-22T19:41:08.283445+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":328,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.281339+0000","last_change":"2024-02-22T19:39:54.559858+0000","last_active":"2024-02-22T19:41:08.281339+0000","last_peered":"2024-02-22T19:41:08.281339+0000","last_clean":"2024-02-22T19:41:08.281339+0000","last_became_active":"2024-02-22T19:39:22.253387+0000","last_became_peered":"2024-02-22T19:39:22.253387+0000","last_unstale":"2024-02-22T19:41:08.281339+0000","last_undegraded":"2024-02-22T19:41:08.281339+0000","last_fullsized":"2024-02-22T19:41:08.281339+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":310,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.295042+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:41:08.295042+0000","last_peered":"2024-02-22T19:41:08.295042+0000","last_clean":"2024-02-22T19:41:08.295042+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:41:08.295042+0000","last_undegraded":"2024-02-22T19:41:08.295042+0000","last_fullsized":"2024-02-22T19:41:08.295042+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"308'6229","reported_seq":6260,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.359123+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:41:14.359123+0000","last_peered":"2024-02-22T19:41:14.359123+0000","last_clean":"2024-02-22T19:41:14.359123+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:41:14.359123+0000","last_undegraded":"2024-02-22T19:41:14.359123+0000","last_fullsized":"2024-02-22T19:41:14.359123+0000","mapping_epoch":285,"log_start":"294'2400","ondisk_log_start":"294'2400","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3829,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408223744,"num_objects":6229,"num_object_clones":0,"num_object_copies":18687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":398656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":132,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278373+0000","last_change":"2024-02-22T19:40:49.502459+0000","last_active":"2024-02-22T19:41:08.278373+0000","last_peered":"2024-02-22T19:41:08.278373+0000","last_clean":"2024-02-22T19:41:08.278373+0000","last_became_active":"2024-02-22T19:39:22.272214+0000","last_became_peered":"2024-02-22T19:39:22.272214+0000","last_unstale":"2024-02-22T19:41:08.278373+0000","last_undegraded":"2024-02-22T19:41:08.278373+0000","last_fullsized":"2024-02-22T19:41:08.278373+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":353,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.284156+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:41:08.284156+0000","last_peered":"2024-02-22T19:41:08.284156+0000","last_clean":"2024-02-22T19:41:08.284156+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:41:08.284156+0000","last_undegraded":"2024-02-22T19:41:08.284156+0000","last_fullsized":"2024-02-22T19:41:08.284156+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":139,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278047+0000","last_change":"2024-02-22T19:40:46.420455+0000","last_active":"2024-02-22T19:41:08.278047+0000","last_peered":"2024-02-22T19:41:08.278047+0000","last_clean":"2024-02-22T19:41:08.278047+0000","last_became_active":"2024-02-22T19:37:57.181199+0000","last_became_peered":"2024-02-22T19:37:57.181199+0000","last_unstale":"2024-02-22T19:41:08.278047+0000","last_undegraded":"2024-02-22T19:41:08.278047+0000","last_fullsized":"2024-02-22T19:41:08.278047+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"251'13","reported_seq":470,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.399789+0000","last_change":"2024-02-22T19:38:50.149948+0000","last_active":"2024-02-22T19:41:09.399789+0000","last_peered":"2024-02-22T19:41:09.399789+0000","last_clean":"2024-02-22T19:41:09.399789+0000","last_became_active":"2024-02-22T19:38:50.148285+0000","last_became_peered":"2024-02-22T19:38:50.148285+0000","last_unstale":"2024-02-22T19:41:09.399789+0000","last_undegraded":"2024-02-22T19:41:09.399789+0000","last_fullsized":"2024-02-22T19:41:09.399789+0000","mapping_epoch":250,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:36:53.503128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.503128+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"308'6259","reported_seq":6290,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.452682+0000","last_change":"2024-02-22T19:40:00.882365+0000","last_active":"2024-02-22T19:41:14.452682+0000","last_peered":"2024-02-22T19:41:14.452682+0000","last_clean":"2024-02-22T19:41:14.452682+0000","last_became_active":"2024-02-22T19:40:00.882035+0000","last_became_peered":"2024-02-22T19:40:00.882035+0000","last_unstale":"2024-02-22T19:41:14.452682+0000","last_undegraded":"2024-02-22T19:41:14.452682+0000","last_fullsized":"2024-02-22T19:41:14.452682+0000","mapping_epoch":285,"log_start":"294'2300","ondisk_log_start":"294'2300","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3959,"ondisk_log_size":3959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":410189824,"num_objects":6259,"num_object_clones":0,"num_object_copies":18777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6259,"num_write_kb":400576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":266,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291705+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:41:08.291705+0000","last_peered":"2024-02-22T19:41:08.291705+0000","last_clean":"2024-02-22T19:41:08.291705+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:41:08.291705+0000","last_undegraded":"2024-02-22T19:41:08.291705+0000","last_fullsized":"2024-02-22T19:41:08.291705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":317,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.284138+0000","last_change":"2024-02-22T19:39:22.248740+0000","last_active":"2024-02-22T19:41:08.284138+0000","last_peered":"2024-02-22T19:41:08.284138+0000","last_clean":"2024-02-22T19:41:08.284138+0000","last_became_active":"2024-02-22T19:39:22.248120+0000","last_became_peered":"2024-02-22T19:39:22.248120+0000","last_unstale":"2024-02-22T19:41:08.284138+0000","last_undegraded":"2024-02-22T19:41:08.284138+0000","last_fullsized":"2024-02-22T19:41:08.284138+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.431527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.431527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":133,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288991+0000","last_change":"2024-02-22T19:40:48.455295+0000","last_active":"2024-02-22T19:41:08.288991+0000","last_peered":"2024-02-22T19:41:08.288991+0000","last_clean":"2024-02-22T19:41:08.288991+0000","last_became_active":"2024-02-22T19:38:47.167662+0000","last_became_peered":"2024-02-22T19:38:47.167662+0000","last_unstale":"2024-02-22T19:41:08.288991+0000","last_undegraded":"2024-02-22T19:41:08.288991+0000","last_fullsized":"2024-02-22T19:41:08.288991+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":321,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.289162+0000","last_change":"2024-02-22T19:39:48.227541+0000","last_active":"2024-02-22T19:41:08.289162+0000","last_peered":"2024-02-22T19:41:08.289162+0000","last_clean":"2024-02-22T19:41:08.289162+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:41:08.289162+0000","last_undegraded":"2024-02-22T19:41:08.289162+0000","last_fullsized":"2024-02-22T19:41:08.289162+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.227439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.227439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"308'6163","reported_seq":6194,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.316210+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:41:14.316210+0000","last_peered":"2024-02-22T19:41:14.316210+0000","last_clean":"2024-02-22T19:41:14.316210+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:41:14.316210+0000","last_undegraded":"2024-02-22T19:41:14.316210+0000","last_fullsized":"2024-02-22T19:41:14.316210+0000","mapping_epoch":285,"log_start":"294'2300","ondisk_log_start":"294'2300","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3863,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":403898368,"num_objects":6163,"num_object_clones":0,"num_object_copies":18489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":394432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":267,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.282942+0000","last_change":"2024-02-22T19:36:54.489612+0000","last_active":"2024-02-22T19:41:08.282942+0000","last_peered":"2024-02-22T19:41:08.282942+0000","last_clean":"2024-02-22T19:41:08.282942+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:41:08.282942+0000","last_undegraded":"2024-02-22T19:41:08.282942+0000","last_fullsized":"2024-02-22T19:41:08.282942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:54.489509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:54.489509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":347,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288443+0000","last_change":"2024-02-22T19:39:39.403851+0000","last_active":"2024-02-22T19:41:08.288443+0000","last_peered":"2024-02-22T19:41:08.288443+0000","last_clean":"2024-02-22T19:41:08.288443+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:41:08.288443+0000","last_undegraded":"2024-02-22T19:41:08.288443+0000","last_fullsized":"2024-02-22T19:41:08.288443+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:39.403775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:39:39.403775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":296,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.284015+0000","last_change":"2024-02-22T19:37:57.185054+0000","last_active":"2024-02-22T19:41:08.284015+0000","last_peered":"2024-02-22T19:41:08.284015+0000","last_clean":"2024-02-22T19:41:08.284015+0000","last_became_active":"2024-02-22T19:37:57.184627+0000","last_became_peered":"2024-02-22T19:37:57.184627+0000","last_unstale":"2024-02-22T19:41:08.284015+0000","last_undegraded":"2024-02-22T19:41:08.284015+0000","last_fullsized":"2024-02-22T19:41:08.284015+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:57.438112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:57.438112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":334,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279617+0000","last_change":"2024-02-22T19:40:31.504524+0000","last_active":"2024-02-22T19:41:08.279617+0000","last_peered":"2024-02-22T19:41:08.279617+0000","last_clean":"2024-02-22T19:41:08.279617+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:41:08.279617+0000","last_undegraded":"2024-02-22T19:41:08.279617+0000","last_fullsized":"2024-02-22T19:41:08.279617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.6","version":"308'6226","reported_seq":6257,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.372014+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:41:14.372014+0000","last_peered":"2024-02-22T19:41:14.372014+0000","last_clean":"2024-02-22T19:41:14.372014+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:41:14.372014+0000","last_undegraded":"2024-02-22T19:41:14.372014+0000","last_fullsized":"2024-02-22T19:41:14.372014+0000","mapping_epoch":285,"log_start":"294'2400","ondisk_log_start":"294'2400","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3826,"ondisk_log_size":3826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408027136,"num_objects":6226,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6226,"num_write_kb":398464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":94,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283117+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:41:08.283117+0000","last_peered":"2024-02-22T19:41:08.283117+0000","last_clean":"2024-02-22T19:41:08.283117+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:41:08.283117+0000","last_undegraded":"2024-02-22T19:41:08.283117+0000","last_fullsized":"2024-02-22T19:41:08.283117+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":326,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.287938+0000","last_change":"2024-02-22T19:39:54.127947+0000","last_active":"2024-02-22T19:41:08.287938+0000","last_peered":"2024-02-22T19:41:08.287938+0000","last_clean":"2024-02-22T19:41:08.287938+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:41:08.287938+0000","last_undegraded":"2024-02-22T19:41:08.287938+0000","last_fullsized":"2024-02-22T19:41:08.287938+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.127876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.127876+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":100,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278923+0000","last_change":"2024-02-22T19:37:57.182314+0000","last_active":"2024-02-22T19:41:08.278923+0000","last_peered":"2024-02-22T19:41:08.278923+0000","last_clean":"2024-02-22T19:41:08.278923+0000","last_became_active":"2024-02-22T19:37:57.181963+0000","last_became_peered":"2024-02-22T19:37:57.181963+0000","last_unstale":"2024-02-22T19:41:08.278923+0000","last_undegraded":"2024-02-22T19:41:08.278923+0000","last_fullsized":"2024-02-22T19:41:08.278923+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.297621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.297621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":106,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288649+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:41:08.288649+0000","last_peered":"2024-02-22T19:41:08.288649+0000","last_clean":"2024-02-22T19:41:08.288649+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:41:08.288649+0000","last_undegraded":"2024-02-22T19:41:08.288649+0000","last_fullsized":"2024-02-22T19:41:08.288649+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"308'6267","reported_seq":6297,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.337612+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:41:14.337612+0000","last_peered":"2024-02-22T19:41:14.337612+0000","last_clean":"2024-02-22T19:41:14.337612+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:41:14.337612+0000","last_undegraded":"2024-02-22T19:41:14.337612+0000","last_fullsized":"2024-02-22T19:41:14.337612+0000","mapping_epoch":285,"log_start":"294'2300","ondisk_log_start":"294'2300","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3967,"ondisk_log_size":3967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":410714112,"num_objects":6267,"num_object_clones":0,"num_object_copies":18801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6267,"num_write_kb":401088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":141,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278229+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:41:08.278229+0000","last_peered":"2024-02-22T19:41:08.278229+0000","last_clean":"2024-02-22T19:41:08.278229+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:41:08.278229+0000","last_undegraded":"2024-02-22T19:41:08.278229+0000","last_fullsized":"2024-02-22T19:41:08.278229+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":275,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291131+0000","last_change":"2024-02-22T19:38:47.171603+0000","last_active":"2024-02-22T19:41:08.291131+0000","last_peered":"2024-02-22T19:41:08.291131+0000","last_clean":"2024-02-22T19:41:08.291131+0000","last_became_active":"2024-02-22T19:38:47.171002+0000","last_became_peered":"2024-02-22T19:38:47.171002+0000","last_unstale":"2024-02-22T19:41:08.291131+0000","last_undegraded":"2024-02-22T19:41:08.291131+0000","last_fullsized":"2024-02-22T19:41:08.291131+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:45.317440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:45.317440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":100,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278341+0000","last_change":"2024-02-22T19:37:57.185637+0000","last_active":"2024-02-22T19:41:08.278341+0000","last_peered":"2024-02-22T19:41:08.278341+0000","last_clean":"2024-02-22T19:41:08.278341+0000","last_became_active":"2024-02-22T19:37:57.185312+0000","last_became_peered":"2024-02-22T19:37:57.185312+0000","last_unstale":"2024-02-22T19:41:08.278341+0000","last_undegraded":"2024-02-22T19:41:08.278341+0000","last_fullsized":"2024-02-22T19:41:08.278341+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":280,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.281465+0000","last_change":"2024-02-22T19:39:51.569245+0000","last_active":"2024-02-22T19:41:08.281465+0000","last_peered":"2024-02-22T19:41:08.281465+0000","last_clean":"2024-02-22T19:41:08.281465+0000","last_became_active":"2024-02-22T19:39:22.251742+0000","last_became_peered":"2024-02-22T19:39:22.251742+0000","last_unstale":"2024-02-22T19:41:08.281465+0000","last_undegraded":"2024-02-22T19:41:08.281465+0000","last_fullsized":"2024-02-22T19:41:08.281465+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":499,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.391870+0000","last_change":"2024-02-22T19:39:37.302965+0000","last_active":"2024-02-22T19:41:09.391870+0000","last_peered":"2024-02-22T19:41:09.391870+0000","last_clean":"2024-02-22T19:41:09.391870+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:41:09.391870+0000","last_undegraded":"2024-02-22T19:41:09.391870+0000","last_fullsized":"2024-02-22T19:41:09.391870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:39:37.302890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.302890+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":103,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.294782+0000","last_change":"2024-02-22T19:39:46.299764+0000","last_active":"2024-02-22T19:41:08.294782+0000","last_peered":"2024-02-22T19:41:08.294782+0000","last_clean":"2024-02-22T19:41:08.294782+0000","last_became_active":"2024-02-22T19:39:22.251614+0000","last_became_peered":"2024-02-22T19:39:22.251614+0000","last_unstale":"2024-02-22T19:41:08.294782+0000","last_undegraded":"2024-02-22T19:41:08.294782+0000","last_fullsized":"2024-02-22T19:41:08.294782+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"308'6153","reported_seq":6184,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.339105+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:41:14.339105+0000","last_peered":"2024-02-22T19:41:14.339105+0000","last_clean":"2024-02-22T19:41:14.339105+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:41:14.339105+0000","last_undegraded":"2024-02-22T19:41:14.339105+0000","last_fullsized":"2024-02-22T19:41:14.339105+0000","mapping_epoch":285,"log_start":"294'2300","ondisk_log_start":"294'2300","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3853,"ondisk_log_size":3853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":403243008,"num_objects":6153,"num_object_clones":0,"num_object_copies":18459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6153,"num_write_kb":393792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":331,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283344+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:41:08.283344+0000","last_peered":"2024-02-22T19:41:08.283344+0000","last_clean":"2024-02-22T19:41:08.283344+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:41:08.283344+0000","last_undegraded":"2024-02-22T19:41:08.283344+0000","last_fullsized":"2024-02-22T19:41:08.283344+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":272,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283974+0000","last_change":"2024-02-22T19:39:47.645026+0000","last_active":"2024-02-22T19:41:08.283974+0000","last_peered":"2024-02-22T19:41:08.283974+0000","last_clean":"2024-02-22T19:41:08.283974+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:41:08.283974+0000","last_undegraded":"2024-02-22T19:41:08.283974+0000","last_fullsized":"2024-02-22T19:41:08.283974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":280,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291403+0000","last_change":"2024-02-22T19:39:45.253946+0000","last_active":"2024-02-22T19:41:08.291403+0000","last_peered":"2024-02-22T19:41:08.291403+0000","last_clean":"2024-02-22T19:41:08.291403+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:41:08.291403+0000","last_undegraded":"2024-02-22T19:41:08.291403+0000","last_fullsized":"2024-02-22T19:41:08.291403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.253853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.253853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":93,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.287029+0000","last_change":"2024-02-22T19:39:52.175249+0000","last_active":"2024-02-22T19:41:08.287029+0000","last_peered":"2024-02-22T19:41:08.287029+0000","last_clean":"2024-02-22T19:41:08.287029+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:41:08.287029+0000","last_undegraded":"2024-02-22T19:41:08.287029+0000","last_fullsized":"2024-02-22T19:41:08.287029+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:39:52.175144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":377,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288542+0000","last_change":"2024-02-22T19:39:34.410627+0000","last_active":"2024-02-22T19:41:08.288542+0000","last_peered":"2024-02-22T19:41:08.288542+0000","last_clean":"2024-02-22T19:41:08.288542+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:41:08.288542+0000","last_undegraded":"2024-02-22T19:41:08.288542+0000","last_fullsized":"2024-02-22T19:41:08.288542+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:39:34.410547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.410547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":320,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288015+0000","last_change":"2024-02-22T19:39:51.129196+0000","last_active":"2024-02-22T19:41:08.288015+0000","last_peered":"2024-02-22T19:41:08.288015+0000","last_clean":"2024-02-22T19:41:08.288015+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:41:08.288015+0000","last_undegraded":"2024-02-22T19:41:08.288015+0000","last_fullsized":"2024-02-22T19:41:08.288015+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.129010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.129010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"308'6235","reported_seq":6266,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.641239+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:41:14.641239+0000","last_peered":"2024-02-22T19:41:14.641239+0000","last_clean":"2024-02-22T19:41:14.641239+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:41:14.641239+0000","last_undegraded":"2024-02-22T19:41:14.641239+0000","last_fullsized":"2024-02-22T19:41:14.641239+0000","mapping_epoch":285,"log_start":"295'2500","ondisk_log_start":"295'2500","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3735,"ondisk_log_size":3735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408616960,"num_objects":6235,"num_object_clones":0,"num_object_copies":18705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6235,"num_write_kb":399040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":114,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.280627+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:41:08.280627+0000","last_peered":"2024-02-22T19:41:08.280627+0000","last_clean":"2024-02-22T19:41:08.280627+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:41:08.280627+0000","last_undegraded":"2024-02-22T19:41:08.280627+0000","last_fullsized":"2024-02-22T19:41:08.280627+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":322,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283204+0000","last_change":"2024-02-22T19:40:47.591722+0000","last_active":"2024-02-22T19:41:08.283204+0000","last_peered":"2024-02-22T19:41:08.283204+0000","last_clean":"2024-02-22T19:41:08.283204+0000","last_became_active":"2024-02-22T19:39:22.252166+0000","last_became_peered":"2024-02-22T19:39:22.252166+0000","last_unstale":"2024-02-22T19:41:08.283204+0000","last_undegraded":"2024-02-22T19:41:08.283204+0000","last_fullsized":"2024-02-22T19:41:08.283204+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":290,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.292175+0000","last_change":"2024-02-22T19:39:47.705812+0000","last_active":"2024-02-22T19:41:08.292175+0000","last_peered":"2024-02-22T19:41:08.292175+0000","last_clean":"2024-02-22T19:41:08.292175+0000","last_became_active":"2024-02-22T19:37:57.180638+0000","last_became_peered":"2024-02-22T19:37:57.180638+0000","last_unstale":"2024-02-22T19:41:08.292175+0000","last_undegraded":"2024-02-22T19:41:08.292175+0000","last_fullsized":"2024-02-22T19:41:08.292175+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":396,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.282174+0000","last_change":"2024-02-22T19:39:22.439994+0000","last_active":"2024-02-22T19:41:08.282174+0000","last_peered":"2024-02-22T19:41:08.282174+0000","last_clean":"2024-02-22T19:41:08.282174+0000","last_became_active":"2024-02-22T19:39:22.256569+0000","last_became_peered":"2024-02-22T19:39:22.256569+0000","last_unstale":"2024-02-22T19:41:08.282174+0000","last_undegraded":"2024-02-22T19:41:08.282174+0000","last_fullsized":"2024-02-22T19:41:08.282174+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'21","last_scrub_stamp":"2024-02-22T19:37:47.421622+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:47.421622+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":292,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283646+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:41:08.283646+0000","last_peered":"2024-02-22T19:41:08.283646+0000","last_clean":"2024-02-22T19:41:08.283646+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:41:08.283646+0000","last_undegraded":"2024-02-22T19:41:08.283646+0000","last_fullsized":"2024-02-22T19:41:08.283646+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"308'6218","reported_seq":6249,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.300494+0000","last_change":"2024-02-22T19:40:01.098193+0000","last_active":"2024-02-22T19:41:14.300494+0000","last_peered":"2024-02-22T19:41:14.300494+0000","last_clean":"2024-02-22T19:41:14.300494+0000","last_became_active":"2024-02-22T19:40:01.097792+0000","last_became_peered":"2024-02-22T19:40:01.097792+0000","last_unstale":"2024-02-22T19:41:14.300494+0000","last_undegraded":"2024-02-22T19:41:14.300494+0000","last_fullsized":"2024-02-22T19:41:14.300494+0000","mapping_epoch":285,"log_start":"292'1800","ondisk_log_start":"292'1800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4418,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":407502848,"num_objects":6218,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6218,"num_write_kb":397952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":176,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.293484+0000","last_change":"2024-02-22T19:40:41.405503+0000","last_active":"2024-02-22T19:41:08.293484+0000","last_peered":"2024-02-22T19:41:08.293484+0000","last_clean":"2024-02-22T19:41:08.293484+0000","last_became_active":"2024-02-22T19:38:47.167994+0000","last_became_peered":"2024-02-22T19:38:47.167994+0000","last_unstale":"2024-02-22T19:41:08.293484+0000","last_undegraded":"2024-02-22T19:41:08.293484+0000","last_fullsized":"2024-02-22T19:41:08.293484+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":292,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290111+0000","last_change":"2024-02-22T19:39:50.386984+0000","last_active":"2024-02-22T19:41:08.290111+0000","last_peered":"2024-02-22T19:41:08.290111+0000","last_clean":"2024-02-22T19:41:08.290111+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:41:08.290111+0000","last_undegraded":"2024-02-22T19:41:08.290111+0000","last_fullsized":"2024-02-22T19:41:08.290111+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.386916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.386916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":82,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290438+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:41:08.290438+0000","last_peered":"2024-02-22T19:41:08.290438+0000","last_clean":"2024-02-22T19:41:08.290438+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:41:08.290438+0000","last_undegraded":"2024-02-22T19:41:08.290438+0000","last_fullsized":"2024-02-22T19:41:08.290438+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1051,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:12.545263+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:41:12.545263+0000","last_peered":"2024-02-22T19:41:12.545263+0000","last_clean":"2024-02-22T19:41:12.545263+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:41:12.545263+0000","last_undegraded":"2024-02-22T19:41:12.545263+0000","last_fullsized":"2024-02-22T19:41:12.545263+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":716,"num_read_kb":711,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":114,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.302213+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:41:08.302213+0000","last_peered":"2024-02-22T19:41:08.302213+0000","last_clean":"2024-02-22T19:41:08.302213+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:41:08.302213+0000","last_undegraded":"2024-02-22T19:41:08.302213+0000","last_fullsized":"2024-02-22T19:41:08.302213+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":159,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.280700+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:41:08.280700+0000","last_peered":"2024-02-22T19:41:08.280700+0000","last_clean":"2024-02-22T19:41:08.280700+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:41:08.280700+0000","last_undegraded":"2024-02-22T19:41:08.280700+0000","last_fullsized":"2024-02-22T19:41:08.280700+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"308'6168","reported_seq":6199,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.333135+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:41:14.333135+0000","last_peered":"2024-02-22T19:41:14.333135+0000","last_clean":"2024-02-22T19:41:14.333135+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:41:14.333135+0000","last_undegraded":"2024-02-22T19:41:14.333135+0000","last_fullsized":"2024-02-22T19:41:14.333135+0000","mapping_epoch":285,"log_start":"294'2200","ondisk_log_start":"294'2200","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3968,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":404226048,"num_objects":6168,"num_object_clones":0,"num_object_copies":18504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6168,"num_write_kb":394752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":217,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.281689+0000","last_change":"2024-02-22T19:39:38.562584+0000","last_active":"2024-02-22T19:41:08.281689+0000","last_peered":"2024-02-22T19:41:08.281689+0000","last_clean":"2024-02-22T19:41:08.281689+0000","last_became_active":"2024-02-22T19:39:22.256349+0000","last_became_peered":"2024-02-22T19:39:22.256349+0000","last_unstale":"2024-02-22T19:41:08.281689+0000","last_undegraded":"2024-02-22T19:41:08.281689+0000","last_fullsized":"2024-02-22T19:41:08.281689+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:38.562500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:39:38.562500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":274,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.289073+0000","last_change":"2024-02-22T19:40:40.450558+0000","last_active":"2024-02-22T19:41:08.289073+0000","last_peered":"2024-02-22T19:41:08.289073+0000","last_clean":"2024-02-22T19:41:08.289073+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:41:08.289073+0000","last_undegraded":"2024-02-22T19:41:08.289073+0000","last_fullsized":"2024-02-22T19:41:08.289073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":298,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290288+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:41:08.290288+0000","last_peered":"2024-02-22T19:41:08.290288+0000","last_clean":"2024-02-22T19:41:08.290288+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:41:08.290288+0000","last_undegraded":"2024-02-22T19:41:08.290288+0000","last_fullsized":"2024-02-22T19:41:08.290288+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":119,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278546+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:41:08.278546+0000","last_peered":"2024-02-22T19:41:08.278546+0000","last_clean":"2024-02-22T19:41:08.278546+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:41:08.278546+0000","last_undegraded":"2024-02-22T19:41:08.278546+0000","last_fullsized":"2024-02-22T19:41:08.278546+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":324,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279659+0000","last_change":"2024-02-22T19:39:49.629194+0000","last_active":"2024-02-22T19:41:08.279659+0000","last_peered":"2024-02-22T19:41:08.279659+0000","last_clean":"2024-02-22T19:41:08.279659+0000","last_became_active":"2024-02-22T19:39:22.263999+0000","last_became_peered":"2024-02-22T19:39:22.263999+0000","last_unstale":"2024-02-22T19:41:08.279659+0000","last_undegraded":"2024-02-22T19:41:08.279659+0000","last_fullsized":"2024-02-22T19:41:08.279659+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"308'6253","reported_seq":6284,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.342348+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:41:14.342348+0000","last_peered":"2024-02-22T19:41:14.342348+0000","last_clean":"2024-02-22T19:41:14.342348+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:41:14.342348+0000","last_undegraded":"2024-02-22T19:41:14.342348+0000","last_fullsized":"2024-02-22T19:41:14.342348+0000","mapping_epoch":285,"log_start":"294'2400","ondisk_log_start":"294'2400","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3853,"ondisk_log_size":3853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":409796608,"num_objects":6253,"num_object_clones":0,"num_object_copies":18759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6253,"num_write_kb":400192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":177,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.282030+0000","last_change":"2024-02-22T19:39:42.557957+0000","last_active":"2024-02-22T19:41:08.282030+0000","last_peered":"2024-02-22T19:41:08.282030+0000","last_clean":"2024-02-22T19:41:08.282030+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:41:08.282030+0000","last_undegraded":"2024-02-22T19:41:08.282030+0000","last_fullsized":"2024-02-22T19:41:08.282030+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:42.557792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.557792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":94,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291677+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:41:08.291677+0000","last_peered":"2024-02-22T19:41:08.291677+0000","last_clean":"2024-02-22T19:41:08.291677+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:41:08.291677+0000","last_undegraded":"2024-02-22T19:41:08.291677+0000","last_fullsized":"2024-02-22T19:41:08.291677+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":100,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279251+0000","last_change":"2024-02-22T19:37:57.190664+0000","last_active":"2024-02-22T19:41:08.279251+0000","last_peered":"2024-02-22T19:41:08.279251+0000","last_clean":"2024-02-22T19:41:08.279251+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:41:08.279251+0000","last_undegraded":"2024-02-22T19:41:08.279251+0000","last_fullsized":"2024-02-22T19:41:08.279251+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:02.261537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:02.261537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":413,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.289365+0000","last_change":"2024-02-22T19:39:40.321965+0000","last_active":"2024-02-22T19:41:08.289365+0000","last_peered":"2024-02-22T19:41:08.289365+0000","last_clean":"2024-02-22T19:41:08.289365+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:41:08.289365+0000","last_undegraded":"2024-02-22T19:41:08.289365+0000","last_fullsized":"2024-02-22T19:41:08.289365+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:40.321851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:40.321851+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"268'13","reported_seq":155,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.413705+0000","last_change":"2024-02-22T19:39:41.566614+0000","last_active":"2024-02-22T19:41:09.413705+0000","last_peered":"2024-02-22T19:41:09.413705+0000","last_clean":"2024-02-22T19:41:09.413705+0000","last_became_active":"2024-02-22T19:39:22.252945+0000","last_became_peered":"2024-02-22T19:39:22.252945+0000","last_unstale":"2024-02-22T19:41:09.413705+0000","last_undegraded":"2024-02-22T19:41:09.413705+0000","last_fullsized":"2024-02-22T19:41:09.413705+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:39:41.566527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.566527+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"308'6218","reported_seq":6245,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.607405+0000","last_change":"2024-02-22T19:40:00.882269+0000","last_active":"2024-02-22T19:41:14.607405+0000","last_peered":"2024-02-22T19:41:14.607405+0000","last_clean":"2024-02-22T19:41:14.607405+0000","last_became_active":"2024-02-22T19:40:00.881943+0000","last_became_peered":"2024-02-22T19:40:00.881943+0000","last_unstale":"2024-02-22T19:41:14.607405+0000","last_undegraded":"2024-02-22T19:41:14.607405+0000","last_fullsized":"2024-02-22T19:41:14.607405+0000","mapping_epoch":285,"log_start":"294'2500","ondisk_log_start":"294'2500","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3718,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":407502848,"num_objects":6218,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6218,"num_write_kb":397952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":176,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290705+0000","last_change":"2024-02-22T19:40:02.507342+0000","last_active":"2024-02-22T19:41:08.290705+0000","last_peered":"2024-02-22T19:41:08.290705+0000","last_clean":"2024-02-22T19:41:08.290705+0000","last_became_active":"2024-02-22T19:40:02.507127+0000","last_became_peered":"2024-02-22T19:40:02.507127+0000","last_unstale":"2024-02-22T19:41:08.290705+0000","last_undegraded":"2024-02-22T19:41:08.290705+0000","last_fullsized":"2024-02-22T19:41:08.290705+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":275,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283872+0000","last_change":"2024-02-22T19:40:31.663872+0000","last_active":"2024-02-22T19:41:08.283872+0000","last_peered":"2024-02-22T19:41:08.283872+0000","last_clean":"2024-02-22T19:41:08.283872+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:41:08.283872+0000","last_undegraded":"2024-02-22T19:41:08.283872+0000","last_fullsized":"2024-02-22T19:41:08.283872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":103,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.302028+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:41:08.302028+0000","last_peered":"2024-02-22T19:41:08.302028+0000","last_clean":"2024-02-22T19:41:08.302028+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:41:08.302028+0000","last_undegraded":"2024-02-22T19:41:08.302028+0000","last_fullsized":"2024-02-22T19:41:08.302028+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":395,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278472+0000","last_change":"2024-02-22T19:39:46.430095+0000","last_active":"2024-02-22T19:41:08.278472+0000","last_peered":"2024-02-22T19:41:08.278472+0000","last_clean":"2024-02-22T19:41:08.278472+0000","last_became_active":"2024-02-22T19:39:22.248416+0000","last_became_peered":"2024-02-22T19:39:22.248416+0000","last_unstale":"2024-02-22T19:41:08.278472+0000","last_undegraded":"2024-02-22T19:41:08.278472+0000","last_fullsized":"2024-02-22T19:41:08.278472+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":464,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.397651+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:41:09.397651+0000","last_peered":"2024-02-22T19:41:09.397651+0000","last_clean":"2024-02-22T19:41:09.397651+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:41:09.397651+0000","last_undegraded":"2024-02-22T19:41:09.397651+0000","last_fullsized":"2024-02-22T19:41:09.397651+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"308'6111","reported_seq":6142,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.307497+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:41:14.307497+0000","last_peered":"2024-02-22T19:41:14.307497+0000","last_clean":"2024-02-22T19:41:14.307497+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:41:14.307497+0000","last_undegraded":"2024-02-22T19:41:14.307497+0000","last_fullsized":"2024-02-22T19:41:14.307497+0000","mapping_epoch":285,"log_start":"294'2100","ondisk_log_start":"294'2100","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4011,"ondisk_log_size":4011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":400490496,"num_objects":6111,"num_object_clones":0,"num_object_copies":18333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6111,"num_write_kb":391104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.286170+0000","last_change":"2024-02-22T19:40:33.048538+0000","last_active":"2024-02-22T19:41:08.286170+0000","last_peered":"2024-02-22T19:41:08.286170+0000","last_clean":"2024-02-22T19:41:08.286170+0000","last_became_active":"2024-02-22T19:38:47.167514+0000","last_became_peered":"2024-02-22T19:38:47.167514+0000","last_unstale":"2024-02-22T19:41:08.286170+0000","last_undegraded":"2024-02-22T19:41:08.286170+0000","last_fullsized":"2024-02-22T19:41:08.286170+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:33.048434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.048434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":72,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.292470+0000","last_change":"2024-02-22T19:38:47.170822+0000","last_active":"2024-02-22T19:41:08.292470+0000","last_peered":"2024-02-22T19:41:08.292470+0000","last_clean":"2024-02-22T19:41:08.292470+0000","last_became_active":"2024-02-22T19:38:47.170472+0000","last_became_peered":"2024-02-22T19:38:47.170472+0000","last_unstale":"2024-02-22T19:41:08.292470+0000","last_undegraded":"2024-02-22T19:41:08.292470+0000","last_fullsized":"2024-02-22T19:41:08.292470+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:20.739750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:20.739750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":299,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.292599+0000","last_change":"2024-02-22T19:39:22.243142+0000","last_active":"2024-02-22T19:41:08.292599+0000","last_peered":"2024-02-22T19:41:08.292599+0000","last_clean":"2024-02-22T19:41:08.292599+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:41:08.292599+0000","last_undegraded":"2024-02-22T19:41:08.292599+0000","last_fullsized":"2024-02-22T19:41:08.292599+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:37:06.317125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":408,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291904+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:41:08.291904+0000","last_peered":"2024-02-22T19:41:08.291904+0000","last_clean":"2024-02-22T19:41:08.291904+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:41:08.291904+0000","last_undegraded":"2024-02-22T19:41:08.291904+0000","last_fullsized":"2024-02-22T19:41:08.291904+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":95,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.293397+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:41:08.293397+0000","last_peered":"2024-02-22T19:41:08.293397+0000","last_clean":"2024-02-22T19:41:08.293397+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:41:08.293397+0000","last_undegraded":"2024-02-22T19:41:08.293397+0000","last_fullsized":"2024-02-22T19:41:08.293397+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"308'6154","reported_seq":6185,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.628615+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:41:14.628615+0000","last_peered":"2024-02-22T19:41:14.628615+0000","last_clean":"2024-02-22T19:41:14.628615+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:41:14.628615+0000","last_undegraded":"2024-02-22T19:41:14.628615+0000","last_fullsized":"2024-02-22T19:41:14.628615+0000","mapping_epoch":285,"log_start":"294'2400","ondisk_log_start":"294'2400","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3754,"ondisk_log_size":3754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":403308544,"num_objects":6154,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6154,"num_write_kb":393856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":108,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278454+0000","last_change":"2024-02-22T19:40:36.433351+0000","last_active":"2024-02-22T19:41:08.278454+0000","last_peered":"2024-02-22T19:41:08.278454+0000","last_clean":"2024-02-22T19:41:08.278454+0000","last_became_active":"2024-02-22T19:37:57.189767+0000","last_became_peered":"2024-02-22T19:37:57.189767+0000","last_unstale":"2024-02-22T19:41:08.278454+0000","last_undegraded":"2024-02-22T19:41:08.278454+0000","last_fullsized":"2024-02-22T19:41:08.278454+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":127,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279129+0000","last_change":"2024-02-22T19:39:22.269372+0000","last_active":"2024-02-22T19:41:08.279129+0000","last_peered":"2024-02-22T19:41:08.279129+0000","last_clean":"2024-02-22T19:41:08.279129+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:41:08.279129+0000","last_undegraded":"2024-02-22T19:41:08.279129+0000","last_fullsized":"2024-02-22T19:41:08.279129+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.979469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.979469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":579,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.406572+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:41:09.406572+0000","last_peered":"2024-02-22T19:41:09.406572+0000","last_clean":"2024-02-22T19:41:09.406572+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:41:09.406572+0000","last_undegraded":"2024-02-22T19:41:09.406572+0000","last_fullsized":"2024-02-22T19:41:09.406572+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":277,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291858+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:41:08.291858+0000","last_peered":"2024-02-22T19:41:08.291858+0000","last_clean":"2024-02-22T19:41:08.291858+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:41:08.291858+0000","last_undegraded":"2024-02-22T19:41:08.291858+0000","last_fullsized":"2024-02-22T19:41:08.291858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":218,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.297856+0000","last_change":"2024-02-22T19:40:53.384009+0000","last_active":"2024-02-22T19:41:08.297856+0000","last_peered":"2024-02-22T19:41:08.297856+0000","last_clean":"2024-02-22T19:41:08.297856+0000","last_became_active":"2024-02-22T19:38:47.093284+0000","last_became_peered":"2024-02-22T19:38:47.093284+0000","last_unstale":"2024-02-22T19:41:08.297856+0000","last_undegraded":"2024-02-22T19:41:08.297856+0000","last_fullsized":"2024-02-22T19:41:08.297856+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"308'6231","reported_seq":6262,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.340657+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:41:14.340657+0000","last_peered":"2024-02-22T19:41:14.340657+0000","last_clean":"2024-02-22T19:41:14.340657+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:41:14.340657+0000","last_undegraded":"2024-02-22T19:41:14.340657+0000","last_fullsized":"2024-02-22T19:41:14.340657+0000","mapping_epoch":285,"log_start":"294'2200","ondisk_log_start":"294'2200","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4031,"ondisk_log_size":4031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408354816,"num_objects":6231,"num_object_clones":0,"num_object_copies":18693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6231,"num_write_kb":398784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":284,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290842+0000","last_change":"2024-02-22T19:40:32.668512+0000","last_active":"2024-02-22T19:41:08.290842+0000","last_peered":"2024-02-22T19:41:08.290842+0000","last_clean":"2024-02-22T19:41:08.290842+0000","last_became_active":"2024-02-22T19:37:57.179488+0000","last_became_peered":"2024-02-22T19:37:57.179488+0000","last_unstale":"2024-02-22T19:41:08.290842+0000","last_undegraded":"2024-02-22T19:41:08.290842+0000","last_fullsized":"2024-02-22T19:41:08.290842+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"251'17","reported_seq":293,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.405549+0000","last_change":"2024-02-22T19:38:50.146296+0000","last_active":"2024-02-22T19:41:09.405549+0000","last_peered":"2024-02-22T19:41:09.405549+0000","last_clean":"2024-02-22T19:41:09.405549+0000","last_became_active":"2024-02-22T19:38:50.145632+0000","last_became_peered":"2024-02-22T19:38:50.145632+0000","last_unstale":"2024-02-22T19:41:09.405549+0000","last_undegraded":"2024-02-22T19:41:09.405549+0000","last_fullsized":"2024-02-22T19:41:09.405549+0000","mapping_epoch":250,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-02-22T19:36:53.427951+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:36:53.427951+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":82,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.286312+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:41:08.286312+0000","last_peered":"2024-02-22T19:41:08.286312+0000","last_clean":"2024-02-22T19:41:08.286312+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:41:08.286312+0000","last_undegraded":"2024-02-22T19:41:08.286312+0000","last_fullsized":"2024-02-22T19:41:08.286312+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":131,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.282428+0000","last_change":"2024-02-22T19:39:43.589553+0000","last_active":"2024-02-22T19:41:08.282428+0000","last_peered":"2024-02-22T19:41:08.282428+0000","last_clean":"2024-02-22T19:41:08.282428+0000","last_became_active":"2024-02-22T19:38:47.183168+0000","last_became_peered":"2024-02-22T19:38:47.183168+0000","last_unstale":"2024-02-22T19:41:08.282428+0000","last_undegraded":"2024-02-22T19:41:08.282428+0000","last_fullsized":"2024-02-22T19:41:08.282428+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"308'6180","reported_seq":6211,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.436671+0000","last_change":"2024-02-22T19:40:01.097839+0000","last_active":"2024-02-22T19:41:14.436671+0000","last_peered":"2024-02-22T19:41:14.436671+0000","last_clean":"2024-02-22T19:41:14.436671+0000","last_became_active":"2024-02-22T19:40:01.097523+0000","last_became_peered":"2024-02-22T19:40:01.097523+0000","last_unstale":"2024-02-22T19:41:14.436671+0000","last_undegraded":"2024-02-22T19:41:14.436671+0000","last_fullsized":"2024-02-22T19:41:14.436671+0000","mapping_epoch":285,"log_start":"294'2200","ondisk_log_start":"294'2200","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3980,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":405012480,"num_objects":6180,"num_object_clones":0,"num_object_copies":18540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6180,"num_write_kb":395520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":105,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.278087+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:41:08.278087+0000","last_peered":"2024-02-22T19:41:08.278087+0000","last_clean":"2024-02-22T19:41:08.278087+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:41:08.278087+0000","last_undegraded":"2024-02-22T19:41:08.278087+0000","last_fullsized":"2024-02-22T19:41:08.278087+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":288,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.288373+0000","last_change":"2024-02-22T19:40:34.092106+0000","last_active":"2024-02-22T19:41:08.288373+0000","last_peered":"2024-02-22T19:41:08.288373+0000","last_clean":"2024-02-22T19:41:08.288373+0000","last_became_active":"2024-02-22T19:38:47.171941+0000","last_became_peered":"2024-02-22T19:38:47.171941+0000","last_unstale":"2024-02-22T19:41:08.288373+0000","last_undegraded":"2024-02-22T19:41:08.288373+0000","last_fullsized":"2024-02-22T19:41:08.288373+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":330,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.290484+0000","last_change":"2024-02-22T19:39:49.403011+0000","last_active":"2024-02-22T19:41:08.290484+0000","last_peered":"2024-02-22T19:41:08.290484+0000","last_clean":"2024-02-22T19:41:08.290484+0000","last_became_active":"2024-02-22T19:39:22.244710+0000","last_became_peered":"2024-02-22T19:39:22.244710+0000","last_unstale":"2024-02-22T19:41:08.290484+0000","last_undegraded":"2024-02-22T19:41:08.290484+0000","last_fullsized":"2024-02-22T19:41:08.290484+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":290,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.279539+0000","last_change":"2024-02-22T19:39:45.615429+0000","last_active":"2024-02-22T19:41:08.279539+0000","last_peered":"2024-02-22T19:41:08.279539+0000","last_clean":"2024-02-22T19:41:08.279539+0000","last_became_active":"2024-02-22T19:38:47.164172+0000","last_became_peered":"2024-02-22T19:38:47.164172+0000","last_unstale":"2024-02-22T19:41:08.279539+0000","last_undegraded":"2024-02-22T19:41:08.279539+0000","last_fullsized":"2024-02-22T19:41:08.279539+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":156,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.287185+0000","last_change":"2024-02-22T19:40:35.038302+0000","last_active":"2024-02-22T19:41:08.287185+0000","last_peered":"2024-02-22T19:41:08.287185+0000","last_clean":"2024-02-22T19:41:08.287185+0000","last_became_active":"2024-02-22T19:38:47.181392+0000","last_became_peered":"2024-02-22T19:38:47.181392+0000","last_unstale":"2024-02-22T19:41:08.287185+0000","last_undegraded":"2024-02-22T19:41:08.287185+0000","last_fullsized":"2024-02-22T19:41:08.287185+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"308'6051","reported_seq":6082,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:14.606777+0000","last_change":"2024-02-22T19:40:00.881912+0000","last_active":"2024-02-22T19:41:14.606777+0000","last_peered":"2024-02-22T19:41:14.606777+0000","last_clean":"2024-02-22T19:41:14.606777+0000","last_became_active":"2024-02-22T19:40:00.881492+0000","last_became_peered":"2024-02-22T19:40:00.881492+0000","last_unstale":"2024-02-22T19:41:14.606777+0000","last_undegraded":"2024-02-22T19:41:14.606777+0000","last_fullsized":"2024-02-22T19:41:14.606777+0000","mapping_epoch":285,"log_start":"294'2300","ondisk_log_start":"294'2300","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3751,"ondisk_log_size":3751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":396558336,"num_objects":6051,"num_object_clones":0,"num_object_copies":18153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6051,"num_write_kb":387264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":290,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.283082+0000","last_change":"2024-02-22T19:36:27.599388+0000","last_active":"2024-02-22T19:41:08.283082+0000","last_peered":"2024-02-22T19:41:08.283082+0000","last_clean":"2024-02-22T19:41:08.283082+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:41:08.283082+0000","last_undegraded":"2024-02-22T19:41:08.283082+0000","last_fullsized":"2024-02-22T19:41:08.283082+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:27.599295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:36:27.599295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":116,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.301905+0000","last_change":"2024-02-22T19:39:32.393413+0000","last_active":"2024-02-22T19:41:08.301905+0000","last_peered":"2024-02-22T19:41:08.301905+0000","last_clean":"2024-02-22T19:41:08.301905+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:41:08.301905+0000","last_undegraded":"2024-02-22T19:41:08.301905+0000","last_fullsized":"2024-02-22T19:41:08.301905+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:39:32.393320+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.393320+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":498,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-02-22T19:41:09.395395+0000","last_change":"2024-02-22T19:39:55.160226+0000","last_active":"2024-02-22T19:41:09.395395+0000","last_peered":"2024-02-22T19:41:09.395395+0000","last_clean":"2024-02-22T19:41:09.395395+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:41:09.395395+0000","last_undegraded":"2024-02-22T19:41:09.395395+0000","last_fullsized":"2024-02-22T19:41:09.395395+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:39:55.160157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:55.160157+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":333,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291569+0000","last_change":"2024-02-22T19:39:22.400760+0000","last_active":"2024-02-22T19:41:08.291569+0000","last_peered":"2024-02-22T19:41:08.291569+0000","last_clean":"2024-02-22T19:41:08.291569+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:41:08.291569+0000","last_undegraded":"2024-02-22T19:41:08.291569+0000","last_fullsized":"2024-02-22T19:41:08.291569+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:37:05.357880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:05.357880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":131,"reported_epoch":307,"state":"active+clean","last_fresh":"2024-02-22T19:41:08.291063+0000","last_change":"2024-02-22T19:39:53.646743+0000","last_active":"2024-02-22T19:41:08.291063+0000","last_peered":"2024-02-22T19:41:08.291063+0000","last_clean":"2024-02-22T19:41:08.291063+0000","last_became_active":"2024-02-22T19:38:47.169571+0000","last_became_peered":"2024-02-22T19:38:47.169571+0000","last_unstale":"2024-02-22T19:41:08.291063+0000","last_undegraded":"2024-02-22T19:41:08.291063+0000","last_fullsized":"2024-02-22T19:41:08.291063+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":6495666176,"num_objects":99116,"num_object_clones":0,"num_object_copies":297348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99116,"num_write_kb":6343424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19488309248,"data_stored":19488309248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62516,"ondisk_log_size":62516,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":716,"num_read_kb":711,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":6257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1109,"num_read_kb":866,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":7172,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266838,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4226056,"kb_used_data":3924736,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":89519096,"statfs":{"total":95995035648,"available":91667554304,"internally_reserved":0,"allocated":4018929664,"data_stored":4018036700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":1.323,"15min":1.258},"min":{"1min":0.27800000000000002,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":3.4340000000000002,"5min":6.8040000000000003,"15min":42.597999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.653,"15min":1.3080000000000001},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":6.7670000000000003,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.8220000000000001,"15min":1.345},"min":{"1min":0.29199999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.4820000000000002,"5min":54.951000000000001,"15min":54.951000000000001},"last":1.123},{"interface":"front","average":{"1min":0.65400000000000003,"5min":2.7200000000000002,"15min":1.7210000000000001},"min":{"1min":0.37,"5min":0.28000000000000003,"15min":0.219},"max":{"1min":2.0249999999999999,"5min":57.012,"15min":57.012},"last":3.8239999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.829,"15min":1.2589999999999999},"min":{"1min":0.32200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.5129999999999999,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.37},{"interface":"front","average":{"1min":0.66400000000000003,"5min":2.02,"15min":1.4359999999999999},"min":{"1min":0.35499999999999998,"5min":0.25700000000000001,"15min":0.23599999999999999},"max":{"1min":2.4550000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":1.276}]},{"osd":3,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":2.0720000000000001,"15min":1.462},"min":{"1min":0.374,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.823,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":2.145,"15min":1.4410000000000001},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":3.0670000000000002,"5min":38.219000000000001,"15min":41.311999999999998},"last":1.2190000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.80300000000000005,"15min":0.59199999999999997},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":2.1440000000000001,"5min":5.5199999999999996,"15min":5.5199999999999996},"last":0.434},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.55800000000000005},"min":{"1min":0.377,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":3.8650000000000002,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.34599999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80600000000000005,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":2.9510000000000001,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.81999999999999995,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":2.1560000000000001,"5min":5.556,"15min":5.556},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:40:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.93300000000000005,"15min":0.72499999999999998},"min":{"1min":0.371,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":2.3530000000000002,"5min":5.5789999999999997,"15min":12.199999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79600000000000004,"15min":0.65600000000000003},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.8439999999999999,"5min":7.6390000000000002,"15min":12.067},"last":0.32000000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528475,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2271992,"kb_used_data":1966576,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91473160,"statfs":{"total":95995035648,"available":93668515840,"internally_reserved":0,"allocated":2013773824,"data_stored":2012895879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.3160000000000001,"15min":1.0780000000000001},"min":{"1min":0.22600000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":2.9009999999999998,"5min":6.6660000000000004,"15min":17.885000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":1.375,"5min":1.4890000000000001,"15min":1.083},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.21199999999999999},"max":{"1min":5.1360000000000001,"5min":8.1120000000000001,"15min":8.2699999999999996},"last":4.2130000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":2.089,"15min":1.2250000000000001},"min":{"1min":0.26100000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":4.7009999999999996,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.8959999999999999,"15min":1.1559999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":6.1280000000000001,"5min":13.590999999999999,"15min":13.590999999999999},"last":2.2389999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":2.0099999999999998,"15min":1.3300000000000001},"min":{"1min":0.28499999999999998,"5min":0.253,"15min":0.249},"max":{"1min":5.266,"5min":11.198,"15min":12.859999999999999},"last":2.649},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.859,"15min":1.1699999999999999},"min":{"1min":0.315,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.3679999999999999,"5min":11.565,"15min":11.565},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.667,"15min":1.3009999999999999},"min":{"1min":0.33100000000000002,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":3.8719999999999999,"5min":10.135,"15min":38.762},"last":2.282},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6519999999999999,"15min":1.452},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":5.1669999999999998,"5min":10.315,"15min":35.612000000000002},"last":2.6259999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65600000000000003,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.21},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.81200000000000006,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.4769999999999999,"5min":9.3949999999999996,"15min":9.3949999999999996},"last":2.5960000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":0.90000000000000002,"15min":0.60499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.1870000000000003,"5min":7.3780000000000001,"15min":7.3780000000000001},"last":4.0899999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.59799999999999998},"min":{"1min":0.35199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6989999999999998,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.373}]},{"osd":7,"last update":"Thu Feb 22 19:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.74099999999999999,"15min":0.59799999999999998},"min":{"1min":0.376,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.4430000000000001,"5min":3.2709999999999999,"15min":3.488},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88400000000000001,"15min":0.68999999999999995},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":2.5110000000000001,"5min":5.7939999999999996,"15min":5.7939999999999996},"last":2.0920000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575028,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4271212,"kb_used_data":3961700,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":89473940,"statfs":{"total":95995035648,"available":91621314560,"internally_reserved":0,"allocated":4056780800,"data_stored":4055851728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.8089999999999999,"15min":1.411},"min":{"1min":0.252,"5min":0.185,"15min":0.185},"max":{"1min":3.8490000000000002,"5min":10.6,"15min":15.613},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.45,"5min":1.9910000000000001,"15min":1.5189999999999999},"min":{"1min":0.246,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":8.157,"5min":9.1259999999999994,"15min":13.515000000000001},"last":0.38}]},{"osd":2,"last update":"Thu Feb 22 19:40:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.673,"15min":1.4650000000000001},"min":{"1min":0.24199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.0700000000000003,"5min":11.682,"15min":16.503},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":2.1560000000000001,"15min":1.556},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.192},"max":{"1min":3.3439999999999999,"5min":14.994,"15min":14.994},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":1.9410000000000001,"15min":1.6459999999999999},"min":{"1min":0.29499999999999998,"5min":0.253,"15min":0.214},"max":{"1min":2.5920000000000001,"5min":10.734999999999999,"15min":25.145},"last":0.24199999999999999},{"interface":"front","average":{"1min":1.226,"5min":2.1850000000000001,"15min":1.7310000000000001},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":9.5820000000000007,"5min":9.5820000000000007,"15min":22.25},"last":0.29299999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:41:05 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.599,"15min":1.2569999999999999},"min":{"1min":0.34399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.6100000000000003,"5min":7.9390000000000001,"15min":15.542999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.232,"15min":1.0640000000000001},"min":{"1min":0.378,"5min":0.30499999999999999,"15min":0.245},"max":{"1min":2.7269999999999999,"5min":10.706,"15min":10.706},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:40:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":1.569,"15min":1.246},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0110000000000001,"5min":8.0579999999999998,"15min":15.587999999999999},"last":1.6899999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.4179999999999999,"15min":1.137},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":4.3730000000000002,"5min":5.4359999999999999,"15min":9.4719999999999995},"last":0.34200000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:40:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.46,"15min":1.1180000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":6.8970000000000002,"5min":10.631,"15min":10.631},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.4950000000000001,"15min":1.23},"min":{"1min":0.38600000000000001,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":3.508,"5min":8.0259999999999998,"15min":15.651999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Feb 22 19:40:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":1.7190000000000001,"15min":1.2250000000000001},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":2.0030000000000001,"5min":13.169,"15min":13.169},"last":1.429},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.4179999999999999,"15min":1.093},"min":{"1min":0.47099999999999997,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":10.673,"15min":10.673},"last":1.3839999999999999}]}]},{"osd":0,"up_from":0,"seq":34359738553,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411502,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1889788,"kb_used_data":1584372,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":91855364,"statfs":{"total":95995035648,"available":94059892736,"internally_reserved":0,"allocated":1622396928,"data_stored":1621514785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":2.6320000000000001,"15min":1.587},"min":{"1min":0.216,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":6.46,"5min":22.338999999999999,"15min":22.338999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":1.508,"5min":2.952,"15min":1.8799999999999999},"min":{"1min":0.25900000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":7.2990000000000004,"5min":30.427,"15min":30.427},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":2.0259999999999998,"15min":1.4770000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.2559999999999998,"5min":12.349,"15min":12.349},"last":0.505},{"interface":"front","average":{"1min":0.89500000000000002,"5min":2.1509999999999998,"15min":1.3819999999999999},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.193},"max":{"1min":3.1749999999999998,"5min":13.782,"15min":13.782},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:40:49 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":2.7389999999999999,"15min":1.7809999999999999},"min":{"1min":0.29099999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.8889999999999998,"5min":22.302,"15min":30.071000000000002},"last":0.376},{"interface":"front","average":{"1min":1.294,"5min":2.827,"15min":1.9219999999999999},"min":{"1min":0.29899999999999999,"5min":0.184,"15min":0.184},"max":{"1min":7.3600000000000003,"5min":30.282,"15min":32.119},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:41:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.5389999999999999,"15min":1.2270000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.4399999999999999,"5min":12.43,"15min":12.43},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.901,"15min":1.143},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.253},"max":{"1min":3.3799999999999999,"5min":10.34,"15min":10.34},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:40:20 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.766,"15min":1.121},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.7199999999999998,"5min":10.300000000000001,"15min":10.300000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.5609999999999999,"15min":1.242},"min":{"1min":0.33200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.992,"5min":12.391999999999999,"15min":12.391999999999999},"last":4.0099999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:41:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.7090000000000001,"15min":1.3220000000000001},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.927,"5min":21.640000000000001,"15min":21.640000000000001},"last":3.714},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.7729999999999999,"15min":1.141},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":2.4990000000000001,"5min":10.369999999999999,"15min":10.369999999999999},"last":2.1640000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:41:14 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.6619999999999999,"15min":1.3280000000000001},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.29099999999999998},"max":{"1min":2.714,"5min":12.484,"15min":12.484},"last":2.1480000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":2.0150000000000001,"15min":1.244},"min":{"1min":0.38900000000000001,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":2.137,"5min":10.539,"15min":10.539},"last":1.5}]}]},{"osd":3,"up_from":26,"seq":111669149864,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4130264,"kb_used_data":3540560,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":89614888,"statfs":{"total":95995035648,"available":91765645312,"internally_reserved":0,"allocated":3625533440,"data_stored":3624624319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":2.1789999999999998,"15min":1.6080000000000001},"min":{"1min":0.23000000000000001,"5min":0.21199999999999999,"15min":0.193},"max":{"1min":5.4960000000000004,"5min":23.48,"15min":23.48},"last":0.26000000000000001},{"interface":"front","average":{"1min":1.96,"5min":2.681,"15min":1.9059999999999999},"min":{"1min":0.315,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":18.172000000000001,"5min":23.411999999999999,"15min":23.411999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":3.3809999999999998,"15min":2.1280000000000001},"min":{"1min":0.27200000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":18.213999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":2.8940000000000001,"15min":1.758},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":12.234,"5min":22.187000000000001,"15min":22.187000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":2.569,"15min":1.8220000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.105,"5min":20.105,"15min":20.105},"last":0.499},{"interface":"front","average":{"1min":1.0880000000000001,"5min":2.52,"15min":1.7629999999999999},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.217},"max":{"1min":7.617,"5min":35.939999999999998,"15min":35.939999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":2.129,"15min":1.528},"min":{"1min":0.38500000000000001,"5min":0.26300000000000001,"15min":0.249},"max":{"1min":2.355,"5min":12.416,"15min":13.680999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":2.238,"15min":1.46},"min":{"1min":0.32300000000000001,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":5.1210000000000004,"5min":13.548,"15min":13.548},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.653,"15min":1.2749999999999999},"min":{"1min":0.434,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":3.6789999999999998,"5min":8.0289999999999999,"15min":12.879},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":2.0659999999999998,"15min":1.48},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":12.372999999999999,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.40899999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:40:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.6559999999999999,"15min":1.2949999999999999},"min":{"1min":0.39900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.0449999999999999,"5min":7.9640000000000004,"15min":12.898999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":2.0920000000000001,"15min":1.5589999999999999},"min":{"1min":0.45200000000000001,"5min":0.34799999999999998,"15min":0.30199999999999999},"max":{"1min":3.9910000000000001,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.44}]},{"osd":7,"last update":"Thu Feb 22 19:41:05 2024","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":2.214,"15min":1.488},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":5.0670000000000002,"5min":13.483000000000001,"15min":13.483000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.9490000000000001,"5min":1.841,"15min":1.3779999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.093,"5min":7.9909999999999997,"15min":12.846},"last":0.56100000000000005}]}]},{"osd":4,"up_from":0,"seq":137438953638,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724704,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4241148,"kb_used_data":3931636,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":89504004,"statfs":{"total":95995035648,"available":91652100096,"internally_reserved":0,"allocated":4025995264,"data_stored":4025113323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.3560000000000001,"15min":1.151},"min":{"1min":0.27800000000000002,"5min":0.23999999999999999,"15min":0.222},"max":{"1min":1.9139999999999999,"5min":17.684000000000001,"15min":17.684000000000001},"last":2.9510000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.605,"15min":1.2250000000000001},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":3.919,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.702,"15min":1.212},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.22500000000000001},"max":{"1min":2.0089999999999999,"5min":29.390999999999998,"15min":29.390999999999998},"last":0.878},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.194},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.218},"max":{"1min":2.7770000000000001,"5min":29.367000000000001,"15min":29.367000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.7230000000000001,"15min":1.2470000000000001},"min":{"1min":0.36299999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.1779999999999999,"5min":11.367000000000001,"15min":14.613},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.722,"15min":1.3029999999999999},"min":{"1min":0.33400000000000002,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":3.8130000000000002,"5min":16.099,"15min":16.099},"last":0.505}]},{"osd":3,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.593,"15min":1.395},"min":{"1min":0.41099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.1399999999999997,"5min":7.0309999999999997,"15min":13.366},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.5349999999999999,"15min":1.3520000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.8839999999999999,"5min":8.6069999999999993,"15min":10.16},"last":2.984}]},{"osd":4,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68600000000000005,"15min":0.51900000000000002},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.219},"max":{"1min":1.6879999999999999,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67800000000000005,"15min":0.52500000000000002},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":1.998,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.623,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":1.853,"5min":3.246,"15min":5.6040000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.60899999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.698,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.35799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":4.6420000000000003,"15min":5.5789999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.61099999999999999},"min":{"1min":0.42399999999999999,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":1.538,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":1.6719999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4064083968,"data_stored":4064083968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1625358336,"data_stored":1625358336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3664838656,"data_stored":3664838656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4064018432,"data_stored":4064018432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2022834176,"data_stored":2022834176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4047175680,"data_stored":4047175680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:41:20.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-02-22T19:41:20.170 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:41:20.170 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:41:20.170 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:41:20.170 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:41:20.232 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 69 16 105786 105770 95.7595 71.75 0.00897965 0.0104305 2024-02-22T19:41:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:20.905 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:19.595625+0000 mgr.y (mgr.14535) 474 : cluster [DBG] pgmap v606: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-02-22T19:41:20.905 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:19.721366+0000 osd.3 (osd.3) 209 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:41:20.905 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:19.844321+0000 osd.3 (osd.3) 210 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:41:20.905 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:19.860110+0000 osd.6 (osd.6) 140 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:41:20.905 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:19.866819+0000 osd.6 (osd.6) 141 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:41:20.905 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:19.878808+0000 mon.c (mon.2) 60 : audit [DBG] from='client.? 172.21.15.53:0/694672505' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:20.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:19.595625+0000 mgr.y (mgr.14535) 474 : cluster [DBG] pgmap v606: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-02-22T19:41:20.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:19.721366+0000 osd.3 (osd.3) 209 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:41:20.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:19.844321+0000 osd.3 (osd.3) 210 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:41:20.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:19.860110+0000 osd.6 (osd.6) 140 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:41:20.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:19.866819+0000 osd.6 (osd.6) 141 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:41:20.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:19.878808+0000 mon.c (mon.2) 60 : audit [DBG] from='client.? 172.21.15.53:0/694672505' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:21.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:19.595625+0000 mgr.y (mgr.14535) 474 : cluster [DBG] pgmap v606: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-02-22T19:41:21.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:19.721366+0000 osd.3 (osd.3) 209 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:41:21.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:19.844321+0000 osd.3 (osd.3) 210 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:41:21.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:19.860110+0000 osd.6 (osd.6) 140 : cluster [DBG] 5.11 scrub starts 2024-02-22T19:41:21.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:19.866819+0000 osd.6 (osd.6) 141 : cluster [DBG] 5.11 scrub ok 2024-02-22T19:41:21.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:19.878808+0000 mon.c (mon.2) 60 : audit [DBG] from='client.? 172.21.15.53:0/694672505' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:21.232 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 70 16 106956 106940 95.4362 73.125 0.00312585 0.0104735 2024-02-22T19:41:21.770 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:20.249426+0000 osd.2 (osd.2) 170 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:41:22.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:20.253270+0000 osd.2 (osd.2) 171 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:41:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:20.249426+0000 osd.2 (osd.2) 170 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:41:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:20.253270+0000 osd.2 (osd.2) 171 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:41:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:20.249426+0000 osd.2 (osd.2) 170 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:41:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:20.253270+0000 osd.2 (osd.2) 171 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:41:22.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 71 16 108155 108139 95.1476 74.9375 0.00989883 0.010506 2024-02-22T19:41:22.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:21.596886+0000 mgr.y (mgr.14535) 475 : cluster [DBG] pgmap v607: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-02-22T19:41:22.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:21.729231+0000 osd.3 (osd.3) 211 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:41:22.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:21.734928+0000 osd.3 (osd.3) 212 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:41:22.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:21.596886+0000 mgr.y (mgr.14535) 475 : cluster [DBG] pgmap v607: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-02-22T19:41:22.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:21.729231+0000 osd.3 (osd.3) 211 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:41:22.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:21.734928+0000 osd.3 (osd.3) 212 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:41:22.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:41:23.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:21.596886+0000 mgr.y (mgr.14535) 475 : cluster [DBG] pgmap v607: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-02-22T19:41:23.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:21.729231+0000 osd.3 (osd.3) 211 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:41:23.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:21.734928+0000 osd.3 (osd.3) 212 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:41:23.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 72 15 109365 109350 94.8773 75.6875 0.00925978 0.0105358 2024-02-22T19:41:24.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:22.211112+0000 osd.1 (osd.1) 194 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:41:24.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:22.216551+0000 osd.1 (osd.1) 195 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:41:24.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:22.643333+0000 osd.7 (osd.7) 199 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:41:24.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:22.650310+0000 osd.7 (osd.7) 200 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:41:24.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:22.828590+0000 osd.6 (osd.6) 142 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:41:24.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:22.833846+0000 osd.6 (osd.6) 143 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:41:24.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:22.211112+0000 osd.1 (osd.1) 194 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:41:24.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:22.216551+0000 osd.1 (osd.1) 195 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:41:24.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:22.643333+0000 osd.7 (osd.7) 199 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:41:24.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:22.650310+0000 osd.7 (osd.7) 200 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:41:24.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:22.828590+0000 osd.6 (osd.6) 142 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:41:24.003 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:22.833846+0000 osd.6 (osd.6) 143 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:41:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:22.211112+0000 osd.1 (osd.1) 194 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:41:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:22.216551+0000 osd.1 (osd.1) 195 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:41:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:22.643333+0000 osd.7 (osd.7) 199 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:41:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:22.650310+0000 osd.7 (osd.7) 200 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:41:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:22.828590+0000 osd.6 (osd.6) 142 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:41:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:22.833846+0000 osd.6 (osd.6) 143 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:41:24.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 73 16 110573 110557 94.611 75.4375 0.0107277 0.0105653 2024-02-22T19:41:24.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:23.268994+0000 osd.2 (osd.2) 172 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:41:24.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:23.338369+0000 osd.2 (osd.2) 173 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:41:24.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:23.598182+0000 mgr.y (mgr.14535) 476 : cluster [DBG] pgmap v608: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 45 MiB/s wr, 720 op/s 2024-02-22T19:41:24.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:23.760911+0000 osd.3 (osd.3) 213 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:41:24.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:23.826936+0000 osd.3 (osd.3) 214 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:41:24.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.566469+0000 osd.7 (osd.7) 201 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.569390+0000 osd.7 (osd.7) 202 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:24.599549+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? 172.21.15.53:0/1967285303' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:23.268994+0000 osd.2 (osd.2) 172 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:23.338369+0000 osd.2 (osd.2) 173 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:23.598182+0000 mgr.y (mgr.14535) 476 : cluster [DBG] pgmap v608: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 45 MiB/s wr, 720 op/s 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:23.760911+0000 osd.3 (osd.3) 213 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:23.826936+0000 osd.3 (osd.3) 214 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.566469+0000 osd.7 (osd.7) 201 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.569390+0000 osd.7 (osd.7) 202 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:41:24.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:24.599549+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? 172.21.15.53:0/1967285303' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:24.790 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:23.268994+0000 osd.2 (osd.2) 172 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:23.338369+0000 osd.2 (osd.2) 173 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:23.598182+0000 mgr.y (mgr.14535) 476 : cluster [DBG] pgmap v608: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 45 MiB/s wr, 720 op/s 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:23.760911+0000 osd.3 (osd.3) 213 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:23.826936+0000 osd.3 (osd.3) 214 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.566469+0000 osd.7 (osd.7) 201 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.569390+0000 osd.7 (osd.7) 202 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:41:24.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:24.599549+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? 172.21.15.53:0/1967285303' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:24.970 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:41:25.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 74 16 111722 111706 94.303 71.8125 0.00891995 0.0106006 2024-02-22T19:41:25.354 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:41:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:41:25.569 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:25.574 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:25.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:25.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:41:25.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:25.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:25.775 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:25.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 9 pgp_num to 16 2024-02-22T19:41:25.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.247798+0000 osd.1 (osd.1) 196 : cluster [DBG] 6.e scrub starts 2024-02-22T19:41:25.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.278746+0000 osd.1 (osd.1) 197 : cluster [DBG] 6.e scrub ok 2024-02-22T19:41:25.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.730201+0000 osd.3 (osd.3) 215 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:41:25.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.739555+0000 osd.3 (osd.3) 216 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.744727+0000 mon.a (mon.0) 1335 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.744768+0000 mon.a (mon.0) 1336 : cluster [INF] Cluster is now healthy 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:24.757261+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? 172.21.15.53:0/1967285303' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.757342+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e311: 8 total, 8 up, 6 in 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.843639+0000 osd.6 (osd.6) 144 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:24.850170+0000 osd.6 (osd.6) 145 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:24.967920+0000 mgr.y (mgr.14535) 477 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:25.119220+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.53:0/1045557240' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:25.385759+0000 mon.a (mon.0) 1340 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [7, 2]}]: dispatch 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:25.386024+0000 mon.a (mon.0) 1341 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.593910+0000 osd.7 (osd.7) 203 : cluster [DBG] 6.8 deep-scrub starts 2024-02-22T19:41:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.602346+0000 osd.7 (osd.7) 204 : cluster [DBG] 6.8 deep-scrub ok 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.247798+0000 osd.1 (osd.1) 196 : cluster [DBG] 6.e scrub starts 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.278746+0000 osd.1 (osd.1) 197 : cluster [DBG] 6.e scrub ok 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.730201+0000 osd.3 (osd.3) 215 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.739555+0000 osd.3 (osd.3) 216 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.744727+0000 mon.a (mon.0) 1335 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.744768+0000 mon.a (mon.0) 1336 : cluster [INF] Cluster is now healthy 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:24.757261+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? 172.21.15.53:0/1967285303' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.757342+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e311: 8 total, 8 up, 6 in 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.843639+0000 osd.6 (osd.6) 144 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:24.850170+0000 osd.6 (osd.6) 145 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:24.967920+0000 mgr.y (mgr.14535) 477 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:25.119220+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.53:0/1045557240' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:25.385759+0000 mon.a (mon.0) 1340 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [7, 2]}]: dispatch 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:25.386024+0000 mon.a (mon.0) 1341 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-02-22T19:41:25.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.593910+0000 osd.7 (osd.7) 203 : cluster [DBG] 6.8 deep-scrub starts 2024-02-22T19:41:25.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.602346+0000 osd.7 (osd.7) 204 : cluster [DBG] 6.8 deep-scrub ok 2024-02-22T19:41:25.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.247798+0000 osd.1 (osd.1) 196 : cluster [DBG] 6.e scrub starts 2024-02-22T19:41:25.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.278746+0000 osd.1 (osd.1) 197 : cluster [DBG] 6.e scrub ok 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.730201+0000 osd.3 (osd.3) 215 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.739555+0000 osd.3 (osd.3) 216 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.744727+0000 mon.a (mon.0) 1335 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.744768+0000 mon.a (mon.0) 1336 : cluster [INF] Cluster is now healthy 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:24.757261+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? 172.21.15.53:0/1967285303' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.757342+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e311: 8 total, 8 up, 6 in 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.843639+0000 osd.6 (osd.6) 144 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:24.850170+0000 osd.6 (osd.6) 145 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:24.967920+0000 mgr.y (mgr.14535) 477 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:25.119220+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.53:0/1045557240' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:25.385759+0000 mon.a (mon.0) 1340 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [7, 2]}]: dispatch 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:25.386024+0000 mon.a (mon.0) 1341 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-02-22T19:41:25.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.593910+0000 osd.7 (osd.7) 203 : cluster [DBG] 6.8 deep-scrub starts 2024-02-22T19:41:25.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.602346+0000 osd.7 (osd.7) 204 : cluster [DBG] 6.8 deep-scrub ok 2024-02-22T19:41:25.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:25.986 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:25.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:26.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:26.138 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:26.204 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:26.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:26.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:26.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 75 16 112907 112891 94.0331 74.0625 0.0210425 0.0106307 2024-02-22T19:41:26.314 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:26.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:26.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:26.507 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:26.555 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:26.924 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:26.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:26.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:26.932 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.258666+0000 osd.2 (osd.2) 174 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.263211+0000 osd.2 (osd.2) 175 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.483730+0000 osd.5 (osd.5) 98 : cluster [DBG] 3.c scrub starts 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.486544+0000 osd.5 (osd.5) 99 : cluster [DBG] 3.c scrub ok 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.600706+0000 mgr.y (mgr.14535) 478 : cluster [DBG] pgmap v610: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.40k op/s 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:25.788092+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? 172.21.15.53:0/1045557240' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:25.788169+0000 mon.a (mon.0) 1343 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [7, 2]}]': finished 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:25.788231+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.788288+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e312: 8 total, 8 up, 6 in 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.814597+0000 osd.6 (osd.6) 146 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:25.816361+0000 osd.6 (osd.6) 147 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:26.600714+0000 osd.7 (osd.7) 205 : cluster [DBG] 3.d scrub starts 2024-02-22T19:41:26.933 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:26.608992+0000 osd.7 (osd.7) 206 : cluster [DBG] 3.d scrub ok 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.258666+0000 osd.2 (osd.2) 174 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.263211+0000 osd.2 (osd.2) 175 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.483730+0000 osd.5 (osd.5) 98 : cluster [DBG] 3.c scrub starts 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.486544+0000 osd.5 (osd.5) 99 : cluster [DBG] 3.c scrub ok 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.600706+0000 mgr.y (mgr.14535) 478 : cluster [DBG] pgmap v610: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.40k op/s 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:25.788092+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? 172.21.15.53:0/1045557240' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:25.788169+0000 mon.a (mon.0) 1343 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [7, 2]}]': finished 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:25.788231+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.788288+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e312: 8 total, 8 up, 6 in 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.814597+0000 osd.6 (osd.6) 146 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:25.816361+0000 osd.6 (osd.6) 147 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:26.600714+0000 osd.7 (osd.7) 205 : cluster [DBG] 3.d scrub starts 2024-02-22T19:41:27.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:26.608992+0000 osd.7 (osd.7) 206 : cluster [DBG] 3.d scrub ok 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.258666+0000 osd.2 (osd.2) 174 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.263211+0000 osd.2 (osd.2) 175 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.483730+0000 osd.5 (osd.5) 98 : cluster [DBG] 3.c scrub starts 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.486544+0000 osd.5 (osd.5) 99 : cluster [DBG] 3.c scrub ok 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.600706+0000 mgr.y (mgr.14535) 478 : cluster [DBG] pgmap v610: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.40k op/s 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:25.788092+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? 172.21.15.53:0/1045557240' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:25.788169+0000 mon.a (mon.0) 1343 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [7, 2]}]': finished 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:25.788231+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.788288+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e312: 8 total, 8 up, 6 in 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.814597+0000 osd.6 (osd.6) 146 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:25.816361+0000 osd.6 (osd.6) 147 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:26.600714+0000 osd.7 (osd.7) 205 : cluster [DBG] 3.d scrub starts 2024-02-22T19:41:27.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:26.608992+0000 osd.7 (osd.7) 206 : cluster [DBG] 3.d scrub ok 2024-02-22T19:41:27.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:27.081 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:27.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 76 16 114156 114140 93.8229 78.0625 0.0102564 0.0106543 2024-02-22T19:41:27.452 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:27.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:27.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:27.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:27.588 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:27.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:41:27.717 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:27.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:27.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:27.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:28.036 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:41:28.036 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:41:28.064 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:26.301556+0000 osd.2 (osd.2) 176 : cluster [DBG] 5.b scrub starts 2024-02-22T19:41:28.064 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:26.309993+0000 osd.2 (osd.2) 177 : cluster [DBG] 5.b scrub ok 2024-02-22T19:41:28.064 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:26.792397+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e313: 8 total, 8 up, 6 in 2024-02-22T19:41:28.064 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:27.278513+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e314: 8 total, 8 up, 6 in 2024-02-22T19:41:28.233 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 77 16 115236 115220 93.4812 67.5 0.00332011 0.0106934 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:26.301556+0000 osd.2 (osd.2) 176 : cluster [DBG] 5.b scrub starts 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:26.309993+0000 osd.2 (osd.2) 177 : cluster [DBG] 5.b scrub ok 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:26.792397+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e313: 8 total, 8 up, 6 in 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:27.278513+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e314: 8 total, 8 up, 6 in 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:26.301556+0000 osd.2 (osd.2) 176 : cluster [DBG] 5.b scrub starts 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:26.309993+0000 osd.2 (osd.2) 177 : cluster [DBG] 5.b scrub ok 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:26.792397+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e313: 8 total, 8 up, 6 in 2024-02-22T19:41:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:27.278513+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e314: 8 total, 8 up, 6 in 2024-02-22T19:41:28.811 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:27.601089+0000 mgr.y (mgr.14535) 479 : cluster [DBG] pgmap v614: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:41:28.811 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:27.815669+0000 osd.6 (osd.6) 148 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:41:28.811 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:27.820929+0000 osd.6 (osd.6) 149 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:41:28.811 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:28.395772+0000 osd.5 (osd.5) 100 : cluster [DBG] 5.f scrub starts 2024-02-22T19:41:28.811 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:28.401556+0000 osd.5 (osd.5) 101 : cluster [DBG] 5.f scrub ok 2024-02-22T19:41:28.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:41:29.234 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 78 16 116481 116465 93.2794 77.8125 0.0143091 0.0107166 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:27.601089+0000 mgr.y (mgr.14535) 479 : cluster [DBG] pgmap v614: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:27.815669+0000 osd.6 (osd.6) 148 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:27.820929+0000 osd.6 (osd.6) 149 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:28.395772+0000 osd.5 (osd.5) 100 : cluster [DBG] 5.f scrub starts 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:28.401556+0000 osd.5 (osd.5) 101 : cluster [DBG] 5.f scrub ok 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:27.601089+0000 mgr.y (mgr.14535) 479 : cluster [DBG] pgmap v614: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-02-22T19:41:29.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:27.815669+0000 osd.6 (osd.6) 148 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:41:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:27.820929+0000 osd.6 (osd.6) 149 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:41:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:28.395772+0000 osd.5 (osd.5) 100 : cluster [DBG] 5.f scrub starts 2024-02-22T19:41:29.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:28.401556+0000 osd.5 (osd.5) 101 : cluster [DBG] 5.f scrub ok 2024-02-22T19:41:29.378 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:29.756 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:29] "GET /metrics HTTP/1.1" 200 176137 "" "Prometheus/2.33.4" 2024-02-22T19:41:29.975 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:28.380906+0000 osd.2 (osd.2) 178 : cluster [DBG] 3.e scrub starts 2024-02-22T19:41:29.976 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:28.385421+0000 osd.2 (osd.2) 179 : cluster [DBG] 3.e scrub ok 2024-02-22T19:41:29.976 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:29.009050+0000 mon.a (mon.0) 1348 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:41:30.234 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 79 16 117656 117640 93.0282 73.4375 0.00890598 0.0107454 2024-02-22T19:41:30.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:28.380906+0000 osd.2 (osd.2) 178 : cluster [DBG] 3.e scrub starts 2024-02-22T19:41:30.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:28.385421+0000 osd.2 (osd.2) 179 : cluster [DBG] 3.e scrub ok 2024-02-22T19:41:30.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:29.009050+0000 mon.a (mon.0) 1348 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:41:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:28.380906+0000 osd.2 (osd.2) 178 : cluster [DBG] 3.e scrub starts 2024-02-22T19:41:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:28.385421+0000 osd.2 (osd.2) 179 : cluster [DBG] 3.e scrub ok 2024-02-22T19:41:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:29.009050+0000 mon.a (mon.0) 1348 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:41:31.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:29.203400+0000 osd.1 (osd.1) 198 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:41:31.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:29.209499+0000 osd.1 (osd.1) 199 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:41:31.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:29.601806+0000 mgr.y (mgr.14535) 480 : cluster [DBG] pgmap v615: 148 pgs: 148 active+clean; 7.0 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.0 KiB/s rd, 56 MiB/s wr, 894 op/s; 22 B/s, 1 objects/s recovering 2024-02-22T19:41:31.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:29.626945+0000 osd.3 (osd.3) 217 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:41:31.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:29.630412+0000 osd.3 (osd.3) 218 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:41:31.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:30.422466+0000 osd.5 (osd.5) 102 : cluster [DBG] 3.5 deep-scrub starts 2024-02-22T19:41:31.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:30.423419+0000 osd.5 (osd.5) 103 : cluster [DBG] 3.5 deep-scrub ok 2024-02-22T19:41:31.234 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:41:31.235130+0000 min lat: 0.00135644 max lat: 0.0951228 avg lat: 0.0107721 2024-02-22T19:41:31.234 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:41:31.234 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 80 16 118838 118822 92.7888 73.875 0.00931758 0.0107721 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:29.203400+0000 osd.1 (osd.1) 198 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:29.209499+0000 osd.1 (osd.1) 199 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:29.601806+0000 mgr.y (mgr.14535) 480 : cluster [DBG] pgmap v615: 148 pgs: 148 active+clean; 7.0 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.0 KiB/s rd, 56 MiB/s wr, 894 op/s; 22 B/s, 1 objects/s recovering 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:29.626945+0000 osd.3 (osd.3) 217 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:29.630412+0000 osd.3 (osd.3) 218 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:30.422466+0000 osd.5 (osd.5) 102 : cluster [DBG] 3.5 deep-scrub starts 2024-02-22T19:41:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:30.423419+0000 osd.5 (osd.5) 103 : cluster [DBG] 3.5 deep-scrub ok 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:29.203400+0000 osd.1 (osd.1) 198 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:29.209499+0000 osd.1 (osd.1) 199 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:29.601806+0000 mgr.y (mgr.14535) 480 : cluster [DBG] pgmap v615: 148 pgs: 148 active+clean; 7.0 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.0 KiB/s rd, 56 MiB/s wr, 894 op/s; 22 B/s, 1 objects/s recovering 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:29.626945+0000 osd.3 (osd.3) 217 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:29.630412+0000 osd.3 (osd.3) 218 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:30.422466+0000 osd.5 (osd.5) 102 : cluster [DBG] 3.5 deep-scrub starts 2024-02-22T19:41:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:30.423419+0000 osd.5 (osd.5) 103 : cluster [DBG] 3.5 deep-scrub ok 2024-02-22T19:41:32.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:30.606829+0000 osd.3 (osd.3) 219 : cluster [DBG] 3.b scrub starts 2024-02-22T19:41:32.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:30.613646+0000 osd.3 (osd.3) 220 : cluster [DBG] 3.b scrub ok 2024-02-22T19:41:32.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.379984+0000 osd.5 (osd.5) 104 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:41:32.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.388139+0000 osd.5 (osd.5) 105 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:41:32.121 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:41:32.121 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:41:32.234 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 81 16 119997 119981 92.5375 72.4375 0.00913162 0.0108025 2024-02-22T19:41:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:30.606829+0000 osd.3 (osd.3) 219 : cluster [DBG] 3.b scrub starts 2024-02-22T19:41:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:30.613646+0000 osd.3 (osd.3) 220 : cluster [DBG] 3.b scrub ok 2024-02-22T19:41:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.379984+0000 osd.5 (osd.5) 104 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:41:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.388139+0000 osd.5 (osd.5) 105 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:41:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:30.606829+0000 osd.3 (osd.3) 219 : cluster [DBG] 3.b scrub starts 2024-02-22T19:41:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:30.613646+0000 osd.3 (osd.3) 220 : cluster [DBG] 3.b scrub ok 2024-02-22T19:41:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.379984+0000 osd.5 (osd.5) 104 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:41:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.388139+0000 osd.5 (osd.5) 105 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:41:32.463 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:41:32.463 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:41:32.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:41:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:32.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:32.696 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:32.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:32.845 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:32.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:32.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:33.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.391361+0000 osd.2 (osd.2) 180 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.395849+0000 osd.2 (osd.2) 181 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.602249+0000 mgr.y (mgr.14535) 481 : cluster [DBG] pgmap v616: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 853 B/s rd, 61 MiB/s wr, 983 op/s; 18 B/s, 1 objects/s recovering 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.634707+0000 osd.7 (osd.7) 207 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:31.641119+0000 osd.7 (osd.7) 208 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:32.604014+0000 osd.7 (osd.7) 209 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:41:33.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:32.611181+0000 osd.7 (osd.7) 210 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:41:33.089 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:33.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:41:33.235 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 82 16 121075 121059 92.2301 67.375 0.015658 0.0108381 2024-02-22T19:41:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.391361+0000 osd.2 (osd.2) 180 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:41:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.395849+0000 osd.2 (osd.2) 181 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:41:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.602249+0000 mgr.y (mgr.14535) 481 : cluster [DBG] pgmap v616: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 853 B/s rd, 61 MiB/s wr, 983 op/s; 18 B/s, 1 objects/s recovering 2024-02-22T19:41:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.634707+0000 osd.7 (osd.7) 207 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:41:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:31.641119+0000 osd.7 (osd.7) 208 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:41:33.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:32.604014+0000 osd.7 (osd.7) 209 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:32.611181+0000 osd.7 (osd.7) 210 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.391361+0000 osd.2 (osd.2) 180 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.395849+0000 osd.2 (osd.2) 181 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.602249+0000 mgr.y (mgr.14535) 481 : cluster [DBG] pgmap v616: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 853 B/s rd, 61 MiB/s wr, 983 op/s; 18 B/s, 1 objects/s recovering 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.634707+0000 osd.7 (osd.7) 207 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:31.641119+0000 osd.7 (osd.7) 208 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:32.604014+0000 osd.7 (osd.7) 209 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:41:33.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:32.611181+0000 osd.7 (osd.7) 210 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:41:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:33.349 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:41:33.382 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:41:33.411 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:41:33.438 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:41:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:33.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:33.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:33.472 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:41:33.509 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:41:33.554 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:41:33.559 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:33.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:33.588 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.594 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.595 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:41:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:33.803 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:33.807 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:33.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:33.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:32.443157+0000 osd.2 (osd.2) 182 : cluster [DBG] 5.d scrub starts 2024-02-22T19:41:33.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:32.453118+0000 osd.2 (osd.2) 183 : cluster [DBG] 5.d scrub ok 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:32.629261+0000 osd.3 (osd.3) 221 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:32.641405+0000 osd.3 (osd.3) 222 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:33.318204+0000 mon.a (mon.0) 1349 : audit [DBG] from='client.? 172.21.15.53:0/861005624' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:32.443157+0000 osd.2 (osd.2) 182 : cluster [DBG] 5.d scrub starts 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:32.453118+0000 osd.2 (osd.2) 183 : cluster [DBG] 5.d scrub ok 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:32.629261+0000 osd.3 (osd.3) 221 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:32.641405+0000 osd.3 (osd.3) 222 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:41:33.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:33.318204+0000 mon.a (mon.0) 1349 : audit [DBG] from='client.? 172.21.15.53:0/861005624' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:33.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:33.952 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:34.025 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:32.443157+0000 osd.2 (osd.2) 182 : cluster [DBG] 5.d scrub starts 2024-02-22T19:41:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:32.453118+0000 osd.2 (osd.2) 183 : cluster [DBG] 5.d scrub ok 2024-02-22T19:41:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:32.629261+0000 osd.3 (osd.3) 221 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:41:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:32.641405+0000 osd.3 (osd.3) 222 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:41:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:33.318204+0000 mon.a (mon.0) 1349 : audit [DBG] from='client.? 172.21.15.53:0/861005624' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:34.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:34.174 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:34.235 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 83 16 122183 122167 91.9527 69.25 0.0118681 0.0108707 2024-02-22T19:41:34.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:34.326 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:34.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:33.602847+0000 mgr.y (mgr.14535) 482 : cluster [DBG] pgmap v617: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 654 B/s rd, 47 MiB/s wr, 754 op/s; 14 B/s, 0 objects/s recovering 2024-02-22T19:41:34.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:33.954174+0000 osd.6 (osd.6) 150 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:34.842 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:33.954232+0000 osd.6 (osd.6) 151 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:34.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:34.583471+0000 osd.7 (osd.7) 211 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:41:34.843 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:34.587613+0000 osd.7 (osd.7) 212 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:41:34.843 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:41:35.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:41:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:33.602847+0000 mgr.y (mgr.14535) 482 : cluster [DBG] pgmap v617: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 654 B/s rd, 47 MiB/s wr, 754 op/s; 14 B/s, 0 objects/s recovering 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:33.954174+0000 osd.6 (osd.6) 150 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:33.954232+0000 osd.6 (osd.6) 151 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:34.583471+0000 osd.7 (osd.7) 211 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:34.587613+0000 osd.7 (osd.7) 212 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:33.602847+0000 mgr.y (mgr.14535) 482 : cluster [DBG] pgmap v617: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 654 B/s rd, 47 MiB/s wr, 754 op/s; 14 B/s, 0 objects/s recovering 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:33.954174+0000 osd.6 (osd.6) 150 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:33.954232+0000 osd.6 (osd.6) 151 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:35.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:34.583471+0000 osd.7 (osd.7) 211 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:41:35.129 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:34.587613+0000 osd.7 (osd.7) 212 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:41:35.237 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 84 16 123396 123380 91.759 75.8125 0.017723 0.0108943 2024-02-22T19:41:36.146 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:34.430355+0000 osd.2 (osd.2) 184 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:34.437999+0000 osd.2 (osd.2) 185 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:34.847978+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e315: 8 total, 8 up, 6 in 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:34.978322+0000 mgr.y (mgr.14535) 483 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:34.430355+0000 osd.2 (osd.2) 184 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:34.437999+0000 osd.2 (osd.2) 185 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:34.847978+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e315: 8 total, 8 up, 6 in 2024-02-22T19:41:36.147 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:34.978322+0000 mgr.y (mgr.14535) 483 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:36.206 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:36.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:36.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:36.237 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 85 16 124657 124641 91.6066 78.8125 0.0171505 0.0109124 2024-02-22T19:41:36.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:34.430355+0000 osd.2 (osd.2) 184 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:41:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:34.437999+0000 osd.2 (osd.2) 185 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:41:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:34.847978+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e315: 8 total, 8 up, 6 in 2024-02-22T19:41:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:34.978322+0000 mgr.y (mgr.14535) 483 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:36.354 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:36.528 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:36.531 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:36.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:36.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:36.868 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:35.208403+0000 osd.1 (osd.1) 200 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:125511 9.3 9:c7dc77c2:::benchmark_data_smithi053_157429_object125510:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e316)", 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:36.956071+0000", 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0072370070000000002, 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072684569999999999, 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:36.967 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 125511 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956071+0000", 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956071+0000", 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:41:36.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956070+0000", 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4062e-05 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956104+0000", 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0385e-05 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956114+0000", 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-02-22T19:41:36.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956120+0000", 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014685100000000001 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956267+0000", 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000117151 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956384+0000", 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000195558 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956580+0000", 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024172479999999999 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958997+0000", 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039474840000000002 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.962944+0000", 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00035083100000000002 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.963295+0000", 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1109000000000001e-05 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.963346+0000", 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072750009999999997 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.972 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:125514 9.d 9:bca16dd1:::benchmark_data_smithi053_157429_object125513:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e316)", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:36.956136+0000", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0071725879999999997, 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073364959999999996, 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 125514 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956136+0000", 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:36.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956136+0000", 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956135+0000", 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5343999999999997e-05 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956190+0000", 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2320000000000004e-06 2024-02-22T19:41:36.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956198+0000", 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.566e-06 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956202+0000", 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000477503 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956680+0000", 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011726999999999999 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956797+0000", 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000165341 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,5", 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.956962+0000", 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0060307340000000003 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.962993+0000", 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9278e-05 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.963022+0000", 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0068866910000000003 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:125515 9.9 9:99998a51:::benchmark_data_smithi053_157429_object125514:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e316)", 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:36.958763+0000", 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0045449959999999999, 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048048259999999999, 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:41:36.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 125515 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958763+0000", 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958763+0000", 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958762+0000", 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7795000000000001e-05 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958790+0000", 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958799+0000", 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3989999999999997e-06 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.958803+0000", 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000238075 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.959041+0000", 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000140595 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.959182+0000", 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019423899999999999 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,5", 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.959376+0000", 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003576657 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.962953+0000", 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00034548400000000001 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.963298+0000", 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045348970000000004 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.15324.0:125512 9.a e316/285 9:500b0544:::benchmark_data_smithi053_157429_object125511:head v 316'7765, mlcod=316'7760)", 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:36.961524+0000", 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001784756, 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021427939999999999, 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961524+0000", 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:41:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961524+0000", 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961523+0000", 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4978e-05 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961548+0000", 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7890000000000006e-06 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961557+0000", 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.879e-06 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961563+0000", 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030676599999999998 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961870+0000", 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1131000000000001e-05 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:41:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:36.961891+0000", 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036756400000000002 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 4 2024-02-22T19:41:36.985 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:41:36.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:35.208403+0000 osd.1 (osd.1) 200 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:35.211859+0000 osd.1 (osd.1) 201 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:35.603263+0000 mgr.y (mgr.14535) 484 : cluster [DBG] pgmap v619: 148 pgs: 148 active+clean; 7.5 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.40k op/s; 13 B/s, 0 objects/s recovering 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:35.632114+0000 osd.5 (osd.5) 106 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:35.632167+0000 osd.5 (osd.5) 107 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:35.868216+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e316: 8 total, 8 up, 6 in 2024-02-22T19:41:37.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:36.163816+0000 osd.1 (osd.1) 202 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:41:37.237 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 86 16 125820 125804 91.3867 72.6875 0.00499261 0.0109386 2024-02-22T19:41:37.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:35.208403+0000 osd.1 (osd.1) 200 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:41:37.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:35.211859+0000 osd.1 (osd.1) 201 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:35.603263+0000 mgr.y (mgr.14535) 484 : cluster [DBG] pgmap v619: 148 pgs: 148 active+clean; 7.5 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.40k op/s; 13 B/s, 0 objects/s recovering 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:35.632114+0000 osd.5 (osd.5) 106 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:35.632167+0000 osd.5 (osd.5) 107 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:35.868216+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e316: 8 total, 8 up, 6 in 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:36.163816+0000 osd.1 (osd.1) 202 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:35.211859+0000 osd.1 (osd.1) 201 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:35.603263+0000 mgr.y (mgr.14535) 484 : cluster [DBG] pgmap v619: 148 pgs: 148 active+clean; 7.5 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.40k op/s; 13 B/s, 0 objects/s recovering 2024-02-22T19:41:37.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:35.632114+0000 osd.5 (osd.5) 106 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:37.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:35.632167+0000 osd.5 (osd.5) 107 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:37.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:35.868216+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e316: 8 total, 8 up, 6 in 2024-02-22T19:41:37.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:36.163816+0000 osd.1 (osd.1) 202 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:41:37.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:41:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:41:38.175 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:38.223 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:41:38.237 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 87 16 126866 126850 91.0877 65.375 0.0151633 0.0109736 2024-02-22T19:41:38.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:36.170043+0000 osd.1 (osd.1) 203 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:41:38.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:36.659916+0000 osd.6 (osd.6) 152 : cluster [DBG] 1.7 deep-scrub starts 2024-02-22T19:41:38.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:36.670296+0000 osd.6 (osd.6) 153 : cluster [DBG] 1.7 deep-scrub ok 2024-02-22T19:41:38.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:36.170043+0000 osd.1 (osd.1) 203 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:41:38.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:36.659916+0000 osd.6 (osd.6) 152 : cluster [DBG] 1.7 deep-scrub starts 2024-02-22T19:41:38.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:36.670296+0000 osd.6 (osd.6) 153 : cluster [DBG] 1.7 deep-scrub ok 2024-02-22T19:41:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:38.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:36.170043+0000 osd.1 (osd.1) 203 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:41:38.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:36.659916+0000 osd.6 (osd.6) 152 : cluster [DBG] 1.7 deep-scrub starts 2024-02-22T19:41:38.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:36.670296+0000 osd.6 (osd.6) 153 : cluster [DBG] 1.7 deep-scrub ok 2024-02-22T19:41:38.264 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:38.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:38.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:38.419 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:38.565 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:38.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:38.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:38.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:38.713 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:38.765 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:38.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:38.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:38.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:37.604057+0000 mgr.y (mgr.14535) 485 : cluster [DBG] pgmap v621: 148 pgs: 148 active+clean; 7.5 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 639 B/s rd, 57 MiB/s wr, 914 op/s 2024-02-22T19:41:38.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:38.132608+0000 mgr.y (mgr.14535) 486 : audit [DBG] from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:41:38.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:38.309845+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:38.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:38.373822+0000 mon.a (mon.0) 1353 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:38.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:38.443314+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? 172.21.15.53:0/4263833330' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:41:38.910 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:39.237 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 88 16 127960 127944 90.8296 68.375 0.0108239 0.011006 2024-02-22T19:41:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:37.604057+0000 mgr.y (mgr.14535) 485 : cluster [DBG] pgmap v621: 148 pgs: 148 active+clean; 7.5 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 639 B/s rd, 57 MiB/s wr, 914 op/s 2024-02-22T19:41:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:38.132608+0000 mgr.y (mgr.14535) 486 : audit [DBG] from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:41:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:38.309845+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:38.373822+0000 mon.a (mon.0) 1353 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:38.443314+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? 172.21.15.53:0/4263833330' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:41:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:37.604057+0000 mgr.y (mgr.14535) 485 : cluster [DBG] pgmap v621: 148 pgs: 148 active+clean; 7.5 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 639 B/s rd, 57 MiB/s wr, 914 op/s 2024-02-22T19:41:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:38.132608+0000 mgr.y (mgr.14535) 486 : audit [DBG] from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:41:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:38.309845+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:38.373822+0000 mon.a (mon.0) 1353 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:38.443314+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? 172.21.15.53:0/4263833330' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:41:39.394 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:41:39.524 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-02-22T19:41:37.603798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8024561316,"num_objects":122637,"num_object_clones":0,"num_object_copies":367911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122637,"num_whiteouts":0,"num_read":1862,"num_read_kb":1614,"num_write":123159,"num_write_kb":7836521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":13776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63499,"ondisk_log_size":63499,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":25570348,"kb_used_data":23449404,"kb_used_omap":0,"kb_used_meta":2120832,"kb_avail":536900564,"statfs":{"total":575970213888,"available":549786177536,"internally_reserved":0,"allocated":24012189696,"data_stored":24006622290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2171731968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":479395840,"num_objects":7315,"num_object_clones":0,"num_object_copies":21945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7315,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7315,"num_write_kb":468160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002563"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":271,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.800815+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:41:26.800815+0000","last_peered":"2024-02-22T19:41:26.800815+0000","last_clean":"2024-02-22T19:41:26.800815+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:41:26.800815+0000","last_undegraded":"2024-02-22T19:41:26.800815+0000","last_fullsized":"2024-02-22T19:41:26.800815+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":292,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.813979+0000","last_change":"2024-02-22T19:39:22.268923+0000","last_active":"2024-02-22T19:41:26.813979+0000","last_peered":"2024-02-22T19:41:26.813979+0000","last_clean":"2024-02-22T19:41:26.813979+0000","last_became_active":"2024-02-22T19:39:22.268587+0000","last_became_peered":"2024-02-22T19:39:22.268587+0000","last_unstale":"2024-02-22T19:41:26.813979+0000","last_undegraded":"2024-02-22T19:41:26.813979+0000","last_fullsized":"2024-02-22T19:41:26.813979+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":134,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.802642+0000","last_change":"2024-02-22T19:41:22.833943+0000","last_active":"2024-02-22T19:41:26.802642+0000","last_peered":"2024-02-22T19:41:26.802642+0000","last_clean":"2024-02-22T19:41:26.802642+0000","last_became_active":"2024-02-22T19:38:47.164599+0000","last_became_peered":"2024-02-22T19:38:47.164599+0000","last_unstale":"2024-02-22T19:41:26.802642+0000","last_undegraded":"2024-02-22T19:41:26.802642+0000","last_fullsized":"2024-02-22T19:41:26.802642+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":277,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807589+0000","last_change":"2024-02-22T19:41:23.827037+0000","last_active":"2024-02-22T19:41:26.807589+0000","last_peered":"2024-02-22T19:41:26.807589+0000","last_clean":"2024-02-22T19:41:26.807589+0000","last_became_active":"2024-02-22T19:29:07.199541+0000","last_became_peered":"2024-02-22T19:29:07.199541+0000","last_unstale":"2024-02-22T19:41:26.807589+0000","last_undegraded":"2024-02-22T19:41:26.807589+0000","last_fullsized":"2024-02-22T19:41:26.807589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":278,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.812877+0000","last_change":"2024-02-22T19:40:34.661155+0000","last_active":"2024-02-22T19:41:26.812877+0000","last_peered":"2024-02-22T19:41:26.812877+0000","last_clean":"2024-02-22T19:41:26.812877+0000","last_became_active":"2024-02-22T19:29:01.013690+0000","last_became_peered":"2024-02-22T19:29:01.013690+0000","last_unstale":"2024-02-22T19:41:26.812877+0000","last_undegraded":"2024-02-22T19:41:26.812877+0000","last_fullsized":"2024-02-22T19:41:26.812877+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":394,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807464+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:41:26.807464+0000","last_peered":"2024-02-22T19:41:26.807464+0000","last_clean":"2024-02-22T19:41:26.807464+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:41:26.807464+0000","last_undegraded":"2024-02-22T19:41:26.807464+0000","last_fullsized":"2024-02-22T19:41:26.807464+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":115,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796240+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:41:26.796240+0000","last_peered":"2024-02-22T19:41:26.796240+0000","last_clean":"2024-02-22T19:41:26.796240+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:41:26.796240+0000","last_undegraded":"2024-02-22T19:41:26.796240+0000","last_fullsized":"2024-02-22T19:41:26.796240+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":115,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.808443+0000","last_change":"2024-02-22T19:38:47.169310+0000","last_active":"2024-02-22T19:41:26.808443+0000","last_peered":"2024-02-22T19:41:26.808443+0000","last_clean":"2024-02-22T19:41:26.808443+0000","last_became_active":"2024-02-22T19:38:47.168764+0000","last_became_peered":"2024-02-22T19:38:47.168764+0000","last_unstale":"2024-02-22T19:41:26.808443+0000","last_undegraded":"2024-02-22T19:41:26.808443+0000","last_fullsized":"2024-02-22T19:41:26.808443+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":172,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.808247+0000","last_change":"2024-02-22T19:40:48.683218+0000","last_active":"2024-02-22T19:41:26.808247+0000","last_peered":"2024-02-22T19:41:26.808247+0000","last_clean":"2024-02-22T19:41:26.808247+0000","last_became_active":"2024-02-22T19:39:22.269400+0000","last_became_peered":"2024-02-22T19:39:22.269400+0000","last_unstale":"2024-02-22T19:41:26.808247+0000","last_undegraded":"2024-02-22T19:41:26.808247+0000","last_fullsized":"2024-02-22T19:41:26.808247+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":111,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.799883+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:41:26.799883+0000","last_peered":"2024-02-22T19:41:26.799883+0000","last_clean":"2024-02-22T19:41:26.799883+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:41:26.799883+0000","last_undegraded":"2024-02-22T19:41:26.799883+0000","last_fullsized":"2024-02-22T19:41:26.799883+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":128,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807501+0000","last_change":"2024-02-22T19:39:53.438381+0000","last_active":"2024-02-22T19:41:26.807501+0000","last_peered":"2024-02-22T19:41:26.807501+0000","last_clean":"2024-02-22T19:41:26.807501+0000","last_became_active":"2024-02-22T19:38:47.179485+0000","last_became_peered":"2024-02-22T19:38:47.179485+0000","last_unstale":"2024-02-22T19:41:26.807501+0000","last_undegraded":"2024-02-22T19:41:26.807501+0000","last_fullsized":"2024-02-22T19:41:26.807501+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":114,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807411+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:41:26.807411+0000","last_peered":"2024-02-22T19:41:26.807411+0000","last_clean":"2024-02-22T19:41:26.807411+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:41:26.807411+0000","last_undegraded":"2024-02-22T19:41:26.807411+0000","last_fullsized":"2024-02-22T19:41:26.807411+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":272,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:27.821025+0000","last_change":"2024-02-22T19:41:27.821025+0000","last_active":"2024-02-22T19:41:27.821025+0000","last_peered":"2024-02-22T19:41:27.821025+0000","last_clean":"2024-02-22T19:41:27.821025+0000","last_became_active":"2024-02-22T19:38:47.168230+0000","last_became_peered":"2024-02-22T19:38:47.168230+0000","last_unstale":"2024-02-22T19:41:27.821025+0000","last_undegraded":"2024-02-22T19:41:27.821025+0000","last_fullsized":"2024-02-22T19:41:27.821025+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":88,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806834+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:41:26.806834+0000","last_peered":"2024-02-22T19:41:26.806834+0000","last_clean":"2024-02-22T19:41:26.806834+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:41:26.806834+0000","last_undegraded":"2024-02-22T19:41:26.806834+0000","last_fullsized":"2024-02-22T19:41:26.806834+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":341,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795337+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:41:26.795337+0000","last_peered":"2024-02-22T19:41:26.795337+0000","last_clean":"2024-02-22T19:41:26.795337+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:41:26.795337+0000","last_undegraded":"2024-02-22T19:41:26.795337+0000","last_fullsized":"2024-02-22T19:41:26.795337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":295,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.813396+0000","last_change":"2024-02-22T19:41:19.844421+0000","last_active":"2024-02-22T19:41:26.813396+0000","last_peered":"2024-02-22T19:41:26.813396+0000","last_clean":"2024-02-22T19:41:26.813396+0000","last_became_active":"2024-02-22T19:37:57.180089+0000","last_became_peered":"2024-02-22T19:37:57.180089+0000","last_unstale":"2024-02-22T19:41:26.813396+0000","last_undegraded":"2024-02-22T19:41:26.813396+0000","last_fullsized":"2024-02-22T19:41:26.813396+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":101,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.794448+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:41:26.794448+0000","last_peered":"2024-02-22T19:41:26.794448+0000","last_clean":"2024-02-22T19:41:26.794448+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:41:26.794448+0000","last_undegraded":"2024-02-22T19:41:26.794448+0000","last_fullsized":"2024-02-22T19:41:26.794448+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":334,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.812434+0000","last_change":"2024-02-22T19:40:39.682028+0000","last_active":"2024-02-22T19:41:26.812434+0000","last_peered":"2024-02-22T19:41:26.812434+0000","last_clean":"2024-02-22T19:41:26.812434+0000","last_became_active":"2024-02-22T19:38:47.180453+0000","last_became_peered":"2024-02-22T19:38:47.180453+0000","last_unstale":"2024-02-22T19:41:26.812434+0000","last_undegraded":"2024-02-22T19:41:26.812434+0000","last_fullsized":"2024-02-22T19:41:26.812434+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":320,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805825+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:41:26.805825+0000","last_peered":"2024-02-22T19:41:26.805825+0000","last_clean":"2024-02-22T19:41:26.805825+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:41:26.805825+0000","last_undegraded":"2024-02-22T19:41:26.805825+0000","last_fullsized":"2024-02-22T19:41:26.805825+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":300,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.794131+0000","last_change":"2024-02-22T19:40:50.388588+0000","last_active":"2024-02-22T19:41:26.794131+0000","last_peered":"2024-02-22T19:41:26.794131+0000","last_clean":"2024-02-22T19:41:26.794131+0000","last_became_active":"2024-02-22T19:38:47.168149+0000","last_became_peered":"2024-02-22T19:38:47.168149+0000","last_unstale":"2024-02-22T19:41:26.794131+0000","last_undegraded":"2024-02-22T19:41:26.794131+0000","last_fullsized":"2024-02-22T19:41:26.794131+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":121,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806250+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:41:26.806250+0000","last_peered":"2024-02-22T19:41:26.806250+0000","last_clean":"2024-02-22T19:41:26.806250+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:41:26.806250+0000","last_undegraded":"2024-02-22T19:41:26.806250+0000","last_fullsized":"2024-02-22T19:41:26.806250+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":309,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795564+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:41:26.795564+0000","last_peered":"2024-02-22T19:41:26.795564+0000","last_clean":"2024-02-22T19:41:26.795564+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:41:26.795564+0000","last_undegraded":"2024-02-22T19:41:26.795564+0000","last_fullsized":"2024-02-22T19:41:26.795564+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":307,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807314+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:41:26.807314+0000","last_peered":"2024-02-22T19:41:26.807314+0000","last_clean":"2024-02-22T19:41:26.807314+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:41:26.807314+0000","last_undegraded":"2024-02-22T19:41:26.807314+0000","last_fullsized":"2024-02-22T19:41:26.807314+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":390,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803715+0000","last_change":"2024-02-22T19:40:49.582930+0000","last_active":"2024-02-22T19:41:26.803715+0000","last_peered":"2024-02-22T19:41:26.803715+0000","last_clean":"2024-02-22T19:41:26.803715+0000","last_became_active":"2024-02-22T19:37:57.190476+0000","last_became_peered":"2024-02-22T19:37:57.190476+0000","last_unstale":"2024-02-22T19:41:26.803715+0000","last_undegraded":"2024-02-22T19:41:26.803715+0000","last_fullsized":"2024-02-22T19:41:26.803715+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":289,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806129+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:41:26.806129+0000","last_peered":"2024-02-22T19:41:26.806129+0000","last_clean":"2024-02-22T19:41:26.806129+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:41:26.806129+0000","last_undegraded":"2024-02-22T19:41:26.806129+0000","last_fullsized":"2024-02-22T19:41:26.806129+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":310,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803181+0000","last_change":"2024-02-22T19:41:19.866939+0000","last_active":"2024-02-22T19:41:26.803181+0000","last_peered":"2024-02-22T19:41:26.803181+0000","last_clean":"2024-02-22T19:41:26.803181+0000","last_became_active":"2024-02-22T19:39:22.256576+0000","last_became_peered":"2024-02-22T19:39:22.256576+0000","last_unstale":"2024-02-22T19:41:26.803181+0000","last_undegraded":"2024-02-22T19:41:26.803181+0000","last_fullsized":"2024-02-22T19:41:26.803181+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":292,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.813864+0000","last_change":"2024-02-22T19:39:22.246647+0000","last_active":"2024-02-22T19:41:26.813864+0000","last_peered":"2024-02-22T19:41:26.813864+0000","last_clean":"2024-02-22T19:41:26.813864+0000","last_became_active":"2024-02-22T19:39:22.246323+0000","last_became_peered":"2024-02-22T19:39:22.246323+0000","last_unstale":"2024-02-22T19:41:26.813864+0000","last_undegraded":"2024-02-22T19:41:26.813864+0000","last_fullsized":"2024-02-22T19:41:26.813864+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":293,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.630524+0000","last_change":"2024-02-22T19:41:29.630524+0000","last_active":"2024-02-22T19:41:29.630524+0000","last_peered":"2024-02-22T19:41:29.630524+0000","last_clean":"2024-02-22T19:41:29.630524+0000","last_became_active":"2024-02-22T19:29:03.083593+0000","last_became_peered":"2024-02-22T19:29:03.083593+0000","last_unstale":"2024-02-22T19:41:29.630524+0000","last_undegraded":"2024-02-22T19:41:29.630524+0000","last_fullsized":"2024-02-22T19:41:29.630524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":301,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807674+0000","last_change":"2024-02-22T19:39:22.250184+0000","last_active":"2024-02-22T19:41:26.807674+0000","last_peered":"2024-02-22T19:41:26.807674+0000","last_clean":"2024-02-22T19:41:26.807674+0000","last_became_active":"2024-02-22T19:39:22.249890+0000","last_became_peered":"2024-02-22T19:39:22.249890+0000","last_unstale":"2024-02-22T19:41:26.807674+0000","last_undegraded":"2024-02-22T19:41:26.807674+0000","last_fullsized":"2024-02-22T19:41:26.807674+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":307,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.804101+0000","last_change":"2024-02-22T19:39:22.253288+0000","last_active":"2024-02-22T19:41:26.804101+0000","last_peered":"2024-02-22T19:41:26.804101+0000","last_clean":"2024-02-22T19:41:26.804101+0000","last_became_active":"2024-02-22T19:39:22.252705+0000","last_became_peered":"2024-02-22T19:39:22.252705+0000","last_unstale":"2024-02-22T19:41:26.804101+0000","last_undegraded":"2024-02-22T19:41:26.804101+0000","last_fullsized":"2024-02-22T19:41:26.804101+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:54.418958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:37:54.418958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.810445+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:41:26.810445+0000","last_peered":"2024-02-22T19:41:26.810445+0000","last_clean":"2024-02-22T19:41:26.810445+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:41:26.810445+0000","last_undegraded":"2024-02-22T19:41:26.810445+0000","last_fullsized":"2024-02-22T19:41:26.810445+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":307,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807385+0000","last_change":"2024-02-22T19:39:37.816765+0000","last_active":"2024-02-22T19:41:26.807385+0000","last_peered":"2024-02-22T19:41:26.807385+0000","last_clean":"2024-02-22T19:41:26.807385+0000","last_became_active":"2024-02-22T19:29:03.446933+0000","last_became_peered":"2024-02-22T19:29:03.446933+0000","last_unstale":"2024-02-22T19:41:26.807385+0000","last_undegraded":"2024-02-22T19:41:26.807385+0000","last_fullsized":"2024-02-22T19:41:26.807385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":283,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.801144+0000","last_change":"2024-02-22T19:39:22.247453+0000","last_active":"2024-02-22T19:41:26.801144+0000","last_peered":"2024-02-22T19:41:26.801144+0000","last_clean":"2024-02-22T19:41:26.801144+0000","last_became_active":"2024-02-22T19:39:22.247266+0000","last_became_peered":"2024-02-22T19:39:22.247266+0000","last_unstale":"2024-02-22T19:41:26.801144+0000","last_undegraded":"2024-02-22T19:41:26.801144+0000","last_fullsized":"2024-02-22T19:41:26.801144+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":301,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807542+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:41:26.807542+0000","last_peered":"2024-02-22T19:41:26.807542+0000","last_clean":"2024-02-22T19:41:26.807542+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:41:26.807542+0000","last_undegraded":"2024-02-22T19:41:26.807542+0000","last_fullsized":"2024-02-22T19:41:26.807542+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":320,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:32.611307+0000","last_change":"2024-02-22T19:41:32.611307+0000","last_active":"2024-02-22T19:41:32.611307+0000","last_peered":"2024-02-22T19:41:32.611307+0000","last_clean":"2024-02-22T19:41:32.611307+0000","last_became_active":"2024-02-22T19:39:22.254411+0000","last_became_peered":"2024-02-22T19:39:22.254411+0000","last_unstale":"2024-02-22T19:41:32.611307+0000","last_undegraded":"2024-02-22T19:41:32.611307+0000","last_fullsized":"2024-02-22T19:41:32.611307+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":314,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796564+0000","last_change":"2024-02-22T19:41:18.275180+0000","last_active":"2024-02-22T19:41:26.796564+0000","last_peered":"2024-02-22T19:41:26.796564+0000","last_clean":"2024-02-22T19:41:26.796564+0000","last_became_active":"2024-02-22T19:29:03.085646+0000","last_became_peered":"2024-02-22T19:29:03.085646+0000","last_unstale":"2024-02-22T19:41:26.796564+0000","last_undegraded":"2024-02-22T19:41:26.796564+0000","last_fullsized":"2024-02-22T19:41:26.796564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":322,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807193+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:41:26.807193+0000","last_peered":"2024-02-22T19:41:26.807193+0000","last_clean":"2024-02-22T19:41:26.807193+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:41:26.807193+0000","last_undegraded":"2024-02-22T19:41:26.807193+0000","last_fullsized":"2024-02-22T19:41:26.807193+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":300,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796455+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:41:26.796455+0000","last_peered":"2024-02-22T19:41:26.796455+0000","last_clean":"2024-02-22T19:41:26.796455+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:41:26.796455+0000","last_undegraded":"2024-02-22T19:41:26.796455+0000","last_fullsized":"2024-02-22T19:41:26.796455+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":83,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805662+0000","last_change":"2024-02-22T19:40:52.560079+0000","last_active":"2024-02-22T19:41:26.805662+0000","last_peered":"2024-02-22T19:41:26.805662+0000","last_clean":"2024-02-22T19:41:26.805662+0000","last_became_active":"2024-02-22T19:38:47.161791+0000","last_became_peered":"2024-02-22T19:38:47.161791+0000","last_unstale":"2024-02-22T19:41:26.805662+0000","last_undegraded":"2024-02-22T19:41:26.805662+0000","last_fullsized":"2024-02-22T19:41:26.805662+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":137,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803508+0000","last_change":"2024-02-22T19:41:17.864550+0000","last_active":"2024-02-22T19:41:26.803508+0000","last_peered":"2024-02-22T19:41:26.803508+0000","last_clean":"2024-02-22T19:41:26.803508+0000","last_became_active":"2024-02-22T19:38:47.168328+0000","last_became_peered":"2024-02-22T19:38:47.168328+0000","last_unstale":"2024-02-22T19:41:26.803508+0000","last_undegraded":"2024-02-22T19:41:26.803508+0000","last_fullsized":"2024-02-22T19:41:26.803508+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":109,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805936+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:41:26.805936+0000","last_peered":"2024-02-22T19:41:26.805936+0000","last_clean":"2024-02-22T19:41:26.805936+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:41:26.805936+0000","last_undegraded":"2024-02-22T19:41:26.805936+0000","last_fullsized":"2024-02-22T19:41:26.805936+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":196,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.318309+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:41:29.318309+0000","last_peered":"2024-02-22T19:41:29.318309+0000","last_clean":"2024-02-22T19:41:29.318309+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:41:29.318309+0000","last_undegraded":"2024-02-22T19:41:29.318309+0000","last_fullsized":"2024-02-22T19:41:29.318309+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":293,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806707+0000","last_change":"2024-02-22T19:41:24.569460+0000","last_active":"2024-02-22T19:41:26.806707+0000","last_peered":"2024-02-22T19:41:26.806707+0000","last_clean":"2024-02-22T19:41:26.806707+0000","last_became_active":"2024-02-22T19:38:47.162182+0000","last_became_peered":"2024-02-22T19:38:47.162182+0000","last_unstale":"2024-02-22T19:41:26.806707+0000","last_undegraded":"2024-02-22T19:41:26.806707+0000","last_fullsized":"2024-02-22T19:41:26.806707+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":316,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.819303+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:41:26.819303+0000","last_peered":"2024-02-22T19:41:26.819303+0000","last_clean":"2024-02-22T19:41:26.819303+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:41:26.819303+0000","last_undegraded":"2024-02-22T19:41:26.819303+0000","last_fullsized":"2024-02-22T19:41:26.819303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":294,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795766+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:41:26.795766+0000","last_peered":"2024-02-22T19:41:26.795766+0000","last_clean":"2024-02-22T19:41:26.795766+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:41:26.795766+0000","last_undegraded":"2024-02-22T19:41:26.795766+0000","last_fullsized":"2024-02-22T19:41:26.795766+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":433,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.317554+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:41:29.317554+0000","last_peered":"2024-02-22T19:41:29.317554+0000","last_clean":"2024-02-22T19:41:29.317554+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:41:29.317554+0000","last_undegraded":"2024-02-22T19:41:29.317554+0000","last_fullsized":"2024-02-22T19:41:29.317554+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":95,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805435+0000","last_change":"2024-02-22T19:39:22.246092+0000","last_active":"2024-02-22T19:41:26.805435+0000","last_peered":"2024-02-22T19:41:26.805435+0000","last_clean":"2024-02-22T19:41:26.805435+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:41:26.805435+0000","last_undegraded":"2024-02-22T19:41:26.805435+0000","last_fullsized":"2024-02-22T19:41:26.805435+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:09.873141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:11.415718+0000","last_clean_scrub_stamp":"2024-02-22T19:37:09.873141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":130,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:31.388212+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:41:31.388212+0000","last_peered":"2024-02-22T19:41:31.388212+0000","last_clean":"2024-02-22T19:41:31.388212+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:41:31.388212+0000","last_undegraded":"2024-02-22T19:41:31.388212+0000","last_fullsized":"2024-02-22T19:41:31.388212+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":312,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795133+0000","last_change":"2024-02-22T19:39:53.126573+0000","last_active":"2024-02-22T19:41:26.795133+0000","last_peered":"2024-02-22T19:41:26.795133+0000","last_clean":"2024-02-22T19:41:26.795133+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:41:26.795133+0000","last_undegraded":"2024-02-22T19:41:26.795133+0000","last_fullsized":"2024-02-22T19:41:26.795133+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:53.126490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.126490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":316,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:31.641232+0000","last_change":"2024-02-22T19:41:31.641232+0000","last_active":"2024-02-22T19:41:31.641232+0000","last_peered":"2024-02-22T19:41:31.641232+0000","last_clean":"2024-02-22T19:41:31.641232+0000","last_became_active":"2024-02-22T19:39:22.254888+0000","last_became_peered":"2024-02-22T19:39:22.254888+0000","last_unstale":"2024-02-22T19:41:31.641232+0000","last_undegraded":"2024-02-22T19:41:31.641232+0000","last_fullsized":"2024-02-22T19:41:31.641232+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":102,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.812957+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:41:26.812957+0000","last_peered":"2024-02-22T19:41:26.812957+0000","last_clean":"2024-02-22T19:41:26.812957+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:41:26.812957+0000","last_undegraded":"2024-02-22T19:41:26.812957+0000","last_fullsized":"2024-02-22T19:41:26.812957+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":287,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.810201+0000","last_change":"2024-02-22T19:38:47.162050+0000","last_active":"2024-02-22T19:41:26.810201+0000","last_peered":"2024-02-22T19:41:26.810201+0000","last_clean":"2024-02-22T19:41:26.810201+0000","last_became_active":"2024-02-22T19:38:47.161672+0000","last_became_peered":"2024-02-22T19:38:47.161672+0000","last_unstale":"2024-02-22T19:41:26.810201+0000","last_undegraded":"2024-02-22T19:41:26.810201+0000","last_fullsized":"2024-02-22T19:41:26.810201+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":334,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.802354+0000","last_change":"2024-02-22T19:39:54.559858+0000","last_active":"2024-02-22T19:41:26.802354+0000","last_peered":"2024-02-22T19:41:26.802354+0000","last_clean":"2024-02-22T19:41:26.802354+0000","last_became_active":"2024-02-22T19:39:22.253387+0000","last_became_peered":"2024-02-22T19:39:22.253387+0000","last_unstale":"2024-02-22T19:41:26.802354+0000","last_undegraded":"2024-02-22T19:41:26.802354+0000","last_fullsized":"2024-02-22T19:41:26.802354+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":316,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.813484+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:41:26.813484+0000","last_peered":"2024-02-22T19:41:26.813484+0000","last_clean":"2024-02-22T19:41:26.813484+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:41:26.813484+0000","last_undegraded":"2024-02-22T19:41:26.813484+0000","last_fullsized":"2024-02-22T19:41:26.813484+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"314'7606","reported_seq":7642,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.382174+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:41:34.382174+0000","last_peered":"2024-02-22T19:41:34.382174+0000","last_clean":"2024-02-22T19:41:34.382174+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:41:34.382174+0000","last_undegraded":"2024-02-22T19:41:34.382174+0000","last_fullsized":"2024-02-22T19:41:34.382174+0000","mapping_epoch":285,"log_start":"300'3810","ondisk_log_start":"300'3810","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3796,"ondisk_log_size":3796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":498466816,"num_objects":7606,"num_object_clones":0,"num_object_copies":22818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7606,"num_write_kb":486784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":138,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.800745+0000","last_change":"2024-02-22T19:40:49.502459+0000","last_active":"2024-02-22T19:41:26.800745+0000","last_peered":"2024-02-22T19:41:26.800745+0000","last_clean":"2024-02-22T19:41:26.800745+0000","last_became_active":"2024-02-22T19:39:22.272214+0000","last_became_peered":"2024-02-22T19:39:22.272214+0000","last_unstale":"2024-02-22T19:41:26.800745+0000","last_undegraded":"2024-02-22T19:41:26.800745+0000","last_fullsized":"2024-02-22T19:41:26.800745+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":359,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805868+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:41:26.805868+0000","last_peered":"2024-02-22T19:41:26.805868+0000","last_clean":"2024-02-22T19:41:26.805868+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:41:26.805868+0000","last_undegraded":"2024-02-22T19:41:26.805868+0000","last_fullsized":"2024-02-22T19:41:26.805868+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":145,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.800950+0000","last_change":"2024-02-22T19:40:46.420455+0000","last_active":"2024-02-22T19:41:26.800950+0000","last_peered":"2024-02-22T19:41:26.800950+0000","last_clean":"2024-02-22T19:41:26.800950+0000","last_became_active":"2024-02-22T19:37:57.181199+0000","last_became_peered":"2024-02-22T19:37:57.181199+0000","last_unstale":"2024-02-22T19:41:26.800950+0000","last_undegraded":"2024-02-22T19:41:26.800950+0000","last_fullsized":"2024-02-22T19:41:26.800950+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"251'13","reported_seq":485,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.323860+0000","last_change":"2024-02-22T19:41:17.712594+0000","last_active":"2024-02-22T19:41:29.323860+0000","last_peered":"2024-02-22T19:41:29.323860+0000","last_clean":"2024-02-22T19:41:29.323860+0000","last_became_active":"2024-02-22T19:38:50.148285+0000","last_became_peered":"2024-02-22T19:38:50.148285+0000","last_unstale":"2024-02-22T19:41:29.323860+0000","last_undegraded":"2024-02-22T19:41:29.323860+0000","last_fullsized":"2024-02-22T19:41:29.323860+0000","mapping_epoch":250,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"314'7729","reported_seq":7766,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.473203+0000","last_change":"2024-02-22T19:40:00.882365+0000","last_active":"2024-02-22T19:41:34.473203+0000","last_peered":"2024-02-22T19:41:34.473203+0000","last_clean":"2024-02-22T19:41:34.473203+0000","last_became_active":"2024-02-22T19:40:00.882035+0000","last_became_peered":"2024-02-22T19:40:00.882035+0000","last_unstale":"2024-02-22T19:41:34.473203+0000","last_undegraded":"2024-02-22T19:41:34.473203+0000","last_fullsized":"2024-02-22T19:41:34.473203+0000","mapping_epoch":285,"log_start":"298'3700","ondisk_log_start":"298'3700","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4029,"ondisk_log_size":4029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":506527744,"num_objects":7729,"num_object_clones":0,"num_object_copies":23187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7729,"num_write_kb":494656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":272,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807630+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:41:26.807630+0000","last_peered":"2024-02-22T19:41:26.807630+0000","last_clean":"2024-02-22T19:41:26.807630+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:41:26.807630+0000","last_undegraded":"2024-02-22T19:41:26.807630+0000","last_fullsized":"2024-02-22T19:41:26.807630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:28.385529+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:41:28.385529+0000","last_peered":"2024-02-22T19:41:28.385529+0000","last_clean":"2024-02-22T19:41:28.385529+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:41:28.385529+0000","last_undegraded":"2024-02-22T19:41:28.385529+0000","last_fullsized":"2024-02-22T19:41:28.385529+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":139,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.794018+0000","last_change":"2024-02-22T19:40:48.455295+0000","last_active":"2024-02-22T19:41:26.794018+0000","last_peered":"2024-02-22T19:41:26.794018+0000","last_clean":"2024-02-22T19:41:26.794018+0000","last_became_active":"2024-02-22T19:38:47.167662+0000","last_became_peered":"2024-02-22T19:38:47.167662+0000","last_unstale":"2024-02-22T19:41:26.794018+0000","last_undegraded":"2024-02-22T19:41:26.794018+0000","last_fullsized":"2024-02-22T19:41:26.794018+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":332,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795150+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:41:26.795150+0000","last_peered":"2024-02-22T19:41:26.795150+0000","last_clean":"2024-02-22T19:41:26.795150+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:41:26.795150+0000","last_undegraded":"2024-02-22T19:41:26.795150+0000","last_fullsized":"2024-02-22T19:41:26.795150+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"314'7644","reported_seq":7681,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.356009+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:41:34.356009+0000","last_peered":"2024-02-22T19:41:34.356009+0000","last_clean":"2024-02-22T19:41:34.356009+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:41:34.356009+0000","last_undegraded":"2024-02-22T19:41:34.356009+0000","last_fullsized":"2024-02-22T19:41:34.356009+0000","mapping_epoch":285,"log_start":"300'3800","ondisk_log_start":"300'3800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3844,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":500957184,"num_objects":7644,"num_object_clones":0,"num_object_copies":22932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7644,"num_write_kb":489216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":278,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803794+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:41:26.803794+0000","last_peered":"2024-02-22T19:41:26.803794+0000","last_clean":"2024-02-22T19:41:26.803794+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:41:26.803794+0000","last_undegraded":"2024-02-22T19:41:26.803794+0000","last_fullsized":"2024-02-22T19:41:26.803794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":358,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795072+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:41:26.795072+0000","last_peered":"2024-02-22T19:41:26.795072+0000","last_clean":"2024-02-22T19:41:26.795072+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:41:26.795072+0000","last_undegraded":"2024-02-22T19:41:26.795072+0000","last_fullsized":"2024-02-22T19:41:26.795072+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":307,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.810523+0000","last_change":"2024-02-22T19:41:26.609086+0000","last_active":"2024-02-22T19:41:26.810523+0000","last_peered":"2024-02-22T19:41:26.810523+0000","last_clean":"2024-02-22T19:41:26.810523+0000","last_became_active":"2024-02-22T19:37:57.184627+0000","last_became_peered":"2024-02-22T19:37:57.184627+0000","last_unstale":"2024-02-22T19:41:26.810523+0000","last_undegraded":"2024-02-22T19:41:26.810523+0000","last_fullsized":"2024-02-22T19:41:26.810523+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":340,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.802854+0000","last_change":"2024-02-22T19:40:31.504524+0000","last_active":"2024-02-22T19:41:26.802854+0000","last_peered":"2024-02-22T19:41:26.802854+0000","last_clean":"2024-02-22T19:41:26.802854+0000","last_became_active":"2024-02-22T19:29:03.447657+0000","last_became_peered":"2024-02-22T19:29:03.447657+0000","last_unstale":"2024-02-22T19:41:26.802854+0000","last_undegraded":"2024-02-22T19:41:26.802854+0000","last_fullsized":"2024-02-22T19:41:26.802854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.6","version":"314'7717","reported_seq":7754,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.299411+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:41:34.299411+0000","last_peered":"2024-02-22T19:41:34.299411+0000","last_clean":"2024-02-22T19:41:34.299411+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:41:34.299411+0000","last_undegraded":"2024-02-22T19:41:34.299411+0000","last_fullsized":"2024-02-22T19:41:34.299411+0000","mapping_epoch":285,"log_start":"300'3914","ondisk_log_start":"300'3914","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3803,"ondisk_log_size":3803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":505741312,"num_objects":7717,"num_object_clones":0,"num_object_copies":23151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7717,"num_write_kb":493888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":100,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805024+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:41:26.805024+0000","last_peered":"2024-02-22T19:41:26.805024+0000","last_clean":"2024-02-22T19:41:26.805024+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:41:26.805024+0000","last_undegraded":"2024-02-22T19:41:26.805024+0000","last_fullsized":"2024-02-22T19:41:26.805024+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":337,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795132+0000","last_change":"2024-02-22T19:41:16.236762+0000","last_active":"2024-02-22T19:41:26.795132+0000","last_peered":"2024-02-22T19:41:26.795132+0000","last_clean":"2024-02-22T19:41:26.795132+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:41:26.795132+0000","last_undegraded":"2024-02-22T19:41:26.795132+0000","last_fullsized":"2024-02-22T19:41:26.795132+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.236664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.236664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":111,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807420+0000","last_change":"2024-02-22T19:41:25.486642+0000","last_active":"2024-02-22T19:41:26.807420+0000","last_peered":"2024-02-22T19:41:26.807420+0000","last_clean":"2024-02-22T19:41:26.807420+0000","last_became_active":"2024-02-22T19:37:57.181963+0000","last_became_peered":"2024-02-22T19:37:57.181963+0000","last_unstale":"2024-02-22T19:41:26.807420+0000","last_undegraded":"2024-02-22T19:41:26.807420+0000","last_fullsized":"2024-02-22T19:41:26.807420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":112,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795307+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:41:26.795307+0000","last_peered":"2024-02-22T19:41:26.795307+0000","last_clean":"2024-02-22T19:41:26.795307+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:41:26.795307+0000","last_undegraded":"2024-02-22T19:41:26.795307+0000","last_fullsized":"2024-02-22T19:41:26.795307+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"314'7735","reported_seq":7771,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.340890+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:41:34.340890+0000","last_peered":"2024-02-22T19:41:34.340890+0000","last_clean":"2024-02-22T19:41:34.340890+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:41:34.340890+0000","last_undegraded":"2024-02-22T19:41:34.340890+0000","last_fullsized":"2024-02-22T19:41:34.340890+0000","mapping_epoch":285,"log_start":"298'3700","ondisk_log_start":"298'3700","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4035,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":506920960,"num_objects":7735,"num_object_clones":0,"num_object_copies":23205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7735,"num_write_kb":495040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":147,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.799817+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:41:26.799817+0000","last_peered":"2024-02-22T19:41:26.799817+0000","last_clean":"2024-02-22T19:41:26.799817+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:41:26.799817+0000","last_undegraded":"2024-02-22T19:41:26.799817+0000","last_fullsized":"2024-02-22T19:41:26.799817+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":286,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:32.641510+0000","last_change":"2024-02-22T19:41:32.641510+0000","last_active":"2024-02-22T19:41:32.641510+0000","last_peered":"2024-02-22T19:41:32.641510+0000","last_clean":"2024-02-22T19:41:32.641510+0000","last_became_active":"2024-02-22T19:38:47.171002+0000","last_became_peered":"2024-02-22T19:38:47.171002+0000","last_unstale":"2024-02-22T19:41:32.641510+0000","last_undegraded":"2024-02-22T19:41:32.641510+0000","last_fullsized":"2024-02-22T19:41:32.641510+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":106,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.800696+0000","last_change":"2024-02-22T19:37:57.185637+0000","last_active":"2024-02-22T19:41:26.800696+0000","last_peered":"2024-02-22T19:41:26.800696+0000","last_clean":"2024-02-22T19:41:26.800696+0000","last_became_active":"2024-02-22T19:37:57.185312+0000","last_became_peered":"2024-02-22T19:37:57.185312+0000","last_unstale":"2024-02-22T19:41:26.800696+0000","last_undegraded":"2024-02-22T19:41:26.800696+0000","last_fullsized":"2024-02-22T19:41:26.800696+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":286,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803374+0000","last_change":"2024-02-22T19:39:51.569245+0000","last_active":"2024-02-22T19:41:26.803374+0000","last_peered":"2024-02-22T19:41:26.803374+0000","last_clean":"2024-02-22T19:41:26.803374+0000","last_became_active":"2024-02-22T19:39:22.251742+0000","last_became_peered":"2024-02-22T19:39:22.251742+0000","last_unstale":"2024-02-22T19:41:26.803374+0000","last_undegraded":"2024-02-22T19:41:26.803374+0000","last_fullsized":"2024-02-22T19:41:26.803374+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":514,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.317948+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:41:29.317948+0000","last_peered":"2024-02-22T19:41:29.317948+0000","last_clean":"2024-02-22T19:41:29.317948+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:41:29.317948+0000","last_undegraded":"2024-02-22T19:41:29.317948+0000","last_fullsized":"2024-02-22T19:41:29.317948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":109,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795479+0000","last_change":"2024-02-22T19:39:46.299764+0000","last_active":"2024-02-22T19:41:26.795479+0000","last_peered":"2024-02-22T19:41:26.795479+0000","last_clean":"2024-02-22T19:41:26.795479+0000","last_became_active":"2024-02-22T19:39:22.251614+0000","last_became_peered":"2024-02-22T19:39:22.251614+0000","last_unstale":"2024-02-22T19:41:26.795479+0000","last_undegraded":"2024-02-22T19:41:26.795479+0000","last_fullsized":"2024-02-22T19:41:26.795479+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"314'7569","reported_seq":7606,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.338900+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:41:34.338900+0000","last_peered":"2024-02-22T19:41:34.338900+0000","last_clean":"2024-02-22T19:41:34.338900+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:41:34.338900+0000","last_undegraded":"2024-02-22T19:41:34.338900+0000","last_fullsized":"2024-02-22T19:41:34.338900+0000","mapping_epoch":285,"log_start":"300'3737","ondisk_log_start":"300'3737","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3832,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":496041984,"num_objects":7569,"num_object_clones":0,"num_object_copies":22707,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7569,"num_write_kb":484416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":337,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806114+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:41:26.806114+0000","last_peered":"2024-02-22T19:41:26.806114+0000","last_clean":"2024-02-22T19:41:26.806114+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:41:26.806114+0000","last_undegraded":"2024-02-22T19:41:26.806114+0000","last_fullsized":"2024-02-22T19:41:26.806114+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":278,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.810794+0000","last_change":"2024-02-22T19:39:47.645026+0000","last_active":"2024-02-22T19:41:26.810794+0000","last_peered":"2024-02-22T19:41:26.810794+0000","last_clean":"2024-02-22T19:41:26.810794+0000","last_became_active":"2024-02-22T19:29:07.197487+0000","last_became_peered":"2024-02-22T19:29:07.197487+0000","last_unstale":"2024-02-22T19:41:26.810794+0000","last_undegraded":"2024-02-22T19:41:26.810794+0000","last_fullsized":"2024-02-22T19:41:26.810794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":291,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796387+0000","last_change":"2024-02-22T19:41:22.216645+0000","last_active":"2024-02-22T19:41:26.796387+0000","last_peered":"2024-02-22T19:41:26.796387+0000","last_clean":"2024-02-22T19:41:26.796387+0000","last_became_active":"2024-02-22T19:29:01.003244+0000","last_became_peered":"2024-02-22T19:29:01.003244+0000","last_unstale":"2024-02-22T19:41:26.796387+0000","last_undegraded":"2024-02-22T19:41:26.796387+0000","last_fullsized":"2024-02-22T19:41:26.796387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":104,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:31.395953+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:41:31.395953+0000","last_peered":"2024-02-22T19:41:31.395953+0000","last_clean":"2024-02-22T19:41:31.395953+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:41:31.395953+0000","last_undegraded":"2024-02-22T19:41:31.395953+0000","last_fullsized":"2024-02-22T19:41:31.395953+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":388,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795810+0000","last_change":"2024-02-22T19:41:20.253383+0000","last_active":"2024-02-22T19:41:26.795810+0000","last_peered":"2024-02-22T19:41:26.795810+0000","last_clean":"2024-02-22T19:41:26.795810+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:41:26.795810+0000","last_undegraded":"2024-02-22T19:41:26.795810+0000","last_fullsized":"2024-02-22T19:41:26.795810+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:41:20.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:20.253277+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":331,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.438109+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:41:34.438109+0000","last_peered":"2024-02-22T19:41:34.438109+0000","last_clean":"2024-02-22T19:41:34.438109+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:41:34.438109+0000","last_undegraded":"2024-02-22T19:41:34.438109+0000","last_fullsized":"2024-02-22T19:41:34.438109+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"314'7733","reported_seq":7770,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.646558+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:41:34.646558+0000","last_peered":"2024-02-22T19:41:34.646558+0000","last_clean":"2024-02-22T19:41:34.646558+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:41:34.646558+0000","last_undegraded":"2024-02-22T19:41:34.646558+0000","last_fullsized":"2024-02-22T19:41:34.646558+0000","mapping_epoch":285,"log_start":"300'4000","ondisk_log_start":"300'4000","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3733,"ondisk_log_size":3733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":506789888,"num_objects":7733,"num_object_clones":0,"num_object_copies":23199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7733,"num_write_kb":494912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":120,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.814314+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:41:26.814314+0000","last_peered":"2024-02-22T19:41:26.814314+0000","last_clean":"2024-02-22T19:41:26.814314+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:41:26.814314+0000","last_undegraded":"2024-02-22T19:41:26.814314+0000","last_fullsized":"2024-02-22T19:41:26.814314+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":328,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803974+0000","last_change":"2024-02-22T19:40:47.591722+0000","last_active":"2024-02-22T19:41:26.803974+0000","last_peered":"2024-02-22T19:41:26.803974+0000","last_clean":"2024-02-22T19:41:26.803974+0000","last_became_active":"2024-02-22T19:39:22.252166+0000","last_became_peered":"2024-02-22T19:39:22.252166+0000","last_unstale":"2024-02-22T19:41:26.803974+0000","last_undegraded":"2024-02-22T19:41:26.803974+0000","last_fullsized":"2024-02-22T19:41:26.803974+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":296,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.808103+0000","last_change":"2024-02-22T19:39:47.705812+0000","last_active":"2024-02-22T19:41:26.808103+0000","last_peered":"2024-02-22T19:41:26.808103+0000","last_clean":"2024-02-22T19:41:26.808103+0000","last_became_active":"2024-02-22T19:37:57.180638+0000","last_became_peered":"2024-02-22T19:37:57.180638+0000","last_unstale":"2024-02-22T19:41:26.808103+0000","last_undegraded":"2024-02-22T19:41:26.808103+0000","last_fullsized":"2024-02-22T19:41:26.808103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":407,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803073+0000","last_change":"2024-02-22T19:41:16.790308+0000","last_active":"2024-02-22T19:41:26.803073+0000","last_peered":"2024-02-22T19:41:26.803073+0000","last_clean":"2024-02-22T19:41:26.803073+0000","last_became_active":"2024-02-22T19:39:22.256569+0000","last_became_peered":"2024-02-22T19:39:22.256569+0000","last_unstale":"2024-02-22T19:41:26.803073+0000","last_undegraded":"2024-02-22T19:41:26.803073+0000","last_fullsized":"2024-02-22T19:41:26.803073+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":298,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.805871+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:41:26.805871+0000","last_peered":"2024-02-22T19:41:26.805871+0000","last_clean":"2024-02-22T19:41:26.805871+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:41:26.805871+0000","last_undegraded":"2024-02-22T19:41:26.805871+0000","last_fullsized":"2024-02-22T19:41:26.805871+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"314'7699","reported_seq":7736,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.299509+0000","last_change":"2024-02-22T19:40:01.098193+0000","last_active":"2024-02-22T19:41:34.299509+0000","last_peered":"2024-02-22T19:41:34.299509+0000","last_clean":"2024-02-22T19:41:34.299509+0000","last_became_active":"2024-02-22T19:40:01.097792+0000","last_became_peered":"2024-02-22T19:40:01.097792+0000","last_unstale":"2024-02-22T19:41:34.299509+0000","last_undegraded":"2024-02-22T19:41:34.299509+0000","last_fullsized":"2024-02-22T19:41:34.299509+0000","mapping_epoch":285,"log_start":"298'3200","ondisk_log_start":"298'3200","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4499,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":504561664,"num_objects":7699,"num_object_clones":0,"num_object_copies":23097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7699,"num_write_kb":492736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":182,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.794194+0000","last_change":"2024-02-22T19:40:41.405503+0000","last_active":"2024-02-22T19:41:26.794194+0000","last_peered":"2024-02-22T19:41:26.794194+0000","last_clean":"2024-02-22T19:41:26.794194+0000","last_became_active":"2024-02-22T19:38:47.167994+0000","last_became_peered":"2024-02-22T19:38:47.167994+0000","last_unstale":"2024-02-22T19:41:26.794194+0000","last_undegraded":"2024-02-22T19:41:26.794194+0000","last_fullsized":"2024-02-22T19:41:26.794194+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":298,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796984+0000","last_change":"2024-02-22T19:39:50.386984+0000","last_active":"2024-02-22T19:41:26.796984+0000","last_peered":"2024-02-22T19:41:26.796984+0000","last_clean":"2024-02-22T19:41:26.796984+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:41:26.796984+0000","last_undegraded":"2024-02-22T19:41:26.796984+0000","last_fullsized":"2024-02-22T19:41:26.796984+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.386916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.386916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":88,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807274+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:41:26.807274+0000","last_peered":"2024-02-22T19:41:26.807274+0000","last_clean":"2024-02-22T19:41:26.807274+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:41:26.807274+0000","last_undegraded":"2024-02-22T19:41:26.807274+0000","last_fullsized":"2024-02-22T19:41:26.807274+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1077,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:32.595225+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:41:32.595225+0000","last_peered":"2024-02-22T19:41:32.595225+0000","last_clean":"2024-02-22T19:41:32.595225+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:41:32.595225+0000","last_undegraded":"2024-02-22T19:41:32.595225+0000","last_fullsized":"2024-02-22T19:41:32.595225+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":736,"num_read_kb":731,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":120,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796099+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:41:26.796099+0000","last_peered":"2024-02-22T19:41:26.796099+0000","last_clean":"2024-02-22T19:41:26.796099+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:41:26.796099+0000","last_undegraded":"2024-02-22T19:41:26.796099+0000","last_fullsized":"2024-02-22T19:41:26.796099+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":165,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.814385+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:41:26.814385+0000","last_peered":"2024-02-22T19:41:26.814385+0000","last_clean":"2024-02-22T19:41:26.814385+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:41:26.814385+0000","last_undegraded":"2024-02-22T19:41:26.814385+0000","last_fullsized":"2024-02-22T19:41:26.814385+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"314'7566","reported_seq":7603,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.327581+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:41:34.327581+0000","last_peered":"2024-02-22T19:41:34.327581+0000","last_clean":"2024-02-22T19:41:34.327581+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:41:34.327581+0000","last_undegraded":"2024-02-22T19:41:34.327581+0000","last_fullsized":"2024-02-22T19:41:34.327581+0000","mapping_epoch":285,"log_start":"298'3500","ondisk_log_start":"298'3500","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4066,"ondisk_log_size":4066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":495845376,"num_objects":7566,"num_object_clones":0,"num_object_copies":22698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7566,"num_write_kb":484224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":228,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803251+0000","last_change":"2024-02-22T19:41:25.816464+0000","last_active":"2024-02-22T19:41:26.803251+0000","last_peered":"2024-02-22T19:41:26.803251+0000","last_clean":"2024-02-22T19:41:26.803251+0000","last_became_active":"2024-02-22T19:39:22.256349+0000","last_became_peered":"2024-02-22T19:39:22.256349+0000","last_unstale":"2024-02-22T19:41:26.803251+0000","last_undegraded":"2024-02-22T19:41:26.803251+0000","last_fullsized":"2024-02-22T19:41:26.803251+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":280,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.793998+0000","last_change":"2024-02-22T19:40:40.450558+0000","last_active":"2024-02-22T19:41:26.793998+0000","last_peered":"2024-02-22T19:41:26.793998+0000","last_clean":"2024-02-22T19:41:26.793998+0000","last_became_active":"2024-02-22T19:29:07.197833+0000","last_became_peered":"2024-02-22T19:29:07.197833+0000","last_unstale":"2024-02-22T19:41:26.793998+0000","last_undegraded":"2024-02-22T19:41:26.793998+0000","last_fullsized":"2024-02-22T19:41:26.793998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":304,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796664+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:41:26.796664+0000","last_peered":"2024-02-22T19:41:26.796664+0000","last_clean":"2024-02-22T19:41:26.796664+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:41:26.796664+0000","last_undegraded":"2024-02-22T19:41:26.796664+0000","last_fullsized":"2024-02-22T19:41:26.796664+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":125,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806122+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:41:26.806122+0000","last_peered":"2024-02-22T19:41:26.806122+0000","last_clean":"2024-02-22T19:41:26.806122+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:41:26.806122+0000","last_undegraded":"2024-02-22T19:41:26.806122+0000","last_fullsized":"2024-02-22T19:41:26.806122+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":330,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.802903+0000","last_change":"2024-02-22T19:39:49.629194+0000","last_active":"2024-02-22T19:41:26.802903+0000","last_peered":"2024-02-22T19:41:26.802903+0000","last_clean":"2024-02-22T19:41:26.802903+0000","last_became_active":"2024-02-22T19:39:22.263999+0000","last_became_peered":"2024-02-22T19:39:22.263999+0000","last_unstale":"2024-02-22T19:41:26.802903+0000","last_undegraded":"2024-02-22T19:41:26.802903+0000","last_fullsized":"2024-02-22T19:41:26.802903+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"314'7697","reported_seq":7733,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.387750+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:41:34.387750+0000","last_peered":"2024-02-22T19:41:34.387750+0000","last_clean":"2024-02-22T19:41:34.387750+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:41:34.387750+0000","last_undegraded":"2024-02-22T19:41:34.387750+0000","last_fullsized":"2024-02-22T19:41:34.387750+0000","mapping_epoch":285,"log_start":"300'3900","ondisk_log_start":"300'3900","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3797,"ondisk_log_size":3797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":504430592,"num_objects":7697,"num_object_clones":0,"num_object_copies":23091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7697,"num_write_kb":492608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":183,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803636+0000","last_change":"2024-02-22T19:39:42.557957+0000","last_active":"2024-02-22T19:41:26.803636+0000","last_peered":"2024-02-22T19:41:26.803636+0000","last_clean":"2024-02-22T19:41:26.803636+0000","last_became_active":"2024-02-22T19:35:58.422993+0000","last_became_peered":"2024-02-22T19:35:58.422993+0000","last_unstale":"2024-02-22T19:41:26.803636+0000","last_undegraded":"2024-02-22T19:41:26.803636+0000","last_fullsized":"2024-02-22T19:41:26.803636+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:42.557792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.557792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":100,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807242+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:41:26.807242+0000","last_peered":"2024-02-22T19:41:26.807242+0000","last_clean":"2024-02-22T19:41:26.807242+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:41:26.807242+0000","last_undegraded":"2024-02-22T19:41:26.807242+0000","last_fullsized":"2024-02-22T19:41:26.807242+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":111,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:30.423493+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:41:30.423493+0000","last_peered":"2024-02-22T19:41:30.423493+0000","last_clean":"2024-02-22T19:41:30.423493+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:41:30.423493+0000","last_undegraded":"2024-02-22T19:41:30.423493+0000","last_fullsized":"2024-02-22T19:41:30.423493+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":419,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796545+0000","last_change":"2024-02-22T19:39:40.321965+0000","last_active":"2024-02-22T19:41:26.796545+0000","last_peered":"2024-02-22T19:41:26.796545+0000","last_clean":"2024-02-22T19:41:26.796545+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:41:26.796545+0000","last_undegraded":"2024-02-22T19:41:26.796545+0000","last_fullsized":"2024-02-22T19:41:26.796545+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:40.321851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:40.321851+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"268'13","reported_seq":170,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.317662+0000","last_change":"2024-02-22T19:41:24.850297+0000","last_active":"2024-02-22T19:41:29.317662+0000","last_peered":"2024-02-22T19:41:29.317662+0000","last_clean":"2024-02-22T19:41:29.317662+0000","last_became_active":"2024-02-22T19:39:22.252945+0000","last_became_peered":"2024-02-22T19:39:22.252945+0000","last_unstale":"2024-02-22T19:41:29.317662+0000","last_undegraded":"2024-02-22T19:41:29.317662+0000","last_fullsized":"2024-02-22T19:41:29.317662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"314'7662","reported_seq":7699,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.646203+0000","last_change":"2024-02-22T19:40:00.882269+0000","last_active":"2024-02-22T19:41:34.646203+0000","last_peered":"2024-02-22T19:41:34.646203+0000","last_clean":"2024-02-22T19:41:34.646203+0000","last_became_active":"2024-02-22T19:40:00.881943+0000","last_became_peered":"2024-02-22T19:40:00.881943+0000","last_unstale":"2024-02-22T19:41:34.646203+0000","last_undegraded":"2024-02-22T19:41:34.646203+0000","last_fullsized":"2024-02-22T19:41:34.646203+0000","mapping_epoch":285,"log_start":"300'4000","ondisk_log_start":"300'4000","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3662,"ondisk_log_size":3662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":502136832,"num_objects":7662,"num_object_clones":0,"num_object_copies":22986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7662,"num_write_kb":490368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":182,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.816668+0000","last_change":"2024-02-22T19:40:02.507342+0000","last_active":"2024-02-22T19:41:26.816668+0000","last_peered":"2024-02-22T19:41:26.816668+0000","last_clean":"2024-02-22T19:41:26.816668+0000","last_became_active":"2024-02-22T19:40:02.507127+0000","last_became_peered":"2024-02-22T19:40:02.507127+0000","last_unstale":"2024-02-22T19:41:26.816668+0000","last_undegraded":"2024-02-22T19:41:26.816668+0000","last_fullsized":"2024-02-22T19:41:26.816668+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":281,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.806964+0000","last_change":"2024-02-22T19:40:31.663872+0000","last_active":"2024-02-22T19:41:26.806964+0000","last_peered":"2024-02-22T19:41:26.806964+0000","last_clean":"2024-02-22T19:41:26.806964+0000","last_became_active":"2024-02-22T19:29:07.199227+0000","last_became_peered":"2024-02-22T19:29:07.199227+0000","last_unstale":"2024-02-22T19:41:26.806964+0000","last_undegraded":"2024-02-22T19:41:26.806964+0000","last_fullsized":"2024-02-22T19:41:26.806964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":109,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795881+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:41:26.795881+0000","last_peered":"2024-02-22T19:41:26.795881+0000","last_clean":"2024-02-22T19:41:26.795881+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:41:26.795881+0000","last_undegraded":"2024-02-22T19:41:26.795881+0000","last_fullsized":"2024-02-22T19:41:26.795881+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":401,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.800025+0000","last_change":"2024-02-22T19:39:46.430095+0000","last_active":"2024-02-22T19:41:26.800025+0000","last_peered":"2024-02-22T19:41:26.800025+0000","last_clean":"2024-02-22T19:41:26.800025+0000","last_became_active":"2024-02-22T19:39:22.248416+0000","last_became_peered":"2024-02-22T19:39:22.248416+0000","last_unstale":"2024-02-22T19:41:26.800025+0000","last_undegraded":"2024-02-22T19:41:26.800025+0000","last_fullsized":"2024-02-22T19:41:26.800025+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":474,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.317770+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:41:29.317770+0000","last_peered":"2024-02-22T19:41:29.317770+0000","last_clean":"2024-02-22T19:41:29.317770+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:41:29.317770+0000","last_undegraded":"2024-02-22T19:41:29.317770+0000","last_fullsized":"2024-02-22T19:41:29.317770+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"314'7573","reported_seq":7610,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.298463+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:41:34.298463+0000","last_peered":"2024-02-22T19:41:34.298463+0000","last_clean":"2024-02-22T19:41:34.298463+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:41:34.298463+0000","last_undegraded":"2024-02-22T19:41:34.298463+0000","last_fullsized":"2024-02-22T19:41:34.298463+0000","mapping_epoch":285,"log_start":"298'3500","ondisk_log_start":"298'3500","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4073,"ondisk_log_size":4073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":496304128,"num_objects":7573,"num_object_clones":0,"num_object_copies":22719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7573,"num_write_kb":484672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795442+0000","last_change":"2024-02-22T19:40:33.048538+0000","last_active":"2024-02-22T19:41:26.795442+0000","last_peered":"2024-02-22T19:41:26.795442+0000","last_clean":"2024-02-22T19:41:26.795442+0000","last_became_active":"2024-02-22T19:38:47.167514+0000","last_became_peered":"2024-02-22T19:38:47.167514+0000","last_unstale":"2024-02-22T19:41:26.795442+0000","last_undegraded":"2024-02-22T19:41:26.795442+0000","last_fullsized":"2024-02-22T19:41:26.795442+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:33.048434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.048434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":83,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.808323+0000","last_change":"2024-02-22T19:41:16.678406+0000","last_active":"2024-02-22T19:41:26.808323+0000","last_peered":"2024-02-22T19:41:26.808323+0000","last_clean":"2024-02-22T19:41:26.808323+0000","last_became_active":"2024-02-22T19:38:47.170472+0000","last_became_peered":"2024-02-22T19:38:47.170472+0000","last_unstale":"2024-02-22T19:41:26.808323+0000","last_undegraded":"2024-02-22T19:41:26.808323+0000","last_fullsized":"2024-02-22T19:41:26.808323+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":310,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.813285+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:41:26.813285+0000","last_peered":"2024-02-22T19:41:26.813285+0000","last_clean":"2024-02-22T19:41:26.813285+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:41:26.813285+0000","last_undegraded":"2024-02-22T19:41:26.813285+0000","last_fullsized":"2024-02-22T19:41:26.813285+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":414,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807606+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:41:26.807606+0000","last_peered":"2024-02-22T19:41:26.807606+0000","last_clean":"2024-02-22T19:41:26.807606+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:41:26.807606+0000","last_undegraded":"2024-02-22T19:41:26.807606+0000","last_fullsized":"2024-02-22T19:41:26.807606+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":101,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807369+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:41:26.807369+0000","last_peered":"2024-02-22T19:41:26.807369+0000","last_clean":"2024-02-22T19:41:26.807369+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:41:26.807369+0000","last_undegraded":"2024-02-22T19:41:26.807369+0000","last_fullsized":"2024-02-22T19:41:26.807369+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"314'7631","reported_seq":7668,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.642218+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:41:34.642218+0000","last_peered":"2024-02-22T19:41:34.642218+0000","last_clean":"2024-02-22T19:41:34.642218+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:41:34.642218+0000","last_undegraded":"2024-02-22T19:41:34.642218+0000","last_fullsized":"2024-02-22T19:41:34.642218+0000","mapping_epoch":285,"log_start":"300'3900","ondisk_log_start":"300'3900","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3731,"ondisk_log_size":3731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":500105216,"num_objects":7631,"num_object_clones":0,"num_object_copies":22893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7631,"num_write_kb":488384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":114,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.800882+0000","last_change":"2024-02-22T19:40:36.433351+0000","last_active":"2024-02-22T19:41:26.800882+0000","last_peered":"2024-02-22T19:41:26.800882+0000","last_clean":"2024-02-22T19:41:26.800882+0000","last_became_active":"2024-02-22T19:37:57.189767+0000","last_became_peered":"2024-02-22T19:37:57.189767+0000","last_unstale":"2024-02-22T19:41:26.800882+0000","last_undegraded":"2024-02-22T19:41:26.800882+0000","last_fullsized":"2024-02-22T19:41:26.800882+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":138,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.808341+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:41:26.808341+0000","last_peered":"2024-02-22T19:41:26.808341+0000","last_clean":"2024-02-22T19:41:26.808341+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:41:26.808341+0000","last_undegraded":"2024-02-22T19:41:26.808341+0000","last_fullsized":"2024-02-22T19:41:26.808341+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":589,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.329362+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:41:29.329362+0000","last_peered":"2024-02-22T19:41:29.329362+0000","last_clean":"2024-02-22T19:41:29.329362+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:41:29.329362+0000","last_undegraded":"2024-02-22T19:41:29.329362+0000","last_fullsized":"2024-02-22T19:41:29.329362+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":283,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.807524+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:41:26.807524+0000","last_peered":"2024-02-22T19:41:26.807524+0000","last_clean":"2024-02-22T19:41:26.807524+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:41:26.807524+0000","last_undegraded":"2024-02-22T19:41:26.807524+0000","last_fullsized":"2024-02-22T19:41:26.807524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":27,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.853728+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:41:26.853728+0000","last_peered":"2024-02-22T19:41:26.853728+0000","last_clean":"2024-02-22T19:41:26.853728+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:41:26.853728+0000","last_undegraded":"2024-02-22T19:41:26.853728+0000","last_fullsized":"2024-02-22T19:41:26.853728+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"314'7655","reported_seq":7692,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.322778+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:41:34.322778+0000","last_peered":"2024-02-22T19:41:34.322778+0000","last_clean":"2024-02-22T19:41:34.322778+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:41:34.322778+0000","last_undegraded":"2024-02-22T19:41:34.322778+0000","last_fullsized":"2024-02-22T19:41:34.322778+0000","mapping_epoch":285,"log_start":"298'3600","ondisk_log_start":"298'3600","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4055,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":501678080,"num_objects":7655,"num_object_clones":0,"num_object_copies":22965,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":489920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":290,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.819212+0000","last_change":"2024-02-22T19:40:32.668512+0000","last_active":"2024-02-22T19:41:26.819212+0000","last_peered":"2024-02-22T19:41:26.819212+0000","last_clean":"2024-02-22T19:41:26.819212+0000","last_became_active":"2024-02-22T19:37:57.179488+0000","last_became_peered":"2024-02-22T19:37:57.179488+0000","last_unstale":"2024-02-22T19:41:26.819212+0000","last_undegraded":"2024-02-22T19:41:26.819212+0000","last_fullsized":"2024-02-22T19:41:26.819212+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"251'17","reported_seq":308,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:29.318318+0000","last_change":"2024-02-22T19:41:28.401645+0000","last_active":"2024-02-22T19:41:29.318318+0000","last_peered":"2024-02-22T19:41:29.318318+0000","last_clean":"2024-02-22T19:41:29.318318+0000","last_became_active":"2024-02-22T19:38:50.145632+0000","last_became_peered":"2024-02-22T19:38:50.145632+0000","last_unstale":"2024-02-22T19:41:29.318318+0000","last_undegraded":"2024-02-22T19:41:29.318318+0000","last_fullsized":"2024-02-22T19:41:29.318318+0000","mapping_epoch":250,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":88,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.794817+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:41:26.794817+0000","last_peered":"2024-02-22T19:41:26.794817+0000","last_clean":"2024-02-22T19:41:26.794817+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:41:26.794817+0000","last_undegraded":"2024-02-22T19:41:26.794817+0000","last_fullsized":"2024-02-22T19:41:26.794817+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":137,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.803399+0000","last_change":"2024-02-22T19:39:43.589553+0000","last_active":"2024-02-22T19:41:26.803399+0000","last_peered":"2024-02-22T19:41:26.803399+0000","last_clean":"2024-02-22T19:41:26.803399+0000","last_became_active":"2024-02-22T19:38:47.183168+0000","last_became_peered":"2024-02-22T19:38:47.183168+0000","last_unstale":"2024-02-22T19:41:26.803399+0000","last_undegraded":"2024-02-22T19:41:26.803399+0000","last_fullsized":"2024-02-22T19:41:26.803399+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"314'7667","reported_seq":7704,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.433075+0000","last_change":"2024-02-22T19:40:01.097839+0000","last_active":"2024-02-22T19:41:34.433075+0000","last_peered":"2024-02-22T19:41:34.433075+0000","last_clean":"2024-02-22T19:41:34.433075+0000","last_became_active":"2024-02-22T19:40:01.097523+0000","last_became_peered":"2024-02-22T19:40:01.097523+0000","last_unstale":"2024-02-22T19:41:34.433075+0000","last_undegraded":"2024-02-22T19:41:34.433075+0000","last_fullsized":"2024-02-22T19:41:34.433075+0000","mapping_epoch":285,"log_start":"298'3600","ondisk_log_start":"298'3600","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":4067,"ondisk_log_size":4067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":502464512,"num_objects":7667,"num_object_clones":0,"num_object_copies":23001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7667,"num_write_kb":490688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":111,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.801069+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:41:26.801069+0000","last_peered":"2024-02-22T19:41:26.801069+0000","last_clean":"2024-02-22T19:41:26.801069+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:41:26.801069+0000","last_undegraded":"2024-02-22T19:41:26.801069+0000","last_fullsized":"2024-02-22T19:41:26.801069+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":294,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795723+0000","last_change":"2024-02-22T19:40:34.092106+0000","last_active":"2024-02-22T19:41:26.795723+0000","last_peered":"2024-02-22T19:41:26.795723+0000","last_clean":"2024-02-22T19:41:26.795723+0000","last_became_active":"2024-02-22T19:38:47.171941+0000","last_became_peered":"2024-02-22T19:38:47.171941+0000","last_unstale":"2024-02-22T19:41:26.795723+0000","last_undegraded":"2024-02-22T19:41:26.795723+0000","last_fullsized":"2024-02-22T19:41:26.795723+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":336,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.796911+0000","last_change":"2024-02-22T19:39:49.403011+0000","last_active":"2024-02-22T19:41:26.796911+0000","last_peered":"2024-02-22T19:41:26.796911+0000","last_clean":"2024-02-22T19:41:26.796911+0000","last_became_active":"2024-02-22T19:39:22.244710+0000","last_became_peered":"2024-02-22T19:39:22.244710+0000","last_unstale":"2024-02-22T19:41:26.796911+0000","last_undegraded":"2024-02-22T19:41:26.796911+0000","last_fullsized":"2024-02-22T19:41:26.796911+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":296,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.802765+0000","last_change":"2024-02-22T19:39:45.615429+0000","last_active":"2024-02-22T19:41:26.802765+0000","last_peered":"2024-02-22T19:41:26.802765+0000","last_clean":"2024-02-22T19:41:26.802765+0000","last_became_active":"2024-02-22T19:38:47.164172+0000","last_became_peered":"2024-02-22T19:38:47.164172+0000","last_unstale":"2024-02-22T19:41:26.802765+0000","last_undegraded":"2024-02-22T19:41:26.802765+0000","last_fullsized":"2024-02-22T19:41:26.802765+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":162,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795469+0000","last_change":"2024-02-22T19:40:35.038302+0000","last_active":"2024-02-22T19:41:26.795469+0000","last_peered":"2024-02-22T19:41:26.795469+0000","last_clean":"2024-02-22T19:41:26.795469+0000","last_became_active":"2024-02-22T19:38:47.181392+0000","last_became_peered":"2024-02-22T19:38:47.181392+0000","last_unstale":"2024-02-22T19:41:26.795469+0000","last_undegraded":"2024-02-22T19:41:26.795469+0000","last_fullsized":"2024-02-22T19:41:26.795469+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"314'7562","reported_seq":7599,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:34.614449+0000","last_change":"2024-02-22T19:40:00.881912+0000","last_active":"2024-02-22T19:41:34.614449+0000","last_peered":"2024-02-22T19:41:34.614449+0000","last_clean":"2024-02-22T19:41:34.614449+0000","last_became_active":"2024-02-22T19:40:00.881492+0000","last_became_peered":"2024-02-22T19:40:00.881492+0000","last_unstale":"2024-02-22T19:41:34.614449+0000","last_undegraded":"2024-02-22T19:41:34.614449+0000","last_fullsized":"2024-02-22T19:41:34.614449+0000","mapping_epoch":285,"log_start":"300'3900","ondisk_log_start":"300'3900","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3662,"ondisk_log_size":3662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":495583232,"num_objects":7562,"num_object_clones":0,"num_object_copies":22686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7562,"num_write_kb":483968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":301,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.804445+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:41:26.804445+0000","last_peered":"2024-02-22T19:41:26.804445+0000","last_clean":"2024-02-22T19:41:26.804445+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:41:26.804445+0000","last_undegraded":"2024-02-22T19:41:26.804445+0000","last_fullsized":"2024-02-22T19:41:26.804445+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":127,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.795702+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:41:26.795702+0000","last_peered":"2024-02-22T19:41:26.795702+0000","last_clean":"2024-02-22T19:41:26.795702+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:41:26.795702+0000","last_undegraded":"2024-02-22T19:41:26.795702+0000","last_fullsized":"2024-02-22T19:41:26.795702+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":513,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:32.453299+0000","last_change":"2024-02-22T19:41:32.453253+0000","last_active":"2024-02-22T19:41:32.453299+0000","last_peered":"2024-02-22T19:41:32.453299+0000","last_clean":"2024-02-22T19:41:32.453299+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:41:32.453299+0000","last_undegraded":"2024-02-22T19:41:32.453299+0000","last_fullsized":"2024-02-22T19:41:32.453299+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":344,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-02-22T19:41:30.613764+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:41:30.613764+0000","last_peered":"2024-02-22T19:41:30.613764+0000","last_clean":"2024-02-22T19:41:30.613764+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:41:30.613764+0000","last_undegraded":"2024-02-22T19:41:30.613764+0000","last_fullsized":"2024-02-22T19:41:30.613764+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":137,"reported_epoch":313,"state":"active+clean","last_fresh":"2024-02-22T19:41:26.819314+0000","last_change":"2024-02-22T19:39:53.646743+0000","last_active":"2024-02-22T19:41:26.819314+0000","last_peered":"2024-02-22T19:41:26.819314+0000","last_clean":"2024-02-22T19:41:26.819314+0000","last_became_active":"2024-02-22T19:38:47.169571+0000","last_became_peered":"2024-02-22T19:38:47.169571+0000","last_unstale":"2024-02-22T19:41:26.819314+0000","last_undegraded":"2024-02-22T19:41:26.819314+0000","last_fullsized":"2024-02-22T19:41:26.819314+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":8024555520,"num_objects":122445,"num_object_clones":0,"num_object_copies":367335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122445,"num_write_kb":7836480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24075894784,"data_stored":24075894784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62684,"ondisk_log_size":62684,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":736,"num_read_kb":731,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":6257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1109,"num_read_kb":866,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":7282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266842,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5155344,"kb_used_data":4845832,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":88589808,"statfs":{"total":95995035648,"available":90715963392,"internally_reserved":0,"allocated":4962131968,"data_stored":4961206634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8400000000000001,"5min":1.534,"15min":1.343},"min":{"1min":0.245,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":7.2919999999999998,"5min":7.2919999999999998,"15min":42.597999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.7430000000000001,"15min":1.3680000000000001},"min":{"1min":0.28199999999999997,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":7.827,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.976,"5min":1.9610000000000001,"15min":1.4379999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":7.2679999999999998,"5min":54.951000000000001,"15min":54.951000000000001},"last":4.2539999999999996},{"interface":"front","average":{"1min":1.9850000000000001,"5min":2.9449999999999998,"15min":1.8169999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.219},"max":{"1min":6.5449999999999999,"5min":57.012,"15min":57.012},"last":0.98299999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.9450000000000001,"15min":1.323},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":6.7489999999999997,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":2.4649999999999999,"5min":2.339,"15min":1.5609999999999999},"min":{"1min":0.314,"5min":0.25700000000000001,"15min":0.23599999999999999},"max":{"1min":8.4879999999999995,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":2.2959999999999998,"15min":1.5589999999999999},"min":{"1min":0.33000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":10.324,"5min":38.692999999999998,"15min":41.457000000000001},"last":1.054},{"interface":"front","average":{"1min":2.177,"5min":2.3769999999999998,"15min":1.5449999999999999},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.29599999999999999},"max":{"1min":7.21,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":0.96699999999999997,"15min":0.65700000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":6.4720000000000004,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":0.88300000000000001,"15min":0.61299999999999999},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":5.1790000000000003,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":2.028,"5min":1.089,"15min":0.69499999999999995},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":6.9039999999999999,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.44},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.032,"15min":0.71399999999999997},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.077,"15min":0.79000000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":6.5,"5min":6.5,"15min":12.199999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.02,"15min":0.74399999999999999},"min":{"1min":0.29799999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.1230000000000002,"5min":7.6390000000000002,"15min":12.067},"last":0.36399999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528479,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2731264,"kb_used_data":2434040,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":91013888,"statfs":{"total":95995035648,"available":93198221312,"internally_reserved":0,"allocated":2492456960,"data_stored":2491546645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.3160000000000001,"15min":1.0780000000000001},"min":{"1min":0.22600000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":2.9009999999999998,"5min":6.6660000000000004,"15min":17.885000000000002},"last":3.2509999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.4890000000000001,"15min":1.083},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.21199999999999999},"max":{"1min":5.1360000000000001,"5min":8.1120000000000001,"15min":8.2699999999999996},"last":2.2850000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":2.089,"15min":1.2250000000000001},"min":{"1min":0.26100000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":4.7009999999999996,"5min":12.808999999999999,"15min":12.808999999999999},"last":3.4580000000000002},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.8959999999999999,"15min":1.1559999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":6.1280000000000001,"5min":13.590999999999999,"15min":13.590999999999999},"last":2.2930000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":2.0099999999999998,"15min":1.3300000000000001},"min":{"1min":0.28499999999999998,"5min":0.253,"15min":0.249},"max":{"1min":5.266,"5min":11.198,"15min":12.859999999999999},"last":3.907},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.859,"15min":1.1699999999999999},"min":{"1min":0.315,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.3679999999999999,"5min":11.565,"15min":11.565},"last":2.3809999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.667,"15min":1.3009999999999999},"min":{"1min":0.33100000000000002,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":3.8719999999999999,"5min":10.135,"15min":38.762},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6519999999999999,"15min":1.452},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":5.1669999999999998,"5min":10.315,"15min":35.612000000000002},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65600000000000003,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.81200000000000006,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.4769999999999999,"5min":9.3949999999999996,"15min":9.3949999999999996},"last":0.34699999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:40:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":0.90000000000000002,"15min":0.60499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.1870000000000003,"5min":7.3780000000000001,"15min":7.3780000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.59799999999999998},"min":{"1min":0.35199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6989999999999998,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:41:25 2024","interfaces":[{"interface":"back","average":{"1min":2.1989999999999998,"5min":1.0269999999999999,"15min":0.70999999999999996},"min":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":22.138999999999999,"5min":22.138999999999999,"15min":22.138999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":2.1640000000000001,"5min":1.1519999999999999,"15min":0.79600000000000004},"min":{"1min":0.38400000000000001,"5min":0.315,"15min":0.315},"max":{"1min":22.678000000000001,"5min":22.678000000000001,"15min":22.678000000000001},"last":0.47399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5186288,"kb_used_data":4880808,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":88558864,"statfs":{"total":95995035648,"available":90684276736,"internally_reserved":0,"allocated":4997947392,"data_stored":4996989936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.8089999999999999,"15min":1.411},"min":{"1min":0.252,"5min":0.185,"15min":0.185},"max":{"1min":3.8490000000000002,"5min":10.6,"15min":15.613},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.9910000000000001,"15min":1.5189999999999999},"min":{"1min":0.246,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":8.157,"5min":9.1259999999999994,"15min":13.515000000000001},"last":3.9470000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:41:15 2024","interfaces":[{"interface":"back","average":{"1min":1.7310000000000001,"5min":1.784,"15min":1.554},"min":{"1min":0.221,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":9.5440000000000005,"5min":11.682,"15min":16.503},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.667,"5min":2.286,"15min":1.641},"min":{"1min":0.29399999999999998,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":4.4530000000000003,"5min":14.994,"15min":14.994},"last":3.883}]},{"osd":3,"last update":"Thu Feb 22 19:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":1.9410000000000001,"15min":1.6459999999999999},"min":{"1min":0.29499999999999998,"5min":0.253,"15min":0.214},"max":{"1min":2.5920000000000001,"5min":10.734999999999999,"15min":25.145},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.226,"5min":2.1850000000000001,"15min":1.7310000000000001},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":9.5820000000000007,"5min":9.5820000000000007,"15min":22.25},"last":4.2050000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:41:05 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.599,"15min":1.2569999999999999},"min":{"1min":0.34399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.6100000000000003,"5min":7.9390000000000001,"15min":15.542999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.232,"15min":1.0640000000000001},"min":{"1min":0.378,"5min":0.30499999999999999,"15min":0.245},"max":{"1min":2.7269999999999999,"5min":10.706,"15min":10.706},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:41:24 2024","interfaces":[{"interface":"back","average":{"1min":1.7929999999999999,"5min":1.702,"15min":1.325},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.1799999999999997,"5min":8.0579999999999998,"15min":15.587999999999999},"last":1.28},{"interface":"front","average":{"1min":1.337,"5min":1.4370000000000001,"15min":1.1879999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":4.125,"5min":5.4359999999999999,"15min":9.4719999999999995},"last":3.9159999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:40:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.46,"15min":1.1180000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":6.8970000000000002,"5min":10.631,"15min":10.631},"last":0.437},{"interface":"front","average":{"1min":1.018,"5min":1.4950000000000001,"15min":1.23},"min":{"1min":0.38600000000000001,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":3.508,"5min":8.0259999999999998,"15min":15.651999999999999},"last":1.2370000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:41:15 2024","interfaces":[{"interface":"back","average":{"1min":2.028,"5min":1.9039999999999999,"15min":1.325},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":4.4080000000000004,"5min":13.169,"15min":13.169},"last":3.839},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.355,"15min":1.147},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":3.444,"5min":10.673,"15min":10.673},"last":0.49399999999999999}]}]},{"osd":0,"up_from":0,"seq":34359738557,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411506,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2276676,"kb_used_data":1967164,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91468476,"statfs":{"total":95995035648,"available":93663719424,"internally_reserved":0,"allocated":2014375936,"data_stored":2013461423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":2.6320000000000001,"15min":1.587},"min":{"1min":0.216,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":6.46,"5min":22.338999999999999,"15min":22.338999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.508,"5min":2.952,"15min":1.8799999999999999},"min":{"1min":0.25900000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":7.2990000000000004,"5min":30.427,"15min":30.427},"last":0.77800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":2.0259999999999998,"15min":1.4770000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.2559999999999998,"5min":12.349,"15min":12.349},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":2.1509999999999998,"15min":1.3819999999999999},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.193},"max":{"1min":3.1749999999999998,"5min":13.782,"15min":13.782},"last":3.6789999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:40:49 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":2.7389999999999999,"15min":1.7809999999999999},"min":{"1min":0.29099999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.8889999999999998,"5min":22.302,"15min":30.071000000000002},"last":0.875},{"interface":"front","average":{"1min":1.294,"5min":2.827,"15min":1.9219999999999999},"min":{"1min":0.29899999999999999,"5min":0.184,"15min":0.184},"max":{"1min":7.3600000000000003,"5min":30.282,"15min":32.119},"last":0.751}]},{"osd":4,"last update":"Thu Feb 22 19:41:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.5389999999999999,"15min":1.2270000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.4399999999999999,"5min":12.43,"15min":12.43},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.901,"15min":1.143},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.253},"max":{"1min":3.3799999999999999,"5min":10.34,"15min":10.34},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:41:23 2024","interfaces":[{"interface":"back","average":{"1min":3.9079999999999999,"5min":2.3980000000000001,"15min":1.3400000000000001},"min":{"1min":0.45300000000000001,"5min":0.317,"15min":0.317},"max":{"1min":40.633000000000003,"5min":40.633000000000003,"15min":40.633000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":4.1189999999999998,"5min":2.173,"15min":1.4770000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":40.997999999999998,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:41:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.7090000000000001,"15min":1.3220000000000001},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.927,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.7729999999999999,"15min":1.141},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":2.4990000000000001,"5min":10.369999999999999,"15min":10.369999999999999},"last":0.502}]},{"osd":7,"last update":"Thu Feb 22 19:41:14 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.6619999999999999,"15min":1.3280000000000001},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.29099999999999998},"max":{"1min":2.714,"5min":12.484,"15min":12.484},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":2.0150000000000001,"15min":1.244},"min":{"1min":0.38900000000000001,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":2.137,"5min":10.539,"15min":10.539},"last":0.52100000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149868,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5024032,"kb_used_data":4426136,"kb_used_omap":0,"kb_used_meta":597888,"kb_avail":88721120,"statfs":{"total":95995035648,"available":90850426880,"internally_reserved":0,"allocated":4532363264,"data_stored":4531421773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":612237312},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":2.1789999999999998,"15min":1.6080000000000001},"min":{"1min":0.23000000000000001,"5min":0.21199999999999999,"15min":0.193},"max":{"1min":5.4960000000000004,"5min":23.48,"15min":23.48},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.96,"5min":2.681,"15min":1.9059999999999999},"min":{"1min":0.315,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":18.172000000000001,"5min":23.411999999999999,"15min":23.411999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":3.3809999999999998,"15min":2.1280000000000001},"min":{"1min":0.27200000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":18.213999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.316},{"interface":"front","average":{"1min":1.3779999999999999,"5min":2.8940000000000001,"15min":1.758},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":12.234,"5min":22.187000000000001,"15min":22.187000000000001},"last":3.1059999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:40:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":2.569,"15min":1.8220000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.105,"5min":20.105,"15min":20.105},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":2.52,"15min":1.7629999999999999},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.217},"max":{"1min":7.617,"5min":35.939999999999998,"15min":35.939999999999998},"last":1.8009999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":2.129,"15min":1.528},"min":{"1min":0.38500000000000001,"5min":0.26300000000000001,"15min":0.249},"max":{"1min":2.355,"5min":12.416,"15min":13.680999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":2.238,"15min":1.46},"min":{"1min":0.32300000000000001,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":5.1210000000000004,"5min":13.548,"15min":13.548},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:41:29 2024","interfaces":[{"interface":"back","average":{"1min":1.8420000000000001,"5min":1.75,"15min":1.3580000000000001},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":11.161,"5min":11.161,"15min":12.879},"last":1.776},{"interface":"front","average":{"1min":1.5649999999999999,"5min":2.1000000000000001,"15min":1.546},"min":{"1min":0.34699999999999998,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":4.8860000000000001,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:40:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.6559999999999999,"15min":1.2949999999999999},"min":{"1min":0.39900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.0449999999999999,"5min":7.9640000000000004,"15min":12.898999999999999},"last":1.823},{"interface":"front","average":{"1min":1.0449999999999999,"5min":2.0920000000000001,"15min":1.5589999999999999},"min":{"1min":0.45200000000000001,"5min":0.34799999999999998,"15min":0.30199999999999999},"max":{"1min":3.9910000000000001,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.40100000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:41:05 2024","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":2.214,"15min":1.488},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":5.0670000000000002,"5min":13.483000000000001,"15min":13.483000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.9490000000000001,"5min":1.841,"15min":1.3779999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.093,"5min":7.9909999999999997,"15min":12.846},"last":5.7149999999999999}]}]},{"osd":4,"up_from":0,"seq":137438953642,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724708,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5196744,"kb_used_data":4895424,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":88548408,"statfs":{"total":95995035648,"available":90673569792,"internally_reserved":0,"allocated":5012914176,"data_stored":5011995879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.3560000000000001,"15min":1.151},"min":{"1min":0.27800000000000002,"5min":0.23999999999999999,"15min":0.222},"max":{"1min":1.9139999999999999,"5min":17.684000000000001,"15min":17.684000000000001},"last":1.339},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.605,"15min":1.2250000000000001},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":3.919,"5min":17.402999999999999,"15min":17.402999999999999},"last":1.194}]},{"osd":1,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.702,"15min":1.212},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.22500000000000001},"max":{"1min":2.0089999999999999,"5min":29.390999999999998,"15min":29.390999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.194},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.218},"max":{"1min":2.7770000000000001,"5min":29.367000000000001,"15min":29.367000000000001},"last":1.171}]},{"osd":2,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.7230000000000001,"15min":1.2470000000000001},"min":{"1min":0.36299999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.1779999999999999,"5min":11.367000000000001,"15min":14.613},"last":1.6080000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.722,"15min":1.3029999999999999},"min":{"1min":0.33400000000000002,"5min":0.27000000000000002,"15min":0.26600000000000001},"max":{"1min":3.8130000000000002,"5min":16.099,"15min":16.099},"last":1.482}]},{"osd":3,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.593,"15min":1.395},"min":{"1min":0.41099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.1399999999999997,"5min":7.0309999999999997,"15min":13.366},"last":1.5369999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.5349999999999999,"15min":1.3520000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.8839999999999999,"5min":8.6069999999999993,"15min":10.16},"last":1.512}]},{"osd":4,"last update":"Thu Feb 22 19:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68600000000000005,"15min":0.51900000000000002},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.219},"max":{"1min":1.6879999999999999,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":1.294},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67800000000000005,"15min":0.52500000000000002},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":1.998,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.879}]},{"osd":6,"last update":"Thu Feb 22 19:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.623,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":1.853,"5min":3.246,"15min":5.6040000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.60899999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.698,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.0249999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:41:22 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.79900000000000004,"15min":0.67500000000000004},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.1449999999999996,"5min":5.1449999999999996,"15min":5.5789999999999997},"last":0.502},{"interface":"front","average":{"1min":1.7110000000000001,"5min":0.98799999999999999,"15min":0.69299999999999995},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":4.9649999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":1.21}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5010096128,"data_stored":5010096128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2015297536,"data_stored":2015297536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4531617792,"data_stored":4531617792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5015601152,"data_stored":5015601152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2507472896,"data_stored":2507472896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4995809280,"data_stored":4995809280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:41:39.755 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:39] "GET /metrics HTTP/1.1" 200 176141 "" "Prometheus/2.33.4" 2024-02-22T19:41:39.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:38.517904+0000 osd.2 (osd.2) 186 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:41:39.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:38.526252+0000 osd.2 (osd.2) 187 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:41:39.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:38.697860+0000 osd.7 (osd.7) 213 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:41:39.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:38.702583+0000 osd.7 (osd.7) 214 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:41:39.978 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:39.370833+0000 mon.a (mon.0) 1355 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:39.978 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:39.376380+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.53:0/4263833330' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:41:39.978 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:39.376434+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e317: 8 total, 8 up, 6 in 2024-02-22T19:41:40.238 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 89 16 129227 129211 90.6981 79.1875 0.0136395 0.011022 2024-02-22T19:41:40.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:38.517904+0000 osd.2 (osd.2) 186 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:41:40.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:38.526252+0000 osd.2 (osd.2) 187 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:38.697860+0000 osd.7 (osd.7) 213 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:38.702583+0000 osd.7 (osd.7) 214 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:39.370833+0000 mon.a (mon.0) 1355 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:39.376380+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.53:0/4263833330' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:39.376434+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e317: 8 total, 8 up, 6 in 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:38.517904+0000 osd.2 (osd.2) 186 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:38.526252+0000 osd.2 (osd.2) 187 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:41:40.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:38.697860+0000 osd.7 (osd.7) 213 : cluster [DBG] 5.10 scrub starts 2024-02-22T19:41:40.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:38.702583+0000 osd.7 (osd.7) 214 : cluster [DBG] 5.10 scrub ok 2024-02-22T19:41:40.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:39.370833+0000 mon.a (mon.0) 1355 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:40.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:39.376380+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.53:0/4263833330' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:41:40.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:39.376434+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e317: 8 total, 8 up, 6 in 2024-02-22T19:41:40.327 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:41:40.328 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:41:40.328 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:41:40.328 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:41:40.328 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:41:40.882 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:40.890 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:40.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:40.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:41.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:39.604598+0000 mgr.y (mgr.14535) 487 : cluster [DBG] pgmap v623: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.7 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-02-22T19:41:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:41.107 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:41.111 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:41.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:41.140 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:41:41.193 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:39.604598+0000 mgr.y (mgr.14535) 487 : cluster [DBG] pgmap v623: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.7 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-02-22T19:41:41.195 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:39.604598+0000 mgr.y (mgr.14535) 487 : cluster [DBG] pgmap v623: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.7 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-02-22T19:41:41.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:41.238 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total time run: 90.0115 2024-02-22T19:41:41.238 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total writes made: 130456 2024-02-22T19:41:41.238 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Write size: 65536 2024-02-22T19:41:41.238 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object size: 65536 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Bandwidth (MB/sec): 90.5828 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Bandwidth: 30.686 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max bandwidth (MB/sec): 207.25 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min bandwidth (MB/sec): 64.875 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average IOPS: 1449 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev IOPS: 490.976 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max IOPS: 3316 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min IOPS: 1038 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average Latency(s): 0.0110363 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Latency(s): 0.00594682 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max latency(s): 0.0951228 2024-02-22T19:41:41.239 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min latency(s): 0.00135644 2024-02-22T19:41:41.245 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Cleaning up (deleting benchmark objects) 2024-02-22T19:41:41.245 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:41.503 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:41.512 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:41.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:41.574 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:41:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:41:41.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:41:41.613 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:41.649 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:41:41.665 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:41.685 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:41:41.724 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:41:41.757 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:41:41.789 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:41:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:41.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:41.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:41.830 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:41:41.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:41.965 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:42.007 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:42.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:42.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:42.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:42.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:41.531119+0000 mon.a (mon.0) 1358 : audit [DBG] from='client.? 172.21.15.53:0/759278662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:42.167 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:42.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:41.531119+0000 mon.a (mon.0) 1358 : audit [DBG] from='client.? 172.21.15.53:0/759278662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:42.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:41.531119+0000 mon.a (mon.0) 1358 : audit [DBG] from='client.? 172.21.15.53:0/759278662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:42.616 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:41:42.927 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:41.605033+0000 mgr.y (mgr.14535) 488 : cluster [DBG] pgmap v624: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 757 B/s rd, 53 MiB/s wr, 856 op/s 2024-02-22T19:41:42.927 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:42.280540+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e318: 8 total, 8 up, 6 in 2024-02-22T19:41:43.032 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:43.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:43.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:43.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:43.180 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:43.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:41.605033+0000 mgr.y (mgr.14535) 488 : cluster [DBG] pgmap v624: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 757 B/s rd, 53 MiB/s wr, 856 op/s 2024-02-22T19:41:43.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:42.280540+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e318: 8 total, 8 up, 6 in 2024-02-22T19:41:43.289 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:43.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:43.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:41.605033+0000 mgr.y (mgr.14535) 488 : cluster [DBG] pgmap v624: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 757 B/s rd, 53 MiB/s wr, 856 op/s 2024-02-22T19:41:43.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:42.280540+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e318: 8 total, 8 up, 6 in 2024-02-22T19:41:43.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:43.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:43.476 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:43.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:43.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:43.648 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:41:43.649 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:41:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:43.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:43.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:43.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:43.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:43.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:44.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:44.404 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:41:44.414 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:44.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:44.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:44.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:44.531 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 7, 5] out_osds: [4, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:41:44.531 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:41:44.531 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [1, 2, 3, 6, 7, 5] 2024-02-22T19:41:44.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 6 2024-02-22T19:41:44.597 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:44.664 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:41:44.881 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:44.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:44.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:44.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:44.992 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:41:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:45.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:45.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:45.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:45.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:41:45.221 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:45.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:43.605314+0000 mgr.y (mgr.14535) 489 : cluster [DBG] pgmap v626: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 661 B/s rd, 47 MiB/s wr, 747 op/s 2024-02-22T19:41:45.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:43.994569+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:45.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:44.471677+0000 mon.a (mon.0) 1361 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:41:45.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:44.472694+0000 mon.a (mon.0) 1362 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:41:45.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:44.483171+0000 mon.a (mon.0) 1363 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:45.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:43.605314+0000 mgr.y (mgr.14535) 489 : cluster [DBG] pgmap v626: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 661 B/s rd, 47 MiB/s wr, 747 op/s 2024-02-22T19:41:45.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:43.994569+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:45.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:44.471677+0000 mon.a (mon.0) 1361 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:41:45.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:44.472694+0000 mon.a (mon.0) 1362 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:41:45.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:44.483171+0000 mon.a (mon.0) 1363 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:45.248 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:45.254 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:45.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:45.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:43.605314+0000 mgr.y (mgr.14535) 489 : cluster [DBG] pgmap v626: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 661 B/s rd, 47 MiB/s wr, 747 op/s 2024-02-22T19:41:45.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:43.994569+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:45.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:44.471677+0000 mon.a (mon.0) 1361 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:41:45.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:44.472694+0000 mon.a (mon.0) 1362 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:41:45.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:44.483171+0000 mon.a (mon.0) 1363 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:45.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:45.415 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:45.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:45.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:45.517 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:45.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:45.814 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:45.821 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:45.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:45.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:45.966 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:46.076 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:46.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:46.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:46.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:46.240 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:44.991695+0000 mgr.y (mgr.14535) 490 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:45.414439+0000 osd.4 (osd.4) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:46.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:45.414484+0000 osd.4 (osd.4) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:44.991695+0000 mgr.y (mgr.14535) 490 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:45.414439+0000 osd.4 (osd.4) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:46.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:45.414484+0000 osd.4 (osd.4) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:46.280 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:46.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:46.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:44.991695+0000 mgr.y (mgr.14535) 490 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:46.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:45.414439+0000 osd.4 (osd.4) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:46.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:45.414484+0000 osd.4 (osd.4) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:46.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:46.430 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:46.449 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:46.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:46.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:46.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:46.626 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:46.803 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:46.807 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:46.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:46.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:46.956 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:47.041 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:41:47.074 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:47.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:47.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:47.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:47.250 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:47.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:47.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:47.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:45.606110+0000 mgr.y (mgr.14535) 491 : cluster [DBG] pgmap v627: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.50k op/s 2024-02-22T19:41:47.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:46.023883+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e319: 8 total, 8 up, 6 in 2024-02-22T19:41:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:46.712569+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? 172.21.15.53:0/802969319' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:47.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:45.606110+0000 mgr.y (mgr.14535) 491 : cluster [DBG] pgmap v627: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.50k op/s 2024-02-22T19:41:47.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:46.023883+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e319: 8 total, 8 up, 6 in 2024-02-22T19:41:47.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:46.712569+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? 172.21.15.53:0/802969319' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:45.606110+0000 mgr.y (mgr.14535) 491 : cluster [DBG] pgmap v627: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.4 KiB/s rd, 63 MiB/s wr, 1.50k op/s 2024-02-22T19:41:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:46.023883+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e319: 8 total, 8 up, 6 in 2024-02-22T19:41:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:46.712569+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? 172.21.15.53:0/802969319' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:41:47.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:47.439 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:47.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:47.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:47.544 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:47.596 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:47.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:47.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:47.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:47.894 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:47.019354+0000 mon.a (mon.0) 1366 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:47.023456+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.53:0/802969319' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:47.023516+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e320: 8 total, 8 up, 6 in 2024-02-22T19:41:48.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:47.019354+0000 mon.a (mon.0) 1366 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:48.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:47.023456+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.53:0/802969319' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:48.450 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:47.023516+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e320: 8 total, 8 up, 6 in 2024-02-22T19:41:48.450 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:47.019354+0000 mon.a (mon.0) 1366 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:41:48.450 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:47.023456+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.53:0/802969319' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:41:48.450 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:47.023516+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e320: 8 total, 8 up, 6 in 2024-02-22T19:41:48.461 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:48.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:48.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:48.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:48.624 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:49.199 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:47.607486+0000 mgr.y (mgr.14535) 492 : cluster [DBG] pgmap v630: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1023 B/s rd, 23 MiB/s wr, 1.04k op/s 2024-02-22T19:41:49.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:47.607486+0000 mgr.y (mgr.14535) 492 : cluster [DBG] pgmap v630: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1023 B/s rd, 23 MiB/s wr, 1.04k op/s 2024-02-22T19:41:49.200 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:49.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:49.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:49.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:47.607486+0000 mgr.y (mgr.14535) 492 : cluster [DBG] pgmap v630: 148 pgs: 148 active+clean; 7.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1023 B/s rd, 23 MiB/s wr, 1.04k op/s 2024-02-22T19:41:49.347 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:49.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:49.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:49.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:49.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:41:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:49] "GET /metrics HTTP/1.1" 200 176141 "" "Prometheus/2.33.4" 2024-02-22T19:41:49.546 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:49.902 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:49.903 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:49.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:49.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:50.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:50.059 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:50.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:50.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:50.155 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:41:50.152+0000 7fe2d794e700 -1 mon.a@0(leader).osd e321 definitely_dead 0 2024-02-22T19:41:50.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:50.417 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:50.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:50.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:50.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:50.606 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:50.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:50.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:50.764 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:50.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:41:51.034 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:51.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:51.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:51.102 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.6. 2024-02-22T19:41:51.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:51.171 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:51.200 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:51.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:51.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:49.608260+0000 mgr.y (mgr.14535) 493 : cluster [DBG] pgmap v631: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.5 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:49.890141+0000 osd.3 (osd.3) 223 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:49.890192+0000 osd.3 (osd.3) 224 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:50.057672+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e321: 8 total, 8 up, 6 in 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:50.152691+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.53:0/1370050993' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:50.153922+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-22T19:41:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:50.154036+0000 mon.a (mon.0) 1371 : cluster [INF] Client client.admin marked osd.6 out, while it was still marked up 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:49.608260+0000 mgr.y (mgr.14535) 493 : cluster [DBG] pgmap v631: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.5 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:49.890141+0000 osd.3 (osd.3) 223 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:49.890192+0000 osd.3 (osd.3) 224 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:50.057672+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e321: 8 total, 8 up, 6 in 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:50.152691+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.53:0/1370050993' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:50.153922+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-22T19:41:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:50.154036+0000 mon.a (mon.0) 1371 : cluster [INF] Client client.admin marked osd.6 out, while it was still marked up 2024-02-22T19:41:51.285 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:41:51.285 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:41:51.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:51.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:49.608260+0000 mgr.y (mgr.14535) 493 : cluster [DBG] pgmap v631: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.5 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-02-22T19:41:51.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:49.890141+0000 osd.3 (osd.3) 223 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:51.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:49.890192+0000 osd.3 (osd.3) 224 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:51.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:50.057672+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e321: 8 total, 8 up, 6 in 2024-02-22T19:41:51.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:50.152691+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.53:0/1370050993' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-22T19:41:51.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:50.153922+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-22T19:41:51.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:50.154036+0000 mon.a (mon.0) 1371 : cluster [INF] Client client.admin marked osd.6 out, while it was still marked up 2024-02-22T19:41:51.334 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:51.355 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:51.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:51.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:51.484 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:51.505 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:51.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:51.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:51.635 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:51.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:41:51.884 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:51.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:51.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:51.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:52.018 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:52.038 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:52.041 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:52.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:52.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:52.180 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:52.207 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:52.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:51.072349+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-02-22T19:41:52.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:51.072400+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e322: 8 total, 8 up, 5 in 2024-02-22T19:41:52.210 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:51.072349+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-02-22T19:41:52.210 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:51.072400+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e322: 8 total, 8 up, 5 in 2024-02-22T19:41:52.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:52.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:52.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:51.072349+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-02-22T19:41:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:51.072400+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e322: 8 total, 8 up, 5 in 2024-02-22T19:41:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:52.671 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:52.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:52.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:52.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:41:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:51.608594+0000 mgr.y (mgr.14535) 494 : cluster [DBG] pgmap v634: 148 pgs: 148 active+clean; 7.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 917 B/s rd, 0 B/s wr, 1.17k op/s 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.077694+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e323: 8 total, 8 up, 5 in 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.282941+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e324: 8 total, 8 up, 5 in 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.283998+0000 osd.7 (osd.7) 215 : cluster [DBG] 5.9 starting backfill to osd.3 from (0'0,0'0] MAX to 251'13 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.287564+0000 osd.5 (osd.5) 108 : cluster [DBG] 5.f starting backfill to osd.1 from (0'0,0'0] MAX to 251'17 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.288109+0000 osd.5 (osd.5) 109 : cluster [DBG] 9.b starting backfill to osd.1 from (0'0,0'0] MAX to 321'9006 2024-02-22T19:41:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.293339+0000 osd.3 (osd.3) 225 : cluster [DBG] 9.c starting backfill to osd.5 from (0'0,0'0] MAX to 321'8883 2024-02-22T19:41:53.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.294102+0000 osd.3 (osd.3) 226 : cluster [DBG] 9.1 starting backfill to osd.2 from (0'0,0'0] MAX to 321'8842 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:51.608594+0000 mgr.y (mgr.14535) 494 : cluster [DBG] pgmap v634: 148 pgs: 148 active+clean; 7.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 917 B/s rd, 0 B/s wr, 1.17k op/s 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.077694+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e323: 8 total, 8 up, 5 in 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.282941+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e324: 8 total, 8 up, 5 in 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.283998+0000 osd.7 (osd.7) 215 : cluster [DBG] 5.9 starting backfill to osd.3 from (0'0,0'0] MAX to 251'13 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.287564+0000 osd.5 (osd.5) 108 : cluster [DBG] 5.f starting backfill to osd.1 from (0'0,0'0] MAX to 251'17 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.288109+0000 osd.5 (osd.5) 109 : cluster [DBG] 9.b starting backfill to osd.1 from (0'0,0'0] MAX to 321'9006 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.293339+0000 osd.3 (osd.3) 225 : cluster [DBG] 9.c starting backfill to osd.5 from (0'0,0'0] MAX to 321'8883 2024-02-22T19:41:53.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.294102+0000 osd.3 (osd.3) 226 : cluster [DBG] 9.1 starting backfill to osd.2 from (0'0,0'0] MAX to 321'8842 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:51.608594+0000 mgr.y (mgr.14535) 494 : cluster [DBG] pgmap v634: 148 pgs: 148 active+clean; 7.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 917 B/s rd, 0 B/s wr, 1.17k op/s 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.077694+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e323: 8 total, 8 up, 5 in 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.282941+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e324: 8 total, 8 up, 5 in 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.283998+0000 osd.7 (osd.7) 215 : cluster [DBG] 5.9 starting backfill to osd.3 from (0'0,0'0] MAX to 251'13 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.287564+0000 osd.5 (osd.5) 108 : cluster [DBG] 5.f starting backfill to osd.1 from (0'0,0'0] MAX to 251'17 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.288109+0000 osd.5 (osd.5) 109 : cluster [DBG] 9.b starting backfill to osd.1 from (0'0,0'0] MAX to 321'9006 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.293339+0000 osd.3 (osd.3) 225 : cluster [DBG] 9.c starting backfill to osd.5 from (0'0,0'0] MAX to 321'8883 2024-02-22T19:41:53.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.294102+0000 osd.3 (osd.3) 226 : cluster [DBG] 9.1 starting backfill to osd.2 from (0'0,0'0] MAX to 321'8842 2024-02-22T19:41:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:41:53.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:53.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:41:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:53.759 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:41:53.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:53.930 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.282843+0000 osd.1 (osd.1) 204 : cluster [DBG] 9.0 starting backfill to osd.5 from (0'0,0'0] MAX to 321'8940 2024-02-22T19:41:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:52.284046+0000 osd.1 (osd.1) 205 : cluster [DBG] 9.4 starting backfill to osd.5 from (0'0,0'0] MAX to 321'9023 2024-02-22T19:41:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:53.270659+0000 osd.7 (osd.7) 216 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:53.270726+0000 osd.7 (osd.7) 217 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:53.295120+0000 mon.a (mon.0) 1376 : cluster [DBG] osdmap e325: 8 total, 8 up, 5 in 2024-02-22T19:41:54.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:53.374844+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:54.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.282843+0000 osd.1 (osd.1) 204 : cluster [DBG] 9.0 starting backfill to osd.5 from (0'0,0'0] MAX to 321'8940 2024-02-22T19:41:54.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:52.284046+0000 osd.1 (osd.1) 205 : cluster [DBG] 9.4 starting backfill to osd.5 from (0'0,0'0] MAX to 321'9023 2024-02-22T19:41:54.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:53.270659+0000 osd.7 (osd.7) 216 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:54.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:53.270726+0000 osd.7 (osd.7) 217 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:54.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:53.295120+0000 mon.a (mon.0) 1376 : cluster [DBG] osdmap e325: 8 total, 8 up, 5 in 2024-02-22T19:41:54.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:53.374844+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:54.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.282843+0000 osd.1 (osd.1) 204 : cluster [DBG] 9.0 starting backfill to osd.5 from (0'0,0'0] MAX to 321'8940 2024-02-22T19:41:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:52.284046+0000 osd.1 (osd.1) 205 : cluster [DBG] 9.4 starting backfill to osd.5 from (0'0,0'0] MAX to 321'9023 2024-02-22T19:41:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:53.270659+0000 osd.7 (osd.7) 216 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:41:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:53.270726+0000 osd.7 (osd.7) 217 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:41:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:53.295120+0000 mon.a (mon.0) 1376 : cluster [DBG] osdmap e325: 8 total, 8 up, 5 in 2024-02-22T19:41:54.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:53.374844+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:41:55.002 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:41:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:41:55.356 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:41:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:41:55.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:53.611879+0000 mgr.y (mgr.14535) 495 : cluster [DBG] pgmap v638: 148 pgs: 148 active+clean; 7.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail 2024-02-22T19:41:55.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:54.299556+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e326: 8 total, 8 up, 5 in 2024-02-22T19:41:55.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:53.611879+0000 mgr.y (mgr.14535) 495 : cluster [DBG] pgmap v638: 148 pgs: 148 active+clean; 7.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail 2024-02-22T19:41:55.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:54.299556+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e326: 8 total, 8 up, 5 in 2024-02-22T19:41:55.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:53.611879+0000 mgr.y (mgr.14535) 495 : cluster [DBG] pgmap v638: 148 pgs: 148 active+clean; 7.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail 2024-02-22T19:41:55.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:54.299556+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e326: 8 total, 8 up, 5 in 2024-02-22T19:41:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:55.911 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:55.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:56.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:56.068 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:56.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:55.002213+0000 mgr.y (mgr.14535) 496 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:56.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:55.002213+0000 mgr.y (mgr.14535) 496 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:56.301 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:56.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:56.438 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:55.002213+0000 mgr.y (mgr.14535) 496 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:41:56.443 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:56.464 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:56.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:56.469 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:56.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:41:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:56.653 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:41:56.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:56.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:41:56.758 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:41:56.819 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:57.388 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:55.612764+0000 mgr.y (mgr.14535) 497 : cluster [DBG] pgmap v640: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 7.1 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 0 B/s wr, 1.09k op/s; 35729/347352 objects misplaced (10.286%); 6.8 MiB/s, 128 objects/s recovering 2024-02-22T19:41:57.388 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:41:57.052567+0000 mon.c (mon.2) 61 : audit [DBG] from='client.? 172.21.15.53:0/2618167653' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:57.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:55.612764+0000 mgr.y (mgr.14535) 497 : cluster [DBG] pgmap v640: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 7.1 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 0 B/s wr, 1.09k op/s; 35729/347352 objects misplaced (10.286%); 6.8 MiB/s, 128 objects/s recovering 2024-02-22T19:41:57.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:41:57.052567+0000 mon.c (mon.2) 61 : audit [DBG] from='client.? 172.21.15.53:0/2618167653' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:57.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:55.612764+0000 mgr.y (mgr.14535) 497 : cluster [DBG] pgmap v640: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 7.1 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 0 B/s wr, 1.09k op/s; 35729/347352 objects misplaced (10.286%); 6.8 MiB/s, 128 objects/s recovering 2024-02-22T19:41:57.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:41:57.052567+0000 mon.c (mon.2) 61 : audit [DBG] from='client.? 172.21.15.53:0/2618167653' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:41:57.690 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:41:57.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 1 1 2024-02-22T19:41:57.894 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:41:57.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:57.903 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:41:58.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:41:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:58.219 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:41:58.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:58.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:41:58.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:58.418 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:58.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:58.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:41:58.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:58.571 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:58.604 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:58.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:58.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:58.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:41:58.763 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:58.803 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:58.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:58.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:41:58.965 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:59.181 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:41:59.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:59.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:41:59.225 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:41:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:57.613182+0000 mgr.y (mgr.14535) 498 : cluster [DBG] pgmap v641: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 7.1 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 0 B/s wr, 787 op/s; 35729/347352 objects misplaced (10.286%); 4.9 MiB/s, 93 objects/s recovering 2024-02-22T19:41:59.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:41:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:57.613182+0000 mgr.y (mgr.14535) 498 : cluster [DBG] pgmap v641: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 7.1 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 0 B/s wr, 787 op/s; 35729/347352 objects misplaced (10.286%); 4.9 MiB/s, 93 objects/s recovering 2024-02-22T19:41:59.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:59.387 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:41:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:57.613182+0000 mgr.y (mgr.14535) 498 : cluster [DBG] pgmap v641: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 7.1 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 0 B/s wr, 787 op/s; 35729/347352 objects misplaced (10.286%); 4.9 MiB/s, 93 objects/s recovering 2024-02-22T19:41:59.387 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:59.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:59.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:41:59.490 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:41:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:41:59] "GET /metrics HTTP/1.1" 200 176147 "" "Prometheus/2.33.4" 2024-02-22T19:41:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:59.564 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:41:59.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:59.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:41:59.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:41:59.715 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:59.723 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:41:59.723 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:41:59.723 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:41:59.723 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:41:59.723 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:41:59.739 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:41:59.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:59.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:41:59.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:41:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:41:59.939 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:41:59.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:41:59.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:00.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:00.115 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:00.695 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:00.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:00.706 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:00.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:00.912 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:00.912 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:01.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:01.073 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:01.112 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:01.119 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:01.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:01.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:42:01.201 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:42:01.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:01.225 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:01.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:01.464 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:01.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:01.465 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:41:59.613870+0000 mgr.y (mgr.14535) 499 : cluster [DBG] pgmap v642: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.8 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 558 B/s rd, 0 B/s wr, 1.15k op/s; 33214/335175 objects misplaced (9.909%); 17 MiB/s, 289 objects/s recovering 2024-02-22T19:42:01.465 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:00.151438+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.53:0/2779524466' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:01.465 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:00.152209+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:01.465 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:41:59.613870+0000 mgr.y (mgr.14535) 499 : cluster [DBG] pgmap v642: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.8 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 558 B/s rd, 0 B/s wr, 1.15k op/s; 33214/335175 objects misplaced (9.909%); 17 MiB/s, 289 objects/s recovering 2024-02-22T19:42:01.465 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:00.151438+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.53:0/2779524466' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:01.465 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:00.152209+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:01.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:01.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:41:59.613870+0000 mgr.y (mgr.14535) 499 : cluster [DBG] pgmap v642: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.8 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 558 B/s rd, 0 B/s wr, 1.15k op/s; 33214/335175 objects misplaced (9.909%); 17 MiB/s, 289 objects/s recovering 2024-02-22T19:42:01.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:00.151438+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.53:0/2779524466' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:01.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:00.152209+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:01.641 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:01.729 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:01.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:01.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:01.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:01.892 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:02.387 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:02.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:02.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:02.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:01.143511+0000 mon.a (mon.0) 1380 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:01.163874+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:42:02.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:01.163944+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e327: 8 total, 8 up, 5 in 2024-02-22T19:42:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:01.143511+0000 mon.a (mon.0) 1380 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:01.163874+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:42:02.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:01.163944+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e327: 8 total, 8 up, 5 in 2024-02-22T19:42:02.538 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:01.143511+0000 mon.a (mon.0) 1380 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:02.538 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:01.163874+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:42:02.538 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:01.163944+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e327: 8 total, 8 up, 5 in 2024-02-22T19:42:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:02.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:02.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:02.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:02.706 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:42:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:02.757 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:02.765 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:02.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:02.797 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:02.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:02.985 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:02.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:02.988 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:03.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:03.188 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:03.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:01.614324+0000 mgr.y (mgr.14535) 500 : cluster [DBG] pgmap v644: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 511 B/s rd, 0 B/s wr, 1.22k op/s; 32411/331056 objects misplaced (9.790%); 17 MiB/s, 279 objects/s recovering 2024-02-22T19:42:03.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:02.289901+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e328: 8 total, 8 up, 5 in 2024-02-22T19:42:03.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:01.614324+0000 mgr.y (mgr.14535) 500 : cluster [DBG] pgmap v644: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 511 B/s rd, 0 B/s wr, 1.22k op/s; 32411/331056 objects misplaced (9.790%); 17 MiB/s, 279 objects/s recovering 2024-02-22T19:42:03.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:02.289901+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e328: 8 total, 8 up, 5 in 2024-02-22T19:42:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:01.614324+0000 mgr.y (mgr.14535) 500 : cluster [DBG] pgmap v644: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 511 B/s rd, 0 B/s wr, 1.22k op/s; 32411/331056 objects misplaced (9.790%); 17 MiB/s, 279 objects/s recovering 2024-02-22T19:42:03.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:02.289901+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e328: 8 total, 8 up, 5 in 2024-02-22T19:42:03.671 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:03.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:03.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:03.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:03.823 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:03.852 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:03.857 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:03.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:03.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:04.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:04.055 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:04.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:04.247 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:04.252 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:04.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:04.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:42:04.353 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:04.402 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:04.892 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:04.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:04.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:04.985 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:42:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:42:04.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:05.045 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:05.098 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:05.105 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:05.106 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:05.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:05.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:03.614736+0000 mgr.y (mgr.14535) 501 : cluster [DBG] pgmap v646: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 679 op/s; 32411/331056 objects misplaced (9.790%); 13 MiB/s, 215 objects/s recovering 2024-02-22T19:42:05.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:03.977669+0000 mon.a (mon.0) 1384 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:03.614736+0000 mgr.y (mgr.14535) 501 : cluster [DBG] pgmap v646: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 679 op/s; 32411/331056 objects misplaced (9.790%); 13 MiB/s, 215 objects/s recovering 2024-02-22T19:42:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:03.977669+0000 mon.a (mon.0) 1384 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:05.270 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:05.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:42:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:42:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:03.614736+0000 mgr.y (mgr.14535) 501 : cluster [DBG] pgmap v646: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 679 op/s; 32411/331056 objects misplaced (9.790%); 13 MiB/s, 215 objects/s recovering 2024-02-22T19:42:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:03.977669+0000 mon.a (mon.0) 1384 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:05.486 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:05.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:05.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:05.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:05.657 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:05.694 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:05.701 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:05.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:05.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:05.851 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:05.852 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:05.882 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:05.882 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:165019 8.8 8:1f0cd79f:::benchmark_data_smithi053_130774_object53285:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:57.162339+0000", 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 248.72671216399999, 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.14804488, 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:42:05.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 165019 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162339+0000", 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162339+0000", 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:42:05.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162337+0000", 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7360000000000007e-06 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162346+0000", 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2283000000000001e-05 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162358+0000", 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7940000000000003e-06 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162366+0000", 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057966390000000001 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168163+0000", 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3740000000000004e-06 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168172+0000", 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.140762957 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.308935+0000", 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014490359999999999 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.310384+0000", 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.14804488 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:165021 8.8 8:1f466dd1:::benchmark_data_smithi053_130774_object53287:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:57.162379+0000", 2024-02-22T19:42:05.898 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 248.72667264899999, 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1630046140000001, 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 165021 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162379+0000", 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162379+0000", 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162378+0000", 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2150000000000001e-06 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162382+0000", 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7640000000000002e-06 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162387+0000", 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.357e-06 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162392+0000", 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058029750000000001 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168195+0000", 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8139999999999997e-06 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168200+0000", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1422123630000001 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.310413+0000", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000122865 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.310536+0000", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014749712 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.325285+0000", 2024-02-22T19:42:05.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8129999999999997e-05 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.325383+0000", 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1630046140000001 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:165028 8.8 8:16e839fd:::benchmark_data_smithi053_130774_object53294:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:57.162399+0000", 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 248.72665280000001, 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1771770770000001, 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 165028 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162399+0000", 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162399+0000", 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162398+0000", 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6460000000000002e-06 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162401+0000", 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4149999999999998e-06 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162406+0000", 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1749999999999998e-06 2024-02-22T19:42:05.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162411+0000", 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058092129999999997 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168220+0000", 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9769999999999997e-06 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168226+0000", 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.142335734 2024-02-22T19:42:05.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.310561+0000", 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029860599999999998 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.310860+0000", 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016085700000000001 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,5,7", 2024-02-22T19:42:05.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.311021+0000", 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0093259050000000007 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.320347+0000", 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001876176 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.322223+0000", 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002669818 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.324893+0000", 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014495528000000001 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.339388+0000", 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7402999999999999e-05 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.339446+0000", 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013003900000000001 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.339576+0000", 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1771770770000001 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:165030 8.8 8:16510b00:::benchmark_data_smithi053_130774_object53296:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:57.162415+0000", 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 248.72663650800001, 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1773631170000001, 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 165030 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162415+0000", 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162415+0000", 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162415+0000", 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.334e-06 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162417+0000", 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1100000000000002e-06 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162422+0000", 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.817e-06 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162426+0000", 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058198010000000003 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168246+0000", 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9649999999999996e-06 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168252+0000", 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144726999 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.312979+0000", 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012400200000000001 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.313103+0000", 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000141619 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,5,7", 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.313244+0000", 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071647330000000004 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.320409+0000", 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001822164 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.322231+0000", 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026972020000000001 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.324928+0000", 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014684954 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.339613+0000", 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4249000000000003e-05 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.339658+0000", 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012057000000000001 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.339778+0000", 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1773631170000001 2024-02-22T19:42:05.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:165046 8.8 8:10970afa:::benchmark_data_smithi053_130774_object53312:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:37:57.162430+0000", 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 248.72662144700001, 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1817662090000001, 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 165046 2024-02-22T19:42:05.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162430+0000", 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162430+0000", 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162430+0000", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6620000000000001e-06 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162432+0000", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9810000000000003e-06 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162437+0000", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9480000000000001e-06 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.162441+0000", 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058293310000000001 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168271+0000", 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0098999999999999e-05 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:57.168281+0000", 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1468578549999999 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.315139+0000", 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012142300000000001 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.315260+0000", 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015167699999999999 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,5,7", 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.315412+0000", 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050327150000000001 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.320444+0000", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017935399999999999 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.322238+0000", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027224839999999998 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.324960+0000", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019042762000000001 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.344003+0000", 2024-02-22T19:42:05.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7658999999999998e-05 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.344061+0000", 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000135494 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:37:58.344196+0000", 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1817662090000001 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209439 8.e 8:72194cde:::benchmark_data_smithi053_130774_object97705:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233144+0000", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.65590759200001, 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.154478621, 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209439 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233144+0000", 2024-02-22T19:42:05.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233144+0000", 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233142+0000", 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2409999999999998e-06 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233149+0000", 2024-02-22T19:42:05.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0718e-05 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233160+0000", 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6279999999999996e-06 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233168+0000", 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000542933 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233711+0000", 2024-02-22T19:42:05.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6080000000000002e-06 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233720+0000", 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.125119301 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.358839+0000", 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017001000000000001 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359009+0000", 2024-02-22T19:42:05.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028542798000000001 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387552+0000", 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0285999999999998e-05 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387623+0000", 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.154478621 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209440 8.e 8:79838a5f:::benchmark_data_smithi053_130774_object97706:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233180+0000", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.65587152699999, 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1543472880000001, 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209440 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233180+0000", 2024-02-22T19:42:05.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233180+0000", 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233179+0000", 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.641e-06 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233183+0000", 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-02-22T19:42:05.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233189+0000", 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4549999999999997e-06 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233193+0000", 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055031999999999998 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233744+0000", 2024-02-22T19:42:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.875e-06 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233752+0000", 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1252841280000001 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359036+0000", 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014295099999999999 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359179+0000", 2024-02-22T19:42:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028263030000000001 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387442+0000", 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5550999999999995e-05 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387527+0000", 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1543472880000001 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209447 8.e 8:7a1e666f:::benchmark_data_smithi053_130774_object97713:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.655853549, 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1516851020000001, 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209447 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233200+0000", 2024-02-22T19:42:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0690000000000001e-06 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233205+0000", 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1099999999999996e-06 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233209+0000", 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056241600000000004 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233772+0000", 2024-02-22T19:42:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3029999999999996e-06 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233778+0000", 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1254261780000001 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359204+0000", 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013111000000000001 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359335+0000", 2024-02-22T19:42:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000186489 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359522+0000", 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0076211359999999997 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367143+0000", 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070781389999999998 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.374221+0000", 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5577999999999997e-05 2024-02-22T19:42:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.374257+0000", 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010431807 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384689+0000", 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1398999999999999e-05 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384750+0000", 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133077 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384883+0000", 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1516851020000001 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209448 8.e 8:7f230ff2:::benchmark_data_smithi053_130774_object97714:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233214+0000", 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.655837937, 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1523798379999999, 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209448 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233214+0000", 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233214+0000", 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233213+0000", 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6010000000000002e-06 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233216+0000", 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1209999999999998e-06 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233221+0000", 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9179999999999996e-06 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233225+0000", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057156999999999998 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233796+0000", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9750000000000004e-06 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233802+0000", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.128199044 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.362001+0000", 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019163900000000001 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.362193+0000", 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000404282 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.362597+0000", 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011586292 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.374184+0000", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042569410000000002 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378441+0000", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0064523219999999999 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384893+0000", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049842899999999999 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385391+0000", 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6817999999999997e-05 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385448+0000", 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000145289 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385593+0000", 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1523798379999999 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209452 8.e 8:755e269c:::benchmark_data_smithi053_130774_object97718:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233229+0000", 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.65582273800001, 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153409022, 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209452 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233229+0000", 2024-02-22T19:42:05.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233229+0000", 2024-02-22T19:42:05.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233228+0000", 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.593e-06 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233231+0000", 2024-02-22T19:42:05.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0420000000000002e-06 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233236+0000", 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8999999999999999e-06 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233240+0000", 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058253799999999996 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233822+0000", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7869999999999998e-06 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233828+0000", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1309483170000001 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.364777+0000", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110528 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.364887+0000", 2024-02-22T19:42:05.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000164594 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.365052+0000", 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013431972 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378484+0000", 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044846099999999999 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378932+0000", 2024-02-22T19:42:05.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005964886 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384897+0000", 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001559932 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386457+0000", 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5113e-05 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386502+0000", 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-02-22T19:42:05.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386638+0000", 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153409022 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209454 8.2 8:480e384d:::benchmark_data_smithi053_130774_object97720:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233244+0000", 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.655807727, 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1015162540000001, 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209454 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233244+0000", 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233244+0000", 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233243+0000", 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233246+0000", 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1050000000000003e-06 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233251+0000", 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2749999999999997e-06 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233255+0000", 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044480709999999996 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.237704+0000", 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5820000000000008e-06 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.237713+0000", 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0658666889999999 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.303580+0000", 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031736720000000002 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.306753+0000", 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033246799999999999 2024-02-22T19:42:05.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,4,5", 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.307086+0000", 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027269775 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334356+0000", 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7673999999999999e-05 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334393+0000", 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0490000000000003e-06 2024-02-22T19:42:05.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334398+0000", 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012682700000000001 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334525+0000", 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6285999999999993e-05 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334612+0000", 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014847300000000001 2024-02-22T19:42:05.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334760+0000", 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1015162540000001 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209465 8.e 8:7bc93309:::benchmark_data_smithi053_130774_object97731:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.65578655900001, 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153583528, 2024-02-22T19:42:05.963 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209465 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5919999999999999e-06 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233267+0000", 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233272+0000", 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6940000000000001e-06 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233276+0000", 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057307899999999999 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233849+0000", 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0809999999999999e-06 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233855+0000", 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333234169999999 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367179+0000", 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000128818 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367307+0000", 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000179359 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367487+0000", 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011036155000000001 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378523+0000", 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017368010000000001 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.380260+0000", 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046411279999999996 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384901+0000", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776489 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386677+0000", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4347000000000001e-05 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386722+0000", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012679 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386848+0000", 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153583528 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209470 8.2 8:4aca6c6d:::benchmark_data_smithi053_130774_object97736:head [delete] snapc 0=[] ondisk+write+known_if_redirected e261)", 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.315545+0000", 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.573506466, 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0310337919999999, 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209470 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315545+0000", 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315545+0000", 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315543+0000", 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2250000000000003e-06 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315551+0000", 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3710000000000006e-06 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315560+0000", 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.127e-06 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315566+0000", 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1929999999999999e-05 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315618+0000", 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315624+0000", 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015601757 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.331225+0000", 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125742 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.331351+0000", 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015536599999999999 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,4,5", 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.331507+0000", 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003654034 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.335161+0000", 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044777899999999998 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.335608+0000", 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016139 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.335770+0000", 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010615696000000001 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.346386+0000", 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9052999999999999e-05 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.346445+0000", 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013429100000000001 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.346579+0000", 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0310337919999999 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.976 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209471 8.e 8:7312cf00:::benchmark_data_smithi053_130774_object97737:head [delete] snapc 0=[] ondisk+write+known_if_redirected e261)", 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 178.57347709999999, 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0714779350000001, 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209471 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2849999999999999e-06 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315577+0000", 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7289999999999997e-06 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315583+0000", 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.216e-06 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315587+0000", 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9706000000000001e-05 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315647+0000", 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8909999999999997e-06 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315651+0000", 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0537560939999999 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.369407+0000", 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014177699999999999 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.369549+0000", 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000205603 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.369754+0000", 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091389360000000003 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378893+0000", 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060116980000000002 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384905+0000", 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107773 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385013+0000", 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001870015 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386883+0000", 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3253000000000001e-05 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386926+0000", 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126515 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387052+0000", 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0714779350000001 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212901 8.e 8:7ce38e7a:::benchmark_data_smithi053_130774_object101167:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.238047+0000", 2024-02-22T19:42:05.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 172.65100493099999, 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0423131109999999, 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212901 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238047+0000", 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238047+0000", 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238045+0000", 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6779999999999995e-06 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238054+0000", 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.715e-05 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238072+0000", 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238079+0000", 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053799999999999996 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238617+0000", 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014364600000000001 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238760+0000", 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000162418 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238923+0000", 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061704639999999996 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245093+0000", 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027255399999999999 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245366+0000", 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040326799999999998 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245769+0000", 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091892870000000008 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254958+0000", 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3540000000000006e-06 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254968+0000", 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0253166010000001 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280284+0000", 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5383000000000007e-05 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280360+0000", 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0423131109999999 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212902 8.e 8:7e8b2c88:::benchmark_data_smithi053_130774_object101168:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.238106+0000", 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 172.65094562499999, 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0426704659999999, 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212902 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238106+0000", 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238106+0000", 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238105+0000", 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5990000000000002e-06 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238109+0000", 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3079999999999998e-06 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238114+0000", 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0799999999999999e-06 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238118+0000", 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033961669999999998 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.241514+0000", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117797 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.241632+0000", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142676 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.241775+0000", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033266820000000001 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:05.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245102+0000", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090548199999999999 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.246007+0000", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049880870000000004 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.250995+0000", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039965390000000003 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254992+0000", 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6800000000000004e-06 2024-02-22T19:42:05.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254998+0000", 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0257239949999999 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280722+0000", 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3993999999999998e-05 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280776+0000", 2024-02-22T19:42:05.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0426704659999999 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212933 8.e 8:7a346efd:::benchmark_data_smithi053_130774_object101199:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.308899+0000", 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 172.58015259199999, 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98675647300000002, 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212933 2024-02-22T19:42:05.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308899+0000", 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:05.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:05.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308899+0000", 2024-02-22T19:42:05.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:42:05.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308898+0000", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0759999999999996e-06 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308902+0000", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8299999999999998e-06 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308909+0000", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2849999999999996e-06 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:05.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308913+0000", 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000246085 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.309159+0000", 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0141e-05 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.309170+0000", 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97163018700000003 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280800+0000", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5624999999999993e-05 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280895+0000", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013860200000000001 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.281034+0000", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0076832949999999997 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288717+0000", 2024-02-22T19:42:06.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6510000000000001e-05 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288754+0000", 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0068142130000000004 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295568+0000", 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6344000000000002e-05 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295614+0000", 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1001999999999998e-05 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295655+0000", 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98675647300000002 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212962 8.e 8:7d2dab33:::benchmark_data_smithi053_130774_object101228:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.351104+0000", 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 172.53794757, 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95283383899999996, 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:06.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212962 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351104+0000", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351104+0000", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351102+0000", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4099999999999997e-06 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351111+0000", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8179e-05 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351129+0000", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1519999999999992e-06 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351138+0000", 2024-02-22T19:42:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016201900000000001 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351300+0000", 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7909999999999999e-06 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351309+0000", 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.93233665300000002 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.283646+0000", 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8783999999999995e-05 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.283745+0000", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012550699999999999 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.283870+0000", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01179499 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295665+0000", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040818579999999998 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.299747+0000", 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041355860000000001 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303883+0000", 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9145999999999998e-05 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303912+0000", 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5894999999999999e-05 2024-02-22T19:42:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303938+0000", 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95283383899999996 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212973 8.e 8:7ea0c169:::benchmark_data_smithi053_130774_object101239:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.361988+0000", 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 172.52706381600001, 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94206076900000002, 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:06.009 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212973 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361988+0000", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361988+0000", 2024-02-22T19:42:06.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361987+0000", 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361990+0000", 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9150000000000004e-06 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361995+0000", 2024-02-22T19:42:06.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6550000000000002e-06 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361999+0000", 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3893000000000006e-05 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.362062+0000", 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1496e-05 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.362074+0000", 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92424356200000002 2024-02-22T19:42:06.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.286318+0000", 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9658e-05 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.286417+0000", 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012773999999999999 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.286545+0000", 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091270350000000004 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295672+0000", 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041171259999999996 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.299789+0000", 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041904109999999998 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303979+0000", 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3694e-05 2024-02-22T19:42:06.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304013+0000", 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5299000000000002e-05 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304048+0000", 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94206076900000002 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.015 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212988 8.e 8:7f780119:::benchmark_data_smithi053_130774_object101254:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.392848+0000", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 172.49620380799999, 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.911307753, 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212988 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392848+0000", 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:06.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392848+0000", 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392847+0000", 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0579999999999997e-06 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392852+0000", 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4080000000000001e-06 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392859+0000", 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392864+0000", 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090847599999999997 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.393772+0000", 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0638999999999999e-05 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.393783+0000", 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89499468199999999 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288778+0000", 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101211 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288879+0000", 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127248 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.289006+0000", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066732939999999998 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295679+0000", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041529660000000001 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.299832+0000", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042760970000000004 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:42:06.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304108+0000", 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5398e-05 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304134+0000", 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1591999999999999e-05 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304155+0000", 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.911307753 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:06.021 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:42:06.022 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-02-22T19:42:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:06.055 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:06.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:06.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:05.012284+0000 mgr.y (mgr.14535) 502 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:05.624750+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? 172.21.15.53:0/610156571' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:42:06.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:06.180 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:05.012284+0000 mgr.y (mgr.14535) 502 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:06.180 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:05.624750+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? 172.21.15.53:0/610156571' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:42:06.180 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:05.012284+0000 mgr.y (mgr.14535) 502 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:06.180 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:05.624750+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? 172.21.15.53:0/610156571' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:42:06.198 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:06.229 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:06.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:06.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:06.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:06.382 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:06.417 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:06.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:06.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:06.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:06.572 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:06.608 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:06.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:06.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:06.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:06.780 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:06.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:06.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:06.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 2 1 2024-02-22T19:42:06.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:07.004 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:07.008 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:07.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:07.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:07.163 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:07.197 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:07.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:05.616882+0000 mgr.y (mgr.14535) 503 : cluster [DBG] pgmap v647: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.4 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 29239/315363 objects misplaced (9.272%); 27 MiB/s, 425 objects/s recovering 2024-02-22T19:42:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:05.987095+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.53:0/610156571' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:42:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:05.987148+0000 mon.a (mon.0) 1387 : cluster [DBG] osdmap e329: 8 total, 8 up, 5 in 2024-02-22T19:42:07.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:05.616882+0000 mgr.y (mgr.14535) 503 : cluster [DBG] pgmap v647: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.4 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 29239/315363 objects misplaced (9.272%); 27 MiB/s, 425 objects/s recovering 2024-02-22T19:42:07.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:05.987095+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.53:0/610156571' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:42:07.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:05.987148+0000 mon.a (mon.0) 1387 : cluster [DBG] osdmap e329: 8 total, 8 up, 5 in 2024-02-22T19:42:07.298 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:05.616882+0000 mgr.y (mgr.14535) 503 : cluster [DBG] pgmap v647: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.4 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 29239/315363 objects misplaced (9.272%); 27 MiB/s, 425 objects/s recovering 2024-02-22T19:42:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:05.987095+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.53:0/610156571' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:42:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:05.987148+0000 mon.a (mon.0) 1387 : cluster [DBG] osdmap e329: 8 total, 8 up, 5 in 2024-02-22T19:42:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:07.542 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:07.554 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:07.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:07.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:07.714 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:07.751 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:07.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:07.758 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:07.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:07.916 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:07.960 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:07.971 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:07.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:07.975 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:08.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:42:08.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:08.178 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:08.380 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:08.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:08.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:08.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:07.290690+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e330: 8 total, 8 up, 5 in 2024-02-22T19:42:08.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:07.618549+0000 mgr.y (mgr.14535) 504 : cluster [DBG] pgmap v650: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.4 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 872 op/s; 29239/315363 objects misplaced (9.272%); 17 MiB/s, 279 objects/s recovering 2024-02-22T19:42:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:07.290690+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e330: 8 total, 8 up, 5 in 2024-02-22T19:42:08.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:07.618549+0000 mgr.y (mgr.14535) 504 : cluster [DBG] pgmap v650: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.4 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 872 op/s; 29239/315363 objects misplaced (9.272%); 17 MiB/s, 279 objects/s recovering 2024-02-22T19:42:08.530 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:08.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:08.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:08.581 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:07.290690+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e330: 8 total, 8 up, 5 in 2024-02-22T19:42:08.581 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:07.618549+0000 mgr.y (mgr.14535) 504 : cluster [DBG] pgmap v650: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.4 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 872 op/s; 29239/315363 objects misplaced (9.272%); 17 MiB/s, 279 objects/s recovering 2024-02-22T19:42:08.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:08.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:08.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:08.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:08.899 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:08.935 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:08.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:08.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:09.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:09.105 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:09.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:08.482610+0000 osd.0 (osd.0) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:09.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:08.482662+0000 osd.0 (osd.0) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:09.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:09.287010+0000 mon.a (mon.0) 1389 : audit [DBG] from='client.? 172.21.15.53:0/2712254938' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:09.488 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:09.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:09.504 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:09.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:42:09.623 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:08.482610+0000 osd.0 (osd.0) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:09.623 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:08.482662+0000 osd.0 (osd.0) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:09.623 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:09.287010+0000 mon.a (mon.0) 1389 : audit [DBG] from='client.? 172.21.15.53:0/2712254938' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:09.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:08.482610+0000 osd.0 (osd.0) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:09.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:08.482662+0000 osd.0 (osd.0) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:09.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:09.287010+0000 mon.a (mon.0) 1389 : audit [DBG] from='client.? 172.21.15.53:0/2712254938' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:09.624 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:42:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:09] "GET /metrics HTTP/1.1" 200 176183 "" "Prometheus/2.33.4" 2024-02-22T19:42:09.641 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:09.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:09.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:09.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:09.845 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:10.317 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:10.322 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:10.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:10.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:10.479 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:09.325881+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e331: 8 total, 8 up, 5 in 2024-02-22T19:42:10.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:09.619228+0000 mgr.y (mgr.14535) 505 : cluster [DBG] pgmap v652: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.2 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.49k op/s; 26893/304197 objects misplaced (8.841%); 34 MiB/s, 551 objects/s recovering 2024-02-22T19:42:10.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:09.325881+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e331: 8 total, 8 up, 5 in 2024-02-22T19:42:10.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:09.619228+0000 mgr.y (mgr.14535) 505 : cluster [DBG] pgmap v652: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.2 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.49k op/s; 26893/304197 objects misplaced (8.841%); 34 MiB/s, 551 objects/s recovering 2024-02-22T19:42:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:09.325881+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e331: 8 total, 8 up, 5 in 2024-02-22T19:42:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:09.619228+0000 mgr.y (mgr.14535) 505 : cluster [DBG] pgmap v652: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.2 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.49k op/s; 26893/304197 objects misplaced (8.841%); 34 MiB/s, 551 objects/s recovering 2024-02-22T19:42:12.332 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:11.620161+0000 mgr.y (mgr.14535) 506 : cluster [DBG] pgmap v653: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 833 op/s; 26145/300366 objects misplaced (8.704%); 18 MiB/s, 291 objects/s recovering 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:11.947070+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.53:0/2863091305' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:11.947754+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:12.286300+0000 mon.a (mon.0) 1392 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:12.286335+0000 mon.a (mon.0) 1393 : cluster [INF] Cluster is now healthy 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:12.298931+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:42:12.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:12.299007+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e332: 8 total, 8 up, 5 in 2024-02-22T19:42:12.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:42:12.991 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:11.620161+0000 mgr.y (mgr.14535) 506 : cluster [DBG] pgmap v653: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 833 op/s; 26145/300366 objects misplaced (8.704%); 18 MiB/s, 291 objects/s recovering 2024-02-22T19:42:12.991 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:11.947070+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.53:0/2863091305' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:12.991 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:11.947754+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:12.991 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:12.286300+0000 mon.a (mon.0) 1392 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:42:12.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:12.286335+0000 mon.a (mon.0) 1393 : cluster [INF] Cluster is now healthy 2024-02-22T19:42:12.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:12.298931+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:42:12.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:12.299007+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e332: 8 total, 8 up, 5 in 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:11.620161+0000 mgr.y (mgr.14535) 506 : cluster [DBG] pgmap v653: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 833 op/s; 26145/300366 objects misplaced (8.704%); 18 MiB/s, 291 objects/s recovering 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:11.947070+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.53:0/2863091305' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:11.947754+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:12.286300+0000 mon.a (mon.0) 1392 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:12.286335+0000 mon.a (mon.0) 1393 : cluster [INF] Cluster is now healthy 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:12.298931+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:42:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:12.299007+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e332: 8 total, 8 up, 5 in 2024-02-22T19:42:14.485 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:14.497 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:14.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:14.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:14.665 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:14.719 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:14.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:14.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:14.752 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:13.620584+0000 mgr.y (mgr.14535) 507 : cluster [DBG] pgmap v655: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 809 B/s rd, 0 B/s wr, 790 op/s; 26145/300366 objects misplaced (8.704%); 17 MiB/s, 276 objects/s recovering 2024-02-22T19:42:14.752 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:13.830202+0000 osd.1 (osd.1) 206 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:14.752 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:13.830238+0000 osd.1 (osd.1) 207 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:14.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:13.620584+0000 mgr.y (mgr.14535) 507 : cluster [DBG] pgmap v655: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 809 B/s rd, 0 B/s wr, 790 op/s; 26145/300366 objects misplaced (8.704%); 17 MiB/s, 276 objects/s recovering 2024-02-22T19:42:14.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:13.830202+0000 osd.1 (osd.1) 206 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:14.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:13.830238+0000 osd.1 (osd.1) 207 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:14.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:14.910 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:14.915 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:14.915 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:13.620584+0000 mgr.y (mgr.14535) 507 : cluster [DBG] pgmap v655: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 6.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 809 B/s rd, 0 B/s wr, 790 op/s; 26145/300366 objects misplaced (8.704%); 17 MiB/s, 276 objects/s recovering 2024-02-22T19:42:14.916 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:13.830202+0000 osd.1 (osd.1) 206 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:14.916 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:13.830238+0000 osd.1 (osd.1) 207 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:14.916 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:42:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:42:14.916 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:15.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:15.078 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:15.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:42:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:42:15.612 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:42:15.612 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:42:15.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:14.702563+0000 mon.a (mon.0) 1396 : cluster [DBG] osdmap e333: 8 total, 8 up, 5 in 2024-02-22T19:42:15.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:15.022191+0000 mgr.y (mgr.14535) 508 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:15.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:15.366912+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.53:0/3808769501' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:42:15.740 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:14.702563+0000 mon.a (mon.0) 1396 : cluster [DBG] osdmap e333: 8 total, 8 up, 5 in 2024-02-22T19:42:15.740 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:15.022191+0000 mgr.y (mgr.14535) 508 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:15.740 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:15.366912+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.53:0/3808769501' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:42:15.863 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:15.867 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:15.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:15.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:15.990 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-02-22T19:42:16.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:14.702563+0000 mon.a (mon.0) 1396 : cluster [DBG] osdmap e333: 8 total, 8 up, 5 in 2024-02-22T19:42:16.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:15.022191+0000 mgr.y (mgr.14535) 508 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:16.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:15.366912+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.53:0/3808769501' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:42:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:16.044 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:16.049 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:16.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:16.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:16.189 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:16.227 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:16.232 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:16.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:16.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:16.378 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:16.412 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:16.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:16.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:16.572 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:16.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:16.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:16.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 3 1 2024-02-22T19:42:16.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:16.743 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:15.621100+0000 mgr.y (mgr.14535) 509 : cluster [DBG] pgmap v657: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.8 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.67k op/s; 23097/283869 objects misplaced (8.136%); 34 MiB/s, 538 objects/s recovering 2024-02-22T19:42:16.743 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:15.725110+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? 172.21.15.53:0/3808769501' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:42:16.743 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:15.725176+0000 mon.a (mon.0) 1399 : cluster [DBG] osdmap e334: 8 total, 8 up, 5 in 2024-02-22T19:42:16.754 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:16.785 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:16.788 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:16.788 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:16.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:16.971 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:16.972 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:17.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:15.621100+0000 mgr.y (mgr.14535) 509 : cluster [DBG] pgmap v657: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.8 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.67k op/s; 23097/283869 objects misplaced (8.136%); 34 MiB/s, 538 objects/s recovering 2024-02-22T19:42:17.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:15.725110+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? 172.21.15.53:0/3808769501' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:42:17.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:15.725176+0000 mon.a (mon.0) 1399 : cluster [DBG] osdmap e334: 8 total, 8 up, 5 in 2024-02-22T19:42:17.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:15.621100+0000 mgr.y (mgr.14535) 509 : cluster [DBG] pgmap v657: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.8 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.67k op/s; 23097/283869 objects misplaced (8.136%); 34 MiB/s, 538 objects/s recovering 2024-02-22T19:42:17.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:15.725110+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? 172.21.15.53:0/3808769501' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:42:17.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:15.725176+0000 mon.a (mon.0) 1399 : cluster [DBG] osdmap e334: 8 total, 8 up, 5 in 2024-02-22T19:42:17.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:17.143 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:17.436 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:17.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:42:17.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:42:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:17.621508+0000 mgr.y (mgr.14535) 510 : cluster [DBG] pgmap v659: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.8 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 917 op/s; 23097/283869 objects misplaced (8.136%); 17 MiB/s, 273 objects/s recovering 2024-02-22T19:42:19.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:17.621508+0000 mgr.y (mgr.14535) 510 : cluster [DBG] pgmap v659: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.8 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 917 op/s; 23097/283869 objects misplaced (8.136%); 17 MiB/s, 273 objects/s recovering 2024-02-22T19:42:19.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:17.621508+0000 mgr.y (mgr.14535) 510 : cluster [DBG] pgmap v659: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.8 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 917 op/s; 23097/283869 objects misplaced (8.136%); 17 MiB/s, 273 objects/s recovering 2024-02-22T19:42:19.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:42:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:19] "GET /metrics HTTP/1.1" 200 176183 "" "Prometheus/2.33.4" 2024-02-22T19:42:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:20.362 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:20.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:20.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:20.536 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:20.566 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:20.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:20.575 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:20.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:20.735 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:21.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:19.622716+0000 mgr.y (mgr.14535) 511 : cluster [DBG] pgmap v660: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.30k op/s; 20986/271794 objects misplaced (7.721%); 27 MiB/s, 425 objects/s recovering 2024-02-22T19:42:21.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:19.622716+0000 mgr.y (mgr.14535) 511 : cluster [DBG] pgmap v660: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.30k op/s; 20986/271794 objects misplaced (7.721%); 27 MiB/s, 425 objects/s recovering 2024-02-22T19:42:21.085 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:19.622716+0000 mgr.y (mgr.14535) 511 : cluster [DBG] pgmap v660: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.30k op/s; 20986/271794 objects misplaced (7.721%); 27 MiB/s, 425 objects/s recovering 2024-02-22T19:42:21.196 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:21.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:21.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:21.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:21.415 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:21.639 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:21.644 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:21.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:21.685 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:21.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:21.796 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:21.829 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:21.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:21.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:21.996 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:21.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:22.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:22.101 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:22.147 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:22.178 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:22.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:22.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:22.320 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:22.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:22.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:22.715 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:22.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:22.723 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:22.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:22.914 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:21.623254+0000 mgr.y (mgr.14535) 512 : cluster [DBG] pgmap v661: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s; 20261/267867 objects misplaced (7.564%); 25 MiB/s, 404 objects/s recovering 2024-02-22T19:42:23.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:21.623254+0000 mgr.y (mgr.14535) 512 : cluster [DBG] pgmap v661: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s; 20261/267867 objects misplaced (7.564%); 25 MiB/s, 404 objects/s recovering 2024-02-22T19:42:23.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:21.623254+0000 mgr.y (mgr.14535) 512 : cluster [DBG] pgmap v661: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s; 20261/267867 objects misplaced (7.564%); 25 MiB/s, 404 objects/s recovering 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:42:23.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:42:23.494 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:23.501 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:23.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:23.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:23.659 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:23.801 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:23.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:23.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:23.948 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:23.979 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:23.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:24.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:22.874507+0000 mon.a (mon.0) 1400 : audit [DBG] from='client.? 172.21.15.53:0/2777635061' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:24.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:23.375467+0000 mon.a (mon.0) 1401 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:22.874507+0000 mon.a (mon.0) 1400 : audit [DBG] from='client.? 172.21.15.53:0/2777635061' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:23.375467+0000 mon.a (mon.0) 1401 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:24.080 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:22.874507+0000 mon.a (mon.0) 1400 : audit [DBG] from='client.? 172.21.15.53:0/2777635061' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:23.375467+0000 mon.a (mon.0) 1401 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:24.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:24.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:24.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:24.331 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:24.363 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:24.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:24.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:24.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:24.515 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:24.547 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:24.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:24.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:24.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:24.705 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:24.730 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:24.736 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:24.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:24.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:24.918 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:24.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:24.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:24.929 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:23.624832+0000 mgr.y (mgr.14535) 513 : cluster [DBG] pgmap v662: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 573 B/s rd, 0 B/s wr, 598 op/s; 20261/267867 objects misplaced (7.564%); 11 MiB/s, 179 objects/s recovering 2024-02-22T19:42:24.930 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:42:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:42:25.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:25.088 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:25.119 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:25.123 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:25.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:25.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:25.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:23.624832+0000 mgr.y (mgr.14535) 513 : cluster [DBG] pgmap v662: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 573 B/s rd, 0 B/s wr, 598 op/s; 20261/267867 objects misplaced (7.564%); 11 MiB/s, 179 objects/s recovering 2024-02-22T19:42:25.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:23.624832+0000 mgr.y (mgr.14535) 513 : cluster [DBG] pgmap v662: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 573 B/s rd, 0 B/s wr, 598 op/s; 20261/267867 objects misplaced (7.564%); 11 MiB/s, 179 objects/s recovering 2024-02-22T19:42:25.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:42:25.280 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:25.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:25.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:25.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:25.456 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:25.490 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:25.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:25.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:25.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:25.683 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:25.687 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:25.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:25.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:25.836 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:25.857 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:25.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:25.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:25.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:26.021 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:24.791144+0000 osd.2 (osd.2) 188 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:26.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:24.791186+0000 osd.2 (osd.2) 189 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:26.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:25.036370+0000 mgr.y (mgr.14535) 514 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:26.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:25.390692+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e335: 8 total, 8 up, 5 in 2024-02-22T19:42:26.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:25.785511+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.53:0/3414577259' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:42:26.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:25.786285+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:42:26.030 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:26.066 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:26.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:26.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:26.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:26.248 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:26.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:26.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:24.791144+0000 osd.2 (osd.2) 188 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:24.791186+0000 osd.2 (osd.2) 189 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:25.036370+0000 mgr.y (mgr.14535) 514 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:25.390692+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e335: 8 total, 8 up, 5 in 2024-02-22T19:42:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:25.785511+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.53:0/3414577259' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:42:26.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:25.786285+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:42:26.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:24.791144+0000 osd.2 (osd.2) 188 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:26.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:24.791186+0000 osd.2 (osd.2) 189 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:26.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:25.036370+0000 mgr.y (mgr.14535) 514 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:26.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:25.390692+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e335: 8 total, 8 up, 5 in 2024-02-22T19:42:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:25.785511+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.53:0/3414577259' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:42:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:25.786285+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:42:26.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:26.447 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:42:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:26.831 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:26.836 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:26.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:26.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:26.986 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:27.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:27.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:25.626680+0000 mgr.y (mgr.14535) 515 : cluster [DBG] pgmap v664: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s; 17540/251916 objects misplaced (6.963%); 20 MiB/s, 319 objects/s recovering 2024-02-22T19:42:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:26.392671+0000 mon.a (mon.0) 1404 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:27.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:26.408182+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:42:27.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:26.408248+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e336: 8 total, 8 up, 5 in 2024-02-22T19:42:27.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:27.200 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:27.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:27.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:25.626680+0000 mgr.y (mgr.14535) 515 : cluster [DBG] pgmap v664: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s; 17540/251916 objects misplaced (6.963%); 20 MiB/s, 319 objects/s recovering 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:26.392671+0000 mon.a (mon.0) 1404 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:26.408182+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:26.408248+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e336: 8 total, 8 up, 5 in 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:25.626680+0000 mgr.y (mgr.14535) 515 : cluster [DBG] pgmap v664: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s; 17540/251916 objects misplaced (6.963%); 20 MiB/s, 319 objects/s recovering 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:26.392671+0000 mon.a (mon.0) 1404 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:26.408182+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:42:27.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:26.408248+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e336: 8 total, 8 up, 5 in 2024-02-22T19:42:27.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:27.390 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:27.417 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:27.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:27.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:27.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:27.591 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:27.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:27.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:27.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:27.734 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:27.769 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:27.774 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:27.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:27.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:42:27.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:28.002 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:28.367 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.3 to 1 (10000) 2024-02-22T19:42:28.390 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:28.394 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:28.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:28.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:28.556 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:28.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:28.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:28.629 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:27.627257+0000 mgr.y (mgr.14535) 516 : cluster [DBG] pgmap v666: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 828 op/s; 17540/251916 objects misplaced (6.963%); 13 MiB/s, 210 objects/s recovering 2024-02-22T19:42:28.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:28.024934+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? 172.21.15.53:0/3872827279' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:42:28.632 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:27.627257+0000 mgr.y (mgr.14535) 516 : cluster [DBG] pgmap v666: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 828 op/s; 17540/251916 objects misplaced (6.963%); 13 MiB/s, 210 objects/s recovering 2024-02-22T19:42:28.632 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:28.024934+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? 172.21.15.53:0/3872827279' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:42:28.731 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:28.773 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:27.627257+0000 mgr.y (mgr.14535) 516 : cluster [DBG] pgmap v666: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 828 op/s; 17540/251916 objects misplaced (6.963%); 13 MiB/s, 210 objects/s recovering 2024-02-22T19:42:28.773 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:28.024934+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? 172.21.15.53:0/3872827279' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:42:28.782 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:28.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:28.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:28.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:29.063 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:42:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:42:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:42:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:42:29.063 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:42:29.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 7 1 2024-02-22T19:42:29.686 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:29.692 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:29.693 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:28.340232+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? 172.21.15.53:0/3872827279' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:42:29.693 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:28.340298+0000 mon.a (mon.0) 1409 : cluster [DBG] osdmap e337: 8 total, 8 up, 5 in 2024-02-22T19:42:29.693 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:42:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:29] "GET /metrics HTTP/1.1" 200 176198 "" "Prometheus/2.33.4" 2024-02-22T19:42:29.693 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:28.340232+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? 172.21.15.53:0/3872827279' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:42:29.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:28.340298+0000 mon.a (mon.0) 1409 : cluster [DBG] osdmap e337: 8 total, 8 up, 5 in 2024-02-22T19:42:29.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:29.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:29.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:28.340232+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? 172.21.15.53:0/3872827279' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:42:29.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:28.340298+0000 mon.a (mon.0) 1409 : cluster [DBG] osdmap e337: 8 total, 8 up, 5 in 2024-02-22T19:42:29.891 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:29.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:42:30.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:42:30.356 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:30.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:30.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:30.377 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:29.629091+0000 mgr.y (mgr.14535) 517 : cluster [DBG] pgmap v668: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s; 15640/240396 objects misplaced (6.506%); 31 MiB/s, 502 objects/s recovering 2024-02-22T19:42:30.378 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:29.629091+0000 mgr.y (mgr.14535) 517 : cluster [DBG] pgmap v668: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s; 15640/240396 objects misplaced (6.506%); 31 MiB/s, 502 objects/s recovering 2024-02-22T19:42:30.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:30.612 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:30.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:30.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:30.722 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:29.629091+0000 mgr.y (mgr.14535) 517 : cluster [DBG] pgmap v668: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s; 15640/240396 objects misplaced (6.506%); 31 MiB/s, 502 objects/s recovering 2024-02-22T19:42:32.171 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:32.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:32.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:32.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:32.336 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:32.469 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:32.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:32.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:32.485 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:32.496 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:32.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:32.622 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:32.663 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:32.667 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:32.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:32.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:31.631822+0000 mgr.y (mgr.14535) 518 : cluster [DBG] pgmap v669: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.8 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 820 B/s rd, 0 B/s wr, 820 op/s; 14909/236562 objects misplaced (6.302%); 16 MiB/s, 250 objects/s recovering 2024-02-22T19:42:32.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:32.311147+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e338: 8 total, 8 up, 5 in 2024-02-22T19:42:32.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:31.631822+0000 mgr.y (mgr.14535) 518 : cluster [DBG] pgmap v669: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.8 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 820 B/s rd, 0 B/s wr, 820 op/s; 14909/236562 objects misplaced (6.302%); 16 MiB/s, 250 objects/s recovering 2024-02-22T19:42:32.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:32.311147+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e338: 8 total, 8 up, 5 in 2024-02-22T19:42:32.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:32.843 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:32.912 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:32.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:33.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:33.016 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:42:33.033 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:33.065 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:42:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:31.631822+0000 mgr.y (mgr.14535) 518 : cluster [DBG] pgmap v669: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.8 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 820 B/s rd, 0 B/s wr, 820 op/s; 14909/236562 objects misplaced (6.302%); 16 MiB/s, 250 objects/s recovering 2024-02-22T19:42:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:32.311147+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e338: 8 total, 8 up, 5 in 2024-02-22T19:42:33.106 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:42:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:33.154 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:42:33.189 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:42:33.216 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:42:33.256 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:42:33.289 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:42:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:42:34.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:32.955439+0000 mon.b (mon.1) 78 : audit [DBG] from='client.? 172.21.15.53:0/419844252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:34.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:32.955439+0000 mon.b (mon.1) 78 : audit [DBG] from='client.? 172.21.15.53:0/419844252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:32.955439+0000 mon.b (mon.1) 78 : audit [DBG] from='client.? 172.21.15.53:0/419844252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:35.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:33.632242+0000 mgr.y (mgr.14535) 519 : cluster [DBG] pgmap v671: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.8 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 708 B/s rd, 0 B/s wr, 708 op/s; 14909/236562 objects misplaced (6.302%); 14 MiB/s, 216 objects/s recovering 2024-02-22T19:42:35.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:33.733073+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e339: 8 total, 8 up, 5 in 2024-02-22T19:42:35.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:33.632242+0000 mgr.y (mgr.14535) 519 : cluster [DBG] pgmap v671: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.8 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 708 B/s rd, 0 B/s wr, 708 op/s; 14909/236562 objects misplaced (6.302%); 14 MiB/s, 216 objects/s recovering 2024-02-22T19:42:35.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:33.733073+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e339: 8 total, 8 up, 5 in 2024-02-22T19:42:35.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:33.632242+0000 mgr.y (mgr.14535) 519 : cluster [DBG] pgmap v671: 148 pgs: 5 active+remapped+backfilling, 143 active+clean; 4.8 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 708 B/s rd, 0 B/s wr, 708 op/s; 14909/236562 objects misplaced (6.302%); 14 MiB/s, 216 objects/s recovering 2024-02-22T19:42:35.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:33.733073+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e339: 8 total, 8 up, 5 in 2024-02-22T19:42:35.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:42:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:42:35.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:42:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:42:35.819 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:34.730047+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e340: 8 total, 8 up, 5 in 2024-02-22T19:42:35.819 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:35.047222+0000 mgr.y (mgr.14535) 520 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:35.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:34.730047+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e340: 8 total, 8 up, 5 in 2024-02-22T19:42:35.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:35.047222+0000 mgr.y (mgr.14535) 520 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:34.730047+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e340: 8 total, 8 up, 5 in 2024-02-22T19:42:36.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:35.047222+0000 mgr.y (mgr.14535) 520 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:42:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:35.632659+0000 mgr.y (mgr.14535) 521 : cluster [DBG] pgmap v674: 148 pgs: 2 peering, 3 active+remapped+backfilling, 143 active+clean; 4.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 949 op/s; 12748/223305 objects misplaced (5.709%); 16 MiB/s, 251 objects/s recovering 2024-02-22T19:42:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:36.418715+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.53:0/534509642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:36.420454+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:37.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:35.632659+0000 mgr.y (mgr.14535) 521 : cluster [DBG] pgmap v674: 148 pgs: 2 peering, 3 active+remapped+backfilling, 143 active+clean; 4.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 949 op/s; 12748/223305 objects misplaced (5.709%); 16 MiB/s, 251 objects/s recovering 2024-02-22T19:42:37.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:36.418715+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.53:0/534509642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:37.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:36.420454+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:37.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:35.632659+0000 mgr.y (mgr.14535) 521 : cluster [DBG] pgmap v674: 148 pgs: 2 peering, 3 active+remapped+backfilling, 143 active+clean; 4.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 949 op/s; 12748/223305 objects misplaced (5.709%); 16 MiB/s, 251 objects/s recovering 2024-02-22T19:42:37.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:36.418715+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.53:0/534509642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:37.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:36.420454+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:42:37.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:42:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:36.750496+0000 mon.a (mon.0) 1414 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:36.757832+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:42:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:36.757907+0000 mon.a (mon.0) 1416 : cluster [DBG] osdmap e341: 8 total, 8 up, 5 in 2024-02-22T19:42:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:37.303760+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e342: 8 total, 8 up, 5 in 2024-02-22T19:42:38.244 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:38.247 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:38.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:36.750496+0000 mon.a (mon.0) 1414 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:36.757832+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:36.757907+0000 mon.a (mon.0) 1416 : cluster [DBG] osdmap e341: 8 total, 8 up, 5 in 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:37.303760+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e342: 8 total, 8 up, 5 in 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:36.750496+0000 mon.a (mon.0) 1414 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:36.757832+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:36.757907+0000 mon.a (mon.0) 1416 : cluster [DBG] osdmap e341: 8 total, 8 up, 5 in 2024-02-22T19:42:38.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:37.303760+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e342: 8 total, 8 up, 5 in 2024-02-22T19:42:38.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:38.391 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:37.635444+0000 mgr.y (mgr.14535) 522 : cluster [DBG] pgmap v677: 148 pgs: 2 peering, 3 active+remapped+backfilling, 143 active+clean; 4.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 12748/223305 objects misplaced (5.709%); 22 MiB/s, 348 objects/s recovering 2024-02-22T19:42:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:38.098811+0000 osd.6 (osd.6) 154 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:38.098863+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:38.381024+0000 mon.a (mon.0) 1418 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:37.635444+0000 mgr.y (mgr.14535) 522 : cluster [DBG] pgmap v677: 148 pgs: 2 peering, 3 active+remapped+backfilling, 143 active+clean; 4.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 12748/223305 objects misplaced (5.709%); 22 MiB/s, 348 objects/s recovering 2024-02-22T19:42:39.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:38.098811+0000 osd.6 (osd.6) 154 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:39.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:38.098863+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:39.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:38.381024+0000 mon.a (mon.0) 1418 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:39.260 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:37.635444+0000 mgr.y (mgr.14535) 522 : cluster [DBG] pgmap v677: 148 pgs: 2 peering, 3 active+remapped+backfilling, 143 active+clean; 4.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 12748/223305 objects misplaced (5.709%); 22 MiB/s, 348 objects/s recovering 2024-02-22T19:42:39.260 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:38.098811+0000 osd.6 (osd.6) 154 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:39.260 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:38.098863+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:39.260 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:38.381024+0000 mon.a (mon.0) 1418 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:39.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:42:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:39] "GET /metrics HTTP/1.1" 200 176203 "" "Prometheus/2.33.4" 2024-02-22T19:42:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:38.807341+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e343: 8 total, 8 up, 5 in 2024-02-22T19:42:40.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:38.807341+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e343: 8 total, 8 up, 5 in 2024-02-22T19:42:40.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:38.807341+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e343: 8 total, 8 up, 5 in 2024-02-22T19:42:41.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:39.636650+0000 mgr.y (mgr.14535) 523 : cluster [DBG] pgmap v679: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.3 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 853 op/s; 12340/210753 objects misplaced (5.855%); 756 KiB/s, 11 objects/s recovering 2024-02-22T19:42:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:39.636650+0000 mgr.y (mgr.14535) 523 : cluster [DBG] pgmap v679: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.3 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 853 op/s; 12340/210753 objects misplaced (5.855%); 756 KiB/s, 11 objects/s recovering 2024-02-22T19:42:41.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:39.636650+0000 mgr.y (mgr.14535) 523 : cluster [DBG] pgmap v679: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.3 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 853 op/s; 12340/210753 objects misplaced (5.855%); 756 KiB/s, 11 objects/s recovering 2024-02-22T19:42:41.287 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:41.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:41.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:41.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:41.419 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:41.460 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:41.467 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:41.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:41.568 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:41.624 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:41.835 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:41.016941+0000 osd.5 (osd.5) 110 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:41.835 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:41.016996+0000 osd.5 (osd.5) 111 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:42.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:41.016941+0000 osd.5 (osd.5) 110 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:42.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:41.016996+0000 osd.5 (osd.5) 111 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:42.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:41.016941+0000 osd.5 (osd.5) 110 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:42.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:41.016996+0000 osd.5 (osd.5) 111 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:41.639931+0000 mgr.y (mgr.14535) 524 : cluster [DBG] pgmap v680: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 939 op/s; 11567/206385 objects misplaced (5.605%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:41.834860+0000 mon.a (mon.0) 1420 : cluster [DBG] osdmap e344: 8 total, 8 up, 5 in 2024-02-22T19:42:43.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:41.639931+0000 mgr.y (mgr.14535) 524 : cluster [DBG] pgmap v680: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 939 op/s; 11567/206385 objects misplaced (5.605%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:43.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:41.834860+0000 mon.a (mon.0) 1420 : cluster [DBG] osdmap e344: 8 total, 8 up, 5 in 2024-02-22T19:42:43.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:41.639931+0000 mgr.y (mgr.14535) 524 : cluster [DBG] pgmap v680: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 939 op/s; 11567/206385 objects misplaced (5.605%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:43.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:41.834860+0000 mon.a (mon.0) 1420 : cluster [DBG] osdmap e344: 8 total, 8 up, 5 in 2024-02-22T19:42:45.060 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:43.640362+0000 mgr.y (mgr.14535) 525 : cluster [DBG] pgmap v682: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 807 B/s rd, 0 B/s wr, 890 op/s; 11567/206385 objects misplaced (5.605%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:42:45.060 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:44.507163+0000 mon.a (mon.0) 1421 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:42:45.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:42:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:42:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:43.640362+0000 mgr.y (mgr.14535) 525 : cluster [DBG] pgmap v682: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 807 B/s rd, 0 B/s wr, 890 op/s; 11567/206385 objects misplaced (5.605%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:42:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:44.507163+0000 mon.a (mon.0) 1421 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:42:45.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:43.640362+0000 mgr.y (mgr.14535) 525 : cluster [DBG] pgmap v682: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 4.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 807 B/s rd, 0 B/s wr, 890 op/s; 11567/206385 objects misplaced (5.605%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:42:45.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:44.507163+0000 mon.a (mon.0) 1421 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:42:45.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:42:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:42:45.804 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:45.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:45.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:45.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:45.987 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:45.991 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:45.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:46.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:46.139 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:45.058775+0000 mgr.y (mgr.14535) 526 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:46.142 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:46.167 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:46.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:46.169 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:46.186 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:45.058775+0000 mgr.y (mgr.14535) 526 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:46.186 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:45.058775+0000 mgr.y (mgr.14535) 526 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:46.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:46.320 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:46.359 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:46.361 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:46.362 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:46.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:46.544 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:47.132 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:42:47.132 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:42:47.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:45.641352+0000 mgr.y (mgr.14535) 527 : cluster [DBG] pgmap v683: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.8 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s; 10396/188508 objects misplaced (5.515%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:46.035719+0000 mon.a (mon.0) 1422 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:46.041550+0000 mon.a (mon.0) 1423 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:46.060180+0000 mon.a (mon.0) 1424 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:45.641352+0000 mgr.y (mgr.14535) 527 : cluster [DBG] pgmap v683: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.8 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s; 10396/188508 objects misplaced (5.515%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:46.035719+0000 mon.a (mon.0) 1422 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:46.041550+0000 mon.a (mon.0) 1423 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:42:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:46.060180+0000 mon.a (mon.0) 1424 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:45.641352+0000 mgr.y (mgr.14535) 527 : cluster [DBG] pgmap v683: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.8 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s; 10396/188508 objects misplaced (5.515%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:42:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:46.035719+0000 mon.a (mon.0) 1422 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:42:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:46.041550+0000 mon.a (mon.0) 1423 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:42:47.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:46.060180+0000 mon.a (mon.0) 1424 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:42:47.812 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:47.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:47.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:47.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:47.981 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:48.014 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:48.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:48.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:48.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:48.175 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:48.220 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:48.229 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:48.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:48.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:42:48.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:48.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:47.641686+0000 mgr.y (mgr.14535) 528 : cluster [DBG] pgmap v684: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.8 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.31k op/s; 10396/188508 objects misplaced (5.515%); 2.5 MiB/s, 39 objects/s recovering 2024-02-22T19:42:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:48.410 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:48.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:48.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:48.563 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:47.641686+0000 mgr.y (mgr.14535) 528 : cluster [DBG] pgmap v684: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.8 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.31k op/s; 10396/188508 objects misplaced (5.515%); 2.5 MiB/s, 39 objects/s recovering 2024-02-22T19:42:48.564 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:47.641686+0000 mgr.y (mgr.14535) 528 : cluster [DBG] pgmap v684: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.8 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.31k op/s; 10396/188508 objects misplaced (5.515%); 2.5 MiB/s, 39 objects/s recovering 2024-02-22T19:42:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:48.581 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:48.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:48.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:48.739 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:48.867 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:48.875 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:48.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:48.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:48.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:49.203 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:49.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:49.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:49.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:49.358 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:49.382 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.7 to 1 (10000) 2024-02-22T19:42:49.392 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:49.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:49.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:49.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:49.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:49.258526+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? 172.21.15.53:0/1650242500' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:42:49.507 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:42:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:49] "GET /metrics HTTP/1.1" 200 176203 "" "Prometheus/2.33.4" 2024-02-22T19:42:49.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:49.258526+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? 172.21.15.53:0/1650242500' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:42:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:49.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:49.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:49.258526+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? 172.21.15.53:0/1650242500' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:42:49.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:49.734 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:49.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:42:49.764 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:49.768 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:49.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:49.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:49.917 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:49.952 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:49.956 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:49.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:50.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:50.141 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:50.146 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:50.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:50.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:50.329 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:50.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:50.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:50.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:50.485 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:50.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:50.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:50.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:50.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:49.364823+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? 172.21.15.53:0/1650242500' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:42:50.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:49.364884+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e345: 8 total, 8 up, 5 in 2024-02-22T19:42:50.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:49.643296+0000 mgr.y (mgr.14535) 529 : cluster [DBG] pgmap v686: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.6 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s; 10056/175479 objects misplaced (5.731%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:50.693 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:49.364823+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? 172.21.15.53:0/1650242500' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:42:50.693 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:49.364884+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e345: 8 total, 8 up, 5 in 2024-02-22T19:42:50.693 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:49.643296+0000 mgr.y (mgr.14535) 529 : cluster [DBG] pgmap v686: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.6 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s; 10056/175479 objects misplaced (5.731%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:50.699 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:49.364823+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? 172.21.15.53:0/1650242500' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:42:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:49.364884+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e345: 8 total, 8 up, 5 in 2024-02-22T19:42:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:49.643296+0000 mgr.y (mgr.14535) 529 : cluster [DBG] pgmap v686: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.6 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s; 10056/175479 objects misplaced (5.731%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:50.865 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:50.871 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:50.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:50.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:51.014 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:51.055 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:51.061 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:51.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:51.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:51.222 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:51.369 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:42:51.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:51.338012+0000 mon.a (mon.0) 1428 : audit [DBG] from='client.? 172.21.15.53:0/2843483552' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:51.409 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:42:51.454 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:42:51.506 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:42:51.551 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:42:51.579 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:42:51.619 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:42:51.656 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:42:51.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.675 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.675 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.675 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.675 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.675 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.675 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:42:51.691 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:51.338012+0000 mon.a (mon.0) 1428 : audit [DBG] from='client.? 172.21.15.53:0/2843483552' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:51.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:51.338012+0000 mon.a (mon.0) 1428 : audit [DBG] from='client.? 172.21.15.53:0/2843483552' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:42:52.007 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:52.013 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:52.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:52.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:52.160 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:52.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:52.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:52.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:42:52.362 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:52.395 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:42:52.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:52.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:42:52.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:51.643858+0000 mgr.y (mgr.14535) 530 : cluster [DBG] pgmap v687: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.20k op/s; 9306/170982 objects misplaced (5.443%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:42:52.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:52.313408+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e346: 8 total, 8 up, 5 in 2024-02-22T19:42:52.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:52.579 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:51.643858+0000 mgr.y (mgr.14535) 530 : cluster [DBG] pgmap v687: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.20k op/s; 9306/170982 objects misplaced (5.443%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:42:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:52.313408+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e346: 8 total, 8 up, 5 in 2024-02-22T19:42:52.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:51.643858+0000 mgr.y (mgr.14535) 530 : cluster [DBG] pgmap v687: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.20k op/s; 9306/170982 objects misplaced (5.443%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:42:52.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:52.313408+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e346: 8 total, 8 up, 5 in 2024-02-22T19:42:53.060 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:53.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:53.064 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:53.076 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:53.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 5 1 2024-02-22T19:42:53.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:53.206 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:53.235 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:53.240 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:53.240 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:53.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:53.454 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:53.568 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:53.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:53.578 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:53.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:54.110 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:54.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:54.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:54.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:54.283 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:54.593 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:42:54.865 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:54.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:54.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:54.926 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:53.645266+0000 mgr.y (mgr.14535) 531 : cluster [DBG] pgmap v689: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 730 op/s; 9306/170982 objects misplaced (5.443%); 1.4 MiB/s, 21 objects/s recovering 2024-02-22T19:42:54.926 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:53.645266+0000 mgr.y (mgr.14535) 531 : cluster [DBG] pgmap v689: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 730 op/s; 9306/170982 objects misplaced (5.443%); 1.4 MiB/s, 21 objects/s recovering 2024-02-22T19:42:54.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:55.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:53.645266+0000 mgr.y (mgr.14535) 531 : cluster [DBG] pgmap v689: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 730 op/s; 9306/170982 objects misplaced (5.443%); 1.4 MiB/s, 21 objects/s recovering 2024-02-22T19:42:55.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:42:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:42:55.027 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:55.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:55.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:55.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:55.243 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:55.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:55.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:55.308 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:42:55.308 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:42:55.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15324.0:210783 9.0 e346/339 9:0966b2c0:::benchmark_data_smithi053_157429_object80324:head v 346'13190, mlcod=346'13186)", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:42:55.300131+0000", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0078186200000000001, 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078503210000000004, 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300131+0000", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300131+0000", 2024-02-22T19:42:55.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300130+0000", 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0069e-05 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300140+0000", 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.287e-05 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300153+0000", 2024-02-22T19:42:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3069999999999999e-06 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300159+0000", 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0552999999999999e-05 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300240+0000", 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1916000000000002e-05 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.300262+0000", 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013066999999999999 2024-02-22T19:42:55.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15324.0:210784 9.1 e346/323 9:89935afc:::benchmark_data_smithi053_157429_object80325:head v 346'12966, mlcod=346'12963)", 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:42:55.305138+0000", 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.002811685, 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00291706, 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.305138+0000", 2024-02-22T19:42:55.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.305138+0000", 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.305136+0000", 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1600000000000001e-05 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.305148+0000", 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9120999999999999e-05 2024-02-22T19:42:55.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.305167+0000", 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9659999999999996e-06 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.305175+0000", 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013379399999999999 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.306513+0000", 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6098000000000001e-05 2024-02-22T19:42:55.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:55.306529+0000", 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013912359999999999 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 2 2024-02-22T19:42:55.315 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:42:55.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:42:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:42:55.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:55.434 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:55.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:55.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:55.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:42:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:55.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:55.070004+0000 mgr.y (mgr.14535) 532 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:55.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:55.070004+0000 mgr.y (mgr.14535) 532 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:55.070004+0000 mgr.y (mgr.14535) 532 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:42:56.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:42:56.600 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:42:56.605 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:56.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:42:56.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:56.804 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:56.870 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:56.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:56.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:56.978 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:42:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:55.645970+0000 mgr.y (mgr.14535) 533 : cluster [DBG] pgmap v690: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.1 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s; 8205/153624 objects misplaced (5.341%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:42:57.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:55.770322+0000 osd.3 (osd.3) 227 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:57.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:55.770372+0000 osd.3 (osd.3) 228 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:55.645970+0000 mgr.y (mgr.14535) 533 : cluster [DBG] pgmap v690: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.1 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s; 8205/153624 objects misplaced (5.341%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:42:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:55.770322+0000 osd.3 (osd.3) 227 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:55.770372+0000 osd.3 (osd.3) 228 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:57.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:55.645970+0000 mgr.y (mgr.14535) 533 : cluster [DBG] pgmap v690: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.1 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s; 8205/153624 objects misplaced (5.341%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:42:57.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:55.770322+0000 osd.3 (osd.3) 227 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:42:57.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:55.770372+0000 osd.3 (osd.3) 228 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:42:57.024 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:57.053 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:42:57.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:57.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:42:57.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:57.213 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:57.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:57.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:57.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:42:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:57.410 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:42:57.414 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:57.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:57.592 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:57.627 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:57.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:57.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:57.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:42:57.751 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:56.751729+0000 mon.a (mon.0) 1430 : cluster [DBG] osdmap e347: 8 total, 8 up, 5 in 2024-02-22T19:42:57.751 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:57.316426+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e348: 8 total, 8 up, 5 in 2024-02-22T19:42:57.789 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:57.809 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:57.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:56.751729+0000 mon.a (mon.0) 1430 : cluster [DBG] osdmap e347: 8 total, 8 up, 5 in 2024-02-22T19:42:57.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:57.316426+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e348: 8 total, 8 up, 5 in 2024-02-22T19:42:57.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:56.751729+0000 mon.a (mon.0) 1430 : cluster [DBG] osdmap e347: 8 total, 8 up, 5 in 2024-02-22T19:42:57.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:57.316426+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e348: 8 total, 8 up, 5 in 2024-02-22T19:42:57.928 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:42:57.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:57.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:42:58.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:58.095 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:58.367 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:58.373 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:58.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:58.474 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:42:58.532 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:42:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:42:58.630 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:58.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:42:58.731 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:58.788 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:42:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:58.803 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:58.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:58.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:58.950 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:57.648471+0000 mgr.y (mgr.14535) 534 : cluster [DBG] pgmap v693: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.1 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 964 op/s; 8205/153624 objects misplaced (5.341%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:58.951 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:57.785903+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.53:0/2801771680' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:58.951 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:57.786666+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:58.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:57.648471+0000 mgr.y (mgr.14535) 534 : cluster [DBG] pgmap v693: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.1 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 964 op/s; 8205/153624 objects misplaced (5.341%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:58.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:57.785903+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.53:0/2801771680' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:58.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:57.786666+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:58.969 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:57.648471+0000 mgr.y (mgr.14535) 534 : cluster [DBG] pgmap v693: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 3.1 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 964 op/s; 8205/153624 objects misplaced (5.341%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:42:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:57.785903+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.53:0/2801771680' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:57.786666+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:42:59.158 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:42:59.162 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:59.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:42:59.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:42:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:42:59.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:42:59.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:42:59.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:42:59.522 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:42:59.609 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:42:59] "GET /metrics HTTP/1.1" 200 176184 "" "Prometheus/2.33.4" 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:58.751644+0000 mon.a (mon.0) 1433 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:42:58.767713+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:58.767775+0000 mon.a (mon.0) 1435 : cluster [DBG] osdmap e349: 8 total, 8 up, 5 in 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:59.586171+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:59.586222+0000 osd.7 (osd.7) 219 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:58.751644+0000 mon.a (mon.0) 1433 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:42:58.767713+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:58.767775+0000 mon.a (mon.0) 1435 : cluster [DBG] osdmap e349: 8 total, 8 up, 5 in 2024-02-22T19:43:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:59.586171+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:00.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:42:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:59.586222+0000 osd.7 (osd.7) 219 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:00.022 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:00.031 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:00.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:58.751644+0000 mon.a (mon.0) 1433 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:42:58.767713+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:43:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:58.767775+0000 mon.a (mon.0) 1435 : cluster [DBG] osdmap e349: 8 total, 8 up, 5 in 2024-02-22T19:43:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:59.586171+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:42:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:59.586222+0000 osd.7 (osd.7) 219 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:00.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:00.191 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:00.464 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:00.470 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:00.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:00.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:00.630 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:00.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:00.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:00.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:00.886 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:01.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:59.649982+0000 mgr.y (mgr.14535) 535 : cluster [DBG] pgmap v695: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.66k op/s; 7846/141096 objects misplaced (5.561%); 2.4 MiB/s, 38 objects/s recovering 2024-02-22T19:43:01.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:42:59.795475+0000 mon.a (mon.0) 1436 : cluster [DBG] osdmap e350: 8 total, 8 up, 5 in 2024-02-22T19:43:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:59.649982+0000 mgr.y (mgr.14535) 535 : cluster [DBG] pgmap v695: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.66k op/s; 7846/141096 objects misplaced (5.561%); 2.4 MiB/s, 38 objects/s recovering 2024-02-22T19:43:01.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:42:59.795475+0000 mon.a (mon.0) 1436 : cluster [DBG] osdmap e350: 8 total, 8 up, 5 in 2024-02-22T19:43:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:59.649982+0000 mgr.y (mgr.14535) 535 : cluster [DBG] pgmap v695: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.66k op/s; 7846/141096 objects misplaced (5.561%); 2.4 MiB/s, 38 objects/s recovering 2024-02-22T19:43:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:42:59.795475+0000 mon.a (mon.0) 1436 : cluster [DBG] osdmap e350: 8 total, 8 up, 5 in 2024-02-22T19:43:02.265 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:02.271 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:02.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:02.300 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:43:02.326 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:43:02.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:43:02.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:02.407 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:43:02.412 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:02.443 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:02.444 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:43:02.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:02.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:43:02.524 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:43:02.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:02.560 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:43:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.580 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.580 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:43:02.609 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:01.650404+0000 mgr.y (mgr.14535) 536 : cluster [DBG] pgmap v697: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.8 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.14k op/s; 7158/136929 objects misplaced (5.228%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:43:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:02.249772+0000 mon.a (mon.0) 1437 : audit [DBG] from='client.? 172.21.15.53:0/3061993813' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:02.480400+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.53:0/2687329314' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:43:03.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:02.481430+0000 mon.a (mon.0) 1438 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:43:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:01.650404+0000 mgr.y (mgr.14535) 536 : cluster [DBG] pgmap v697: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.8 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.14k op/s; 7158/136929 objects misplaced (5.228%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:43:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:02.249772+0000 mon.a (mon.0) 1437 : audit [DBG] from='client.? 172.21.15.53:0/3061993813' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:02.480400+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.53:0/2687329314' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:43:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:02.481430+0000 mon.a (mon.0) 1438 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:43:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:01.650404+0000 mgr.y (mgr.14535) 536 : cluster [DBG] pgmap v697: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.8 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.14k op/s; 7158/136929 objects misplaced (5.228%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:43:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:02.249772+0000 mon.a (mon.0) 1437 : audit [DBG] from='client.? 172.21.15.53:0/3061993813' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:02.480400+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.53:0/2687329314' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:43:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:02.481430+0000 mon.a (mon.0) 1438 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:43:03.334 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-02-22T19:43:03.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:43:04.483 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-22T19:43:04.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:43:04.494 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:04.502 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:04.503 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:02.743034+0000 osd.4 (osd.4) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:02.743081+0000 osd.4 (osd.4) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:03.323943+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:43:04.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:03.324019+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e351: 8 total, 8 up, 5 in 2024-02-22T19:43:04.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:03.650808+0000 mgr.y (mgr.14535) 537 : cluster [DBG] pgmap v699: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.8 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 927 op/s; 7158/136929 objects misplaced (5.228%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:43:04.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:04.647 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:04.680 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:04.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:04.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:04.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:02.743034+0000 osd.4 (osd.4) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:04.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:02.743081+0000 osd.4 (osd.4) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:04.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:03.323943+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:43:04.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:03.324019+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e351: 8 total, 8 up, 5 in 2024-02-22T19:43:04.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:03.650808+0000 mgr.y (mgr.14535) 537 : cluster [DBG] pgmap v699: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.8 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 927 op/s; 7158/136929 objects misplaced (5.228%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:43:04.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:02.743034+0000 osd.4 (osd.4) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:04.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:02.743081+0000 osd.4 (osd.4) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:04.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:03.323943+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:43:04.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:03.324019+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e351: 8 total, 8 up, 5 in 2024-02-22T19:43:04.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:03.650808+0000 mgr.y (mgr.14535) 537 : cluster [DBG] pgmap v699: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.8 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 927 op/s; 7158/136929 objects misplaced (5.228%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:43:04.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:04.849 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:43:04.849 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:43:04.849 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:04.899 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:05.086 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:43:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:43:05.119 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:05.129 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:05.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:05.230 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:05.302 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:05.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:43:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:43:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:05.083934+0000 mgr.y (mgr.14535) 538 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:05.374 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:05.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:05.383 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:05.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:43:05.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:05.520 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:05.558 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:05.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:05.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:05.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:05.719 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:05.751 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:05.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:05.083934+0000 mgr.y (mgr.14535) 538 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:05.756 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:05.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:05.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:05.083934+0000 mgr.y (mgr.14535) 538 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:05.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:05.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:05.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:05.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:43:06.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:06.389 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:06.400 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:06.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:06.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:06.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:06.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:06.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:06.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:06.735 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:05.651743+0000 mgr.y (mgr.14535) 539 : cluster [DBG] pgmap v700: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.4 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.68k op/s; 6030/118878 objects misplaced (5.072%); 3.2 MiB/s, 50 objects/s recovering 2024-02-22T19:43:06.738 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:06.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:05.651743+0000 mgr.y (mgr.14535) 539 : cluster [DBG] pgmap v700: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.4 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.68k op/s; 6030/118878 objects misplaced (5.072%); 3.2 MiB/s, 50 objects/s recovering 2024-02-22T19:43:06.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:05.651743+0000 mgr.y (mgr.14535) 539 : cluster [DBG] pgmap v700: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.4 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.68k op/s; 6030/118878 objects misplaced (5.072%); 3.2 MiB/s, 50 objects/s recovering 2024-02-22T19:43:06.772 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:06.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:06.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:06.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:06.930 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:06.964 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:06.970 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:06.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:07.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:07.142 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:07.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:07.146 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:07.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:07.301 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:07.874 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:07.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:07.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:07.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:08.016 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:08.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:07.325584+0000 mon.a (mon.0) 1441 : cluster [DBG] osdmap e352: 8 total, 8 up, 5 in 2024-02-22T19:43:08.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:07.652183+0000 mgr.y (mgr.14535) 540 : cluster [DBG] pgmap v702: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.4 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 651 B/s rd, 0 B/s wr, 766 op/s; 6030/118878 objects misplaced (5.072%); 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:07.325584+0000 mon.a (mon.0) 1441 : cluster [DBG] osdmap e352: 8 total, 8 up, 5 in 2024-02-22T19:43:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:07.652183+0000 mgr.y (mgr.14535) 540 : cluster [DBG] pgmap v702: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.4 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 651 B/s rd, 0 B/s wr, 766 op/s; 6030/118878 objects misplaced (5.072%); 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:07.325584+0000 mon.a (mon.0) 1441 : cluster [DBG] osdmap e352: 8 total, 8 up, 5 in 2024-02-22T19:43:08.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:07.652183+0000 mgr.y (mgr.14535) 540 : cluster [DBG] pgmap v702: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.4 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 651 B/s rd, 0 B/s wr, 766 op/s; 6030/118878 objects misplaced (5.072%); 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:09.561 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:43:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:09] "GET /metrics HTTP/1.1" 200 176187 "" "Prometheus/2.33.4" 2024-02-22T19:43:10.161 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:10.638 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:10.644 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:10.653 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:10.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:10.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:10.812 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:09.654025+0000 mgr.y (mgr.14535) 541 : cluster [DBG] pgmap v703: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.2 GiB used, 439 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s; 5661/105708 objects misplaced (5.355%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:43:10.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:09.654025+0000 mgr.y (mgr.14535) 541 : cluster [DBG] pgmap v703: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.2 GiB used, 439 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s; 5661/105708 objects misplaced (5.355%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:43:10.819 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:09.654025+0000 mgr.y (mgr.14535) 541 : cluster [DBG] pgmap v703: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.2 GiB used, 439 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s; 5661/105708 objects misplaced (5.355%); 1.8 MiB/s, 29 objects/s recovering 2024-02-22T19:43:11.754 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:43:11.910 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:10.819620+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? 172.21.15.53:0/2035917899' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:43:11.910 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:10.819620+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? 172.21.15.53:0/2035917899' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:43:12.003 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:12.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:12.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:10.819620+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? 172.21.15.53:0/2035917899' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:43:12.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:12.170 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:12.201 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:12.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:12.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:12.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:12.356 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:12.395 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:12.398 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:12.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:12.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:12.556 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:12.710 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:12.718 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:12.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:12.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:12.869 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:12.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:12.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:12.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:11.654600+0000 mgr.y (mgr.14535) 542 : cluster [DBG] pgmap v704: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 5016/101487 objects misplaced (4.943%); 2.6 MiB/s, 41 objects/s recovering 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:11.720276+0000 mon.a (mon.0) 1443 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:11.720311+0000 mon.a (mon.0) 1444 : cluster [INF] Cluster is now healthy 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:11.743333+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.53:0/2035917899' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:11.743404+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e353: 8 total, 8 up, 5 in 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:12.334461+0000 mon.a (mon.0) 1447 : cluster [DBG] osdmap e354: 8 total, 8 up, 5 in 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:11.654600+0000 mgr.y (mgr.14535) 542 : cluster [DBG] pgmap v704: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 5016/101487 objects misplaced (4.943%); 2.6 MiB/s, 41 objects/s recovering 2024-02-22T19:43:12.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:11.720276+0000 mon.a (mon.0) 1443 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:43:12.985 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:11.720311+0000 mon.a (mon.0) 1444 : cluster [INF] Cluster is now healthy 2024-02-22T19:43:12.985 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:11.743333+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.53:0/2035917899' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:43:12.985 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:11.743404+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e353: 8 total, 8 up, 5 in 2024-02-22T19:43:12.985 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:12.334461+0000 mon.a (mon.0) 1447 : cluster [DBG] osdmap e354: 8 total, 8 up, 5 in 2024-02-22T19:43:13.006 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:13.052 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:13.100 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:11.654600+0000 mgr.y (mgr.14535) 542 : cluster [DBG] pgmap v704: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 5016/101487 objects misplaced (4.943%); 2.6 MiB/s, 41 objects/s recovering 2024-02-22T19:43:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:11.720276+0000 mon.a (mon.0) 1443 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:43:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:11.720311+0000 mon.a (mon.0) 1444 : cluster [INF] Cluster is now healthy 2024-02-22T19:43:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:11.743333+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.53:0/2035917899' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:43:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:11.743404+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e353: 8 total, 8 up, 5 in 2024-02-22T19:43:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:12.334461+0000 mon.a (mon.0) 1447 : cluster [DBG] osdmap e354: 8 total, 8 up, 5 in 2024-02-22T19:43:13.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:13.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:13.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:13.285 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:13.743 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:43:13.747 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:43:14.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:13.654998+0000 mgr.y (mgr.14535) 543 : cluster [DBG] pgmap v707: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 809 B/s rd, 0 B/s wr, 916 op/s; 5016/101487 objects misplaced (4.943%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:43:14.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:13.707324+0000 mgr.y (mgr.14535) 544 : audit [DBG] from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:14.957 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:13.654998+0000 mgr.y (mgr.14535) 543 : cluster [DBG] pgmap v707: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 809 B/s rd, 0 B/s wr, 916 op/s; 5016/101487 objects misplaced (4.943%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:43:14.958 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:13.707324+0000 mgr.y (mgr.14535) 544 : audit [DBG] from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:14.960 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:14.967 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:14.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:15.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:15.097 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:43:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:43:15.098 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:43:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:43:15.099 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:13.654998+0000 mgr.y (mgr.14535) 543 : cluster [DBG] pgmap v707: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 2.1 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 809 B/s rd, 0 B/s wr, 916 op/s; 5016/101487 objects misplaced (4.943%); 1.7 MiB/s, 27 objects/s recovering 2024-02-22T19:43:15.099 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:13.707324+0000 mgr.y (mgr.14535) 544 : audit [DBG] from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:15.124 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:15.161 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:15.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:15.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:15.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:15.320 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:15.438 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:15.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:15.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:15.457 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":707,"stamp":"2024-02-22T19:43:13.654765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2204374716,"num_objects":33829,"num_object_clones":0,"num_object_copies":101487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5016,"num_objects_unfound":0,"num_objects_dirty":33829,"num_whiteouts":0,"num_read":1952,"num_read_kb":1704,"num_write":227991,"num_write_kb":8349226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15022,"num_bytes_recovered":954940730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53314,"ondisk_log_size":53314,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":8406432,"kb_used_data":6310584,"kb_used_omap":0,"kb_used_meta":2095616,"kb_avail":460319328,"statfs":{"total":479975178240,"available":471366991872,"internally_reserved":0,"allocated":6462038016,"data_stored":6456306420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2145910784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-379912192,"num_objects":-5797,"num_object_clones":0,"num_object_copies":-17391,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1014,"num_objects_unfound":0,"num_objects_dirty":-5797,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5797,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11403264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.328304"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":309,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335867+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:43:03.335867+0000","last_peered":"2024-02-22T19:43:03.335867+0000","last_clean":"2024-02-22T19:43:03.335867+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:43:03.335867+0000","last_undegraded":"2024-02-22T19:43:03.335867+0000","last_fullsized":"2024-02-22T19:43:03.335867+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":339,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336400+0000","last_change":"2024-02-22T19:41:52.114804+0000","last_active":"2024-02-22T19:43:03.336400+0000","last_peered":"2024-02-22T19:43:03.336400+0000","last_clean":"2024-02-22T19:43:03.336400+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:43:03.336400+0000","last_undegraded":"2024-02-22T19:43:03.336400+0000","last_fullsized":"2024-02-22T19:43:03.336400+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":123,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.332482+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:43:03.332482+0000","last_peered":"2024-02-22T19:43:03.332482+0000","last_clean":"2024-02-22T19:43:03.332482+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:43:03.332482+0000","last_undegraded":"2024-02-22T19:43:03.332482+0000","last_fullsized":"2024-02-22T19:43:03.332482+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":324,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341651+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:43:03.341651+0000","last_peered":"2024-02-22T19:43:03.341651+0000","last_clean":"2024-02-22T19:43:03.341651+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:43:03.341651+0000","last_undegraded":"2024-02-22T19:43:03.341651+0000","last_fullsized":"2024-02-22T19:43:03.341651+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":325,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.345446+0000","last_change":"2024-02-22T19:41:52.112817+0000","last_active":"2024-02-22T19:43:03.345446+0000","last_peered":"2024-02-22T19:43:03.345446+0000","last_clean":"2024-02-22T19:43:03.345446+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:43:03.345446+0000","last_undegraded":"2024-02-22T19:43:03.345446+0000","last_fullsized":"2024-02-22T19:43:03.345446+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":432,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344358+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:43:03.344358+0000","last_peered":"2024-02-22T19:43:03.344358+0000","last_clean":"2024-02-22T19:43:03.344358+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:43:03.344358+0000","last_undegraded":"2024-02-22T19:43:03.344358+0000","last_fullsized":"2024-02-22T19:43:03.344358+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":153,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.329995+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:43:03.329995+0000","last_peered":"2024-02-22T19:43:03.329995+0000","last_clean":"2024-02-22T19:43:03.329995+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:43:03.329995+0000","last_undegraded":"2024-02-22T19:43:03.329995+0000","last_fullsized":"2024-02-22T19:43:03.329995+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":162,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336501+0000","last_change":"2024-02-22T19:41:52.113562+0000","last_active":"2024-02-22T19:43:03.336501+0000","last_peered":"2024-02-22T19:43:03.336501+0000","last_clean":"2024-02-22T19:43:03.336501+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:43:03.336501+0000","last_undegraded":"2024-02-22T19:43:03.336501+0000","last_fullsized":"2024-02-22T19:43:03.336501+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":229,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344226+0000","last_change":"2024-02-22T19:41:52.752324+0000","last_active":"2024-02-22T19:43:03.344226+0000","last_peered":"2024-02-22T19:43:03.344226+0000","last_clean":"2024-02-22T19:43:03.344226+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:43:03.344226+0000","last_undegraded":"2024-02-22T19:43:03.344226+0000","last_fullsized":"2024-02-22T19:43:03.344226+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":149,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335112+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:43:03.335112+0000","last_peered":"2024-02-22T19:43:03.335112+0000","last_clean":"2024-02-22T19:43:03.335112+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:43:03.335112+0000","last_undegraded":"2024-02-22T19:43:03.335112+0000","last_fullsized":"2024-02-22T19:43:03.335112+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":179,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335984+0000","last_change":"2024-02-22T19:41:52.116983+0000","last_active":"2024-02-22T19:43:03.335984+0000","last_peered":"2024-02-22T19:43:03.335984+0000","last_clean":"2024-02-22T19:43:03.335984+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:43:03.335984+0000","last_undegraded":"2024-02-22T19:43:03.335984+0000","last_fullsized":"2024-02-22T19:43:03.335984+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":152,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341408+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:43:03.341408+0000","last_peered":"2024-02-22T19:43:03.341408+0000","last_clean":"2024-02-22T19:43:03.341408+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:43:03.341408+0000","last_undegraded":"2024-02-22T19:43:03.341408+0000","last_fullsized":"2024-02-22T19:43:03.341408+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":227,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336650+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:43:03.336650+0000","last_peered":"2024-02-22T19:43:03.336650+0000","last_clean":"2024-02-22T19:43:03.336650+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:43:03.336650+0000","last_undegraded":"2024-02-22T19:43:03.336650+0000","last_fullsized":"2024-02-22T19:43:03.336650+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":126,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339006+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:43:03.339006+0000","last_peered":"2024-02-22T19:43:03.339006+0000","last_clean":"2024-02-22T19:43:03.339006+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:43:03.339006+0000","last_undegraded":"2024-02-22T19:43:03.339006+0000","last_fullsized":"2024-02-22T19:43:03.339006+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":379,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.354979+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:43:03.354979+0000","last_peered":"2024-02-22T19:43:03.354979+0000","last_clean":"2024-02-22T19:43:03.354979+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:43:03.354979+0000","last_undegraded":"2024-02-22T19:43:03.354979+0000","last_fullsized":"2024-02-22T19:43:03.354979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":342,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.343444+0000","last_change":"2024-02-22T19:41:52.121546+0000","last_active":"2024-02-22T19:43:03.343444+0000","last_peered":"2024-02-22T19:43:03.343444+0000","last_clean":"2024-02-22T19:43:03.343444+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:43:03.343444+0000","last_undegraded":"2024-02-22T19:43:03.343444+0000","last_fullsized":"2024-02-22T19:43:03.343444+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":139,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.333786+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:43:03.333786+0000","last_peered":"2024-02-22T19:43:03.333786+0000","last_clean":"2024-02-22T19:43:03.333786+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:43:03.333786+0000","last_undegraded":"2024-02-22T19:43:03.333786+0000","last_fullsized":"2024-02-22T19:43:03.333786+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":393,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344957+0000","last_change":"2024-02-22T19:41:53.187026+0000","last_active":"2024-02-22T19:43:03.344957+0000","last_peered":"2024-02-22T19:43:03.344957+0000","last_clean":"2024-02-22T19:43:03.344957+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:43:03.344957+0000","last_undegraded":"2024-02-22T19:43:03.344957+0000","last_fullsized":"2024-02-22T19:43:03.344957+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":358,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338806+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:43:03.338806+0000","last_peered":"2024-02-22T19:43:03.338806+0000","last_clean":"2024-02-22T19:43:03.338806+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:43:03.338806+0000","last_undegraded":"2024-02-22T19:43:03.338806+0000","last_fullsized":"2024-02-22T19:43:03.338806+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":347,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.332943+0000","last_change":"2024-02-22T19:41:52.110049+0000","last_active":"2024-02-22T19:43:03.332943+0000","last_peered":"2024-02-22T19:43:03.332943+0000","last_clean":"2024-02-22T19:43:03.332943+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:43:03.332943+0000","last_undegraded":"2024-02-22T19:43:03.332943+0000","last_fullsized":"2024-02-22T19:43:03.332943+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":159,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339552+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:43:03.339552+0000","last_peered":"2024-02-22T19:43:03.339552+0000","last_clean":"2024-02-22T19:43:03.339552+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:43:03.339552+0000","last_undegraded":"2024-02-22T19:43:03.339552+0000","last_fullsized":"2024-02-22T19:43:03.339552+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":347,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.333969+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:43:03.333969+0000","last_peered":"2024-02-22T19:43:03.333969+0000","last_clean":"2024-02-22T19:43:03.333969+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:43:03.333969+0000","last_undegraded":"2024-02-22T19:43:03.333969+0000","last_fullsized":"2024-02-22T19:43:03.333969+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":345,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340870+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:43:03.340870+0000","last_peered":"2024-02-22T19:43:03.340870+0000","last_clean":"2024-02-22T19:43:03.340870+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:43:03.340870+0000","last_undegraded":"2024-02-22T19:43:03.340870+0000","last_fullsized":"2024-02-22T19:43:03.340870+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":387,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336960+0000","last_change":"2024-02-22T19:41:52.743206+0000","last_active":"2024-02-22T19:43:03.336960+0000","last_peered":"2024-02-22T19:43:03.336960+0000","last_clean":"2024-02-22T19:43:03.336960+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:43:03.336960+0000","last_undegraded":"2024-02-22T19:43:03.336960+0000","last_fullsized":"2024-02-22T19:43:03.336960+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":327,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338843+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:43:03.338843+0000","last_peered":"2024-02-22T19:43:03.338843+0000","last_clean":"2024-02-22T19:43:03.338843+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:43:03.338843+0000","last_undegraded":"2024-02-22T19:43:03.338843+0000","last_fullsized":"2024-02-22T19:43:03.338843+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":292,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339502+0000","last_change":"2024-02-22T19:41:52.111808+0000","last_active":"2024-02-22T19:43:03.339502+0000","last_peered":"2024-02-22T19:43:03.339502+0000","last_clean":"2024-02-22T19:43:03.339502+0000","last_became_active":"2024-02-22T19:41:52.110758+0000","last_became_peered":"2024-02-22T19:41:52.110758+0000","last_unstale":"2024-02-22T19:43:03.339502+0000","last_undegraded":"2024-02-22T19:43:03.339502+0000","last_fullsized":"2024-02-22T19:43:03.339502+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":339,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336229+0000","last_change":"2024-02-22T19:41:52.114607+0000","last_active":"2024-02-22T19:43:03.336229+0000","last_peered":"2024-02-22T19:43:03.336229+0000","last_clean":"2024-02-22T19:43:03.336229+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:43:03.336229+0000","last_undegraded":"2024-02-22T19:43:03.336229+0000","last_fullsized":"2024-02-22T19:43:03.336229+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":346,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340967+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:43:03.340967+0000","last_peered":"2024-02-22T19:43:03.340967+0000","last_clean":"2024-02-22T19:43:03.340967+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:43:03.340967+0000","last_undegraded":"2024-02-22T19:43:03.340967+0000","last_fullsized":"2024-02-22T19:43:03.340967+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":348,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341805+0000","last_change":"2024-02-22T19:41:52.118842+0000","last_active":"2024-02-22T19:43:03.341805+0000","last_peered":"2024-02-22T19:43:03.341805+0000","last_clean":"2024-02-22T19:43:03.341805+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:43:03.341805+0000","last_undegraded":"2024-02-22T19:43:03.341805+0000","last_fullsized":"2024-02-22T19:43:03.341805+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":359,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338326+0000","last_change":"2024-02-22T19:41:52.104771+0000","last_active":"2024-02-22T19:43:03.338326+0000","last_peered":"2024-02-22T19:43:03.338326+0000","last_clean":"2024-02-22T19:43:03.338326+0000","last_became_active":"2024-02-22T19:41:52.104234+0000","last_became_peered":"2024-02-22T19:41:52.104234+0000","last_unstale":"2024-02-22T19:43:03.338326+0000","last_undegraded":"2024-02-22T19:43:03.338326+0000","last_fullsized":"2024-02-22T19:43:03.338326+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.702590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.702590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":144,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336487+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:43:03.336487+0000","last_peered":"2024-02-22T19:43:03.336487+0000","last_clean":"2024-02-22T19:43:03.336487+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:43:03.336487+0000","last_undegraded":"2024-02-22T19:43:03.336487+0000","last_fullsized":"2024-02-22T19:43:03.336487+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":366,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344206+0000","last_change":"2024-02-22T19:41:53.508908+0000","last_active":"2024-02-22T19:43:03.344206+0000","last_peered":"2024-02-22T19:43:03.344206+0000","last_clean":"2024-02-22T19:43:03.344206+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:43:03.344206+0000","last_undegraded":"2024-02-22T19:43:03.344206+0000","last_fullsized":"2024-02-22T19:43:03.344206+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":330,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334926+0000","last_change":"2024-02-22T19:41:52.112028+0000","last_active":"2024-02-22T19:43:03.334926+0000","last_peered":"2024-02-22T19:43:03.334926+0000","last_clean":"2024-02-22T19:43:03.334926+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:43:03.334926+0000","last_undegraded":"2024-02-22T19:43:03.334926+0000","last_fullsized":"2024-02-22T19:43:03.334926+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":339,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341574+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:43:03.341574+0000","last_peered":"2024-02-22T19:43:03.341574+0000","last_clean":"2024-02-22T19:43:03.341574+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:43:03.341574+0000","last_undegraded":"2024-02-22T19:43:03.341574+0000","last_fullsized":"2024-02-22T19:43:03.341574+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":367,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340172+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:43:03.340172+0000","last_peered":"2024-02-22T19:43:03.340172+0000","last_clean":"2024-02-22T19:43:03.340172+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:43:03.340172+0000","last_undegraded":"2024-02-22T19:43:03.340172+0000","last_fullsized":"2024-02-22T19:43:03.340172+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":371,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334259+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:43:03.334259+0000","last_peered":"2024-02-22T19:43:03.334259+0000","last_clean":"2024-02-22T19:43:03.334259+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:43:03.334259+0000","last_undegraded":"2024-02-22T19:43:03.334259+0000","last_fullsized":"2024-02-22T19:43:03.334259+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":360,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.343770+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:43:03.343770+0000","last_peered":"2024-02-22T19:43:03.343770+0000","last_clean":"2024-02-22T19:43:03.343770+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:43:03.343770+0000","last_undegraded":"2024-02-22T19:43:03.343770+0000","last_fullsized":"2024-02-22T19:43:03.343770+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":338,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334073+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:43:03.334073+0000","last_peered":"2024-02-22T19:43:03.334073+0000","last_clean":"2024-02-22T19:43:03.334073+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:43:03.334073+0000","last_undegraded":"2024-02-22T19:43:03.334073+0000","last_fullsized":"2024-02-22T19:43:03.334073+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338655+0000","last_change":"2024-02-22T19:41:52.104077+0000","last_active":"2024-02-22T19:43:03.338655+0000","last_peered":"2024-02-22T19:43:03.338655+0000","last_clean":"2024-02-22T19:43:03.338655+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:43:03.338655+0000","last_undegraded":"2024-02-22T19:43:03.338655+0000","last_fullsized":"2024-02-22T19:43:03.338655+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":134,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.333832+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:43:03.333832+0000","last_peered":"2024-02-22T19:43:03.333832+0000","last_clean":"2024-02-22T19:43:03.333832+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:43:03.333832+0000","last_undegraded":"2024-02-22T19:43:03.333832+0000","last_fullsized":"2024-02-22T19:43:03.333832+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":147,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339443+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:43:03.339443+0000","last_peered":"2024-02-22T19:43:03.339443+0000","last_clean":"2024-02-22T19:43:03.339443+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:43:03.339443+0000","last_undegraded":"2024-02-22T19:43:03.339443+0000","last_fullsized":"2024-02-22T19:43:03.339443+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":253,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.340750+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:43:07.340750+0000","last_peered":"2024-02-22T19:43:07.340750+0000","last_clean":"2024-02-22T19:43:07.340750+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:43:07.340750+0000","last_undegraded":"2024-02-22T19:43:07.340750+0000","last_fullsized":"2024-02-22T19:43:07.340750+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":340,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338760+0000","last_change":"2024-02-22T19:41:52.103816+0000","last_active":"2024-02-22T19:43:03.338760+0000","last_peered":"2024-02-22T19:43:03.338760+0000","last_clean":"2024-02-22T19:43:03.338760+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:43:03.338760+0000","last_undegraded":"2024-02-22T19:43:03.338760+0000","last_fullsized":"2024-02-22T19:43:03.338760+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":354,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.342328+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:43:03.342328+0000","last_peered":"2024-02-22T19:43:03.342328+0000","last_clean":"2024-02-22T19:43:03.342328+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:43:03.342328+0000","last_undegraded":"2024-02-22T19:43:03.342328+0000","last_fullsized":"2024-02-22T19:43:03.342328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":332,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.356043+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:43:03.356043+0000","last_peered":"2024-02-22T19:43:03.356043+0000","last_clean":"2024-02-22T19:43:03.356043+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:43:03.356043+0000","last_undegraded":"2024-02-22T19:43:03.356043+0000","last_fullsized":"2024-02-22T19:43:03.356043+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":490,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.346165+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:43:07.346165+0000","last_peered":"2024-02-22T19:43:07.346165+0000","last_clean":"2024-02-22T19:43:07.346165+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:43:07.346165+0000","last_undegraded":"2024-02-22T19:43:07.346165+0000","last_fullsized":"2024-02-22T19:43:07.346165+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":138,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338530+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:43:03.338530+0000","last_peered":"2024-02-22T19:43:03.338530+0000","last_clean":"2024-02-22T19:43:03.338530+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:43:03.338530+0000","last_undegraded":"2024-02-22T19:43:03.338530+0000","last_fullsized":"2024-02-22T19:43:03.338530+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":168,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336662+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:43:03.336662+0000","last_peered":"2024-02-22T19:43:03.336662+0000","last_clean":"2024-02-22T19:43:03.336662+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:43:03.336662+0000","last_undegraded":"2024-02-22T19:43:03.336662+0000","last_fullsized":"2024-02-22T19:43:03.336662+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":355,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355253+0000","last_change":"2024-02-22T19:41:38.526359+0000","last_active":"2024-02-22T19:43:03.355253+0000","last_peered":"2024-02-22T19:43:03.355253+0000","last_clean":"2024-02-22T19:43:03.355253+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:03.355253+0000","last_undegraded":"2024-02-22T19:43:03.355253+0000","last_fullsized":"2024-02-22T19:43:03.355253+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.526259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.526259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":363,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339310+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:43:03.339310+0000","last_peered":"2024-02-22T19:43:03.339310+0000","last_clean":"2024-02-22T19:43:03.339310+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:43:03.339310+0000","last_undegraded":"2024-02-22T19:43:03.339310+0000","last_fullsized":"2024-02-22T19:43:03.339310+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":140,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.345587+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:43:03.345587+0000","last_peered":"2024-02-22T19:43:03.345587+0000","last_clean":"2024-02-22T19:43:03.345587+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:43:03.345587+0000","last_undegraded":"2024-02-22T19:43:03.345587+0000","last_fullsized":"2024-02-22T19:43:03.345587+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":352,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340127+0000","last_change":"2024-02-22T19:43:02.804580+0000","last_active":"2024-02-22T19:43:03.340127+0000","last_peered":"2024-02-22T19:43:03.340127+0000","last_clean":"2024-02-22T19:43:03.340127+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:43:03.340127+0000","last_undegraded":"2024-02-22T19:43:03.340127+0000","last_fullsized":"2024-02-22T19:43:03.340127+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":316,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334755+0000","last_change":"2024-02-22T19:41:52.112840+0000","last_active":"2024-02-22T19:43:03.334755+0000","last_peered":"2024-02-22T19:43:03.334755+0000","last_clean":"2024-02-22T19:43:03.334755+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:43:03.334755+0000","last_undegraded":"2024-02-22T19:43:03.334755+0000","last_fullsized":"2024-02-22T19:43:03.334755+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":354,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.343520+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:43:03.343520+0000","last_peered":"2024-02-22T19:43:03.343520+0000","last_clean":"2024-02-22T19:43:03.343520+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:43:03.343520+0000","last_undegraded":"2024-02-22T19:43:03.343520+0000","last_fullsized":"2024-02-22T19:43:03.343520+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"352'14182","reported_seq":14257,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.424055+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:43:09.424055+0000","last_peered":"2024-02-22T19:43:09.424055+0000","last_clean":"2024-02-22T19:43:09.424055+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:43:09.424055+0000","last_undegraded":"2024-02-22T19:43:09.424055+0000","last_fullsized":"2024-02-22T19:43:09.424055+0000","mapping_epoch":285,"log_start":"334'11040","ondisk_log_start":"334'11040","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3142,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":133693440,"num_objects":2040,"num_object_clones":0,"num_object_copies":6120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14182,"num_write_kb":519104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":185,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335991+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:43:03.335991+0000","last_peered":"2024-02-22T19:43:03.335991+0000","last_clean":"2024-02-22T19:43:03.335991+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:43:03.335991+0000","last_undegraded":"2024-02-22T19:43:03.335991+0000","last_fullsized":"2024-02-22T19:43:03.335991+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":397,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339118+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:43:03.339118+0000","last_peered":"2024-02-22T19:43:03.339118+0000","last_clean":"2024-02-22T19:43:03.339118+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:43:03.339118+0000","last_undegraded":"2024-02-22T19:43:03.339118+0000","last_fullsized":"2024-02-22T19:43:03.339118+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":208,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334736+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:43:03.334736+0000","last_peered":"2024-02-22T19:43:03.334736+0000","last_clean":"2024-02-22T19:43:03.334736+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:43:03.334736+0000","last_undegraded":"2024-02-22T19:43:03.334736+0000","last_fullsized":"2024-02-22T19:43:03.334736+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":572,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.350469+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:43:07.350469+0000","last_peered":"2024-02-22T19:43:07.350469+0000","last_clean":"2024-02-22T19:43:07.350469+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:43:07.350469+0000","last_undegraded":"2024-02-22T19:43:07.350469+0000","last_fullsized":"2024-02-22T19:43:07.350469+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"352'14336","reported_seq":26215,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.512252+0000","last_change":"2024-02-22T19:42:34.755451+0000","last_active":"2024-02-22T19:43:09.512252+0000","last_peered":"2024-02-22T19:43:09.512252+0000","last_clean":"2024-02-22T19:43:09.512252+0000","last_became_active":"2024-02-22T19:42:34.737692+0000","last_became_peered":"2024-02-22T19:42:34.737692+0000","last_unstale":"2024-02-22T19:43:09.512252+0000","last_undegraded":"2024-02-22T19:43:09.512252+0000","last_fullsized":"2024-02-22T19:43:09.512252+0000","mapping_epoch":339,"log_start":"334'10953","ondisk_log_start":"334'10953","created":285,"last_epoch_clean":340,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3383,"ondisk_log_size":3383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":136970240,"num_objects":2090,"num_object_clones":0,"num_object_copies":6270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14336,"num_write_kb":525632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5891,"num_bytes_recovered":386072576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":310,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341727+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:43:03.341727+0000","last_peered":"2024-02-22T19:43:03.341727+0000","last_clean":"2024-02-22T19:43:03.341727+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:43:03.341727+0000","last_undegraded":"2024-02-22T19:43:03.341727+0000","last_fullsized":"2024-02-22T19:43:03.341727+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":54,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355194+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:43:03.355194+0000","last_peered":"2024-02-22T19:43:03.355194+0000","last_clean":"2024-02-22T19:43:03.355194+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:43:03.355194+0000","last_undegraded":"2024-02-22T19:43:03.355194+0000","last_fullsized":"2024-02-22T19:43:03.355194+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":198,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.332708+0000","last_change":"2024-02-22T19:41:52.597069+0000","last_active":"2024-02-22T19:43:03.332708+0000","last_peered":"2024-02-22T19:43:03.332708+0000","last_clean":"2024-02-22T19:43:03.332708+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:43:03.332708+0000","last_undegraded":"2024-02-22T19:43:03.332708+0000","last_fullsized":"2024-02-22T19:43:03.332708+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":370,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.349251+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:43:03.349251+0000","last_peered":"2024-02-22T19:43:03.349251+0000","last_clean":"2024-02-22T19:43:03.349251+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:43:03.349251+0000","last_undegraded":"2024-02-22T19:43:03.349251+0000","last_fullsized":"2024-02-22T19:43:03.349251+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"352'14169","reported_seq":14243,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.414346+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:43:09.414346+0000","last_peered":"2024-02-22T19:43:09.414346+0000","last_clean":"2024-02-22T19:43:09.414346+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:43:09.414346+0000","last_undegraded":"2024-02-22T19:43:09.414346+0000","last_fullsized":"2024-02-22T19:43:09.414346+0000","mapping_epoch":285,"log_start":"334'11100","ondisk_log_start":"334'11100","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3069,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":140050432,"num_objects":2137,"num_object_clones":0,"num_object_copies":6411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14169,"num_write_kb":521792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":316,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338416+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:43:03.338416+0000","last_peered":"2024-02-22T19:43:03.338416+0000","last_clean":"2024-02-22T19:43:03.338416+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:43:03.338416+0000","last_undegraded":"2024-02-22T19:43:03.338416+0000","last_fullsized":"2024-02-22T19:43:03.338416+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":396,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.354557+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:43:03.354557+0000","last_peered":"2024-02-22T19:43:03.354557+0000","last_clean":"2024-02-22T19:43:03.354557+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:43:03.354557+0000","last_undegraded":"2024-02-22T19:43:03.354557+0000","last_fullsized":"2024-02-22T19:43:03.354557+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":354,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340288+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:43:03.340288+0000","last_peered":"2024-02-22T19:43:03.340288+0000","last_clean":"2024-02-22T19:43:03.340288+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:43:03.340288+0000","last_undegraded":"2024-02-22T19:43:03.340288+0000","last_fullsized":"2024-02-22T19:43:03.340288+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":345,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334027+0000","last_change":"2024-02-22T19:41:52.135450+0000","last_active":"2024-02-22T19:43:03.334027+0000","last_peered":"2024-02-22T19:43:03.334027+0000","last_clean":"2024-02-22T19:43:03.334027+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:43:03.334027+0000","last_undegraded":"2024-02-22T19:43:03.334027+0000","last_fullsized":"2024-02-22T19:43:03.334027+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"352'14280","reported_seq":14355,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.436788+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:43:09.436788+0000","last_peered":"2024-02-22T19:43:09.436788+0000","last_clean":"2024-02-22T19:43:09.436788+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:43:09.436788+0000","last_undegraded":"2024-02-22T19:43:09.436788+0000","last_fullsized":"2024-02-22T19:43:09.436788+0000","mapping_epoch":285,"log_start":"334'11131","ondisk_log_start":"334'11131","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3149,"ondisk_log_size":3149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":140247040,"num_objects":2140,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14280,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":138,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339078+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:43:03.339078+0000","last_peered":"2024-02-22T19:43:03.339078+0000","last_clean":"2024-02-22T19:43:03.339078+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:43:03.339078+0000","last_undegraded":"2024-02-22T19:43:03.339078+0000","last_fullsized":"2024-02-22T19:43:03.339078+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":375,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355017+0000","last_change":"2024-02-22T19:41:16.236762+0000","last_active":"2024-02-22T19:43:03.355017+0000","last_peered":"2024-02-22T19:43:03.355017+0000","last_clean":"2024-02-22T19:43:03.355017+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:43:03.355017+0000","last_undegraded":"2024-02-22T19:43:03.355017+0000","last_fullsized":"2024-02-22T19:43:03.355017+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.236664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.236664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":158,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335872+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:43:03.335872+0000","last_peered":"2024-02-22T19:43:03.335872+0000","last_clean":"2024-02-22T19:43:03.335872+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:43:03.335872+0000","last_undegraded":"2024-02-22T19:43:03.335872+0000","last_fullsized":"2024-02-22T19:43:03.335872+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":150,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355025+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:43:03.355025+0000","last_peered":"2024-02-22T19:43:03.355025+0000","last_clean":"2024-02-22T19:43:03.355025+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:43:03.355025+0000","last_undegraded":"2024-02-22T19:43:03.355025+0000","last_fullsized":"2024-02-22T19:43:03.355025+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"352'14384","reported_seq":14459,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.382471+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:43:09.382471+0000","last_peered":"2024-02-22T19:43:09.382471+0000","last_clean":"2024-02-22T19:43:09.382471+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:43:09.382471+0000","last_undegraded":"2024-02-22T19:43:09.382471+0000","last_fullsized":"2024-02-22T19:43:09.382471+0000","mapping_epoch":285,"log_start":"334'10843","ondisk_log_start":"334'10843","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3541,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":141295616,"num_objects":2156,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14384,"num_write_kb":529280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":185,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335035+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:43:03.335035+0000","last_peered":"2024-02-22T19:43:03.335035+0000","last_clean":"2024-02-22T19:43:03.335035+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:43:03.335035+0000","last_undegraded":"2024-02-22T19:43:03.335035+0000","last_fullsized":"2024-02-22T19:43:03.335035+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":333,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.345666+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:43:03.345666+0000","last_peered":"2024-02-22T19:43:03.345666+0000","last_clean":"2024-02-22T19:43:03.345666+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:43:03.345666+0000","last_undegraded":"2024-02-22T19:43:03.345666+0000","last_fullsized":"2024-02-22T19:43:03.345666+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":153,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335774+0000","last_change":"2024-02-22T19:41:52.111972+0000","last_active":"2024-02-22T19:43:03.335774+0000","last_peered":"2024-02-22T19:43:03.335774+0000","last_clean":"2024-02-22T19:43:03.335774+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:43:03.335774+0000","last_undegraded":"2024-02-22T19:43:03.335774+0000","last_fullsized":"2024-02-22T19:43:03.335774+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":268,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.333304+0000","last_change":"2024-02-22T19:41:52.113125+0000","last_active":"2024-02-22T19:43:03.333304+0000","last_peered":"2024-02-22T19:43:03.333304+0000","last_clean":"2024-02-22T19:43:03.333304+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:43:03.333304+0000","last_undegraded":"2024-02-22T19:43:03.333304+0000","last_fullsized":"2024-02-22T19:43:03.333304+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":571,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.333221+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:43:07.333221+0000","last_peered":"2024-02-22T19:43:07.333221+0000","last_clean":"2024-02-22T19:43:07.333221+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:43:07.333221+0000","last_undegraded":"2024-02-22T19:43:07.333221+0000","last_fullsized":"2024-02-22T19:43:07.333221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":156,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.333887+0000","last_change":"2024-02-22T19:41:52.101375+0000","last_active":"2024-02-22T19:43:03.333887+0000","last_peered":"2024-02-22T19:43:03.333887+0000","last_clean":"2024-02-22T19:43:03.333887+0000","last_became_active":"2024-02-22T19:41:52.101126+0000","last_became_peered":"2024-02-22T19:41:52.101126+0000","last_unstale":"2024-02-22T19:43:03.333887+0000","last_undegraded":"2024-02-22T19:43:03.333887+0000","last_fullsized":"2024-02-22T19:43:03.333887+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"352'14074","reported_seq":14149,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.401070+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:43:09.401070+0000","last_peered":"2024-02-22T19:43:09.401070+0000","last_clean":"2024-02-22T19:43:09.401070+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:43:09.401070+0000","last_undegraded":"2024-02-22T19:43:09.401070+0000","last_fullsized":"2024-02-22T19:43:09.401070+0000","mapping_epoch":285,"log_start":"334'10935","ondisk_log_start":"334'10935","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3139,"ondisk_log_size":3139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":132382720,"num_objects":2020,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":515008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":375,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338756+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:43:03.338756+0000","last_peered":"2024-02-22T19:43:03.338756+0000","last_clean":"2024-02-22T19:43:03.338756+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:43:03.338756+0000","last_undegraded":"2024-02-22T19:43:03.338756+0000","last_fullsized":"2024-02-22T19:43:03.338756+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":325,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340388+0000","last_change":"2024-02-22T19:41:52.109613+0000","last_active":"2024-02-22T19:43:03.340388+0000","last_peered":"2024-02-22T19:43:03.340388+0000","last_clean":"2024-02-22T19:43:03.340388+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:43:03.340388+0000","last_undegraded":"2024-02-22T19:43:03.340388+0000","last_fullsized":"2024-02-22T19:43:03.340388+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":338,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.332281+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:43:03.332281+0000","last_peered":"2024-02-22T19:43:03.332281+0000","last_clean":"2024-02-22T19:43:03.332281+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:43:03.332281+0000","last_undegraded":"2024-02-22T19:43:03.332281+0000","last_fullsized":"2024-02-22T19:43:03.332281+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.353192+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:43:03.353192+0000","last_peered":"2024-02-22T19:43:03.353192+0000","last_clean":"2024-02-22T19:43:03.353192+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:43:03.353192+0000","last_undegraded":"2024-02-22T19:43:03.353192+0000","last_fullsized":"2024-02-22T19:43:03.353192+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":426,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.356165+0000","last_change":"2024-02-22T19:41:20.253383+0000","last_active":"2024-02-22T19:43:03.356165+0000","last_peered":"2024-02-22T19:43:03.356165+0000","last_clean":"2024-02-22T19:43:03.356165+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:03.356165+0000","last_undegraded":"2024-02-22T19:43:03.356165+0000","last_fullsized":"2024-02-22T19:43:03.356165+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:41:20.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:20.253277+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":369,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355628+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:43:03.355628+0000","last_peered":"2024-02-22T19:43:03.355628+0000","last_clean":"2024-02-22T19:43:03.355628+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:43:03.355628+0000","last_undegraded":"2024-02-22T19:43:03.355628+0000","last_fullsized":"2024-02-22T19:43:03.355628+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"352'14288","reported_seq":14363,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.704178+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:43:09.704178+0000","last_peered":"2024-02-22T19:43:09.704178+0000","last_clean":"2024-02-22T19:43:09.704178+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:43:09.704178+0000","last_undegraded":"2024-02-22T19:43:09.704178+0000","last_fullsized":"2024-02-22T19:43:09.704178+0000","mapping_epoch":285,"log_start":"335'11145","ondisk_log_start":"335'11145","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3143,"ondisk_log_size":3143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":138280960,"num_objects":2110,"num_object_clones":0,"num_object_copies":6330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14288,"num_write_kb":524736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":158,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.337129+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:43:03.337129+0000","last_peered":"2024-02-22T19:43:03.337129+0000","last_clean":"2024-02-22T19:43:03.337129+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:43:03.337129+0000","last_undegraded":"2024-02-22T19:43:03.337129+0000","last_fullsized":"2024-02-22T19:43:03.337129+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":375,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338034+0000","last_change":"2024-02-22T19:41:52.104562+0000","last_active":"2024-02-22T19:43:03.338034+0000","last_peered":"2024-02-22T19:43:03.338034+0000","last_clean":"2024-02-22T19:43:03.338034+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:43:03.338034+0000","last_undegraded":"2024-02-22T19:43:03.338034+0000","last_fullsized":"2024-02-22T19:43:03.338034+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":343,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344058+0000","last_change":"2024-02-22T19:41:52.111795+0000","last_active":"2024-02-22T19:43:03.344058+0000","last_peered":"2024-02-22T19:43:03.344058+0000","last_clean":"2024-02-22T19:43:03.344058+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:43:03.344058+0000","last_undegraded":"2024-02-22T19:43:03.344058+0000","last_fullsized":"2024-02-22T19:43:03.344058+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":383,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340218+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:43:03.340218+0000","last_peered":"2024-02-22T19:43:03.340218+0000","last_clean":"2024-02-22T19:43:03.340218+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:43:03.340218+0000","last_undegraded":"2024-02-22T19:43:03.340218+0000","last_fullsized":"2024-02-22T19:43:03.340218+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":336,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339193+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:43:03.339193+0000","last_peered":"2024-02-22T19:43:03.339193+0000","last_clean":"2024-02-22T19:43:03.339193+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:43:03.339193+0000","last_undegraded":"2024-02-22T19:43:03.339193+0000","last_fullsized":"2024-02-22T19:43:03.339193+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"352'14277","reported_seq":16144,"reported_epoch":352,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:43:09.398684+0000","last_change":"2024-02-22T19:41:52.760268+0000","last_active":"2024-02-22T19:43:09.398684+0000","last_peered":"2024-02-22T19:43:09.398684+0000","last_clean":"2024-02-22T19:41:51.046554+0000","last_became_active":"2024-02-22T19:41:52.395683+0000","last_became_peered":"2024-02-22T19:41:52.395683+0000","last_unstale":"2024-02-22T19:43:09.398684+0000","last_undegraded":"2024-02-22T19:43:09.398684+0000","last_fullsized":"2024-02-22T19:43:09.398684+0000","mapping_epoch":323,"log_start":"334'11145","ondisk_log_start":"334'11145","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3132,"ondisk_log_size":3132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":138608640,"num_objects":2115,"num_object_clones":0,"num_object_copies":6345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1677,"num_objects_unfound":0,"num_objects_dirty":2115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":524544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":58392576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,6],"avail_no_missing":["5","2","6"],"object_location_counts":[{"shards":"2,5,6","objects":2115}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":229,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.333608+0000","last_change":"2024-02-22T19:41:52.114247+0000","last_active":"2024-02-22T19:43:03.333608+0000","last_peered":"2024-02-22T19:43:03.333608+0000","last_clean":"2024-02-22T19:43:03.333608+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:43:03.333608+0000","last_undegraded":"2024-02-22T19:43:03.333608+0000","last_fullsized":"2024-02-22T19:43:03.333608+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":341,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334378+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:43:03.334378+0000","last_peered":"2024-02-22T19:43:03.334378+0000","last_clean":"2024-02-22T19:43:03.334378+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:43:03.334378+0000","last_undegraded":"2024-02-22T19:43:03.334378+0000","last_fullsized":"2024-02-22T19:43:03.334378+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":126,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.343913+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:43:03.343913+0000","last_peered":"2024-02-22T19:43:03.343913+0000","last_clean":"2024-02-22T19:43:03.343913+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:43:03.343913+0000","last_undegraded":"2024-02-22T19:43:03.343913+0000","last_fullsized":"2024-02-22T19:43:03.343913+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1210,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.822240+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:43:07.822240+0000","last_peered":"2024-02-22T19:43:07.822240+0000","last_clean":"2024-02-22T19:43:07.822240+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:43:07.822240+0000","last_undegraded":"2024-02-22T19:43:07.822240+0000","last_fullsized":"2024-02-22T19:43:07.822240+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":831,"num_read_kb":826,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":158,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335422+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:43:03.335422+0000","last_peered":"2024-02-22T19:43:03.335422+0000","last_clean":"2024-02-22T19:43:03.335422+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:43:03.335422+0000","last_undegraded":"2024-02-22T19:43:03.335422+0000","last_fullsized":"2024-02-22T19:43:03.335422+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":203,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.337205+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:43:03.337205+0000","last_peered":"2024-02-22T19:43:03.337205+0000","last_clean":"2024-02-22T19:43:03.337205+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:43:03.337205+0000","last_undegraded":"2024-02-22T19:43:03.337205+0000","last_fullsized":"2024-02-22T19:43:03.337205+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"352'14099","reported_seq":14174,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.382952+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:43:09.382952+0000","last_peered":"2024-02-22T19:43:09.382952+0000","last_clean":"2024-02-22T19:43:09.382952+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:43:09.382952+0000","last_undegraded":"2024-02-22T19:43:09.382952+0000","last_fullsized":"2024-02-22T19:43:09.382952+0000","mapping_epoch":285,"log_start":"334'10564","ondisk_log_start":"334'10564","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3535,"ondisk_log_size":3535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":134414336,"num_objects":2051,"num_object_clones":0,"num_object_copies":6153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14099,"num_write_kb":516800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":205,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.345371+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:43:03.345371+0000","last_peered":"2024-02-22T19:43:03.345371+0000","last_clean":"2024-02-22T19:43:03.345371+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:43:03.345371+0000","last_undegraded":"2024-02-22T19:43:03.345371+0000","last_fullsized":"2024-02-22T19:43:03.345371+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":327,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.332801+0000","last_change":"2024-02-22T19:41:52.094945+0000","last_active":"2024-02-22T19:43:03.332801+0000","last_peered":"2024-02-22T19:43:03.332801+0000","last_clean":"2024-02-22T19:43:03.332801+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:43:03.332801+0000","last_undegraded":"2024-02-22T19:43:03.332801+0000","last_fullsized":"2024-02-22T19:43:03.332801+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":342,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334532+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:43:03.334532+0000","last_peered":"2024-02-22T19:43:03.334532+0000","last_clean":"2024-02-22T19:43:03.334532+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:43:03.334532+0000","last_undegraded":"2024-02-22T19:43:03.334532+0000","last_fullsized":"2024-02-22T19:43:03.334532+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":163,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335795+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:43:03.335795+0000","last_peered":"2024-02-22T19:43:03.335795+0000","last_clean":"2024-02-22T19:43:03.335795+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:43:03.335795+0000","last_undegraded":"2024-02-22T19:43:03.335795+0000","last_fullsized":"2024-02-22T19:43:03.335795+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":312,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335679+0000","last_change":"2024-02-22T19:41:52.107671+0000","last_active":"2024-02-22T19:43:03.335679+0000","last_peered":"2024-02-22T19:43:03.335679+0000","last_clean":"2024-02-22T19:43:03.335679+0000","last_became_active":"2024-02-22T19:41:52.106767+0000","last_became_peered":"2024-02-22T19:41:52.106767+0000","last_unstale":"2024-02-22T19:43:03.335679+0000","last_undegraded":"2024-02-22T19:43:03.335679+0000","last_fullsized":"2024-02-22T19:43:03.335679+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"352'14342","reported_seq":14417,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.398788+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:43:09.398788+0000","last_peered":"2024-02-22T19:43:09.398788+0000","last_clean":"2024-02-22T19:43:09.398788+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:43:09.398788+0000","last_undegraded":"2024-02-22T19:43:09.398788+0000","last_fullsized":"2024-02-22T19:43:09.398788+0000","mapping_epoch":285,"log_start":"334'11190","ondisk_log_start":"334'11190","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3152,"ondisk_log_size":3152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":136183808,"num_objects":2078,"num_object_clones":0,"num_object_copies":6234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14342,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.343002+0000","last_change":"2024-02-22T19:41:52.110027+0000","last_active":"2024-02-22T19:43:03.343002+0000","last_peered":"2024-02-22T19:43:03.343002+0000","last_clean":"2024-02-22T19:43:03.343002+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:43:03.343002+0000","last_undegraded":"2024-02-22T19:43:03.343002+0000","last_fullsized":"2024-02-22T19:43:03.343002+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.670303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":138,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.340849+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:43:03.340849+0000","last_peered":"2024-02-22T19:43:03.340849+0000","last_clean":"2024-02-22T19:43:03.340849+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:43:03.340849+0000","last_undegraded":"2024-02-22T19:43:03.340849+0000","last_fullsized":"2024-02-22T19:43:03.340849+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":149,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336580+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:43:03.336580+0000","last_peered":"2024-02-22T19:43:03.336580+0000","last_clean":"2024-02-22T19:43:03.336580+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:43:03.336580+0000","last_undegraded":"2024-02-22T19:43:03.336580+0000","last_fullsized":"2024-02-22T19:43:03.336580+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":462,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334157+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:43:03.334157+0000","last_peered":"2024-02-22T19:43:03.334157+0000","last_clean":"2024-02-22T19:43:03.334157+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:43:03.334157+0000","last_undegraded":"2024-02-22T19:43:03.334157+0000","last_fullsized":"2024-02-22T19:43:03.334157+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"326'14","reported_seq":368,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.340857+0000","last_change":"2024-02-22T19:41:52.698274+0000","last_active":"2024-02-22T19:43:07.340857+0000","last_peered":"2024-02-22T19:43:07.340857+0000","last_clean":"2024-02-22T19:43:07.340857+0000","last_became_active":"2024-02-22T19:41:52.112897+0000","last_became_peered":"2024-02-22T19:41:52.112897+0000","last_unstale":"2024-02-22T19:43:07.340857+0000","last_undegraded":"2024-02-22T19:43:07.340857+0000","last_fullsized":"2024-02-22T19:43:07.340857+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"352'14205","reported_seq":16062,"reported_epoch":352,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:43:09.711640+0000","last_change":"2024-02-22T19:41:53.515251+0000","last_active":"2024-02-22T19:43:09.711640+0000","last_peered":"2024-02-22T19:43:09.711640+0000","last_clean":"2024-02-22T19:41:51.078339+0000","last_became_active":"2024-02-22T19:41:52.386374+0000","last_became_peered":"2024-02-22T19:41:52.386374+0000","last_unstale":"2024-02-22T19:43:09.711640+0000","last_undegraded":"2024-02-22T19:43:09.711640+0000","last_fullsized":"2024-02-22T19:43:09.711640+0000","mapping_epoch":323,"log_start":"334'11058","ondisk_log_start":"334'11058","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3147,"ondisk_log_size":3147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":137035776,"num_objects":2091,"num_object_clones":0,"num_object_copies":6273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1652,"num_objects_unfound":0,"num_objects_dirty":2091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14205,"num_write_kb":521472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":886,"num_bytes_recovered":58064896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,6],"avail_no_missing":["3","6","7"],"object_location_counts":[{"shards":"3,6,7","objects":2091}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":229,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.342111+0000","last_change":"2024-02-22T19:41:52.103880+0000","last_active":"2024-02-22T19:43:03.342111+0000","last_peered":"2024-02-22T19:43:03.342111+0000","last_clean":"2024-02-22T19:43:03.342111+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:43:03.342111+0000","last_undegraded":"2024-02-22T19:43:03.342111+0000","last_fullsized":"2024-02-22T19:43:03.342111+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":328,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.339210+0000","last_change":"2024-02-22T19:41:52.103942+0000","last_active":"2024-02-22T19:43:03.339210+0000","last_peered":"2024-02-22T19:43:03.339210+0000","last_clean":"2024-02-22T19:43:03.339210+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:43:03.339210+0000","last_undegraded":"2024-02-22T19:43:03.339210+0000","last_fullsized":"2024-02-22T19:43:03.339210+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":147,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335195+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:43:03.335195+0000","last_peered":"2024-02-22T19:43:03.335195+0000","last_clean":"2024-02-22T19:43:03.335195+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:43:03.335195+0000","last_undegraded":"2024-02-22T19:43:03.335195+0000","last_fullsized":"2024-02-22T19:43:03.335195+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":464,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335243+0000","last_change":"2024-02-22T19:41:52.715987+0000","last_active":"2024-02-22T19:43:03.335243+0000","last_peered":"2024-02-22T19:43:03.335243+0000","last_clean":"2024-02-22T19:43:03.335243+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:43:03.335243+0000","last_undegraded":"2024-02-22T19:43:03.335243+0000","last_fullsized":"2024-02-22T19:43:03.335243+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":531,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.344789+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:43:07.344789+0000","last_peered":"2024-02-22T19:43:07.344789+0000","last_clean":"2024-02-22T19:43:07.344789+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:43:07.344789+0000","last_undegraded":"2024-02-22T19:43:07.344789+0000","last_fullsized":"2024-02-22T19:43:07.344789+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"352'14049","reported_seq":14125,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.313953+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:43:09.313953+0000","last_peered":"2024-02-22T19:43:09.313953+0000","last_clean":"2024-02-22T19:43:09.313953+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:43:09.313953+0000","last_undegraded":"2024-02-22T19:43:09.313953+0000","last_fullsized":"2024-02-22T19:43:09.313953+0000","mapping_epoch":285,"log_start":"334'10435","ondisk_log_start":"334'10435","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3614,"ondisk_log_size":3614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":139198488,"num_objects":2125,"num_object_clones":0,"num_object_copies":6375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14049,"num_write_kb":517505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355541+0000","last_change":"2024-02-22T19:41:52.102626+0000","last_active":"2024-02-22T19:43:03.355541+0000","last_peered":"2024-02-22T19:43:03.355541+0000","last_clean":"2024-02-22T19:43:03.355541+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:43:03.355541+0000","last_undegraded":"2024-02-22T19:43:03.355541+0000","last_fullsized":"2024-02-22T19:43:03.355541+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:33.048434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.048434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":130,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344356+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:43:03.344356+0000","last_peered":"2024-02-22T19:43:03.344356+0000","last_clean":"2024-02-22T19:43:03.344356+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:43:03.344356+0000","last_undegraded":"2024-02-22T19:43:03.344356+0000","last_fullsized":"2024-02-22T19:43:03.344356+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":348,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.343184+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:43:03.343184+0000","last_peered":"2024-02-22T19:43:03.343184+0000","last_clean":"2024-02-22T19:43:03.343184+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:43:03.343184+0000","last_undegraded":"2024-02-22T19:43:03.343184+0000","last_fullsized":"2024-02-22T19:43:03.343184+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":452,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341237+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:43:03.341237+0000","last_peered":"2024-02-22T19:43:03.341237+0000","last_clean":"2024-02-22T19:43:03.341237+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:43:03.341237+0000","last_undegraded":"2024-02-22T19:43:03.341237+0000","last_fullsized":"2024-02-22T19:43:03.341237+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":139,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.344117+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:43:03.344117+0000","last_peered":"2024-02-22T19:43:03.344117+0000","last_clean":"2024-02-22T19:43:03.344117+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:43:03.344117+0000","last_undegraded":"2024-02-22T19:43:03.344117+0000","last_fullsized":"2024-02-22T19:43:03.344117+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"352'14102","reported_seq":14177,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.663862+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:43:09.663862+0000","last_peered":"2024-02-22T19:43:09.663862+0000","last_clean":"2024-02-22T19:43:09.663862+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:43:09.663862+0000","last_undegraded":"2024-02-22T19:43:09.663862+0000","last_fullsized":"2024-02-22T19:43:09.663862+0000","mapping_epoch":285,"log_start":"334'10958","ondisk_log_start":"334'10958","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3144,"ondisk_log_size":3144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":135921664,"num_objects":2074,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14102,"num_write_kb":517632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":161,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336067+0000","last_change":"2024-02-22T19:41:52.107922+0000","last_active":"2024-02-22T19:43:03.336067+0000","last_peered":"2024-02-22T19:43:03.336067+0000","last_clean":"2024-02-22T19:43:03.336067+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:43:03.336067+0000","last_undegraded":"2024-02-22T19:43:03.336067+0000","last_fullsized":"2024-02-22T19:43:03.336067+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":176,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336289+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:43:03.336289+0000","last_peered":"2024-02-22T19:43:03.336289+0000","last_clean":"2024-02-22T19:43:03.336289+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:43:03.336289+0000","last_undegraded":"2024-02-22T19:43:03.336289+0000","last_fullsized":"2024-02-22T19:43:03.336289+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":646,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.339623+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:43:07.339623+0000","last_peered":"2024-02-22T19:43:07.339623+0000","last_clean":"2024-02-22T19:43:07.339623+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:43:07.339623+0000","last_undegraded":"2024-02-22T19:43:07.339623+0000","last_fullsized":"2024-02-22T19:43:07.339623+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":321,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341141+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:43:03.341141+0000","last_peered":"2024-02-22T19:43:03.341141+0000","last_clean":"2024-02-22T19:43:03.341141+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:43:03.341141+0000","last_undegraded":"2024-02-22T19:43:03.341141+0000","last_fullsized":"2024-02-22T19:43:03.341141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":65,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.336319+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:43:03.336319+0000","last_peered":"2024-02-22T19:43:03.336319+0000","last_clean":"2024-02-22T19:43:03.336319+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:43:03.336319+0000","last_undegraded":"2024-02-22T19:43:03.336319+0000","last_fullsized":"2024-02-22T19:43:03.336319+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"352'14289","reported_seq":14364,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.287579+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:43:09.287579+0000","last_peered":"2024-02-22T19:43:09.287579+0000","last_clean":"2024-02-22T19:43:09.287579+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:43:09.287579+0000","last_undegraded":"2024-02-22T19:43:09.287579+0000","last_fullsized":"2024-02-22T19:43:09.287579+0000","mapping_epoch":285,"log_start":"334'10701","ondisk_log_start":"334'10701","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3588,"ondisk_log_size":3588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":140312576,"num_objects":2141,"num_object_clones":0,"num_object_copies":6423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14289,"num_write_kb":525760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":337,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.342248+0000","last_change":"2024-02-22T19:41:52.112553+0000","last_active":"2024-02-22T19:43:03.342248+0000","last_peered":"2024-02-22T19:43:03.342248+0000","last_clean":"2024-02-22T19:43:03.342248+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:43:03.342248+0000","last_undegraded":"2024-02-22T19:43:03.342248+0000","last_fullsized":"2024-02-22T19:43:03.342248+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"326'20","reported_seq":392,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.341278+0000","last_change":"2024-02-22T19:41:54.322898+0000","last_active":"2024-02-22T19:43:07.341278+0000","last_peered":"2024-02-22T19:43:07.341278+0000","last_clean":"2024-02-22T19:43:07.341278+0000","last_became_active":"2024-02-22T19:41:54.322531+0000","last_became_peered":"2024-02-22T19:41:54.322531+0000","last_unstale":"2024-02-22T19:43:07.341278+0000","last_undegraded":"2024-02-22T19:43:07.341278+0000","last_fullsized":"2024-02-22T19:43:07.341278+0000","mapping_epoch":325,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":126,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355934+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:43:03.355934+0000","last_peered":"2024-02-22T19:43:03.355934+0000","last_clean":"2024-02-22T19:43:03.355934+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:43:03.355934+0000","last_undegraded":"2024-02-22T19:43:03.355934+0000","last_fullsized":"2024-02-22T19:43:03.355934+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":134,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335334+0000","last_change":"2024-02-22T19:41:52.358275+0000","last_active":"2024-02-22T19:43:03.335334+0000","last_peered":"2024-02-22T19:43:03.335334+0000","last_clean":"2024-02-22T19:43:03.335334+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:43:03.335334+0000","last_undegraded":"2024-02-22T19:43:03.335334+0000","last_fullsized":"2024-02-22T19:43:03.335334+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"352'14224","reported_seq":26349,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:09.521030+0000","last_change":"2024-02-22T19:42:34.754313+0000","last_active":"2024-02-22T19:43:09.521030+0000","last_peered":"2024-02-22T19:43:09.521030+0000","last_clean":"2024-02-22T19:43:09.521030+0000","last_became_active":"2024-02-22T19:42:34.737475+0000","last_became_peered":"2024-02-22T19:42:34.737475+0000","last_unstale":"2024-02-22T19:43:09.521030+0000","last_undegraded":"2024-02-22T19:43:09.521030+0000","last_fullsized":"2024-02-22T19:43:09.521030+0000","mapping_epoch":339,"log_start":"334'10770","ondisk_log_start":"334'10770","created":285,"last_epoch_clean":340,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3454,"ondisk_log_size":3454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":140115968,"num_objects":2138,"num_object_clones":0,"num_object_copies":6414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":523584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6014,"num_bytes_recovered":394133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":149,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334833+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:43:03.334833+0000","last_peered":"2024-02-22T19:43:03.334833+0000","last_clean":"2024-02-22T19:43:03.334833+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:43:03.334833+0000","last_undegraded":"2024-02-22T19:43:03.334833+0000","last_fullsized":"2024-02-22T19:43:03.334833+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":341,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355968+0000","last_change":"2024-02-22T19:41:52.107535+0000","last_active":"2024-02-22T19:43:03.355968+0000","last_peered":"2024-02-22T19:43:03.355968+0000","last_clean":"2024-02-22T19:43:03.355968+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:43:03.355968+0000","last_undegraded":"2024-02-22T19:43:03.355968+0000","last_fullsized":"2024-02-22T19:43:03.355968+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":383,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334708+0000","last_change":"2024-02-22T19:41:52.092333+0000","last_active":"2024-02-22T19:43:03.334708+0000","last_peered":"2024-02-22T19:43:03.334708+0000","last_clean":"2024-02-22T19:43:03.334708+0000","last_became_active":"2024-02-22T19:41:52.092045+0000","last_became_peered":"2024-02-22T19:41:52.092045+0000","last_unstale":"2024-02-22T19:43:03.334708+0000","last_undegraded":"2024-02-22T19:43:03.334708+0000","last_fullsized":"2024-02-22T19:43:03.334708+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":258,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.334327+0000","last_change":"2024-02-22T19:41:52.110141+0000","last_active":"2024-02-22T19:43:03.334327+0000","last_peered":"2024-02-22T19:43:03.334327+0000","last_clean":"2024-02-22T19:43:03.334327+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:43:03.334327+0000","last_undegraded":"2024-02-22T19:43:03.334327+0000","last_fullsized":"2024-02-22T19:43:03.334327+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":227,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.355627+0000","last_change":"2024-02-22T19:41:52.810562+0000","last_active":"2024-02-22T19:43:03.355627+0000","last_peered":"2024-02-22T19:43:03.355627+0000","last_clean":"2024-02-22T19:43:03.355627+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:43:03.355627+0000","last_undegraded":"2024-02-22T19:43:03.355627+0000","last_fullsized":"2024-02-22T19:43:03.355627+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"352'13977","reported_seq":15841,"reported_epoch":352,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:43:09.655079+0000","last_change":"2024-02-22T19:41:52.755946+0000","last_active":"2024-02-22T19:43:09.655079+0000","last_peered":"2024-02-22T19:43:09.655079+0000","last_clean":"2024-02-22T19:41:51.044126+0000","last_became_active":"2024-02-22T19:41:52.387533+0000","last_became_peered":"2024-02-22T19:41:52.387533+0000","last_unstale":"2024-02-22T19:43:09.655079+0000","last_undegraded":"2024-02-22T19:43:09.655079+0000","last_fullsized":"2024-02-22T19:43:09.655079+0000","mapping_epoch":323,"log_start":"334'10817","ondisk_log_start":"334'10817","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3160,"ondisk_log_size":3160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":139657216,"num_objects":2131,"num_object_clones":0,"num_object_copies":6393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1687,"num_objects_unfound":0,"num_objects_dirty":2131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13977,"num_write_kb":515456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":889,"num_bytes_recovered":58261504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,6],"avail_no_missing":["3","6","7"],"object_location_counts":[{"shards":"3,6,7","objects":2131}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":339,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.338991+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:43:03.338991+0000","last_peered":"2024-02-22T19:43:03.338991+0000","last_clean":"2024-02-22T19:43:03.338991+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:43:03.338991+0000","last_undegraded":"2024-02-22T19:43:03.338991+0000","last_fullsized":"2024-02-22T19:43:03.338991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":165,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.335003+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:43:03.335003+0000","last_peered":"2024-02-22T19:43:03.335003+0000","last_clean":"2024-02-22T19:43:03.335003+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:43:03.335003+0000","last_undegraded":"2024-02-22T19:43:03.335003+0000","last_fullsized":"2024-02-22T19:43:03.335003+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":570,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-02-22T19:43:07.336367+0000","last_change":"2024-02-22T19:41:32.453253+0000","last_active":"2024-02-22T19:43:07.336367+0000","last_peered":"2024-02-22T19:43:07.336367+0000","last_clean":"2024-02-22T19:43:07.336367+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:43:07.336367+0000","last_undegraded":"2024-02-22T19:43:07.336367+0000","last_fullsized":"2024-02-22T19:43:07.336367+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":382,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.341498+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:43:03.341498+0000","last_peered":"2024-02-22T19:43:03.341498+0000","last_clean":"2024-02-22T19:43:03.341498+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:43:03.341498+0000","last_undegraded":"2024-02-22T19:43:03.341498+0000","last_fullsized":"2024-02-22T19:43:03.341498+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":196,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-02-22T19:43:03.342503+0000","last_change":"2024-02-22T19:41:52.575055+0000","last_active":"2024-02-22T19:43:03.342503+0000","last_peered":"2024-02-22T19:43:03.342503+0000","last_clean":"2024-02-22T19:43:03.342503+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:43:03.342503+0000","last_undegraded":"2024-02-22T19:43:03.342503+0000","last_fullsized":"2024-02-22T19:43:03.342503+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":2204368920,"num_objects":33637,"num_object_clones":0,"num_object_copies":100911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5016,"num_objects_unfound":0,"num_objects_dirty":33637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227277,"num_write_kb":8349185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14571,"num_bytes_recovered":954925056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7176925184,"data_stored":7176912968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52492,"ondisk_log_size":52492,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":831,"num_read_kb":826,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":4934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266861,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1687208,"kb_used_data":1385888,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":92057944,"statfs":{"total":95995035648,"available":94267334656,"internally_reserved":0,"allocated":1419149312,"data_stored":1418008435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3979999999999999,"15min":1.3919999999999999},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":5.2320000000000002,"5min":7.2919999999999998,"15min":42.597999999999999},"last":7.056},{"interface":"front","average":{"1min":0.79300000000000004,"5min":1.528,"15min":1.385},"min":{"1min":0.27600000000000002,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":3.0750000000000002,"5min":8.0489999999999995,"15min":42.155999999999999},"last":3.1600000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":3.5600000000000001,"5min":2.3639999999999999,"15min":1.637},"min":{"1min":0.26300000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":36.509,"5min":54.951000000000001,"15min":54.951000000000001},"last":4.835},{"interface":"front","average":{"1min":1.4239999999999999,"5min":2.786,"15min":1.8759999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":5.6139999999999999,"5min":57.012,"15min":57.012},"last":3.2010000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.885,"15min":1.373},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.23999999999999999},"max":{"1min":4.165,"5min":40.606000000000002,"15min":41.640999999999998},"last":5.1200000000000001},{"interface":"front","average":{"1min":2.0289999999999999,"5min":2.4670000000000001,"15min":1.6559999999999999},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.23599999999999999},"max":{"1min":7.2140000000000004,"5min":38.411000000000001,"15min":41.722000000000001},"last":4.8949999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":2.6059999999999999,"15min":1.7549999999999999},"min":{"1min":0.372,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":36.491999999999997,"5min":38.692999999999998,"15min":41.457000000000001},"last":7.1289999999999996},{"interface":"front","average":{"1min":3.6179999999999999,"5min":2.653,"15min":1.744},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":36.732999999999997,"5min":38.219000000000001,"15min":41.311999999999998},"last":9.4039999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.95699999999999996,"15min":0.66900000000000004},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":2.238,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":3.1179999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.085,"15min":0.69099999999999995},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":4.3449999999999998,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":5.0910000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6120000000000001,"5min":1.232,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":5.1699999999999999,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":1.784},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.046,"15min":0.73499999999999999},"min":{"1min":0.371,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":5.282,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":7.3239999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.0309999999999999,"15min":0.79400000000000004},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":1.512,"5min":6.5,"15min":12.199999999999999},"last":3.044},{"interface":"front","average":{"1min":2.0329999999999999,"5min":1.3160000000000001,"15min":0.84299999999999997},"min":{"1min":0.33900000000000002,"5min":0.29699999999999999,"15min":0.28100000000000003},"max":{"1min":6.3869999999999996,"5min":7.6390000000000002,"15min":12.067},"last":3.29}]}]},{"osd":6,"up_from":0,"seq":193273528498,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575051,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1995516,"kb_used_data":1409908,"kb_used_omap":0,"kb_used_meta":585600,"kb_avail":91749636,"statfs":{"total":95995035648,"available":93951627264,"internally_reserved":0,"allocated":1443745792,"data_stored":1442584503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":599654400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":5.7480000000000002,"5min":2.391,"15min":1.8580000000000001},"min":{"1min":0.38900000000000001,"5min":0.185,"15min":0.185},"max":{"1min":51.091000000000001,"5min":51.091000000000001,"15min":51.091000000000001},"last":6.6760000000000002},{"interface":"front","average":{"1min":5.1799999999999997,"5min":2.3860000000000001,"15min":1.952},"min":{"1min":0.38300000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":51.289999999999999,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:42:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.7370000000000001,"15min":1.633},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":7.8339999999999996,"5min":11.682,"15min":16.503},"last":6.7069999999999999},{"interface":"front","average":{"1min":2.597,"5min":1.9399999999999999,"15min":1.788},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":9.1370000000000005,"5min":9.9559999999999995,"15min":14.994},"last":2.6909999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:42:53 2024","interfaces":[{"interface":"back","average":{"1min":5.6390000000000002,"5min":2.5960000000000001,"15min":2.0630000000000002},"min":{"1min":0.39800000000000002,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":51.755000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":5.8040000000000003},{"interface":"front","average":{"1min":5.1509999999999998,"5min":2.6339999999999999,"15min":2.1360000000000001},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":51.741,"5min":51.741,"15min":51.741},"last":6.6449999999999996}]},{"osd":4,"last update":"Thu Feb 22 19:42:06 2024","interfaces":[{"interface":"back","average":{"1min":1.9610000000000001,"5min":1.679,"15min":1.3460000000000001},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.8789999999999996,"5min":7.9390000000000001,"15min":15.542999999999999},"last":6.5869999999999997},{"interface":"front","average":{"1min":2.129,"5min":1.329,"15min":1.167},"min":{"1min":0.316,"5min":0.316,"15min":0.245},"max":{"1min":11.137,"5min":11.137,"15min":11.137},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:42:28 2024","interfaces":[{"interface":"back","average":{"1min":3.1619999999999999,"5min":2.0499999999999998,"15min":1.4950000000000001},"min":{"1min":0.375,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":16.213000000000001,"5min":16.213000000000001,"15min":16.213000000000001},"last":6.5460000000000003},{"interface":"front","average":{"1min":3.3370000000000002,"5min":1.6819999999999999,"15min":1.3720000000000001},"min":{"1min":0.34899999999999998,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":16.381,"5min":16.381,"15min":16.381},"last":0.63}]},{"osd":6,"last update":"Thu Feb 22 19:42:55 2024","interfaces":[{"interface":"back","average":{"1min":4.1929999999999996,"5min":1.9059999999999999,"15min":1.4370000000000001},"min":{"1min":0.433,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":51.079999999999998,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":6.1859999999999999,"5min":2.4590000000000001,"15min":1.6899999999999999},"min":{"1min":0.44,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":51.119999999999997,"5min":51.119999999999997,"15min":51.119999999999997},"last":6.6159999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:42:15 2024","interfaces":[{"interface":"back","average":{"1min":3.2349999999999999,"5min":1.9590000000000001,"15min":1.506},"min":{"1min":0.40799999999999997,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":16.422000000000001,"5min":16.422000000000001,"15min":16.422000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2649999999999999,"15min":1.1970000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":6.4340000000000002,"5min":6.4340000000000002,"15min":10.673},"last":0.39700000000000002}]}]},{"osd":0,"up_from":0,"seq":34359738576,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411525,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":897492,"kb_used_data":587980,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92847660,"statfs":{"total":95995035648,"available":95076003840,"internally_reserved":0,"allocated":602091520,"data_stored":600969274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":2.286,"15min":1.641},"min":{"1min":0.215,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":4.3630000000000004,"5min":16.763999999999999,"15min":22.338999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":1.931,"5min":2.641,"15min":1.9850000000000001},"min":{"1min":0.23200000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":27.361999999999998,"5min":30.427,"15min":30.427},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:42:15 2024","interfaces":[{"interface":"back","average":{"1min":2.3220000000000001,"5min":1.9910000000000001,"15min":1.6060000000000001},"min":{"1min":0.23300000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":12.101000000000001,"5min":12.349,"15min":12.349},"last":0.54600000000000004},{"interface":"front","average":{"1min":2.3090000000000002,"5min":2.1400000000000001,"15min":1.5089999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.193},"max":{"1min":12.18,"5min":13.782,"15min":13.782},"last":0.496}]},{"osd":3,"last update":"Thu Feb 22 19:42:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":2.3740000000000001,"15min":1.9410000000000001},"min":{"1min":0.28299999999999997,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":12.151999999999999,"5min":16.076000000000001,"15min":30.071000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":4.5179999999999998,"5min":3.0430000000000001,"15min":2.258},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.184},"max":{"1min":28.263999999999999,"5min":30.282,"15min":32.119},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:43:06 2024","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.585,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.0140000000000002,"5min":12.43,"15min":12.43},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.861,"15min":1.2529999999999999},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":4.6420000000000003,"5min":10.34,"15min":10.34},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:42:24 2024","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":2.339,"15min":1.399},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.8239999999999998,"5min":40.633000000000003,"15min":40.633000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":2.3580000000000001,"5min":2.2549999999999999,"15min":1.595},"min":{"1min":0.41799999999999998,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":7.2030000000000003,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.67600000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:43:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2669999999999999,"5min":1.8140000000000001,"15min":1.4990000000000001},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.311},"max":{"1min":7.1550000000000002,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.9530000000000001,"15min":1.304},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":4.5940000000000003,"5min":10.369999999999999,"15min":10.369999999999999},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:42:15 2024","interfaces":[{"interface":"back","average":{"1min":2.214,"5min":1.681,"15min":1.4370000000000001},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.29099999999999998},"max":{"1min":8.5649999999999995,"5min":12.484,"15min":12.484},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.744,"5min":1.9830000000000001,"15min":1.321},"min":{"1min":0.312,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":4.827,"5min":10.539,"15min":10.539},"last":0.436}]}]},{"osd":3,"up_from":26,"seq":111669149887,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1828144,"kb_used_data":1242408,"kb_used_omap":0,"kb_used_meta":585600,"kb_avail":91917008,"statfs":{"total":95995035648,"available":94123016192,"internally_reserved":0,"allocated":1272225792,"data_stored":1271076568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":599654400},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":2.544,"5min":1.964,"15min":1.8169999999999999},"min":{"1min":0.22,"5min":0.191,"15min":0.191},"max":{"1min":7.2560000000000002,"5min":9.9789999999999992,"15min":23.48},"last":0.53200000000000003},{"interface":"front","average":{"1min":2.7469999999999999,"5min":2.4209999999999998,"15min":2.1139999999999999},"min":{"1min":0.254,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":7.1980000000000004,"5min":18.172000000000001,"15min":23.411999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":3.395,"15min":2.427},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.749000000000001,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":2.7189999999999999,"5min":2.851,"15min":1.974},"min":{"1min":0.38400000000000001,"5min":0.23000000000000001,"15min":0.214},"max":{"1min":8.0869999999999997,"5min":22.187000000000001,"15min":22.187000000000001},"last":0.621}]},{"osd":2,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":2.5139999999999998,"5min":2.367,"15min":1.994},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.2800000000000002,"5min":20.105,"15min":20.105},"last":0.316},{"interface":"front","average":{"1min":2.823,"5min":2.5270000000000001,"15min":1.978},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.217},"max":{"1min":9.1479999999999997,"5min":35.939999999999998,"15min":35.939999999999998},"last":4.7519999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":2.3740000000000001,"5min":2.1699999999999999,"15min":1.73},"min":{"1min":0.30599999999999999,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":9.968,"5min":12.416,"15min":13.680999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.9890000000000001,"15min":1.5429999999999999},"min":{"1min":0.40400000000000003,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":3.6019999999999999,"5min":13.548,"15min":13.548},"last":0.34999999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:42:32 2024","interfaces":[{"interface":"back","average":{"1min":2.3959999999999999,"5min":1.776,"15min":1.478},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":6.4009999999999998,"5min":11.161,"15min":12.879},"last":0.81200000000000006},{"interface":"front","average":{"1min":2.0230000000000001,"5min":2.1019999999999999,"15min":1.6419999999999999},"min":{"1min":0.32900000000000001,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":7.1289999999999996,"5min":12.372999999999999,"15min":13.707000000000001},"last":1.2629999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:42:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3119999999999998,"5min":1.9019999999999999,"15min":1.514},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":7.1440000000000001,"5min":11.263,"15min":12.898999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":2.573,"5min":2.1230000000000002,"15min":1.772},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":10.02,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:43:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":2.0920000000000001,"15min":1.595},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":3.6259999999999999,"5min":13.483000000000001,"15min":13.483000000000001},"last":1.6140000000000001},{"interface":"front","average":{"1min":2.2770000000000001,"5min":2.0659999999999998,"15min":1.621},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.2240000000000002,"5min":11.539,"15min":12.846},"last":4.71}]}]},{"osd":4,"up_from":0,"seq":137438953661,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724727,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1998072,"kb_used_data":1684400,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":91747080,"statfs":{"total":95995035648,"available":93949009920,"internally_reserved":0,"allocated":1724825600,"data_stored":1723667640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.5249999999999999,"15min":1.278},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":4.4290000000000003,"5min":17.684000000000001,"15min":17.684000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":2.9049999999999998,"5min":1.831,"15min":1.4590000000000001},"min":{"1min":0.26100000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":11.9,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.29099999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4470000000000001,"15min":1.3420000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":6.0750000000000002,"5min":11.202,"15min":29.390999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":2.1640000000000001,"5min":1.6779999999999999,"15min":1.4159999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.218},"max":{"1min":5.4329999999999998,"5min":8.7490000000000006,"15min":29.367000000000001},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.9339999999999999,"15min":1.4059999999999999},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.774,"5min":11.367000000000001,"15min":14.613},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.8640000000000001,"15min":1.4650000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5579999999999998,"5min":16.099,"15min":16.099},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.573,"15min":1.5429999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.4459999999999997,"5min":6.0439999999999996,"15min":13.366},"last":3.7829999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.7490000000000001,"15min":1.5229999999999999},"min":{"1min":0.34999999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":7.2729999999999997,"5min":8.6069999999999993,"15min":10.16},"last":2.246}]},{"osd":4,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.0089999999999999,"15min":0.65900000000000003},"min":{"1min":0.29799999999999999,"5min":0.223,"15min":0.219},"max":{"1min":3.6669999999999998,"5min":5.75,"15min":5.75},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.855,"5min":1.2070000000000001,"15min":0.72099999999999997},"min":{"1min":0.32800000000000001,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":10.407,"5min":10.407,"15min":10.407},"last":0.373}]},{"osd":6,"last update":"Thu Feb 22 19:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":0.64600000000000002},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":2.0169999999999999,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.438},{"interface":"front","average":{"1min":1.756,"5min":1.238,"15min":0.78800000000000003},"min":{"1min":0.36699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.436,"5min":10.436,"15min":10.436},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:42:24 2024","interfaces":[{"interface":"back","average":{"1min":2.0099999999999998,"5min":1.0569999999999999,"15min":0.76200000000000001},"min":{"1min":0.32100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.7119999999999997,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.1879999999999999,"15min":0.78200000000000003},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":6.9630000000000001,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1393299456,"data_stored":1393295384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":583467008,"data_stored":583467008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1238302720,"data_stored":1238302720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681915928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":897318912,"data_stored":897318912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1382617088,"data_stored":1382613016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:43:15.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:43:15.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:15.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:15.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:15.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:15.765 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:15.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:15.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:15.097510+0000 mgr.y (mgr.14535) 545 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:15.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:43:15.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:15.955 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:15.983 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:15.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:15.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:16.050 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:15.097510+0000 mgr.y (mgr.14535) 545 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:16.051 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:15.097510+0000 mgr.y (mgr.14535) 545 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:16.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:16.258 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:16.263 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:16.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:16.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:16.395 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:43:16.395 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:43:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:16.430 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:16.434 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:16.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:16.498 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:43:16.511 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:43:16.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:16.543 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:43:16.578 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:43:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:16.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:16.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:16.630 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:43:16.671 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:43:16.702 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:43:16.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:16.745 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:43:16.802 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:17.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:17.083 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:17.083 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:15.655847+0000 mgr.y (mgr.14535) 546 : cluster [DBG] pgmap v708: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.7 GiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 3980/83556 objects misplaced (4.763%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:43:17.083 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:16.421765+0000 mon.a (mon.0) 1448 : audit [DBG] from='client.? 172.21.15.53:0/848328113' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:17.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:15.655847+0000 mgr.y (mgr.14535) 546 : cluster [DBG] pgmap v708: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.7 GiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 3980/83556 objects misplaced (4.763%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:43:17.084 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:16.421765+0000 mon.a (mon.0) 1448 : audit [DBG] from='client.? 172.21.15.53:0/848328113' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:15.655847+0000 mgr.y (mgr.14535) 546 : cluster [DBG] pgmap v708: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.7 GiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 3980/83556 objects misplaced (4.763%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:43:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:16.421765+0000 mon.a (mon.0) 1448 : audit [DBG] from='client.? 172.21.15.53:0/848328113' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:17.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:17.237 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:17.633 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:17.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:17.642 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:17.650 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:43:17.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:17.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:17.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:17.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:18.032 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:18.053 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:18.058 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:18.115 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:18.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:18.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:18.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:18.555 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:18.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:18.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:18.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:18.738 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:18.742 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:18.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:18.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:17.658224+0000 mgr.y (mgr.14535) 547 : cluster [DBG] pgmap v709: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.7 GiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 923 op/s; 3980/83556 objects misplaced (4.763%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:18.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:17.658224+0000 mgr.y (mgr.14535) 547 : cluster [DBG] pgmap v709: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.7 GiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 923 op/s; 3980/83556 objects misplaced (4.763%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:18.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:18.897 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:17.658224+0000 mgr.y (mgr.14535) 547 : cluster [DBG] pgmap v709: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.7 GiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 923 op/s; 3980/83556 objects misplaced (4.763%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:19.158 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:19.161 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:19.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:19.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:19.314 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:19.480 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:19.484 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:19.493 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:19.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:19.495 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:43:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:19] "GET /metrics HTTP/1.1" 200 176187 "" "Prometheus/2.33.4" 2024-02-22T19:43:19.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:19.648 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:20.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:43:20.221 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:20.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:20.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:20.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:20.370 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:20.402 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:20.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:20.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:20.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:20.564 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:20.825 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:20.830 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:20.830 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:19.660259+0000 mgr.y (mgr.14535) 548 : cluster [DBG] pgmap v710: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.4 GiB data, 6.2 GiB used, 441 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 3638/70632 objects misplaced (5.151%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:43:20.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:19.660259+0000 mgr.y (mgr.14535) 548 : cluster [DBG] pgmap v710: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.4 GiB data, 6.2 GiB used, 441 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 3638/70632 objects misplaced (5.151%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:43:20.831 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:20.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:21.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:19.660259+0000 mgr.y (mgr.14535) 548 : cluster [DBG] pgmap v710: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.4 GiB data, 6.2 GiB used, 441 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 3638/70632 objects misplaced (5.151%); 1.8 MiB/s, 28 objects/s recovering 2024-02-22T19:43:21.366 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:21.371 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:21.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:21.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:21.525 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:21.582 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:21.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:21.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:21.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:21.750 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:22.529 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:22.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:22.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:22.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:22.736 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:22.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:21.661805+0000 mgr.y (mgr.14535) 549 : cluster [DBG] pgmap v711: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.3 GiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s; 2997/66489 objects misplaced (4.508%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:43:22.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:22.795783+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.53:0/910096789' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:43:22.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:22.796254+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:43:22.840 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:21.661805+0000 mgr.y (mgr.14535) 549 : cluster [DBG] pgmap v711: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.3 GiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s; 2997/66489 objects misplaced (4.508%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:43:22.840 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:22.795783+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.53:0/910096789' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:43:22.840 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:22.796254+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:43:22.873 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:43:22.881 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:22.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:22.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:22.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:23.030 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:43:23.054 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:43:23.057 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:23.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:21.661805+0000 mgr.y (mgr.14535) 549 : cluster [DBG] pgmap v711: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.3 GiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s; 2997/66489 objects misplaced (4.508%); 2.2 MiB/s, 35 objects/s recovering 2024-02-22T19:43:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:22.795783+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.53:0/910096789' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:43:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:22.796254+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:43:23.330 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:23.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:23.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:23.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:23.513 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:23.790 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:23.791 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":712,"stamp":"2024-02-22T19:43:22.847842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1439831740,"num_objects":22163,"num_object_clones":0,"num_object_copies":66489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2997,"num_objects_unfound":0,"num_objects_dirty":22163,"num_whiteouts":0,"num_read":1962,"num_read_kb":1714,"num_write":239657,"num_write_kb":8349226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15372,"num_bytes_recovered":977878330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53180,"ondisk_log_size":53180,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":6305176,"kb_used_data":4200752,"kb_used_omap":0,"kb_used_meta":2103808,"kb_avail":462420584,"statfs":{"total":479975178240,"available":473518678016,"internally_reserved":0,"allocated":4301570048,"data_stored":4295785092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2154299392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-764542976,"num_objects":-11666,"num_object_clones":0,"num_object_copies":-34998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2019,"num_objects_unfound":0,"num_objects_dirty":-11666,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11666,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":22937600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.509654"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":311,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751844+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:43:11.751844+0000","last_peered":"2024-02-22T19:43:11.751844+0000","last_clean":"2024-02-22T19:43:11.751844+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:43:11.751844+0000","last_undegraded":"2024-02-22T19:43:11.751844+0000","last_fullsized":"2024-02-22T19:43:11.751844+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":341,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751602+0000","last_change":"2024-02-22T19:41:52.114804+0000","last_active":"2024-02-22T19:43:11.751602+0000","last_peered":"2024-02-22T19:43:11.751602+0000","last_clean":"2024-02-22T19:43:11.751602+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:43:11.751602+0000","last_undegraded":"2024-02-22T19:43:11.751602+0000","last_fullsized":"2024-02-22T19:43:11.751602+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":125,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744870+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:43:11.744870+0000","last_peered":"2024-02-22T19:43:11.744870+0000","last_clean":"2024-02-22T19:43:11.744870+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:43:11.744870+0000","last_undegraded":"2024-02-22T19:43:11.744870+0000","last_fullsized":"2024-02-22T19:43:11.744870+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":326,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765151+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:43:11.765151+0000","last_peered":"2024-02-22T19:43:11.765151+0000","last_clean":"2024-02-22T19:43:11.765151+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:43:11.765151+0000","last_undegraded":"2024-02-22T19:43:11.765151+0000","last_fullsized":"2024-02-22T19:43:11.765151+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":327,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.764020+0000","last_change":"2024-02-22T19:41:52.112817+0000","last_active":"2024-02-22T19:43:11.764020+0000","last_peered":"2024-02-22T19:43:11.764020+0000","last_clean":"2024-02-22T19:43:11.764020+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:43:11.764020+0000","last_undegraded":"2024-02-22T19:43:11.764020+0000","last_fullsized":"2024-02-22T19:43:11.764020+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":434,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765415+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:43:11.765415+0000","last_peered":"2024-02-22T19:43:11.765415+0000","last_clean":"2024-02-22T19:43:11.765415+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:43:11.765415+0000","last_undegraded":"2024-02-22T19:43:11.765415+0000","last_fullsized":"2024-02-22T19:43:11.765415+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":155,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744432+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:43:11.744432+0000","last_peered":"2024-02-22T19:43:11.744432+0000","last_clean":"2024-02-22T19:43:11.744432+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:43:11.744432+0000","last_undegraded":"2024-02-22T19:43:11.744432+0000","last_fullsized":"2024-02-22T19:43:11.744432+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":164,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752523+0000","last_change":"2024-02-22T19:41:52.113562+0000","last_active":"2024-02-22T19:43:11.752523+0000","last_peered":"2024-02-22T19:43:11.752523+0000","last_clean":"2024-02-22T19:43:11.752523+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:43:11.752523+0000","last_undegraded":"2024-02-22T19:43:11.752523+0000","last_fullsized":"2024-02-22T19:43:11.752523+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":231,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.762567+0000","last_change":"2024-02-22T19:41:52.752324+0000","last_active":"2024-02-22T19:43:11.762567+0000","last_peered":"2024-02-22T19:43:11.762567+0000","last_clean":"2024-02-22T19:43:11.762567+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:43:11.762567+0000","last_undegraded":"2024-02-22T19:43:11.762567+0000","last_fullsized":"2024-02-22T19:43:11.762567+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":151,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750714+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:43:11.750714+0000","last_peered":"2024-02-22T19:43:11.750714+0000","last_clean":"2024-02-22T19:43:11.750714+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:43:11.750714+0000","last_undegraded":"2024-02-22T19:43:11.750714+0000","last_fullsized":"2024-02-22T19:43:11.750714+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":181,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751751+0000","last_change":"2024-02-22T19:41:52.116983+0000","last_active":"2024-02-22T19:43:11.751751+0000","last_peered":"2024-02-22T19:43:11.751751+0000","last_clean":"2024-02-22T19:43:11.751751+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:43:11.751751+0000","last_undegraded":"2024-02-22T19:43:11.751751+0000","last_fullsized":"2024-02-22T19:43:11.751751+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":154,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.764895+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:43:11.764895+0000","last_peered":"2024-02-22T19:43:11.764895+0000","last_clean":"2024-02-22T19:43:11.764895+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:43:11.764895+0000","last_undegraded":"2024-02-22T19:43:11.764895+0000","last_fullsized":"2024-02-22T19:43:11.764895+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":229,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751993+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:43:11.751993+0000","last_peered":"2024-02-22T19:43:11.751993+0000","last_clean":"2024-02-22T19:43:11.751993+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:43:11.751993+0000","last_undegraded":"2024-02-22T19:43:11.751993+0000","last_fullsized":"2024-02-22T19:43:11.751993+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":128,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.749508+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:43:11.749508+0000","last_peered":"2024-02-22T19:43:11.749508+0000","last_clean":"2024-02-22T19:43:11.749508+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:43:11.749508+0000","last_undegraded":"2024-02-22T19:43:11.749508+0000","last_fullsized":"2024-02-22T19:43:11.749508+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":381,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.772353+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:43:11.772353+0000","last_peered":"2024-02-22T19:43:11.772353+0000","last_clean":"2024-02-22T19:43:11.772353+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:43:11.772353+0000","last_undegraded":"2024-02-22T19:43:11.772353+0000","last_fullsized":"2024-02-22T19:43:11.772353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":344,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.769756+0000","last_change":"2024-02-22T19:41:52.121546+0000","last_active":"2024-02-22T19:43:11.769756+0000","last_peered":"2024-02-22T19:43:11.769756+0000","last_clean":"2024-02-22T19:43:11.769756+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:43:11.769756+0000","last_undegraded":"2024-02-22T19:43:11.769756+0000","last_fullsized":"2024-02-22T19:43:11.769756+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":141,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745309+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:43:11.745309+0000","last_peered":"2024-02-22T19:43:11.745309+0000","last_clean":"2024-02-22T19:43:11.745309+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:43:11.745309+0000","last_undegraded":"2024-02-22T19:43:11.745309+0000","last_fullsized":"2024-02-22T19:43:11.745309+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":395,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.763492+0000","last_change":"2024-02-22T19:41:53.187026+0000","last_active":"2024-02-22T19:43:11.763492+0000","last_peered":"2024-02-22T19:43:11.763492+0000","last_clean":"2024-02-22T19:43:11.763492+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:43:11.763492+0000","last_undegraded":"2024-02-22T19:43:11.763492+0000","last_fullsized":"2024-02-22T19:43:11.763492+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":360,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750549+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:43:11.750549+0000","last_peered":"2024-02-22T19:43:11.750549+0000","last_clean":"2024-02-22T19:43:11.750549+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:43:11.750549+0000","last_undegraded":"2024-02-22T19:43:11.750549+0000","last_fullsized":"2024-02-22T19:43:11.750549+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":349,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744435+0000","last_change":"2024-02-22T19:41:52.110049+0000","last_active":"2024-02-22T19:43:11.744435+0000","last_peered":"2024-02-22T19:43:11.744435+0000","last_clean":"2024-02-22T19:43:11.744435+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:43:11.744435+0000","last_undegraded":"2024-02-22T19:43:11.744435+0000","last_fullsized":"2024-02-22T19:43:11.744435+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":161,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754481+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:43:11.754481+0000","last_peered":"2024-02-22T19:43:11.754481+0000","last_clean":"2024-02-22T19:43:11.754481+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:43:11.754481+0000","last_undegraded":"2024-02-22T19:43:11.754481+0000","last_fullsized":"2024-02-22T19:43:11.754481+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":349,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.747773+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:43:11.747773+0000","last_peered":"2024-02-22T19:43:11.747773+0000","last_clean":"2024-02-22T19:43:11.747773+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:43:11.747773+0000","last_undegraded":"2024-02-22T19:43:11.747773+0000","last_fullsized":"2024-02-22T19:43:11.747773+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":347,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.766412+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:43:11.766412+0000","last_peered":"2024-02-22T19:43:11.766412+0000","last_clean":"2024-02-22T19:43:11.766412+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:43:11.766412+0000","last_undegraded":"2024-02-22T19:43:11.766412+0000","last_fullsized":"2024-02-22T19:43:11.766412+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":389,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752525+0000","last_change":"2024-02-22T19:41:52.743206+0000","last_active":"2024-02-22T19:43:11.752525+0000","last_peered":"2024-02-22T19:43:11.752525+0000","last_clean":"2024-02-22T19:43:11.752525+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:43:11.752525+0000","last_undegraded":"2024-02-22T19:43:11.752525+0000","last_fullsized":"2024-02-22T19:43:11.752525+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":329,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750285+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:43:11.750285+0000","last_peered":"2024-02-22T19:43:11.750285+0000","last_clean":"2024-02-22T19:43:11.750285+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:43:11.750285+0000","last_undegraded":"2024-02-22T19:43:11.750285+0000","last_fullsized":"2024-02-22T19:43:11.750285+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":294,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750889+0000","last_change":"2024-02-22T19:41:52.111808+0000","last_active":"2024-02-22T19:43:11.750889+0000","last_peered":"2024-02-22T19:43:11.750889+0000","last_clean":"2024-02-22T19:43:11.750889+0000","last_became_active":"2024-02-22T19:41:52.110758+0000","last_became_peered":"2024-02-22T19:41:52.110758+0000","last_unstale":"2024-02-22T19:43:11.750889+0000","last_undegraded":"2024-02-22T19:43:11.750889+0000","last_fullsized":"2024-02-22T19:43:11.750889+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":341,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751507+0000","last_change":"2024-02-22T19:41:52.114607+0000","last_active":"2024-02-22T19:43:11.751507+0000","last_peered":"2024-02-22T19:43:11.751507+0000","last_clean":"2024-02-22T19:43:11.751507+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:43:11.751507+0000","last_undegraded":"2024-02-22T19:43:11.751507+0000","last_fullsized":"2024-02-22T19:43:11.751507+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":348,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.766458+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:43:11.766458+0000","last_peered":"2024-02-22T19:43:11.766458+0000","last_clean":"2024-02-22T19:43:11.766458+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:43:11.766458+0000","last_undegraded":"2024-02-22T19:43:11.766458+0000","last_fullsized":"2024-02-22T19:43:11.766458+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":350,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765317+0000","last_change":"2024-02-22T19:41:52.118842+0000","last_active":"2024-02-22T19:43:11.765317+0000","last_peered":"2024-02-22T19:43:11.765317+0000","last_clean":"2024-02-22T19:43:11.765317+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:43:11.765317+0000","last_undegraded":"2024-02-22T19:43:11.765317+0000","last_fullsized":"2024-02-22T19:43:11.765317+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":361,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.749400+0000","last_change":"2024-02-22T19:41:52.104771+0000","last_active":"2024-02-22T19:43:11.749400+0000","last_peered":"2024-02-22T19:43:11.749400+0000","last_clean":"2024-02-22T19:43:11.749400+0000","last_became_active":"2024-02-22T19:41:52.104234+0000","last_became_peered":"2024-02-22T19:41:52.104234+0000","last_unstale":"2024-02-22T19:43:11.749400+0000","last_undegraded":"2024-02-22T19:43:11.749400+0000","last_fullsized":"2024-02-22T19:43:11.749400+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.702590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.702590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":146,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751845+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:43:11.751845+0000","last_peered":"2024-02-22T19:43:11.751845+0000","last_clean":"2024-02-22T19:43:11.751845+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:43:11.751845+0000","last_undegraded":"2024-02-22T19:43:11.751845+0000","last_fullsized":"2024-02-22T19:43:11.751845+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":368,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765269+0000","last_change":"2024-02-22T19:41:53.508908+0000","last_active":"2024-02-22T19:43:11.765269+0000","last_peered":"2024-02-22T19:43:11.765269+0000","last_clean":"2024-02-22T19:43:11.765269+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:43:11.765269+0000","last_undegraded":"2024-02-22T19:43:11.765269+0000","last_fullsized":"2024-02-22T19:43:11.765269+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":332,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751072+0000","last_change":"2024-02-22T19:41:52.112028+0000","last_active":"2024-02-22T19:43:11.751072+0000","last_peered":"2024-02-22T19:43:11.751072+0000","last_clean":"2024-02-22T19:43:11.751072+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:43:11.751072+0000","last_undegraded":"2024-02-22T19:43:11.751072+0000","last_fullsized":"2024-02-22T19:43:11.751072+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":341,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765061+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:43:11.765061+0000","last_peered":"2024-02-22T19:43:11.765061+0000","last_clean":"2024-02-22T19:43:11.765061+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:43:11.765061+0000","last_undegraded":"2024-02-22T19:43:11.765061+0000","last_fullsized":"2024-02-22T19:43:11.765061+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":369,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754572+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:43:11.754572+0000","last_peered":"2024-02-22T19:43:11.754572+0000","last_clean":"2024-02-22T19:43:11.754572+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:43:11.754572+0000","last_undegraded":"2024-02-22T19:43:11.754572+0000","last_fullsized":"2024-02-22T19:43:11.754572+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":373,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745627+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:43:11.745627+0000","last_peered":"2024-02-22T19:43:11.745627+0000","last_clean":"2024-02-22T19:43:11.745627+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:43:11.745627+0000","last_undegraded":"2024-02-22T19:43:11.745627+0000","last_fullsized":"2024-02-22T19:43:11.745627+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":362,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.762661+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:43:11.762661+0000","last_peered":"2024-02-22T19:43:11.762661+0000","last_clean":"2024-02-22T19:43:11.762661+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:43:11.762661+0000","last_undegraded":"2024-02-22T19:43:11.762661+0000","last_fullsized":"2024-02-22T19:43:11.762661+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":340,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745518+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:43:11.745518+0000","last_peered":"2024-02-22T19:43:11.745518+0000","last_clean":"2024-02-22T19:43:11.745518+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:43:11.745518+0000","last_undegraded":"2024-02-22T19:43:11.745518+0000","last_fullsized":"2024-02-22T19:43:11.745518+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":132,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.749514+0000","last_change":"2024-02-22T19:41:52.104077+0000","last_active":"2024-02-22T19:43:11.749514+0000","last_peered":"2024-02-22T19:43:11.749514+0000","last_clean":"2024-02-22T19:43:11.749514+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:43:11.749514+0000","last_undegraded":"2024-02-22T19:43:11.749514+0000","last_fullsized":"2024-02-22T19:43:11.749514+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":136,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745689+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:43:11.745689+0000","last_peered":"2024-02-22T19:43:11.745689+0000","last_clean":"2024-02-22T19:43:11.745689+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:43:11.745689+0000","last_undegraded":"2024-02-22T19:43:11.745689+0000","last_fullsized":"2024-02-22T19:43:11.745689+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":149,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754201+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:43:11.754201+0000","last_peered":"2024-02-22T19:43:11.754201+0000","last_clean":"2024-02-22T19:43:11.754201+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:43:11.754201+0000","last_undegraded":"2024-02-22T19:43:11.754201+0000","last_fullsized":"2024-02-22T19:43:11.754201+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":257,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345421+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:43:14.345421+0000","last_peered":"2024-02-22T19:43:14.345421+0000","last_clean":"2024-02-22T19:43:14.345421+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:43:14.345421+0000","last_undegraded":"2024-02-22T19:43:14.345421+0000","last_fullsized":"2024-02-22T19:43:14.345421+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":342,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.749575+0000","last_change":"2024-02-22T19:41:52.103816+0000","last_active":"2024-02-22T19:43:11.749575+0000","last_peered":"2024-02-22T19:43:11.749575+0000","last_clean":"2024-02-22T19:43:11.749575+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:43:11.749575+0000","last_undegraded":"2024-02-22T19:43:11.749575+0000","last_fullsized":"2024-02-22T19:43:11.749575+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":356,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.763239+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:43:11.763239+0000","last_peered":"2024-02-22T19:43:11.763239+0000","last_clean":"2024-02-22T19:43:11.763239+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:43:11.763239+0000","last_undegraded":"2024-02-22T19:43:11.763239+0000","last_fullsized":"2024-02-22T19:43:11.763239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":334,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.771642+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:43:11.771642+0000","last_peered":"2024-02-22T19:43:11.771642+0000","last_clean":"2024-02-22T19:43:11.771642+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:43:11.771642+0000","last_undegraded":"2024-02-22T19:43:11.771642+0000","last_fullsized":"2024-02-22T19:43:11.771642+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":494,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.347448+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:43:14.347448+0000","last_peered":"2024-02-22T19:43:14.347448+0000","last_clean":"2024-02-22T19:43:14.347448+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:43:14.347448+0000","last_undegraded":"2024-02-22T19:43:14.347448+0000","last_fullsized":"2024-02-22T19:43:14.347448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":140,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.749190+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:43:11.749190+0000","last_peered":"2024-02-22T19:43:11.749190+0000","last_clean":"2024-02-22T19:43:11.749190+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:43:11.749190+0000","last_undegraded":"2024-02-22T19:43:11.749190+0000","last_fullsized":"2024-02-22T19:43:11.749190+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":170,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752605+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:43:11.752605+0000","last_peered":"2024-02-22T19:43:11.752605+0000","last_clean":"2024-02-22T19:43:11.752605+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:43:11.752605+0000","last_undegraded":"2024-02-22T19:43:11.752605+0000","last_fullsized":"2024-02-22T19:43:11.752605+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":357,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.772610+0000","last_change":"2024-02-22T19:41:38.526359+0000","last_active":"2024-02-22T19:43:11.772610+0000","last_peered":"2024-02-22T19:43:11.772610+0000","last_clean":"2024-02-22T19:43:11.772610+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:11.772610+0000","last_undegraded":"2024-02-22T19:43:11.772610+0000","last_fullsized":"2024-02-22T19:43:11.772610+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.526259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.526259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":365,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750672+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:43:11.750672+0000","last_peered":"2024-02-22T19:43:11.750672+0000","last_clean":"2024-02-22T19:43:11.750672+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:43:11.750672+0000","last_undegraded":"2024-02-22T19:43:11.750672+0000","last_fullsized":"2024-02-22T19:43:11.750672+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":142,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.764172+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:43:11.764172+0000","last_peered":"2024-02-22T19:43:11.764172+0000","last_clean":"2024-02-22T19:43:11.764172+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:43:11.764172+0000","last_undegraded":"2024-02-22T19:43:11.764172+0000","last_fullsized":"2024-02-22T19:43:11.764172+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":356,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754525+0000","last_change":"2024-02-22T19:43:09.640164+0000","last_active":"2024-02-22T19:43:11.754525+0000","last_peered":"2024-02-22T19:43:11.754525+0000","last_clean":"2024-02-22T19:43:11.754525+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:43:11.754525+0000","last_undegraded":"2024-02-22T19:43:11.754525+0000","last_fullsized":"2024-02-22T19:43:11.754525+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":318,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750802+0000","last_change":"2024-02-22T19:41:52.112840+0000","last_active":"2024-02-22T19:43:11.750802+0000","last_peered":"2024-02-22T19:43:11.750802+0000","last_clean":"2024-02-22T19:43:11.750802+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:43:11.750802+0000","last_undegraded":"2024-02-22T19:43:11.750802+0000","last_fullsized":"2024-02-22T19:43:11.750802+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":356,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.769804+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:43:11.769804+0000","last_peered":"2024-02-22T19:43:11.769804+0000","last_clean":"2024-02-22T19:43:11.769804+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:43:11.769804+0000","last_undegraded":"2024-02-22T19:43:11.769804+0000","last_fullsized":"2024-02-22T19:43:11.769804+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"354'14915","reported_seq":14992,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.407995+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:43:19.407995+0000","last_peered":"2024-02-22T19:43:19.407995+0000","last_clean":"2024-02-22T19:43:19.407995+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:43:19.407995+0000","last_undegraded":"2024-02-22T19:43:19.407995+0000","last_fullsized":"2024-02-22T19:43:19.407995+0000","mapping_epoch":285,"log_start":"342'11840","ondisk_log_start":"342'11840","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3075,"ondisk_log_size":3075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85655552,"num_objects":1307,"num_object_clones":0,"num_object_copies":3921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14915,"num_write_kb":519104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":187,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751783+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:43:11.751783+0000","last_peered":"2024-02-22T19:43:11.751783+0000","last_clean":"2024-02-22T19:43:11.751783+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:43:11.751783+0000","last_undegraded":"2024-02-22T19:43:11.751783+0000","last_fullsized":"2024-02-22T19:43:11.751783+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":399,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750713+0000","last_change":"2024-02-22T19:39:41.754077+0000","last_active":"2024-02-22T19:43:11.750713+0000","last_peered":"2024-02-22T19:43:11.750713+0000","last_clean":"2024-02-22T19:43:11.750713+0000","last_became_active":"2024-02-22T19:39:22.246613+0000","last_became_peered":"2024-02-22T19:39:22.246613+0000","last_unstale":"2024-02-22T19:43:11.750713+0000","last_undegraded":"2024-02-22T19:43:11.750713+0000","last_fullsized":"2024-02-22T19:43:11.750713+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":210,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750713+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:43:11.750713+0000","last_peered":"2024-02-22T19:43:11.750713+0000","last_clean":"2024-02-22T19:43:11.750713+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:43:11.750713+0000","last_undegraded":"2024-02-22T19:43:11.750713+0000","last_fullsized":"2024-02-22T19:43:11.750713+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":576,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345546+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:43:14.345546+0000","last_peered":"2024-02-22T19:43:14.345546+0000","last_clean":"2024-02-22T19:43:14.345546+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:43:14.345546+0000","last_undegraded":"2024-02-22T19:43:14.345546+0000","last_fullsized":"2024-02-22T19:43:14.345546+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"354'15072","reported_seq":26953,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.494303+0000","last_change":"2024-02-22T19:42:34.755451+0000","last_active":"2024-02-22T19:43:19.494303+0000","last_peered":"2024-02-22T19:43:19.494303+0000","last_clean":"2024-02-22T19:43:19.494303+0000","last_became_active":"2024-02-22T19:42:34.737692+0000","last_became_peered":"2024-02-22T19:42:34.737692+0000","last_unstale":"2024-02-22T19:43:19.494303+0000","last_undegraded":"2024-02-22T19:43:19.494303+0000","last_fullsized":"2024-02-22T19:43:19.494303+0000","mapping_epoch":339,"log_start":"337'11653","ondisk_log_start":"337'11653","created":285,"last_epoch_clean":340,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3419,"ondisk_log_size":3419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88735744,"num_objects":1354,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15072,"num_write_kb":525632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5891,"num_bytes_recovered":386072576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":312,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765233+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:43:11.765233+0000","last_peered":"2024-02-22T19:43:11.765233+0000","last_clean":"2024-02-22T19:43:11.765233+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:43:11.765233+0000","last_undegraded":"2024-02-22T19:43:11.765233+0000","last_fullsized":"2024-02-22T19:43:11.765233+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":56,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.772302+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:43:11.772302+0000","last_peered":"2024-02-22T19:43:11.772302+0000","last_clean":"2024-02-22T19:43:11.772302+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:43:11.772302+0000","last_undegraded":"2024-02-22T19:43:11.772302+0000","last_fullsized":"2024-02-22T19:43:11.772302+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":200,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744173+0000","last_change":"2024-02-22T19:41:52.597069+0000","last_active":"2024-02-22T19:43:11.744173+0000","last_peered":"2024-02-22T19:43:11.744173+0000","last_clean":"2024-02-22T19:43:11.744173+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:43:11.744173+0000","last_undegraded":"2024-02-22T19:43:11.744173+0000","last_fullsized":"2024-02-22T19:43:11.744173+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":372,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.775616+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:43:11.775616+0000","last_peered":"2024-02-22T19:43:11.775616+0000","last_clean":"2024-02-22T19:43:11.775616+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:43:11.775616+0000","last_undegraded":"2024-02-22T19:43:11.775616+0000","last_fullsized":"2024-02-22T19:43:11.775616+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"354'14910","reported_seq":14986,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.432369+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:43:19.432369+0000","last_peered":"2024-02-22T19:43:19.432369+0000","last_clean":"2024-02-22T19:43:19.432369+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:43:19.432369+0000","last_undegraded":"2024-02-22T19:43:19.432369+0000","last_fullsized":"2024-02-22T19:43:19.432369+0000","mapping_epoch":285,"log_start":"340'11800","ondisk_log_start":"340'11800","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3110,"ondisk_log_size":3110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":91488256,"num_objects":1396,"num_object_clones":0,"num_object_copies":4188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14910,"num_write_kb":521792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":318,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.748874+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:43:11.748874+0000","last_peered":"2024-02-22T19:43:11.748874+0000","last_clean":"2024-02-22T19:43:11.748874+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:43:11.748874+0000","last_undegraded":"2024-02-22T19:43:11.748874+0000","last_fullsized":"2024-02-22T19:43:11.748874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":398,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.773009+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:43:11.773009+0000","last_peered":"2024-02-22T19:43:11.773009+0000","last_clean":"2024-02-22T19:43:11.773009+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:43:11.773009+0000","last_undegraded":"2024-02-22T19:43:11.773009+0000","last_fullsized":"2024-02-22T19:43:11.773009+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":356,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754668+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:43:11.754668+0000","last_peered":"2024-02-22T19:43:11.754668+0000","last_clean":"2024-02-22T19:43:11.754668+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:43:11.754668+0000","last_undegraded":"2024-02-22T19:43:11.754668+0000","last_fullsized":"2024-02-22T19:43:11.754668+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":347,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745528+0000","last_change":"2024-02-22T19:41:52.135450+0000","last_active":"2024-02-22T19:43:11.745528+0000","last_peered":"2024-02-22T19:43:11.745528+0000","last_clean":"2024-02-22T19:43:11.745528+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:43:11.745528+0000","last_undegraded":"2024-02-22T19:43:11.745528+0000","last_fullsized":"2024-02-22T19:43:11.745528+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"354'15023","reported_seq":15100,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.432268+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:43:19.432268+0000","last_peered":"2024-02-22T19:43:19.432268+0000","last_clean":"2024-02-22T19:43:19.432268+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:43:19.432268+0000","last_undegraded":"2024-02-22T19:43:19.432268+0000","last_fullsized":"2024-02-22T19:43:19.432268+0000","mapping_epoch":285,"log_start":"341'11931","ondisk_log_start":"341'11931","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3092,"ondisk_log_size":3092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":91553792,"num_objects":1397,"num_object_clones":0,"num_object_copies":4191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15023,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":140,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750436+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:43:11.750436+0000","last_peered":"2024-02-22T19:43:11.750436+0000","last_clean":"2024-02-22T19:43:11.750436+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:43:11.750436+0000","last_undegraded":"2024-02-22T19:43:11.750436+0000","last_fullsized":"2024-02-22T19:43:11.750436+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":377,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.774273+0000","last_change":"2024-02-22T19:41:16.236762+0000","last_active":"2024-02-22T19:43:11.774273+0000","last_peered":"2024-02-22T19:43:11.774273+0000","last_clean":"2024-02-22T19:43:11.774273+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:43:11.774273+0000","last_undegraded":"2024-02-22T19:43:11.774273+0000","last_fullsized":"2024-02-22T19:43:11.774273+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.236664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.236664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":160,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751669+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:43:11.751669+0000","last_peered":"2024-02-22T19:43:11.751669+0000","last_clean":"2024-02-22T19:43:11.751669+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:43:11.751669+0000","last_undegraded":"2024-02-22T19:43:11.751669+0000","last_fullsized":"2024-02-22T19:43:11.751669+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":152,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.773264+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:43:11.773264+0000","last_peered":"2024-02-22T19:43:11.773264+0000","last_clean":"2024-02-22T19:43:11.773264+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:43:11.773264+0000","last_undegraded":"2024-02-22T19:43:11.773264+0000","last_fullsized":"2024-02-22T19:43:11.773264+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"354'15146","reported_seq":15223,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.372160+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:43:19.372160+0000","last_peered":"2024-02-22T19:43:19.372160+0000","last_clean":"2024-02-22T19:43:19.372160+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:43:19.372160+0000","last_undegraded":"2024-02-22T19:43:19.372160+0000","last_fullsized":"2024-02-22T19:43:19.372160+0000","mapping_epoch":285,"log_start":"337'11543","ondisk_log_start":"337'11543","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3603,"ondisk_log_size":3603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":91357184,"num_objects":1394,"num_object_clones":0,"num_object_copies":4182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15146,"num_write_kb":529280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":187,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750626+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:43:11.750626+0000","last_peered":"2024-02-22T19:43:11.750626+0000","last_clean":"2024-02-22T19:43:11.750626+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:43:11.750626+0000","last_undegraded":"2024-02-22T19:43:11.750626+0000","last_fullsized":"2024-02-22T19:43:11.750626+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":335,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.764273+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:43:11.764273+0000","last_peered":"2024-02-22T19:43:11.764273+0000","last_clean":"2024-02-22T19:43:11.764273+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:43:11.764273+0000","last_undegraded":"2024-02-22T19:43:11.764273+0000","last_fullsized":"2024-02-22T19:43:11.764273+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751733+0000","last_change":"2024-02-22T19:41:52.111972+0000","last_active":"2024-02-22T19:43:11.751733+0000","last_peered":"2024-02-22T19:43:11.751733+0000","last_clean":"2024-02-22T19:43:11.751733+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:43:11.751733+0000","last_undegraded":"2024-02-22T19:43:11.751733+0000","last_fullsized":"2024-02-22T19:43:11.751733+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":270,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745201+0000","last_change":"2024-02-22T19:41:52.113125+0000","last_active":"2024-02-22T19:43:11.745201+0000","last_peered":"2024-02-22T19:43:11.745201+0000","last_clean":"2024-02-22T19:43:11.745201+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:43:11.745201+0000","last_undegraded":"2024-02-22T19:43:11.745201+0000","last_fullsized":"2024-02-22T19:43:11.745201+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":575,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345639+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:43:14.345639+0000","last_peered":"2024-02-22T19:43:14.345639+0000","last_clean":"2024-02-22T19:43:14.345639+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:43:14.345639+0000","last_undegraded":"2024-02-22T19:43:14.345639+0000","last_fullsized":"2024-02-22T19:43:14.345639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":158,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.747691+0000","last_change":"2024-02-22T19:41:52.101375+0000","last_active":"2024-02-22T19:43:11.747691+0000","last_peered":"2024-02-22T19:43:11.747691+0000","last_clean":"2024-02-22T19:43:11.747691+0000","last_became_active":"2024-02-22T19:41:52.101126+0000","last_became_peered":"2024-02-22T19:41:52.101126+0000","last_unstale":"2024-02-22T19:43:11.747691+0000","last_undegraded":"2024-02-22T19:43:11.747691+0000","last_fullsized":"2024-02-22T19:43:11.747691+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"354'14766","reported_seq":14843,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.420997+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:43:19.420997+0000","last_peered":"2024-02-22T19:43:19.420997+0000","last_clean":"2024-02-22T19:43:19.420997+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:43:19.420997+0000","last_undegraded":"2024-02-22T19:43:19.420997+0000","last_fullsized":"2024-02-22T19:43:19.420997+0000","mapping_epoch":285,"log_start":"340'11635","ondisk_log_start":"340'11635","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3131,"ondisk_log_size":3131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":87031808,"num_objects":1328,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14766,"num_write_kb":515008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":377,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750007+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:43:11.750007+0000","last_peered":"2024-02-22T19:43:11.750007+0000","last_clean":"2024-02-22T19:43:11.750007+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:43:11.750007+0000","last_undegraded":"2024-02-22T19:43:11.750007+0000","last_fullsized":"2024-02-22T19:43:11.750007+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":327,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754781+0000","last_change":"2024-02-22T19:41:52.109613+0000","last_active":"2024-02-22T19:43:11.754781+0000","last_peered":"2024-02-22T19:43:11.754781+0000","last_clean":"2024-02-22T19:43:11.754781+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:43:11.754781+0000","last_undegraded":"2024-02-22T19:43:11.754781+0000","last_fullsized":"2024-02-22T19:43:11.754781+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":340,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744675+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:43:11.744675+0000","last_peered":"2024-02-22T19:43:11.744675+0000","last_clean":"2024-02-22T19:43:11.744675+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:43:11.744675+0000","last_undegraded":"2024-02-22T19:43:11.744675+0000","last_fullsized":"2024-02-22T19:43:11.744675+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.772036+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:43:11.772036+0000","last_peered":"2024-02-22T19:43:11.772036+0000","last_clean":"2024-02-22T19:43:11.772036+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:43:11.772036+0000","last_undegraded":"2024-02-22T19:43:11.772036+0000","last_fullsized":"2024-02-22T19:43:11.772036+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":428,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.771723+0000","last_change":"2024-02-22T19:41:20.253383+0000","last_active":"2024-02-22T19:43:11.771723+0000","last_peered":"2024-02-22T19:43:11.771723+0000","last_clean":"2024-02-22T19:43:11.771723+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:11.771723+0000","last_undegraded":"2024-02-22T19:43:11.771723+0000","last_fullsized":"2024-02-22T19:43:11.771723+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:41:20.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:20.253277+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":371,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.772678+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:43:11.772678+0000","last_peered":"2024-02-22T19:43:11.772678+0000","last_clean":"2024-02-22T19:43:11.772678+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:43:11.772678+0000","last_undegraded":"2024-02-22T19:43:11.772678+0000","last_fullsized":"2024-02-22T19:43:11.772678+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"354'15008","reported_seq":15085,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.654214+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:43:19.654214+0000","last_peered":"2024-02-22T19:43:19.654214+0000","last_clean":"2024-02-22T19:43:19.654214+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:43:19.654214+0000","last_undegraded":"2024-02-22T19:43:19.654214+0000","last_fullsized":"2024-02-22T19:43:19.654214+0000","mapping_epoch":285,"log_start":"341'11845","ondisk_log_start":"341'11845","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3163,"ondisk_log_size":3163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":91095040,"num_objects":1390,"num_object_clones":0,"num_object_copies":4170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15008,"num_write_kb":524736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":160,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752637+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:43:11.752637+0000","last_peered":"2024-02-22T19:43:11.752637+0000","last_clean":"2024-02-22T19:43:11.752637+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:43:11.752637+0000","last_undegraded":"2024-02-22T19:43:11.752637+0000","last_fullsized":"2024-02-22T19:43:11.752637+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":377,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.749159+0000","last_change":"2024-02-22T19:41:52.104562+0000","last_active":"2024-02-22T19:43:11.749159+0000","last_peered":"2024-02-22T19:43:11.749159+0000","last_clean":"2024-02-22T19:43:11.749159+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:43:11.749159+0000","last_undegraded":"2024-02-22T19:43:11.749159+0000","last_fullsized":"2024-02-22T19:43:11.749159+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":345,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.762429+0000","last_change":"2024-02-22T19:41:52.111795+0000","last_active":"2024-02-22T19:43:11.762429+0000","last_peered":"2024-02-22T19:43:11.762429+0000","last_clean":"2024-02-22T19:43:11.762429+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:43:11.762429+0000","last_undegraded":"2024-02-22T19:43:11.762429+0000","last_fullsized":"2024-02-22T19:43:11.762429+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":385,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.754608+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:43:11.754608+0000","last_peered":"2024-02-22T19:43:11.754608+0000","last_clean":"2024-02-22T19:43:11.754608+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:43:11.754608+0000","last_undegraded":"2024-02-22T19:43:11.754608+0000","last_fullsized":"2024-02-22T19:43:11.754608+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":338,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750418+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:43:11.750418+0000","last_peered":"2024-02-22T19:43:11.750418+0000","last_clean":"2024-02-22T19:43:11.750418+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:43:11.750418+0000","last_undegraded":"2024-02-22T19:43:11.750418+0000","last_fullsized":"2024-02-22T19:43:11.750418+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"354'15028","reported_seq":17129,"reported_epoch":354,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:43:19.403070+0000","last_change":"2024-02-22T19:41:52.760268+0000","last_active":"2024-02-22T19:43:19.403070+0000","last_peered":"2024-02-22T19:43:19.403070+0000","last_clean":"2024-02-22T19:41:51.046554+0000","last_became_active":"2024-02-22T19:41:52.395683+0000","last_became_peered":"2024-02-22T19:41:52.395683+0000","last_unstale":"2024-02-22T19:43:19.403070+0000","last_undegraded":"2024-02-22T19:43:19.403070+0000","last_fullsized":"2024-02-22T19:43:19.403070+0000","mapping_epoch":323,"log_start":"342'11945","ondisk_log_start":"342'11945","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3083,"ondisk_log_size":3083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":89391104,"num_objects":1364,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1002,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15028,"num_write_kb":524544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1007,"num_bytes_recovered":65994752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,6],"avail_no_missing":["5","2","6"],"object_location_counts":[{"shards":"2,5,6","objects":1364}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":231,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745471+0000","last_change":"2024-02-22T19:41:52.114247+0000","last_active":"2024-02-22T19:43:11.745471+0000","last_peered":"2024-02-22T19:43:11.745471+0000","last_clean":"2024-02-22T19:43:11.745471+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:43:11.745471+0000","last_undegraded":"2024-02-22T19:43:11.745471+0000","last_fullsized":"2024-02-22T19:43:11.745471+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":343,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744578+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:43:11.744578+0000","last_peered":"2024-02-22T19:43:11.744578+0000","last_clean":"2024-02-22T19:43:11.744578+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:43:11.744578+0000","last_undegraded":"2024-02-22T19:43:11.744578+0000","last_fullsized":"2024-02-22T19:43:11.744578+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":128,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765584+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:43:11.765584+0000","last_peered":"2024-02-22T19:43:11.765584+0000","last_clean":"2024-02-22T19:43:11.765584+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:43:11.765584+0000","last_undegraded":"2024-02-22T19:43:11.765584+0000","last_fullsized":"2024-02-22T19:43:11.765584+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1222,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:17.844247+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:43:17.844247+0000","last_peered":"2024-02-22T19:43:17.844247+0000","last_clean":"2024-02-22T19:43:17.844247+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:43:17.844247+0000","last_undegraded":"2024-02-22T19:43:17.844247+0000","last_fullsized":"2024-02-22T19:43:17.844247+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":841,"num_read_kb":836,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":160,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750692+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:43:11.750692+0000","last_peered":"2024-02-22T19:43:11.750692+0000","last_clean":"2024-02-22T19:43:11.750692+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:43:11.750692+0000","last_undegraded":"2024-02-22T19:43:11.750692+0000","last_fullsized":"2024-02-22T19:43:11.750692+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":205,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752677+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:43:11.752677+0000","last_peered":"2024-02-22T19:43:11.752677+0000","last_clean":"2024-02-22T19:43:11.752677+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:43:11.752677+0000","last_undegraded":"2024-02-22T19:43:11.752677+0000","last_fullsized":"2024-02-22T19:43:11.752677+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"354'14805","reported_seq":14882,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.386046+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:43:19.386046+0000","last_peered":"2024-02-22T19:43:19.386046+0000","last_clean":"2024-02-22T19:43:19.386046+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:43:19.386046+0000","last_undegraded":"2024-02-22T19:43:19.386046+0000","last_fullsized":"2024-02-22T19:43:19.386046+0000","mapping_epoch":285,"log_start":"336'11264","ondisk_log_start":"336'11264","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3541,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88145920,"num_objects":1345,"num_object_clones":0,"num_object_copies":4035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14805,"num_write_kb":516800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":207,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.763935+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:43:11.763935+0000","last_peered":"2024-02-22T19:43:11.763935+0000","last_clean":"2024-02-22T19:43:11.763935+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:43:11.763935+0000","last_undegraded":"2024-02-22T19:43:11.763935+0000","last_fullsized":"2024-02-22T19:43:11.763935+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":329,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744287+0000","last_change":"2024-02-22T19:41:52.094945+0000","last_active":"2024-02-22T19:43:11.744287+0000","last_peered":"2024-02-22T19:43:11.744287+0000","last_clean":"2024-02-22T19:43:11.744287+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:43:11.744287+0000","last_undegraded":"2024-02-22T19:43:11.744287+0000","last_fullsized":"2024-02-22T19:43:11.744287+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":344,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.744773+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:43:11.744773+0000","last_peered":"2024-02-22T19:43:11.744773+0000","last_clean":"2024-02-22T19:43:11.744773+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:43:11.744773+0000","last_undegraded":"2024-02-22T19:43:11.744773+0000","last_fullsized":"2024-02-22T19:43:11.744773+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":165,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.753034+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:43:11.753034+0000","last_peered":"2024-02-22T19:43:11.753034+0000","last_clean":"2024-02-22T19:43:11.753034+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:43:11.753034+0000","last_undegraded":"2024-02-22T19:43:11.753034+0000","last_fullsized":"2024-02-22T19:43:11.753034+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":314,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751656+0000","last_change":"2024-02-22T19:41:52.107671+0000","last_active":"2024-02-22T19:43:11.751656+0000","last_peered":"2024-02-22T19:43:11.751656+0000","last_clean":"2024-02-22T19:43:11.751656+0000","last_became_active":"2024-02-22T19:41:52.106767+0000","last_became_peered":"2024-02-22T19:41:52.106767+0000","last_unstale":"2024-02-22T19:43:11.751656+0000","last_undegraded":"2024-02-22T19:43:11.751656+0000","last_fullsized":"2024-02-22T19:43:11.751656+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"354'15082","reported_seq":15159,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.422121+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:43:19.422121+0000","last_peered":"2024-02-22T19:43:19.422121+0000","last_clean":"2024-02-22T19:43:19.422121+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:43:19.422121+0000","last_undegraded":"2024-02-22T19:43:19.422121+0000","last_fullsized":"2024-02-22T19:43:19.422121+0000","mapping_epoch":285,"log_start":"341'11990","ondisk_log_start":"341'11990","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3092,"ondisk_log_size":3092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":87687168,"num_objects":1338,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15082,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":59,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.769498+0000","last_change":"2024-02-22T19:41:52.110027+0000","last_active":"2024-02-22T19:43:11.769498+0000","last_peered":"2024-02-22T19:43:11.769498+0000","last_clean":"2024-02-22T19:43:11.769498+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:43:11.769498+0000","last_undegraded":"2024-02-22T19:43:11.769498+0000","last_fullsized":"2024-02-22T19:43:11.769498+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.670303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":140,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.766370+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:43:11.766370+0000","last_peered":"2024-02-22T19:43:11.766370+0000","last_clean":"2024-02-22T19:43:11.766370+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:43:11.766370+0000","last_undegraded":"2024-02-22T19:43:11.766370+0000","last_fullsized":"2024-02-22T19:43:11.766370+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":151,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752523+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:43:11.752523+0000","last_peered":"2024-02-22T19:43:11.752523+0000","last_clean":"2024-02-22T19:43:11.752523+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:43:11.752523+0000","last_undegraded":"2024-02-22T19:43:11.752523+0000","last_fullsized":"2024-02-22T19:43:11.752523+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":464,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745613+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:43:11.745613+0000","last_peered":"2024-02-22T19:43:11.745613+0000","last_clean":"2024-02-22T19:43:11.745613+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:43:11.745613+0000","last_undegraded":"2024-02-22T19:43:11.745613+0000","last_fullsized":"2024-02-22T19:43:11.745613+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"326'14","reported_seq":372,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345470+0000","last_change":"2024-02-22T19:41:52.698274+0000","last_active":"2024-02-22T19:43:14.345470+0000","last_peered":"2024-02-22T19:43:14.345470+0000","last_clean":"2024-02-22T19:43:14.345470+0000","last_became_active":"2024-02-22T19:41:52.112897+0000","last_became_peered":"2024-02-22T19:41:52.112897+0000","last_unstale":"2024-02-22T19:43:14.345470+0000","last_undegraded":"2024-02-22T19:43:14.345470+0000","last_fullsized":"2024-02-22T19:43:14.345470+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"354'14915","reported_seq":17009,"reported_epoch":354,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:43:19.718749+0000","last_change":"2024-02-22T19:41:53.515251+0000","last_active":"2024-02-22T19:43:19.718749+0000","last_peered":"2024-02-22T19:43:19.718749+0000","last_clean":"2024-02-22T19:41:51.078339+0000","last_became_active":"2024-02-22T19:41:52.386374+0000","last_became_peered":"2024-02-22T19:41:52.386374+0000","last_unstale":"2024-02-22T19:43:19.718749+0000","last_undegraded":"2024-02-22T19:43:19.718749+0000","last_fullsized":"2024-02-22T19:43:19.718749+0000","mapping_epoch":323,"log_start":"340'11758","ondisk_log_start":"340'11758","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3157,"ondisk_log_size":3157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90505216,"num_objects":1381,"num_object_clones":0,"num_object_copies":4143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1000,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14915,"num_write_kb":521472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1003,"num_bytes_recovered":65732608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,6],"avail_no_missing":["3","6","7"],"object_location_counts":[{"shards":"3,6,7","objects":1381}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":231,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.763012+0000","last_change":"2024-02-22T19:41:52.103880+0000","last_active":"2024-02-22T19:43:11.763012+0000","last_peered":"2024-02-22T19:43:11.763012+0000","last_clean":"2024-02-22T19:43:11.763012+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:43:11.763012+0000","last_undegraded":"2024-02-22T19:43:11.763012+0000","last_fullsized":"2024-02-22T19:43:11.763012+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":330,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750132+0000","last_change":"2024-02-22T19:41:52.103942+0000","last_active":"2024-02-22T19:43:11.750132+0000","last_peered":"2024-02-22T19:43:11.750132+0000","last_clean":"2024-02-22T19:43:11.750132+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:43:11.750132+0000","last_undegraded":"2024-02-22T19:43:11.750132+0000","last_fullsized":"2024-02-22T19:43:11.750132+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":149,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750573+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:43:11.750573+0000","last_peered":"2024-02-22T19:43:11.750573+0000","last_clean":"2024-02-22T19:43:11.750573+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:43:11.750573+0000","last_undegraded":"2024-02-22T19:43:11.750573+0000","last_fullsized":"2024-02-22T19:43:11.750573+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":466,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750885+0000","last_change":"2024-02-22T19:41:52.715987+0000","last_active":"2024-02-22T19:43:11.750885+0000","last_peered":"2024-02-22T19:43:11.750885+0000","last_clean":"2024-02-22T19:43:11.750885+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:43:11.750885+0000","last_undegraded":"2024-02-22T19:43:11.750885+0000","last_fullsized":"2024-02-22T19:43:11.750885+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":535,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.350179+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:43:14.350179+0000","last_peered":"2024-02-22T19:43:14.350179+0000","last_clean":"2024-02-22T19:43:14.350179+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:43:14.350179+0000","last_undegraded":"2024-02-22T19:43:14.350179+0000","last_fullsized":"2024-02-22T19:43:14.350179+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"354'14783","reported_seq":14861,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.319360+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:43:19.319360+0000","last_peered":"2024-02-22T19:43:19.319360+0000","last_clean":"2024-02-22T19:43:19.319360+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:43:19.319360+0000","last_undegraded":"2024-02-22T19:43:19.319360+0000","last_fullsized":"2024-02-22T19:43:19.319360+0000","mapping_epoch":285,"log_start":"337'11235","ondisk_log_start":"337'11235","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3548,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":91095064,"num_objects":1391,"num_object_clones":0,"num_object_copies":4173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14783,"num_write_kb":517505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.771121+0000","last_change":"2024-02-22T19:41:52.102626+0000","last_active":"2024-02-22T19:43:11.771121+0000","last_peered":"2024-02-22T19:43:11.771121+0000","last_clean":"2024-02-22T19:43:11.771121+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:43:11.771121+0000","last_undegraded":"2024-02-22T19:43:11.771121+0000","last_fullsized":"2024-02-22T19:43:11.771121+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:33.048434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.048434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":132,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.762697+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:43:11.762697+0000","last_peered":"2024-02-22T19:43:11.762697+0000","last_clean":"2024-02-22T19:43:11.762697+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:43:11.762697+0000","last_undegraded":"2024-02-22T19:43:11.762697+0000","last_fullsized":"2024-02-22T19:43:11.762697+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":350,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.769616+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:43:11.769616+0000","last_peered":"2024-02-22T19:43:11.769616+0000","last_clean":"2024-02-22T19:43:11.769616+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:43:11.769616+0000","last_undegraded":"2024-02-22T19:43:11.769616+0000","last_fullsized":"2024-02-22T19:43:11.769616+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":454,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.766600+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:43:11.766600+0000","last_peered":"2024-02-22T19:43:11.766600+0000","last_clean":"2024-02-22T19:43:11.766600+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:43:11.766600+0000","last_undegraded":"2024-02-22T19:43:11.766600+0000","last_fullsized":"2024-02-22T19:43:11.766600+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":141,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.765120+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:43:11.765120+0000","last_peered":"2024-02-22T19:43:11.765120+0000","last_clean":"2024-02-22T19:43:11.765120+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:43:11.765120+0000","last_undegraded":"2024-02-22T19:43:11.765120+0000","last_fullsized":"2024-02-22T19:43:11.765120+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"354'14826","reported_seq":14903,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.702197+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:43:19.702197+0000","last_peered":"2024-02-22T19:43:19.702197+0000","last_clean":"2024-02-22T19:43:19.702197+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:43:19.702197+0000","last_undegraded":"2024-02-22T19:43:19.702197+0000","last_fullsized":"2024-02-22T19:43:19.702197+0000","mapping_epoch":285,"log_start":"340'11658","ondisk_log_start":"340'11658","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3168,"ondisk_log_size":3168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88473600,"num_objects":1350,"num_object_clones":0,"num_object_copies":4050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14826,"num_write_kb":517632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":163,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751911+0000","last_change":"2024-02-22T19:41:52.107922+0000","last_active":"2024-02-22T19:43:11.751911+0000","last_peered":"2024-02-22T19:43:11.751911+0000","last_clean":"2024-02-22T19:43:11.751911+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:43:11.751911+0000","last_undegraded":"2024-02-22T19:43:11.751911+0000","last_fullsized":"2024-02-22T19:43:11.751911+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":178,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.752431+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:43:11.752431+0000","last_peered":"2024-02-22T19:43:11.752431+0000","last_clean":"2024-02-22T19:43:11.752431+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:43:11.752431+0000","last_undegraded":"2024-02-22T19:43:11.752431+0000","last_fullsized":"2024-02-22T19:43:11.752431+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":650,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345485+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:43:14.345485+0000","last_peered":"2024-02-22T19:43:14.345485+0000","last_clean":"2024-02-22T19:43:14.345485+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:43:14.345485+0000","last_undegraded":"2024-02-22T19:43:14.345485+0000","last_fullsized":"2024-02-22T19:43:14.345485+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":323,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.766531+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:43:11.766531+0000","last_peered":"2024-02-22T19:43:11.766531+0000","last_clean":"2024-02-22T19:43:11.766531+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:43:11.766531+0000","last_undegraded":"2024-02-22T19:43:11.766531+0000","last_fullsized":"2024-02-22T19:43:11.766531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":67,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.751554+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:43:11.751554+0000","last_peered":"2024-02-22T19:43:11.751554+0000","last_clean":"2024-02-22T19:43:11.751554+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:43:11.751554+0000","last_undegraded":"2024-02-22T19:43:11.751554+0000","last_fullsized":"2024-02-22T19:43:11.751554+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"354'15002","reported_seq":15079,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.374689+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:43:19.374689+0000","last_peered":"2024-02-22T19:43:19.374689+0000","last_clean":"2024-02-22T19:43:19.374689+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:43:19.374689+0000","last_undegraded":"2024-02-22T19:43:19.374689+0000","last_fullsized":"2024-02-22T19:43:19.374689+0000","mapping_epoch":285,"log_start":"337'11401","ondisk_log_start":"337'11401","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3601,"ondisk_log_size":3601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":93585408,"num_objects":1428,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15002,"num_write_kb":525760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":339,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.763157+0000","last_change":"2024-02-22T19:41:52.112553+0000","last_active":"2024-02-22T19:43:11.763157+0000","last_peered":"2024-02-22T19:43:11.763157+0000","last_clean":"2024-02-22T19:43:11.763157+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:43:11.763157+0000","last_undegraded":"2024-02-22T19:43:11.763157+0000","last_fullsized":"2024-02-22T19:43:11.763157+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"326'20","reported_seq":396,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345447+0000","last_change":"2024-02-22T19:41:54.322898+0000","last_active":"2024-02-22T19:43:14.345447+0000","last_peered":"2024-02-22T19:43:14.345447+0000","last_clean":"2024-02-22T19:43:14.345447+0000","last_became_active":"2024-02-22T19:41:54.322531+0000","last_became_peered":"2024-02-22T19:41:54.322531+0000","last_unstale":"2024-02-22T19:43:14.345447+0000","last_undegraded":"2024-02-22T19:43:14.345447+0000","last_fullsized":"2024-02-22T19:43:14.345447+0000","mapping_epoch":325,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":128,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.772316+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:43:11.772316+0000","last_peered":"2024-02-22T19:43:11.772316+0000","last_clean":"2024-02-22T19:43:11.772316+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:43:11.772316+0000","last_undegraded":"2024-02-22T19:43:11.772316+0000","last_fullsized":"2024-02-22T19:43:11.772316+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":136,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750647+0000","last_change":"2024-02-22T19:41:52.358275+0000","last_active":"2024-02-22T19:43:11.750647+0000","last_peered":"2024-02-22T19:43:11.750647+0000","last_clean":"2024-02-22T19:43:11.750647+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:43:11.750647+0000","last_undegraded":"2024-02-22T19:43:11.750647+0000","last_fullsized":"2024-02-22T19:43:11.750647+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"354'14945","reported_seq":27072,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:19.541461+0000","last_change":"2024-02-22T19:42:34.754313+0000","last_active":"2024-02-22T19:43:19.541461+0000","last_peered":"2024-02-22T19:43:19.541461+0000","last_clean":"2024-02-22T19:43:19.541461+0000","last_became_active":"2024-02-22T19:42:34.737475+0000","last_became_peered":"2024-02-22T19:42:34.737475+0000","last_unstale":"2024-02-22T19:43:19.541461+0000","last_undegraded":"2024-02-22T19:43:19.541461+0000","last_fullsized":"2024-02-22T19:43:19.541461+0000","mapping_epoch":339,"log_start":"338'11570","ondisk_log_start":"338'11570","created":285,"last_epoch_clean":340,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3375,"ondisk_log_size":3375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92864512,"num_objects":1417,"num_object_clones":0,"num_object_copies":4251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14945,"num_write_kb":523584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6014,"num_bytes_recovered":394133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":151,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750977+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:43:11.750977+0000","last_peered":"2024-02-22T19:43:11.750977+0000","last_clean":"2024-02-22T19:43:11.750977+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:43:11.750977+0000","last_undegraded":"2024-02-22T19:43:11.750977+0000","last_fullsized":"2024-02-22T19:43:11.750977+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":343,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.771565+0000","last_change":"2024-02-22T19:41:52.107535+0000","last_active":"2024-02-22T19:43:11.771565+0000","last_peered":"2024-02-22T19:43:11.771565+0000","last_clean":"2024-02-22T19:43:11.771565+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:43:11.771565+0000","last_undegraded":"2024-02-22T19:43:11.771565+0000","last_fullsized":"2024-02-22T19:43:11.771565+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":385,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745008+0000","last_change":"2024-02-22T19:41:52.092333+0000","last_active":"2024-02-22T19:43:11.745008+0000","last_peered":"2024-02-22T19:43:11.745008+0000","last_clean":"2024-02-22T19:43:11.745008+0000","last_became_active":"2024-02-22T19:41:52.092045+0000","last_became_peered":"2024-02-22T19:41:52.092045+0000","last_unstale":"2024-02-22T19:43:11.745008+0000","last_undegraded":"2024-02-22T19:43:11.745008+0000","last_fullsized":"2024-02-22T19:43:11.745008+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":260,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.745396+0000","last_change":"2024-02-22T19:41:52.110141+0000","last_active":"2024-02-22T19:43:11.745396+0000","last_peered":"2024-02-22T19:43:11.745396+0000","last_clean":"2024-02-22T19:43:11.745396+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:43:11.745396+0000","last_undegraded":"2024-02-22T19:43:11.745396+0000","last_fullsized":"2024-02-22T19:43:11.745396+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":229,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.771213+0000","last_change":"2024-02-22T19:41:52.810562+0000","last_active":"2024-02-22T19:43:11.771213+0000","last_peered":"2024-02-22T19:43:11.771213+0000","last_clean":"2024-02-22T19:43:11.771213+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:43:11.771213+0000","last_undegraded":"2024-02-22T19:43:11.771213+0000","last_fullsized":"2024-02-22T19:43:11.771213+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"354'14717","reported_seq":16817,"reported_epoch":354,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:43:19.706832+0000","last_change":"2024-02-22T19:41:52.755946+0000","last_active":"2024-02-22T19:43:19.706832+0000","last_peered":"2024-02-22T19:43:19.706832+0000","last_clean":"2024-02-22T19:41:51.044126+0000","last_became_active":"2024-02-22T19:41:52.387533+0000","last_became_peered":"2024-02-22T19:41:52.387533+0000","last_unstale":"2024-02-22T19:43:19.706832+0000","last_undegraded":"2024-02-22T19:43:19.706832+0000","last_fullsized":"2024-02-22T19:43:19.706832+0000","mapping_epoch":323,"log_start":"338'11517","ondisk_log_start":"338'11517","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3200,"ondisk_log_size":3200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":91160576,"num_objects":1391,"num_object_clones":0,"num_object_copies":4173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":995,"num_objects_unfound":0,"num_objects_dirty":1391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":515456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1006,"num_bytes_recovered":65929216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,6],"avail_no_missing":["3","6","7"],"object_location_counts":[{"shards":"3,6,7","objects":1391}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":341,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750347+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:43:11.750347+0000","last_peered":"2024-02-22T19:43:11.750347+0000","last_clean":"2024-02-22T19:43:11.750347+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:43:11.750347+0000","last_undegraded":"2024-02-22T19:43:11.750347+0000","last_fullsized":"2024-02-22T19:43:11.750347+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":167,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.750497+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:43:11.750497+0000","last_peered":"2024-02-22T19:43:11.750497+0000","last_clean":"2024-02-22T19:43:11.750497+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:43:11.750497+0000","last_undegraded":"2024-02-22T19:43:11.750497+0000","last_fullsized":"2024-02-22T19:43:11.750497+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":574,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-22T19:43:14.345804+0000","last_change":"2024-02-22T19:41:32.453253+0000","last_active":"2024-02-22T19:43:14.345804+0000","last_peered":"2024-02-22T19:43:14.345804+0000","last_clean":"2024-02-22T19:43:14.345804+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:43:14.345804+0000","last_undegraded":"2024-02-22T19:43:14.345804+0000","last_fullsized":"2024-02-22T19:43:14.345804+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":384,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.764981+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:43:11.764981+0000","last_peered":"2024-02-22T19:43:11.764981+0000","last_clean":"2024-02-22T19:43:11.764981+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:43:11.764981+0000","last_undegraded":"2024-02-22T19:43:11.764981+0000","last_fullsized":"2024-02-22T19:43:11.764981+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":198,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-22T19:43:11.763377+0000","last_change":"2024-02-22T19:41:52.575055+0000","last_active":"2024-02-22T19:43:11.763377+0000","last_peered":"2024-02-22T19:43:11.763377+0000","last_clean":"2024-02-22T19:43:11.763377+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:43:11.763377+0000","last_undegraded":"2024-02-22T19:43:11.763377+0000","last_fullsized":"2024-02-22T19:43:11.763377+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":1439825944,"num_objects":21971,"num_object_clones":0,"num_object_copies":65913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2997,"num_objects_unfound":0,"num_objects_dirty":21971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238943,"num_write_kb":8349185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14921,"num_bytes_recovered":977862656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4834799616,"data_stored":4834787400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52358,"ondisk_log_size":52358,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":841,"num_read_kb":836,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":4934,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266863,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1224896,"kb_used_data":919480,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92520256,"statfs":{"total":95995035648,"available":94740742144,"internally_reserved":0,"allocated":941547520,"data_stored":940395971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3979999999999999,"15min":1.3919999999999999},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":5.2320000000000002,"5min":7.2919999999999998,"15min":42.597999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":1.528,"15min":1.385},"min":{"1min":0.27600000000000002,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":3.0750000000000002,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":3.5600000000000001,"5min":2.3639999999999999,"15min":1.637},"min":{"1min":0.26300000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":36.509,"5min":54.951000000000001,"15min":54.951000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.4239999999999999,"5min":2.786,"15min":1.8759999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":5.6139999999999999,"5min":57.012,"15min":57.012},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.885,"15min":1.373},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.23999999999999999},"max":{"1min":4.165,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":2.0289999999999999,"5min":2.4670000000000001,"15min":1.6559999999999999},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.23599999999999999},"max":{"1min":7.2140000000000004,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":2.6059999999999999,"15min":1.7549999999999999},"min":{"1min":0.372,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":36.491999999999997,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":3.6179999999999999,"5min":2.653,"15min":1.744},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":36.732999999999997,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.95699999999999996,"15min":0.66900000000000004},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":2.238,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.31},{"interface":"front","average":{"1min":1.621,"5min":1.085,"15min":0.69099999999999995},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":4.3449999999999998,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.29899999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6120000000000001,"5min":1.232,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.27000000000000002},"max":{"1min":5.1699999999999999,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.371},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.046,"15min":0.73499999999999999},"min":{"1min":0.371,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":5.282,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.0309999999999999,"15min":0.79400000000000004},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":1.512,"5min":6.5,"15min":12.199999999999999},"last":0.437},{"interface":"front","average":{"1min":2.0329999999999999,"5min":1.3160000000000001,"15min":0.84299999999999997},"min":{"1min":0.33900000000000002,"5min":0.29699999999999999,"15min":0.28100000000000003},"max":{"1min":6.3869999999999996,"5min":7.6390000000000002,"15min":12.067},"last":0.38500000000000001}]}]},{"osd":6,"up_from":0,"seq":193273528500,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575053,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1535764,"kb_used_data":941836,"kb_used_omap":0,"kb_used_meta":593792,"kb_avail":92209388,"statfs":{"total":95995035648,"available":94422413312,"internally_reserved":0,"allocated":964440064,"data_stored":963268103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":608043008},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":5.7480000000000002,"5min":2.391,"15min":1.8580000000000001},"min":{"1min":0.38900000000000001,"5min":0.185,"15min":0.185},"max":{"1min":51.091000000000001,"5min":51.091000000000001,"15min":51.091000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":5.1799999999999997,"5min":2.3860000000000001,"15min":1.952},"min":{"1min":0.38300000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":51.289999999999999,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":5.4669999999999996,"5min":2.4630000000000001,"15min":1.9710000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":51.027000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":4.7220000000000004,"5min":2.4950000000000001,"15min":2.077},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":51.369999999999997,"5min":51.369999999999997,"15min":51.369999999999997},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:42:53 2024","interfaces":[{"interface":"back","average":{"1min":5.6390000000000002,"5min":2.5960000000000001,"15min":2.0630000000000002},"min":{"1min":0.39800000000000002,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":51.755000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":5.2809999999999997},{"interface":"front","average":{"1min":5.1509999999999998,"5min":2.6339999999999999,"15min":2.1360000000000001},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":51.741,"5min":51.741,"15min":51.741},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:43:10 2024","interfaces":[{"interface":"back","average":{"1min":6.1319999999999997,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.32600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":51.158999999999999,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":3.843,"5min":1.853,"15min":1.385},"min":{"1min":0.311,"5min":0.311,"15min":0.245},"max":{"1min":51.276000000000003,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Feb 22 19:42:28 2024","interfaces":[{"interface":"back","average":{"1min":3.1619999999999999,"5min":2.0499999999999998,"15min":1.4950000000000001},"min":{"1min":0.375,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":16.213000000000001,"5min":16.213000000000001,"15min":16.213000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":3.3370000000000002,"5min":1.6819999999999999,"15min":1.3720000000000001},"min":{"1min":0.34899999999999998,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":16.381,"5min":16.381,"15min":16.381},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:42:55 2024","interfaces":[{"interface":"back","average":{"1min":4.1929999999999996,"5min":1.9059999999999999,"15min":1.4370000000000001},"min":{"1min":0.433,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":51.079999999999998,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":6.1859999999999999,"5min":2.4590000000000001,"15min":1.6899999999999999},"min":{"1min":0.44,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":51.119999999999997,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":4.7080000000000002,"5min":2.5830000000000002,"15min":1.784},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":51.244999999999997,"5min":51.244999999999997,"15min":51.244999999999997},"last":1.595},{"interface":"front","average":{"1min":4.4050000000000002,"5min":1.8540000000000001,"15min":1.454},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":51.317999999999998,"5min":51.317999999999998,"15min":51.317999999999998},"last":1.169}]}]},{"osd":0,"up_from":0,"seq":34359738578,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411527,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":702444,"kb_used_data":397028,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93042708,"statfs":{"total":95995035648,"available":95275732992,"internally_reserved":0,"allocated":406556672,"data_stored":405423754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":2.0699999999999998,"15min":1.706},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.18099999999999999},"max":{"1min":4.7000000000000002,"5min":16.763999999999999,"15min":22.338999999999999},"last":0.23999999999999999},{"interface":"front","average":{"1min":2.2789999999999999,"5min":2.5640000000000001,"15min":2.1120000000000001},"min":{"1min":0.26700000000000002,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":14.603999999999999,"5min":30.427,"15min":30.427},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.8560000000000001,"15min":1.679},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.0819999999999999,"5min":12.349,"15min":12.349},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.889,"15min":1.585},"min":{"1min":0.215,"5min":0.215,"15min":0.193},"max":{"1min":5.1900000000000004,"5min":13.782,"15min":13.782},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:42:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":2.3740000000000001,"15min":1.9410000000000001},"min":{"1min":0.28299999999999997,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":12.151999999999999,"5min":16.076000000000001,"15min":30.071000000000002},"last":0.27000000000000002},{"interface":"front","average":{"1min":4.5179999999999998,"5min":3.0430000000000001,"15min":2.258},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.184},"max":{"1min":28.263999999999999,"5min":30.282,"15min":32.119},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Feb 22 19:43:06 2024","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.585,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.0140000000000002,"5min":12.43,"15min":12.43},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.861,"15min":1.2529999999999999},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":4.6420000000000003,"5min":10.34,"15min":10.34},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:42:24 2024","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":2.339,"15min":1.399},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.8239999999999998,"5min":40.633000000000003,"15min":40.633000000000003},"last":2.2069999999999999},{"interface":"front","average":{"1min":2.3580000000000001,"5min":2.2549999999999999,"15min":1.595},"min":{"1min":0.41799999999999998,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":7.2030000000000003,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:43:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2669999999999999,"5min":1.8140000000000001,"15min":1.4990000000000001},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.311},"max":{"1min":7.1550000000000002,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.698,"5min":1.9530000000000001,"15min":1.304},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":4.5940000000000003,"5min":10.369999999999999,"15min":10.369999999999999},"last":2.242}]},{"osd":7,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.736,"15min":1.5049999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.1580000000000004,"5min":12.484,"15min":12.484},"last":0.495},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.9299999999999999,"15min":1.407},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":4.5999999999999996,"5min":10.539,"15min":10.539},"last":3.5499999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149889,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1408328,"kb_used_data":818304,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":92336824,"statfs":{"total":95995035648,"available":94552907776,"internally_reserved":0,"allocated":837943296,"data_stored":836783400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":2.544,"5min":1.964,"15min":1.8169999999999999},"min":{"1min":0.22,"5min":0.191,"15min":0.191},"max":{"1min":7.2560000000000002,"5min":9.9789999999999992,"15min":23.48},"last":0.746},{"interface":"front","average":{"1min":2.7469999999999999,"5min":2.4209999999999998,"15min":2.1139999999999999},"min":{"1min":0.254,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":7.1980000000000004,"5min":18.172000000000001,"15min":23.411999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":3.395,"15min":2.427},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.749000000000001,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.626},{"interface":"front","average":{"1min":2.7189999999999999,"5min":2.851,"15min":1.974},"min":{"1min":0.38400000000000001,"5min":0.23000000000000001,"15min":0.214},"max":{"1min":8.0869999999999997,"5min":22.187000000000001,"15min":22.187000000000001},"last":5.1449999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":2.5139999999999998,"5min":2.367,"15min":1.994},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.2800000000000002,"5min":20.105,"15min":20.105},"last":5.1710000000000003},{"interface":"front","average":{"1min":2.823,"5min":2.5270000000000001,"15min":1.978},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.217},"max":{"1min":9.1479999999999997,"5min":35.939999999999998,"15min":35.939999999999998},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":2.3740000000000001,"5min":2.1699999999999999,"15min":1.73},"min":{"1min":0.30599999999999999,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":9.968,"5min":12.416,"15min":13.680999999999999},"last":5.3760000000000003},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.9890000000000001,"15min":1.5429999999999999},"min":{"1min":0.40400000000000003,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":3.6019999999999999,"5min":13.548,"15min":13.548},"last":7.6479999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:42:32 2024","interfaces":[{"interface":"back","average":{"1min":2.3959999999999999,"5min":1.776,"15min":1.478},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.33600000000000002},"max":{"1min":6.4009999999999998,"5min":11.161,"15min":12.879},"last":5.0910000000000002},{"interface":"front","average":{"1min":2.0230000000000001,"5min":2.1019999999999999,"15min":1.6419999999999999},"min":{"1min":0.32900000000000001,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":7.1289999999999996,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.433}]},{"osd":6,"last update":"Thu Feb 22 19:42:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3119999999999998,"5min":1.9019999999999999,"15min":1.514},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":7.1440000000000001,"5min":11.263,"15min":12.898999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":2.573,"5min":2.1230000000000002,"15min":1.772},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":10.02,"5min":12.396000000000001,"15min":13.803000000000001},"last":5.2859999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:43:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":2.0920000000000001,"15min":1.595},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":3.6259999999999999,"5min":13.483000000000001,"15min":13.483000000000001},"last":8.0879999999999992},{"interface":"front","average":{"1min":2.2770000000000001,"5min":2.0659999999999998,"15min":1.621},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.2240000000000002,"5min":11.539,"15min":12.846},"last":5.0570000000000004}]}]},{"osd":4,"up_from":0,"seq":137438953663,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724729,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1433744,"kb_used_data":1124104,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":92311408,"statfs":{"total":95995035648,"available":94526881792,"internally_reserved":0,"allocated":1151082496,"data_stored":1149913864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.5249999999999999,"15min":1.278},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":4.4290000000000003,"5min":17.684000000000001,"15min":17.684000000000001},"last":8.4299999999999997},{"interface":"front","average":{"1min":2.9049999999999998,"5min":1.831,"15min":1.4590000000000001},"min":{"1min":0.26100000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":11.9,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4470000000000001,"15min":1.3420000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":6.0750000000000002,"5min":11.202,"15min":29.390999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":2.1640000000000001,"5min":1.6779999999999999,"15min":1.4159999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.218},"max":{"1min":5.4329999999999998,"5min":8.7490000000000006,"15min":29.367000000000001},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.9339999999999999,"15min":1.4059999999999999},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.774,"5min":11.367000000000001,"15min":14.613},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.8640000000000001,"15min":1.4650000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5579999999999998,"5min":16.099,"15min":16.099},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.573,"15min":1.5429999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.4459999999999997,"5min":6.0439999999999996,"15min":13.366},"last":1.002},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.7490000000000001,"15min":1.5229999999999999},"min":{"1min":0.34999999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":7.2729999999999997,"5min":8.6069999999999993,"15min":10.16},"last":2.6899999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:42:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.0089999999999999,"15min":0.65900000000000003},"min":{"1min":0.29799999999999999,"5min":0.223,"15min":0.219},"max":{"1min":3.6669999999999998,"5min":5.75,"15min":5.75},"last":2.0150000000000001},{"interface":"front","average":{"1min":1.855,"5min":1.2070000000000001,"15min":0.72099999999999997},"min":{"1min":0.32800000000000001,"5min":0.27500000000000002,"15min":0.23499999999999999},"max":{"1min":10.407,"5min":10.407,"15min":10.407},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":0.64600000000000002},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":2.0169999999999999,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.756,"5min":1.238,"15min":0.78800000000000003},"min":{"1min":0.36699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.436,"5min":10.436,"15min":10.436},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:42:24 2024","interfaces":[{"interface":"back","average":{"1min":2.0099999999999998,"5min":1.0569999999999999,"15min":0.76200000000000001},"min":{"1min":0.32100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.7119999999999997,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.876},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.1879999999999999,"15min":0.78200000000000003},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":6.9630000000000001,"5min":7.6379999999999999,"15min":7.6379999999999999},"last":0.46600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":915083264,"data_stored":915079192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387710976,"data_stored":387710976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":807993344,"data_stored":807993344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1103040512,"data_stored":1103036440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":713818112,"data_stored":713818112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":907153408,"data_stored":907149336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:43:23.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph status --format=json 2024-02-22T19:43:23.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:23.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:23.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:23.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:23.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:24.031 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:22.818226+0000 mon.a (mon.0) 1450 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:24.031 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:22.846446+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:43:24.031 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:22.846498+0000 mon.a (mon.0) 1452 : cluster [DBG] osdmap e355: 8 total, 8 up, 5 in 2024-02-22T19:43:24.031 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:22.959786+0000 osd.1 (osd.1) 208 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:24.031 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:22.959841+0000 osd.1 (osd.1) 209 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:23.000363+0000 mgr.y (mgr.14535) 550 : audit [DBG] from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:22.818226+0000 mon.a (mon.0) 1450 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:22.846446+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:22.846498+0000 mon.a (mon.0) 1452 : cluster [DBG] osdmap e355: 8 total, 8 up, 5 in 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:22.959786+0000 osd.1 (osd.1) 208 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:22.959841+0000 osd.1 (osd.1) 209 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:23.000363+0000 mgr.y (mgr.14535) 550 : audit [DBG] from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:24.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:22.818226+0000 mon.a (mon.0) 1450 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:22.846446+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:43:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:22.846498+0000 mon.a (mon.0) 1452 : cluster [DBG] osdmap e355: 8 total, 8 up, 5 in 2024-02-22T19:43:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:22.959786+0000 osd.1 (osd.1) 208 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:22.959841+0000 osd.1 (osd.1) 209 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:23.000363+0000 mgr.y (mgr.14535) 550 : audit [DBG] from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15324.0:244169 9.8 9:1e4f8e21:::benchmark_data_smithi053_157429_object113710:head [delete] snapc 0=[] ondisk+write+known_if_redirected e356)", 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:43:24.137214+0000", 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0086045770000000004, 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0086358960000000005, 2024-02-22T19:43:24.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15324", 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 244169 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137214+0000", 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137214+0000", 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137212+0000", 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0999999999999998e-06 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137219+0000", 2024-02-22T19:43:24.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0984e-05 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137230+0000", 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3180000000000003e-06 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137236+0000", 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0559000000000002e-05 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:24.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137287+0000", 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105443 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137392+0000", 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000121313 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5,7", 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137514+0000", 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057223689999999997 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.143236+0000", 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060223120000000002 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15324.0:244165 9.4 e356/339 9:2b50e98f:::benchmark_data_smithi053_157429_object113706:head v 356'15376, mlcod=356'15373)", 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:43:24.137782+0000", 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0080362839999999994, 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0081730889999999997, 2024-02-22T19:43:24.151 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137782+0000", 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137782+0000", 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137782+0000", 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.788e-06 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137785+0000", 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0930000000000002e-06 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137790+0000", 2024-02-22T19:43:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9770000000000002e-06 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.137794+0000", 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003509819 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.141304+0000", 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8182e-05 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.141322+0000", 2024-02-22T19:43:24.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035405549999999999 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.15324.0:244169 9.8 e356/285)", 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:43:24.139687+0000", 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0061313319999999998, 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006340872, 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.139687+0000", 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.139687+0000", 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.139686+0000", 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5700000000000003e-06 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.139691+0000", 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8800000000000008e-06 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.139699+0000", 2024-02-22T19:43:24.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6840000000000004e-06 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:24.139703+0000", 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6453999999999999e-05 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 3 2024-02-22T19:43:24.158 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:43:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:24.458 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:24.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:24.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:24.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:24.629 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:24.695 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:24.700 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:24.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:24.707 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:24.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:24.872 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:24.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:24.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:24.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:25.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:23.662256+0000 mgr.y (mgr.14535) 551 : cluster [DBG] pgmap v713: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.3 GiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s; 2997/66489 objects misplaced (4.508%); 2.2 MiB/s, 34 objects/s recovering 2024-02-22T19:43:25.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:23.859441+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e356: 8 total, 8 up, 5 in 2024-02-22T19:43:25.032 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:43:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:43:25.036 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:25.068 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:25.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:25.075 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:25.103 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:23.662256+0000 mgr.y (mgr.14535) 551 : cluster [DBG] pgmap v713: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.3 GiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s; 2997/66489 objects misplaced (4.508%); 2.2 MiB/s, 34 objects/s recovering 2024-02-22T19:43:25.104 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:23.859441+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e356: 8 total, 8 up, 5 in 2024-02-22T19:43:25.104 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:23.662256+0000 mgr.y (mgr.14535) 551 : cluster [DBG] pgmap v713: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1.3 GiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s; 2997/66489 objects misplaced (4.508%); 2.2 MiB/s, 34 objects/s recovering 2024-02-22T19:43:25.105 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:23.859441+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e356: 8 total, 8 up, 5 in 2024-02-22T19:43:25.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:43:25.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:25.219 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:25.254 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:25.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:25.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:25.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:43:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:43:25.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:25.428 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:25.768 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:25.803 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:25.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:25.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:25.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:25.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:25.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:25.109028+0000 mgr.y (mgr.14535) 552 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:25.803914+0000 mon.a (mon.0) 1454 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [1, 2]}]: dispatch 2024-02-22T19:43:26.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:25.109028+0000 mgr.y (mgr.14535) 552 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:26.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:25.803914+0000 mon.a (mon.0) 1454 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [1, 2]}]: dispatch 2024-02-22T19:43:26.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:26.169 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:26.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:26.170 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:25.109028+0000 mgr.y (mgr.14535) 552 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:26.170 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:25.803914+0000 mon.a (mon.0) 1454 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [1, 2]}]: dispatch 2024-02-22T19:43:26.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:26.327 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:26.494 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:43:26.925 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:25.664738+0000 mgr.y (mgr.14535) 553 : cluster [DBG] pgmap v715: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1020 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 2074/49545 objects misplaced (4.186%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:43:26.926 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:25.899151+0000 mon.a (mon.0) 1455 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [1, 2]}]': finished 2024-02-22T19:43:26.926 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:25.899230+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e357: 8 total, 8 up, 5 in 2024-02-22T19:43:27.179 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:25.664738+0000 mgr.y (mgr.14535) 553 : cluster [DBG] pgmap v715: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1020 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 2074/49545 objects misplaced (4.186%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:43:27.180 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:25.899151+0000 mon.a (mon.0) 1455 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [1, 2]}]': finished 2024-02-22T19:43:27.180 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:25.899230+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e357: 8 total, 8 up, 5 in 2024-02-22T19:43:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:25.664738+0000 mgr.y (mgr.14535) 553 : cluster [DBG] pgmap v715: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1020 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 2074/49545 objects misplaced (4.186%); 2.7 MiB/s, 43 objects/s recovering 2024-02-22T19:43:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:25.899151+0000 mon.a (mon.0) 1455 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [1, 2]}]': finished 2024-02-22T19:43:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:25.899230+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e357: 8 total, 8 up, 5 in 2024-02-22T19:43:27.607 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:27.616 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:27.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:27.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:27.775 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:27.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:27.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:28.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:28.111 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:28.112 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:28.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:26.579124+0000 osd.2 (osd.2) 190 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:26.579177+0000 osd.2 (osd.2) 191 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:28.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:26.930579+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e358: 8 total, 8 up, 5 in 2024-02-22T19:43:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:26.579124+0000 osd.2 (osd.2) 190 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:26.579177+0000 osd.2 (osd.2) 191 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:28.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:26.930579+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e358: 8 total, 8 up, 5 in 2024-02-22T19:43:28.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:26.579124+0000 osd.2 (osd.2) 190 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:28.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:26.579177+0000 osd.2 (osd.2) 191 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:28.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:26.930579+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e358: 8 total, 8 up, 5 in 2024-02-22T19:43:28.282 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:28.311 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:28.319 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:28.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:28.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:28.471 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:28.507 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:28.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:28.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:28.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:28.659 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:28.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:28.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:28.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:28.871 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:28.937 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:27.665666+0000 mgr.y (mgr.14535) 554 : cluster [DBG] pgmap v718: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1020 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s; 2074/49545 objects misplaced (4.186%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:28.937 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:28.908774+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? 172.21.15.53:0/3730622036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:28.965 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:28.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:28.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:29.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:43:29.106 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:29.131 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:27.665666+0000 mgr.y (mgr.14535) 554 : cluster [DBG] pgmap v718: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1020 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s; 2074/49545 objects misplaced (4.186%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:29.132 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:28.908774+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? 172.21.15.53:0/3730622036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:29.132 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:27.665666+0000 mgr.y (mgr.14535) 554 : cluster [DBG] pgmap v718: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 1020 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s; 2074/49545 objects misplaced (4.186%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:29.132 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:28.908774+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? 172.21.15.53:0/3730622036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:29.161 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:43:29.188 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:43:29.213 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:29.216 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:29.217 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:29.231 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:43:29.317 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:29.403 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:29.406 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:43:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:29] "GET /metrics HTTP/1.1" 200 176194 "" "Prometheus/2.33.4" 2024-02-22T19:43:29.542 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:29.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:29.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:29.648 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:29.733 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:29.828 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:29.833 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:29.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:43:29.840 INFO:teuthology.orchestra.run.smithi053.stdout:{"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1084,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":358,"num_osds":8,"num_up_osds":8,"osd_up_since":1708630133,"num_in_osds":5,"osd_in_since":1708630911,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":145},{"state_name":"active+remapped+backfilling","count":3}],"num_pgs":148,"num_pools":7,"num_objects":16515,"data_bytes":1069684412,"bytes_used":5378060288,"bytes_avail":474597117952,"bytes_total":479975178240,"misplaced_objects":2074,"misplaced_total":49545,"misplaced_ratio":0.041860934503986272,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":2380614,"recovering_keys_per_sec":0,"num_objects_recovered":175,"num_bytes_recovered":11468800,"num_keys_recovered":0,"read_bytes_sec":1062,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1172},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.53:8443/","prometheus":"http://172.21.15.53:9283/"}},"servicemap":{"epoch":13,"modified":"2024-02-22T19:30:25.194204+0000","services":{"rgw":{"daemons":{"summary":"","24371":{"start_epoch":12,"start_stamp":"2024-02-22T19:29:08.193725+0000","gid":24371,"addr":"172.21.15.53:0/65330745","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi053","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi053","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"8","realm_id":"","realm_name":"","zone_id":"2d4caf58-b951-44e6-ad1d-667e6ee8308c","zone_name":"default","zonegroup_id":"a2ce501a-2ab0-4645-bd76-4ac5a6518222","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{"79c6a7c2-f061-48ce-88ec-ac17dd60b0ea":{"message":"Global Recovery Event (85s)\n [===========================.] (remaining: 1s)","progress":0.97972971200942993,"add_to_ceph_s":true}}} 2024-02-22T19:43:29.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:30.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:29.839951+0000 mon.c (mon.2) 67 : audit [DBG] from='client.? 172.21.15.53:0/2962148005' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-22T19:43:30.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:29.839951+0000 mon.c (mon.2) 67 : audit [DBG] from='client.? 172.21.15.53:0/2962148005' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-22T19:43:30.300 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:30.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:30.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:29.839951+0000 mon.c (mon.2) 67 : audit [DBG] from='client.? 172.21.15.53:0/2962148005' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-22T19:43:30.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:30.460 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:30.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:30.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:30.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:30.683 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:30.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:30.689 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:30.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:30.844 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:29.666581+0000 mgr.y (mgr.14535) 555 : cluster [DBG] pgmap v719: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 764 MiB data, 4.2 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.62k op/s; 1782/37260 objects misplaced (4.783%); 2.4 MiB/s, 39 objects/s recovering 2024-02-22T19:43:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:29.666581+0000 mgr.y (mgr.14535) 555 : cluster [DBG] pgmap v719: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 764 MiB data, 4.2 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.62k op/s; 1782/37260 objects misplaced (4.783%); 2.4 MiB/s, 39 objects/s recovering 2024-02-22T19:43:31.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:29.666581+0000 mgr.y (mgr.14535) 555 : cluster [DBG] pgmap v719: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 764 MiB data, 4.2 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.62k op/s; 1782/37260 objects misplaced (4.783%); 2.4 MiB/s, 39 objects/s recovering 2024-02-22T19:43:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:31.667112+0000 mgr.y (mgr.14535) 556 : cluster [DBG] pgmap v720: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 674 MiB data, 4.0 GiB used, 443 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s; 1181/32940 objects misplaced (3.585%); 2.8 MiB/s, 44 objects/s recovering 2024-02-22T19:43:33.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:31.667112+0000 mgr.y (mgr.14535) 556 : cluster [DBG] pgmap v720: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 674 MiB data, 4.0 GiB used, 443 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s; 1181/32940 objects misplaced (3.585%); 2.8 MiB/s, 44 objects/s recovering 2024-02-22T19:43:33.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:31.667112+0000 mgr.y (mgr.14535) 556 : cluster [DBG] pgmap v720: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 674 MiB data, 4.0 GiB used, 443 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s; 1181/32940 objects misplaced (3.585%); 2.8 MiB/s, 44 objects/s recovering 2024-02-22T19:43:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:32.994904+0000 osd.0 (osd.0) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:32.994968+0000 osd.0 (osd.0) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:33.016564+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e359: 8 total, 8 up, 5 in 2024-02-22T19:43:34.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:32.994904+0000 osd.0 (osd.0) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:34.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:32.994968+0000 osd.0 (osd.0) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:34.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:33.016564+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e359: 8 total, 8 up, 5 in 2024-02-22T19:43:34.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:32.994904+0000 osd.0 (osd.0) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:34.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:32.994968+0000 osd.0 (osd.0) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:34.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:33.016564+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e359: 8 total, 8 up, 5 in 2024-02-22T19:43:35.026 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:43:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:43:35.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:33.669369+0000 mgr.y (mgr.14535) 557 : cluster [DBG] pgmap v722: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 674 MiB data, 4.0 GiB used, 443 GiB / 447 GiB avail; 658 B/s rd, 0 B/s wr, 712 op/s; 1181/32940 objects misplaced (3.585%); 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:35.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:43:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:43:35.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:33.669369+0000 mgr.y (mgr.14535) 557 : cluster [DBG] pgmap v722: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 674 MiB data, 4.0 GiB used, 443 GiB / 447 GiB avail; 658 B/s rd, 0 B/s wr, 712 op/s; 1181/32940 objects misplaced (3.585%); 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:35.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:33.669369+0000 mgr.y (mgr.14535) 557 : cluster [DBG] pgmap v722: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 674 MiB data, 4.0 GiB used, 443 GiB / 447 GiB avail; 658 B/s rd, 0 B/s wr, 712 op/s; 1181/32940 objects misplaced (3.585%); 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:35.119216+0000 mgr.y (mgr.14535) 558 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:36.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:35.119216+0000 mgr.y (mgr.14535) 558 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:36.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:35.119216+0000 mgr.y (mgr.14535) 558 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:35.670210+0000 mgr.y (mgr.14535) 559 : cluster [DBG] pgmap v723: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 311 MiB data, 3.1 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.30k op/s; 448/15525 objects misplaced (2.886%); 2.5 MiB/s, 40 objects/s recovering 2024-02-22T19:43:37.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:35.670210+0000 mgr.y (mgr.14535) 559 : cluster [DBG] pgmap v723: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 311 MiB data, 3.1 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.30k op/s; 448/15525 objects misplaced (2.886%); 2.5 MiB/s, 40 objects/s recovering 2024-02-22T19:43:37.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:35.670210+0000 mgr.y (mgr.14535) 559 : cluster [DBG] pgmap v723: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 311 MiB data, 3.1 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.30k op/s; 448/15525 objects misplaced (2.886%); 2.5 MiB/s, 40 objects/s recovering 2024-02-22T19:43:38.252 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-02-22T19:43:38.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:43:38.258 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:38.264 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:38.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:38.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:38.451 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:37.334599+0000 mon.a (mon.0) 1459 : cluster [DBG] osdmap e360: 8 total, 8 up, 5 in 2024-02-22T19:43:38.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:37.670558+0000 mgr.y (mgr.14535) 560 : cluster [DBG] pgmap v725: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 311 MiB data, 3.1 GiB used, 444 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 905 op/s; 448/15525 objects misplaced (2.886%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:38.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:37.334599+0000 mon.a (mon.0) 1459 : cluster [DBG] osdmap e360: 8 total, 8 up, 5 in 2024-02-22T19:43:38.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:37.670558+0000 mgr.y (mgr.14535) 560 : cluster [DBG] pgmap v725: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 311 MiB data, 3.1 GiB used, 444 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 905 op/s; 448/15525 objects misplaced (2.886%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:38.643 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Removed 130456 objects 2024-02-22T19:43:38.644 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Clean up completed and total clean up time :117.399 2024-02-22T19:43:38.649 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-02-22T19:43:38.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-02-22T19:43:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:37.334599+0000 mon.a (mon.0) 1459 : cluster [DBG] osdmap e360: 8 total, 8 up, 5 in 2024-02-22T19:43:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:37.670558+0000 mgr.y (mgr.14535) 560 : cluster [DBG] pgmap v725: 148 pgs: 3 active+remapped+backfilling, 145 active+clean; 311 MiB data, 3.1 GiB used, 444 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 905 op/s; 448/15525 objects misplaced (2.886%); 2.3 MiB/s, 36 objects/s recovering 2024-02-22T19:43:39.519 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:39.523 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:39.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:39.526 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:39.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:39.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:39.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:39.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:43:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:39] "GET /metrics HTTP/1.1" 200 176185 "" "Prometheus/2.33.4" 2024-02-22T19:43:39.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:39.840 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:39.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:39.941 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:39.975 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:40.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:39.361380+0000 mon.a (mon.0) 1460 : cluster [DBG] osdmap e361: 8 total, 8 up, 5 in 2024-02-22T19:43:40.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:39.671320+0000 mgr.y (mgr.14535) 561 : cluster [DBG] pgmap v727: 148 pgs: 1 peering, 2 active+remapped+backfilling, 145 active+clean; 71 MiB data, 2.7 GiB used, 444 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.45k op/s; 282/3963 objects misplaced (7.116%); 2.1 MiB/s, 32 objects/s recovering 2024-02-22T19:43:40.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:39.361380+0000 mon.a (mon.0) 1460 : cluster [DBG] osdmap e361: 8 total, 8 up, 5 in 2024-02-22T19:43:40.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:39.671320+0000 mgr.y (mgr.14535) 561 : cluster [DBG] pgmap v727: 148 pgs: 1 peering, 2 active+remapped+backfilling, 145 active+clean; 71 MiB data, 2.7 GiB used, 444 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.45k op/s; 282/3963 objects misplaced (7.116%); 2.1 MiB/s, 32 objects/s recovering 2024-02-22T19:43:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:39.361380+0000 mon.a (mon.0) 1460 : cluster [DBG] osdmap e361: 8 total, 8 up, 5 in 2024-02-22T19:43:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:39.671320+0000 mgr.y (mgr.14535) 561 : cluster [DBG] pgmap v727: 148 pgs: 1 peering, 2 active+remapped+backfilling, 145 active+clean; 71 MiB data, 2.7 GiB used, 444 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.45k op/s; 282/3963 objects misplaced (7.116%); 2.1 MiB/s, 32 objects/s recovering 2024-02-22T19:43:41.427 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:41.431 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:41.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:41.433 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:41.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:43:41.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:41.560 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:41.700 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:40.366699+0000 mon.a (mon.0) 1461 : cluster [DBG] osdmap e362: 8 total, 8 up, 5 in 2024-02-22T19:43:41.701 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:40.366699+0000 mon.a (mon.0) 1461 : cluster [DBG] osdmap e362: 8 total, 8 up, 5 in 2024-02-22T19:43:41.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:40.366699+0000 mon.a (mon.0) 1461 : cluster [DBG] osdmap e362: 8 total, 8 up, 5 in 2024-02-22T19:43:42.139 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:43:42.140 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:43:42.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:41.671603+0000 mgr.y (mgr.14535) 562 : cluster [DBG] pgmap v729: 148 pgs: 3 peering, 145 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 831 op/s; 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:42.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:41.671603+0000 mgr.y (mgr.14535) 562 : cluster [DBG] pgmap v729: 148 pgs: 3 peering, 145 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 831 op/s; 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:41.671603+0000 mgr.y (mgr.14535) 562 : cluster [DBG] pgmap v729: 148 pgs: 3 peering, 145 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 831 op/s; 1.4 MiB/s, 22 objects/s recovering 2024-02-22T19:43:43.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:43:44.341 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:44.811 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:44.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:44.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:44.838 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:43.671903+0000 mgr.y (mgr.14535) 563 : cluster [DBG] pgmap v730: 148 pgs: 3 peering, 145 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 807 B/s rd, 0 B/s wr, 786 op/s; 1.3 MiB/s, 20 objects/s recovering 2024-02-22T19:43:44.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:43.671903+0000 mgr.y (mgr.14535) 563 : cluster [DBG] pgmap v730: 148 pgs: 3 peering, 145 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 807 B/s rd, 0 B/s wr, 786 op/s; 1.3 MiB/s, 20 objects/s recovering 2024-02-22T19:43:44.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:44.946 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:43.671903+0000 mgr.y (mgr.14535) 563 : cluster [DBG] pgmap v730: 148 pgs: 3 peering, 145 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 807 B/s rd, 0 B/s wr, 786 op/s; 1.3 MiB/s, 20 objects/s recovering 2024-02-22T19:43:44.946 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:43:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:43:44.948 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:44.971 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:44.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:44.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:45.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:45.106 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:45.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:45.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:45.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:45.277 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:45.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:43:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:43:45.382 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:45.401 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:45.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:45.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:45.419 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:45.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:45.642 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:45.645 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:45.645 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:45.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:45.780 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:45.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:45.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:45.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:45.935 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:45.956 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:45.959 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:45.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:46.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:45.129953+0000 mgr.y (mgr.14535) 564 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:46.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:45.129953+0000 mgr.y (mgr.14535) 564 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:46.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:46.089 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:45.129953+0000 mgr.y (mgr.14535) 564 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:46.839 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:46.842 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:46.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:46.944 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:46.975 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:46.998 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:45.672502+0000 mgr.y (mgr.14535) 565 : cluster [DBG] pgmap v731: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 623 op/s; 1.0 MiB/s, 16 objects/s recovering 2024-02-22T19:43:46.998 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:46.066307+0000 mon.a (mon.0) 1462 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:43:46.998 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:46.461534+0000 mon.a (mon.0) 1463 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:43:46.999 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:46.462210+0000 mon.a (mon.0) 1464 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:43:46.999 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:46.466375+0000 mon.a (mon.0) 1465 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:43:46.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:45.672502+0000 mgr.y (mgr.14535) 565 : cluster [DBG] pgmap v731: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 623 op/s; 1.0 MiB/s, 16 objects/s recovering 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:46.066307+0000 mon.a (mon.0) 1462 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:46.461534+0000 mon.a (mon.0) 1463 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:46.462210+0000 mon.a (mon.0) 1464 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:46.466375+0000 mon.a (mon.0) 1465 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:45.672502+0000 mgr.y (mgr.14535) 565 : cluster [DBG] pgmap v731: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 623 op/s; 1.0 MiB/s, 16 objects/s recovering 2024-02-22T19:43:47.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:46.066307+0000 mon.a (mon.0) 1462 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:43:47.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:46.461534+0000 mon.a (mon.0) 1463 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:43:47.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:46.462210+0000 mon.a (mon.0) 1464 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:43:47.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:46.466375+0000 mon.a (mon.0) 1465 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:43:47.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:47.129 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:47.149 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:47.152 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:47.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:47.284 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:47.307 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:47.311 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:47.412 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:47.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.466 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:47.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:47.618 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:47.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:47.766 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:47.768 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:47.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:47.897 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:47.916 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:47.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:47.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:47.929 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:43:47.946 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:43:47.964 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:43:47.984 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:43:48.002 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:43:48.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:43:48.020 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:43:48.038 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:43:48.047 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:48.056 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:43:48.057 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.057 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.057 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.057 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.058 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.058 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.058 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.058 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:43:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:43:48.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:48.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:43:48.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:48.203 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:43:48.406 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:48.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:48.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:47.672805+0000 mgr.y (mgr.14535) 566 : cluster [DBG] pgmap v732: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 600 op/s; 1024 KiB/s, 15 objects/s recovering 2024-02-22T19:43:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:47.732857+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? 172.21.15.53:0/117835441' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:43:48.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:47.911561+0000 mon.a (mon.0) 1467 : audit [DBG] from='client.? 172.21.15.53:0/2874550897' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:47.672805+0000 mgr.y (mgr.14535) 566 : cluster [DBG] pgmap v732: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 600 op/s; 1024 KiB/s, 15 objects/s recovering 2024-02-22T19:43:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:47.732857+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? 172.21.15.53:0/117835441' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:43:48.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:47.911561+0000 mon.a (mon.0) 1467 : audit [DBG] from='client.? 172.21.15.53:0/2874550897' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:48.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:48.546 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:47.672805+0000 mgr.y (mgr.14535) 566 : cluster [DBG] pgmap v732: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 600 op/s; 1024 KiB/s, 15 objects/s recovering 2024-02-22T19:43:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:47.732857+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? 172.21.15.53:0/117835441' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:43:48.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:47.911561+0000 mon.a (mon.0) 1467 : audit [DBG] from='client.? 172.21.15.53:0/2874550897' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:43:48.745 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:48.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:48.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:48.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:48.906 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:49.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:49.213 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:43:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:43:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:47.730319+0000 osd.5 (osd.5) 112 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:47.730366+0000 osd.5 (osd.5) 113 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:48.334723+0000 mon.a (mon.0) 1468 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:48.343636+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? 172.21.15.53:0/117835441' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:43:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:48.343696+0000 mon.a (mon.0) 1470 : cluster [DBG] osdmap e363: 8 total, 8 up, 5 in 2024-02-22T19:43:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:49.197708+0000 mgr.y (mgr.14535) 567 : audit [DBG] from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:49.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:49] "GET /metrics HTTP/1.1" 200 176185 "" "Prometheus/2.33.4" 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:47.730319+0000 osd.5 (osd.5) 112 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:47.730366+0000 osd.5 (osd.5) 113 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:48.334723+0000 mon.a (mon.0) 1468 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:48.343636+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? 172.21.15.53:0/117835441' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:48.343696+0000 mon.a (mon.0) 1470 : cluster [DBG] osdmap e363: 8 total, 8 up, 5 in 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:49.197708+0000 mgr.y (mgr.14535) 567 : audit [DBG] from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:47.730319+0000 osd.5 (osd.5) 112 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:43:49.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:47.730366+0000 osd.5 (osd.5) 113 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:43:49.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:48.334723+0000 mon.a (mon.0) 1468 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:49.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:48.343636+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? 172.21.15.53:0/117835441' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:43:49.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:48.343696+0000 mon.a (mon.0) 1470 : cluster [DBG] osdmap e363: 8 total, 8 up, 5 in 2024-02-22T19:43:49.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:49.197708+0000 mgr.y (mgr.14535) 567 : audit [DBG] from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:43:50.397 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_2' removed 2024-02-22T19:43:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:49.498636+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.53:0/2358865950' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:43:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:49.498911+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:43:50.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:49.673525+0000 mgr.y (mgr.14535) 568 : cluster [DBG] pgmap v734: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:50.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:49.498636+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.53:0/2358865950' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:43:50.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:49.498911+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:43:50.683 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:49.673525+0000 mgr.y (mgr.14535) 568 : cluster [DBG] pgmap v734: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:50.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:49.498636+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.53:0/2358865950' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:43:50.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:49.498911+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:43:50.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:49.673525+0000 mgr.y (mgr.14535) 568 : cluster [DBG] pgmap v734: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:50.701 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":733,"stamp":"2024-02-22T19:43:48.339451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":1987,"num_read_kb":1739,"num_write":261628,"num_write_kb":8349226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16030,"num_bytes_recovered":1020870038,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53551,"ondisk_log_size":53551,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":2975020,"kb_used_data":12932,"kb_used_omap":0,"kb_used_meta":2962048,"kb_avail":465750740,"statfs":{"total":479975178240,"available":476928757760,"internally_reserved":0,"allocated":13242368,"data_stored":7257478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3033137152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-73990144,"num_objects":-1129,"num_object_clones":0,"num_object_copies":-3387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-1129,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1129,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":5767168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.668684"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":319,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364337+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:43:39.364337+0000","last_peered":"2024-02-22T19:43:39.364337+0000","last_clean":"2024-02-22T19:43:39.364337+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:43:39.364337+0000","last_undegraded":"2024-02-22T19:43:39.364337+0000","last_fullsized":"2024-02-22T19:43:39.364337+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":349,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365300+0000","last_change":"2024-02-22T19:41:52.114804+0000","last_active":"2024-02-22T19:43:39.365300+0000","last_peered":"2024-02-22T19:43:39.365300+0000","last_clean":"2024-02-22T19:43:39.365300+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:43:39.365300+0000","last_undegraded":"2024-02-22T19:43:39.365300+0000","last_fullsized":"2024-02-22T19:43:39.365300+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:55.686995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:36:55.686995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":133,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360436+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:43:39.360436+0000","last_peered":"2024-02-22T19:43:39.360436+0000","last_clean":"2024-02-22T19:43:39.360436+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:43:39.360436+0000","last_undegraded":"2024-02-22T19:43:39.360436+0000","last_fullsized":"2024-02-22T19:43:39.360436+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":334,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365287+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:43:39.365287+0000","last_peered":"2024-02-22T19:43:39.365287+0000","last_clean":"2024-02-22T19:43:39.365287+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:43:39.365287+0000","last_undegraded":"2024-02-22T19:43:39.365287+0000","last_fullsized":"2024-02-22T19:43:39.365287+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":335,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365811+0000","last_change":"2024-02-22T19:41:52.112817+0000","last_active":"2024-02-22T19:43:39.365811+0000","last_peered":"2024-02-22T19:43:39.365811+0000","last_clean":"2024-02-22T19:43:39.365811+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:43:39.365811+0000","last_undegraded":"2024-02-22T19:43:39.365811+0000","last_fullsized":"2024-02-22T19:43:39.365811+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":442,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.369453+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:43:39.369453+0000","last_peered":"2024-02-22T19:43:39.369453+0000","last_clean":"2024-02-22T19:43:39.369453+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:43:39.369453+0000","last_undegraded":"2024-02-22T19:43:39.369453+0000","last_fullsized":"2024-02-22T19:43:39.369453+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":163,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.359831+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:43:39.359831+0000","last_peered":"2024-02-22T19:43:39.359831+0000","last_clean":"2024-02-22T19:43:39.359831+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:43:39.359831+0000","last_undegraded":"2024-02-22T19:43:39.359831+0000","last_fullsized":"2024-02-22T19:43:39.359831+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":172,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365268+0000","last_change":"2024-02-22T19:41:52.113562+0000","last_active":"2024-02-22T19:43:39.365268+0000","last_peered":"2024-02-22T19:43:39.365268+0000","last_clean":"2024-02-22T19:43:39.365268+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:43:39.365268+0000","last_undegraded":"2024-02-22T19:43:39.365268+0000","last_fullsized":"2024-02-22T19:43:39.365268+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":239,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364918+0000","last_change":"2024-02-22T19:41:52.752324+0000","last_active":"2024-02-22T19:43:39.364918+0000","last_peered":"2024-02-22T19:43:39.364918+0000","last_clean":"2024-02-22T19:43:39.364918+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:43:39.364918+0000","last_undegraded":"2024-02-22T19:43:39.364918+0000","last_fullsized":"2024-02-22T19:43:39.364918+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":159,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363997+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:43:39.363997+0000","last_peered":"2024-02-22T19:43:39.363997+0000","last_clean":"2024-02-22T19:43:39.363997+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:43:39.363997+0000","last_undegraded":"2024-02-22T19:43:39.363997+0000","last_fullsized":"2024-02-22T19:43:39.363997+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":189,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364719+0000","last_change":"2024-02-22T19:41:52.116983+0000","last_active":"2024-02-22T19:43:39.364719+0000","last_peered":"2024-02-22T19:43:39.364719+0000","last_clean":"2024-02-22T19:43:39.364719+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:43:39.364719+0000","last_undegraded":"2024-02-22T19:43:39.364719+0000","last_fullsized":"2024-02-22T19:43:39.364719+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":162,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365043+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:43:39.365043+0000","last_peered":"2024-02-22T19:43:39.365043+0000","last_clean":"2024-02-22T19:43:39.365043+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:43:39.365043+0000","last_undegraded":"2024-02-22T19:43:39.365043+0000","last_fullsized":"2024-02-22T19:43:39.365043+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":237,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364994+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:43:39.364994+0000","last_peered":"2024-02-22T19:43:39.364994+0000","last_clean":"2024-02-22T19:43:39.364994+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:43:39.364994+0000","last_undegraded":"2024-02-22T19:43:39.364994+0000","last_fullsized":"2024-02-22T19:43:39.364994+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":136,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371327+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:43:39.371327+0000","last_peered":"2024-02-22T19:43:39.371327+0000","last_clean":"2024-02-22T19:43:39.371327+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:43:39.371327+0000","last_undegraded":"2024-02-22T19:43:39.371327+0000","last_fullsized":"2024-02-22T19:43:39.371327+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":389,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360491+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:43:39.360491+0000","last_peered":"2024-02-22T19:43:39.360491+0000","last_clean":"2024-02-22T19:43:39.360491+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:43:39.360491+0000","last_undegraded":"2024-02-22T19:43:39.360491+0000","last_fullsized":"2024-02-22T19:43:39.360491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":352,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.366348+0000","last_change":"2024-02-22T19:41:52.121546+0000","last_active":"2024-02-22T19:43:39.366348+0000","last_peered":"2024-02-22T19:43:39.366348+0000","last_clean":"2024-02-22T19:43:39.366348+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:43:39.366348+0000","last_undegraded":"2024-02-22T19:43:39.366348+0000","last_fullsized":"2024-02-22T19:43:39.366348+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":149,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360351+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:43:39.360351+0000","last_peered":"2024-02-22T19:43:39.360351+0000","last_clean":"2024-02-22T19:43:39.360351+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:43:39.360351+0000","last_undegraded":"2024-02-22T19:43:39.360351+0000","last_fullsized":"2024-02-22T19:43:39.360351+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":403,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365429+0000","last_change":"2024-02-22T19:41:53.187026+0000","last_active":"2024-02-22T19:43:39.365429+0000","last_peered":"2024-02-22T19:43:39.365429+0000","last_clean":"2024-02-22T19:43:39.365429+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:43:39.365429+0000","last_undegraded":"2024-02-22T19:43:39.365429+0000","last_fullsized":"2024-02-22T19:43:39.365429+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":368,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372003+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:43:39.372003+0000","last_peered":"2024-02-22T19:43:39.372003+0000","last_clean":"2024-02-22T19:43:39.372003+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:43:39.372003+0000","last_undegraded":"2024-02-22T19:43:39.372003+0000","last_fullsized":"2024-02-22T19:43:39.372003+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":357,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360670+0000","last_change":"2024-02-22T19:41:52.110049+0000","last_active":"2024-02-22T19:43:39.360670+0000","last_peered":"2024-02-22T19:43:39.360670+0000","last_clean":"2024-02-22T19:43:39.360670+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:43:39.360670+0000","last_undegraded":"2024-02-22T19:43:39.360670+0000","last_fullsized":"2024-02-22T19:43:39.360670+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":169,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372576+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:43:39.372576+0000","last_peered":"2024-02-22T19:43:39.372576+0000","last_clean":"2024-02-22T19:43:39.372576+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:43:39.372576+0000","last_undegraded":"2024-02-22T19:43:39.372576+0000","last_fullsized":"2024-02-22T19:43:39.372576+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":357,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361894+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:43:39.361894+0000","last_peered":"2024-02-22T19:43:39.361894+0000","last_clean":"2024-02-22T19:43:39.361894+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:43:39.361894+0000","last_undegraded":"2024-02-22T19:43:39.361894+0000","last_fullsized":"2024-02-22T19:43:39.361894+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":355,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364494+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:43:39.364494+0000","last_peered":"2024-02-22T19:43:39.364494+0000","last_clean":"2024-02-22T19:43:39.364494+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:43:39.364494+0000","last_undegraded":"2024-02-22T19:43:39.364494+0000","last_fullsized":"2024-02-22T19:43:39.364494+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":397,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365423+0000","last_change":"2024-02-22T19:41:52.743206+0000","last_active":"2024-02-22T19:43:39.365423+0000","last_peered":"2024-02-22T19:43:39.365423+0000","last_clean":"2024-02-22T19:43:39.365423+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:43:39.365423+0000","last_undegraded":"2024-02-22T19:43:39.365423+0000","last_fullsized":"2024-02-22T19:43:39.365423+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":337,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371849+0000","last_change":"2024-02-22T19:38:47.148029+0000","last_active":"2024-02-22T19:43:39.371849+0000","last_peered":"2024-02-22T19:43:39.371849+0000","last_clean":"2024-02-22T19:43:39.371849+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:43:39.371849+0000","last_undegraded":"2024-02-22T19:43:39.371849+0000","last_fullsized":"2024-02-22T19:43:39.371849+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:56.406760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:56.406760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":302,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372433+0000","last_change":"2024-02-22T19:41:52.111808+0000","last_active":"2024-02-22T19:43:39.372433+0000","last_peered":"2024-02-22T19:43:39.372433+0000","last_clean":"2024-02-22T19:43:39.372433+0000","last_became_active":"2024-02-22T19:41:52.110758+0000","last_became_peered":"2024-02-22T19:41:52.110758+0000","last_unstale":"2024-02-22T19:43:39.372433+0000","last_undegraded":"2024-02-22T19:43:39.372433+0000","last_fullsized":"2024-02-22T19:43:39.372433+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":349,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365109+0000","last_change":"2024-02-22T19:41:52.114607+0000","last_active":"2024-02-22T19:43:39.365109+0000","last_peered":"2024-02-22T19:43:39.365109+0000","last_clean":"2024-02-22T19:43:39.365109+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:43:39.365109+0000","last_undegraded":"2024-02-22T19:43:39.365109+0000","last_fullsized":"2024-02-22T19:43:39.365109+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":356,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364570+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:43:39.364570+0000","last_peered":"2024-02-22T19:43:39.364570+0000","last_clean":"2024-02-22T19:43:39.364570+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:43:39.364570+0000","last_undegraded":"2024-02-22T19:43:39.364570+0000","last_fullsized":"2024-02-22T19:43:39.364570+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":358,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365508+0000","last_change":"2024-02-22T19:41:52.118842+0000","last_active":"2024-02-22T19:43:39.365508+0000","last_peered":"2024-02-22T19:43:39.365508+0000","last_clean":"2024-02-22T19:43:39.365508+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:43:39.365508+0000","last_undegraded":"2024-02-22T19:43:39.365508+0000","last_fullsized":"2024-02-22T19:43:39.365508+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:51.454678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:51.454678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":369,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371496+0000","last_change":"2024-02-22T19:41:52.104771+0000","last_active":"2024-02-22T19:43:39.371496+0000","last_peered":"2024-02-22T19:43:39.371496+0000","last_clean":"2024-02-22T19:43:39.371496+0000","last_became_active":"2024-02-22T19:41:52.104234+0000","last_became_peered":"2024-02-22T19:41:52.104234+0000","last_unstale":"2024-02-22T19:43:39.371496+0000","last_undegraded":"2024-02-22T19:43:39.371496+0000","last_fullsized":"2024-02-22T19:43:39.371496+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.702590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.702590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":154,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364841+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:43:39.364841+0000","last_peered":"2024-02-22T19:43:39.364841+0000","last_clean":"2024-02-22T19:43:39.364841+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:43:39.364841+0000","last_undegraded":"2024-02-22T19:43:39.364841+0000","last_fullsized":"2024-02-22T19:43:39.364841+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":376,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.369390+0000","last_change":"2024-02-22T19:41:53.508908+0000","last_active":"2024-02-22T19:43:39.369390+0000","last_peered":"2024-02-22T19:43:39.369390+0000","last_clean":"2024-02-22T19:43:39.369390+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:43:39.369390+0000","last_undegraded":"2024-02-22T19:43:39.369390+0000","last_fullsized":"2024-02-22T19:43:39.369390+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":340,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363839+0000","last_change":"2024-02-22T19:41:52.112028+0000","last_active":"2024-02-22T19:43:39.363839+0000","last_peered":"2024-02-22T19:43:39.363839+0000","last_clean":"2024-02-22T19:43:39.363839+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:43:39.363839+0000","last_undegraded":"2024-02-22T19:43:39.363839+0000","last_fullsized":"2024-02-22T19:43:39.363839+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":349,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365177+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:43:39.365177+0000","last_peered":"2024-02-22T19:43:39.365177+0000","last_clean":"2024-02-22T19:43:39.365177+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:43:39.365177+0000","last_undegraded":"2024-02-22T19:43:39.365177+0000","last_fullsized":"2024-02-22T19:43:39.365177+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":377,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371936+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:43:39.371936+0000","last_peered":"2024-02-22T19:43:39.371936+0000","last_clean":"2024-02-22T19:43:39.371936+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:43:39.371936+0000","last_undegraded":"2024-02-22T19:43:39.371936+0000","last_fullsized":"2024-02-22T19:43:39.371936+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":381,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360936+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:43:39.360936+0000","last_peered":"2024-02-22T19:43:39.360936+0000","last_clean":"2024-02-22T19:43:39.360936+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:43:39.360936+0000","last_undegraded":"2024-02-22T19:43:39.360936+0000","last_fullsized":"2024-02-22T19:43:39.360936+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":370,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.369200+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:43:39.369200+0000","last_peered":"2024-02-22T19:43:39.369200+0000","last_clean":"2024-02-22T19:43:39.369200+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:43:39.369200+0000","last_undegraded":"2024-02-22T19:43:39.369200+0000","last_fullsized":"2024-02-22T19:43:39.369200+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":348,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361078+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:43:39.361078+0000","last_peered":"2024-02-22T19:43:39.361078+0000","last_clean":"2024-02-22T19:43:39.361078+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:43:39.361078+0000","last_undegraded":"2024-02-22T19:43:39.361078+0000","last_fullsized":"2024-02-22T19:43:39.361078+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":140,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371669+0000","last_change":"2024-02-22T19:41:52.104077+0000","last_active":"2024-02-22T19:43:39.371669+0000","last_peered":"2024-02-22T19:43:39.371669+0000","last_clean":"2024-02-22T19:43:39.371669+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:43:39.371669+0000","last_undegraded":"2024-02-22T19:43:39.371669+0000","last_fullsized":"2024-02-22T19:43:39.371669+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":144,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363329+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:43:39.363329+0000","last_peered":"2024-02-22T19:43:39.363329+0000","last_clean":"2024-02-22T19:43:39.363329+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:43:39.363329+0000","last_undegraded":"2024-02-22T19:43:39.363329+0000","last_fullsized":"2024-02-22T19:43:39.363329+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":157,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372361+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:43:39.372361+0000","last_peered":"2024-02-22T19:43:39.372361+0000","last_clean":"2024-02-22T19:43:39.372361+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:43:39.372361+0000","last_undegraded":"2024-02-22T19:43:39.372361+0000","last_fullsized":"2024-02-22T19:43:39.372361+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":270,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.372147+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:43:40.372147+0000","last_peered":"2024-02-22T19:43:40.372147+0000","last_clean":"2024-02-22T19:43:40.372147+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:43:40.372147+0000","last_undegraded":"2024-02-22T19:43:40.372147+0000","last_fullsized":"2024-02-22T19:43:40.372147+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":350,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371139+0000","last_change":"2024-02-22T19:41:52.103816+0000","last_active":"2024-02-22T19:43:39.371139+0000","last_peered":"2024-02-22T19:43:39.371139+0000","last_clean":"2024-02-22T19:43:39.371139+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:43:39.371139+0000","last_undegraded":"2024-02-22T19:43:39.371139+0000","last_fullsized":"2024-02-22T19:43:39.371139+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":364,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365520+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:43:39.365520+0000","last_peered":"2024-02-22T19:43:39.365520+0000","last_clean":"2024-02-22T19:43:39.365520+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:43:39.365520+0000","last_undegraded":"2024-02-22T19:43:39.365520+0000","last_fullsized":"2024-02-22T19:43:39.365520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":342,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363999+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:43:39.363999+0000","last_peered":"2024-02-22T19:43:39.363999+0000","last_clean":"2024-02-22T19:43:39.363999+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:43:39.363999+0000","last_undegraded":"2024-02-22T19:43:39.363999+0000","last_fullsized":"2024-02-22T19:43:39.363999+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":507,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.367735+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:43:40.367735+0000","last_peered":"2024-02-22T19:43:40.367735+0000","last_clean":"2024-02-22T19:43:40.367735+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:43:40.367735+0000","last_undegraded":"2024-02-22T19:43:40.367735+0000","last_fullsized":"2024-02-22T19:43:40.367735+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":148,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371869+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:43:39.371869+0000","last_peered":"2024-02-22T19:43:39.371869+0000","last_clean":"2024-02-22T19:43:39.371869+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:43:39.371869+0000","last_undegraded":"2024-02-22T19:43:39.371869+0000","last_fullsized":"2024-02-22T19:43:39.371869+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":178,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365376+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:43:39.365376+0000","last_peered":"2024-02-22T19:43:39.365376+0000","last_clean":"2024-02-22T19:43:39.365376+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:43:39.365376+0000","last_undegraded":"2024-02-22T19:43:39.365376+0000","last_fullsized":"2024-02-22T19:43:39.365376+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":365,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360760+0000","last_change":"2024-02-22T19:41:38.526359+0000","last_active":"2024-02-22T19:43:39.360760+0000","last_peered":"2024-02-22T19:43:39.360760+0000","last_clean":"2024-02-22T19:43:39.360760+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:39.360760+0000","last_undegraded":"2024-02-22T19:43:39.360760+0000","last_fullsized":"2024-02-22T19:43:39.360760+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.526259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.526259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":373,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372308+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:43:39.372308+0000","last_peered":"2024-02-22T19:43:39.372308+0000","last_clean":"2024-02-22T19:43:39.372308+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:43:39.372308+0000","last_undegraded":"2024-02-22T19:43:39.372308+0000","last_fullsized":"2024-02-22T19:43:39.372308+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":150,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365950+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:43:39.365950+0000","last_peered":"2024-02-22T19:43:39.365950+0000","last_clean":"2024-02-22T19:43:39.365950+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:43:39.365950+0000","last_undegraded":"2024-02-22T19:43:39.365950+0000","last_fullsized":"2024-02-22T19:43:39.365950+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":370,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371651+0000","last_change":"2024-02-22T19:43:22.588560+0000","last_active":"2024-02-22T19:43:39.371651+0000","last_peered":"2024-02-22T19:43:39.371651+0000","last_clean":"2024-02-22T19:43:39.371651+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:43:39.371651+0000","last_undegraded":"2024-02-22T19:43:39.371651+0000","last_fullsized":"2024-02-22T19:43:39.371651+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:53.392179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:53.392179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":326,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363556+0000","last_change":"2024-02-22T19:41:52.112840+0000","last_active":"2024-02-22T19:43:39.363556+0000","last_peered":"2024-02-22T19:43:39.363556+0000","last_clean":"2024-02-22T19:43:39.363556+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:43:39.363556+0000","last_undegraded":"2024-02-22T19:43:39.363556+0000","last_fullsized":"2024-02-22T19:43:39.363556+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":364,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.366413+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:43:39.366413+0000","last_peered":"2024-02-22T19:43:39.366413+0000","last_clean":"2024-02-22T19:43:39.366413+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:43:39.366413+0000","last_undegraded":"2024-02-22T19:43:39.366413+0000","last_fullsized":"2024-02-22T19:43:39.366413+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"360'16222","reported_seq":16307,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364492+0000","last_change":"2024-02-22T19:40:00.879678+0000","last_active":"2024-02-22T19:43:39.364492+0000","last_peered":"2024-02-22T19:43:39.364492+0000","last_clean":"2024-02-22T19:43:39.364492+0000","last_became_active":"2024-02-22T19:40:00.879348+0000","last_became_peered":"2024-02-22T19:40:00.879348+0000","last_unstale":"2024-02-22T19:43:39.364492+0000","last_undegraded":"2024-02-22T19:43:39.364492+0000","last_fullsized":"2024-02-22T19:43:39.364492+0000","mapping_epoch":285,"log_start":"346'13140","ondisk_log_start":"346'13140","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3082,"ondisk_log_size":3082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16222,"num_write_kb":519104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":195,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364216+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:43:39.364216+0000","last_peered":"2024-02-22T19:43:39.364216+0000","last_clean":"2024-02-22T19:43:39.364216+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:43:39.364216+0000","last_undegraded":"2024-02-22T19:43:39.364216+0000","last_fullsized":"2024-02-22T19:43:39.364216+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":422,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371644+0000","last_change":"2024-02-22T19:43:26.953378+0000","last_active":"2024-02-22T19:43:39.371644+0000","last_peered":"2024-02-22T19:43:39.371644+0000","last_clean":"2024-02-22T19:43:39.371644+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:43:39.371644+0000","last_undegraded":"2024-02-22T19:43:39.371644+0000","last_fullsized":"2024-02-22T19:43:39.371644+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":218,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363648+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:43:39.363648+0000","last_peered":"2024-02-22T19:43:39.363648+0000","last_clean":"2024-02-22T19:43:39.363648+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:43:39.363648+0000","last_undegraded":"2024-02-22T19:43:39.363648+0000","last_fullsized":"2024-02-22T19:43:39.363648+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":589,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.370163+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:43:40.370163+0000","last_peered":"2024-02-22T19:43:40.370163+0000","last_clean":"2024-02-22T19:43:40.370163+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:43:40.370163+0000","last_undegraded":"2024-02-22T19:43:40.370163+0000","last_fullsized":"2024-02-22T19:43:40.370163+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"360'16426","reported_seq":28315,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360407+0000","last_change":"2024-02-22T19:42:34.755451+0000","last_active":"2024-02-22T19:43:39.360407+0000","last_peered":"2024-02-22T19:43:39.360407+0000","last_clean":"2024-02-22T19:43:39.360407+0000","last_became_active":"2024-02-22T19:42:34.737692+0000","last_became_peered":"2024-02-22T19:42:34.737692+0000","last_unstale":"2024-02-22T19:43:39.360407+0000","last_undegraded":"2024-02-22T19:43:39.360407+0000","last_fullsized":"2024-02-22T19:43:39.360407+0000","mapping_epoch":339,"log_start":"345'12953","ondisk_log_start":"345'12953","created":285,"last_epoch_clean":340,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3473,"ondisk_log_size":3473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":525632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5891,"num_bytes_recovered":386072576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":320,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365410+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:43:39.365410+0000","last_peered":"2024-02-22T19:43:39.365410+0000","last_clean":"2024-02-22T19:43:39.365410+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:43:39.365410+0000","last_undegraded":"2024-02-22T19:43:39.365410+0000","last_fullsized":"2024-02-22T19:43:39.365410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":64,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362231+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:43:39.362231+0000","last_peered":"2024-02-22T19:43:39.362231+0000","last_clean":"2024-02-22T19:43:39.362231+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:43:39.362231+0000","last_undegraded":"2024-02-22T19:43:39.362231+0000","last_fullsized":"2024-02-22T19:43:39.362231+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":208,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360404+0000","last_change":"2024-02-22T19:41:52.597069+0000","last_active":"2024-02-22T19:43:39.360404+0000","last_peered":"2024-02-22T19:43:39.360404+0000","last_clean":"2024-02-22T19:43:39.360404+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:43:39.360404+0000","last_undegraded":"2024-02-22T19:43:39.360404+0000","last_fullsized":"2024-02-22T19:43:39.360404+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":380,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362053+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:43:39.362053+0000","last_peered":"2024-02-22T19:43:39.362053+0000","last_clean":"2024-02-22T19:43:39.362053+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:43:39.362053+0000","last_undegraded":"2024-02-22T19:43:39.362053+0000","last_fullsized":"2024-02-22T19:43:39.362053+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"360'16306","reported_seq":16391,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364777+0000","last_change":"2024-02-22T19:40:00.879590+0000","last_active":"2024-02-22T19:43:39.364777+0000","last_peered":"2024-02-22T19:43:39.364777+0000","last_clean":"2024-02-22T19:43:39.364777+0000","last_became_active":"2024-02-22T19:40:00.879278+0000","last_became_peered":"2024-02-22T19:40:00.879278+0000","last_unstale":"2024-02-22T19:43:39.364777+0000","last_undegraded":"2024-02-22T19:43:39.364777+0000","last_fullsized":"2024-02-22T19:43:39.364777+0000","mapping_epoch":285,"log_start":"346'13200","ondisk_log_start":"346'13200","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3106,"ondisk_log_size":3106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16306,"num_write_kb":521792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":326,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.370440+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:43:39.370440+0000","last_peered":"2024-02-22T19:43:39.370440+0000","last_clean":"2024-02-22T19:43:39.370440+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:43:39.370440+0000","last_undegraded":"2024-02-22T19:43:39.370440+0000","last_fullsized":"2024-02-22T19:43:39.370440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":406,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362311+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:43:39.362311+0000","last_peered":"2024-02-22T19:43:39.362311+0000","last_clean":"2024-02-22T19:43:39.362311+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:43:39.362311+0000","last_undegraded":"2024-02-22T19:43:39.362311+0000","last_fullsized":"2024-02-22T19:43:39.362311+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":364,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372429+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:43:39.372429+0000","last_peered":"2024-02-22T19:43:39.372429+0000","last_clean":"2024-02-22T19:43:39.372429+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:43:39.372429+0000","last_undegraded":"2024-02-22T19:43:39.372429+0000","last_fullsized":"2024-02-22T19:43:39.372429+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":355,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360704+0000","last_change":"2024-02-22T19:41:52.135450+0000","last_active":"2024-02-22T19:43:39.360704+0000","last_peered":"2024-02-22T19:43:39.360704+0000","last_clean":"2024-02-22T19:43:39.360704+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:43:39.360704+0000","last_undegraded":"2024-02-22T19:43:39.360704+0000","last_fullsized":"2024-02-22T19:43:39.360704+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"360'16420","reported_seq":16505,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364441+0000","last_change":"2024-02-22T19:40:01.098246+0000","last_active":"2024-02-22T19:43:39.364441+0000","last_peered":"2024-02-22T19:43:39.364441+0000","last_clean":"2024-02-22T19:43:39.364441+0000","last_became_active":"2024-02-22T19:40:01.097791+0000","last_became_peered":"2024-02-22T19:40:01.097791+0000","last_unstale":"2024-02-22T19:43:39.364441+0000","last_undegraded":"2024-02-22T19:43:39.364441+0000","last_fullsized":"2024-02-22T19:43:39.364441+0000","mapping_epoch":285,"log_start":"346'13331","ondisk_log_start":"346'13331","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3089,"ondisk_log_size":3089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16420,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":148,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372125+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:43:39.372125+0000","last_peered":"2024-02-22T19:43:39.372125+0000","last_clean":"2024-02-22T19:43:39.372125+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:43:39.372125+0000","last_undegraded":"2024-02-22T19:43:39.372125+0000","last_fullsized":"2024-02-22T19:43:39.372125+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":385,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361009+0000","last_change":"2024-02-22T19:41:16.236762+0000","last_active":"2024-02-22T19:43:39.361009+0000","last_peered":"2024-02-22T19:43:39.361009+0000","last_clean":"2024-02-22T19:43:39.361009+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:43:39.361009+0000","last_undegraded":"2024-02-22T19:43:39.361009+0000","last_fullsized":"2024-02-22T19:43:39.361009+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.236664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.236664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":168,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364629+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:43:39.364629+0000","last_peered":"2024-02-22T19:43:39.364629+0000","last_clean":"2024-02-22T19:43:39.364629+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:43:39.364629+0000","last_undegraded":"2024-02-22T19:43:39.364629+0000","last_fullsized":"2024-02-22T19:43:39.364629+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":160,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362544+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:43:39.362544+0000","last_peered":"2024-02-22T19:43:39.362544+0000","last_clean":"2024-02-22T19:43:39.362544+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:43:39.362544+0000","last_undegraded":"2024-02-22T19:43:39.362544+0000","last_fullsized":"2024-02-22T19:43:39.362544+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"360'16540","reported_seq":16625,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371699+0000","last_change":"2024-02-22T19:40:00.881433+0000","last_active":"2024-02-22T19:43:39.371699+0000","last_peered":"2024-02-22T19:43:39.371699+0000","last_clean":"2024-02-22T19:43:39.371699+0000","last_became_active":"2024-02-22T19:40:00.880998+0000","last_became_peered":"2024-02-22T19:40:00.880998+0000","last_unstale":"2024-02-22T19:43:39.371699+0000","last_undegraded":"2024-02-22T19:43:39.371699+0000","last_fullsized":"2024-02-22T19:43:39.371699+0000","mapping_epoch":285,"log_start":"345'12943","ondisk_log_start":"345'12943","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3597,"ondisk_log_size":3597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":529280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":195,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363803+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:43:39.363803+0000","last_peered":"2024-02-22T19:43:39.363803+0000","last_clean":"2024-02-22T19:43:39.363803+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:43:39.363803+0000","last_undegraded":"2024-02-22T19:43:39.363803+0000","last_fullsized":"2024-02-22T19:43:39.363803+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":343,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365960+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:43:39.365960+0000","last_peered":"2024-02-22T19:43:39.365960+0000","last_clean":"2024-02-22T19:43:39.365960+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:43:39.365960+0000","last_undegraded":"2024-02-22T19:43:39.365960+0000","last_fullsized":"2024-02-22T19:43:39.365960+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":163,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364194+0000","last_change":"2024-02-22T19:41:52.111972+0000","last_active":"2024-02-22T19:43:39.364194+0000","last_peered":"2024-02-22T19:43:39.364194+0000","last_clean":"2024-02-22T19:43:39.364194+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:43:39.364194+0000","last_undegraded":"2024-02-22T19:43:39.364194+0000","last_fullsized":"2024-02-22T19:43:39.364194+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.036671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.036671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":278,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362322+0000","last_change":"2024-02-22T19:41:52.113125+0000","last_active":"2024-02-22T19:43:39.362322+0000","last_peered":"2024-02-22T19:43:39.362322+0000","last_clean":"2024-02-22T19:43:39.362322+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:43:39.362322+0000","last_undegraded":"2024-02-22T19:43:39.362322+0000","last_fullsized":"2024-02-22T19:43:39.362322+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":588,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.371671+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:43:40.371671+0000","last_peered":"2024-02-22T19:43:40.371671+0000","last_clean":"2024-02-22T19:43:40.371671+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:43:40.371671+0000","last_undegraded":"2024-02-22T19:43:40.371671+0000","last_fullsized":"2024-02-22T19:43:40.371671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":166,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361877+0000","last_change":"2024-02-22T19:41:52.101375+0000","last_active":"2024-02-22T19:43:39.361877+0000","last_peered":"2024-02-22T19:43:39.361877+0000","last_clean":"2024-02-22T19:43:39.361877+0000","last_became_active":"2024-02-22T19:41:52.101126+0000","last_became_peered":"2024-02-22T19:41:52.101126+0000","last_unstale":"2024-02-22T19:43:39.361877+0000","last_undegraded":"2024-02-22T19:43:39.361877+0000","last_fullsized":"2024-02-22T19:43:39.361877+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.a","version":"360'16094","reported_seq":16179,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363717+0000","last_change":"2024-02-22T19:40:00.880902+0000","last_active":"2024-02-22T19:43:39.363717+0000","last_peered":"2024-02-22T19:43:39.363717+0000","last_clean":"2024-02-22T19:43:39.363717+0000","last_became_active":"2024-02-22T19:40:00.880596+0000","last_became_peered":"2024-02-22T19:40:00.880596+0000","last_unstale":"2024-02-22T19:43:39.363717+0000","last_undegraded":"2024-02-22T19:43:39.363717+0000","last_fullsized":"2024-02-22T19:43:39.363717+0000","mapping_epoch":285,"log_start":"346'12935","ondisk_log_start":"346'12935","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3159,"ondisk_log_size":3159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":515008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":385,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371749+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:43:39.371749+0000","last_peered":"2024-02-22T19:43:39.371749+0000","last_clean":"2024-02-22T19:43:39.371749+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:43:39.371749+0000","last_undegraded":"2024-02-22T19:43:39.371749+0000","last_fullsized":"2024-02-22T19:43:39.371749+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":335,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372575+0000","last_change":"2024-02-22T19:41:52.109613+0000","last_active":"2024-02-22T19:43:39.372575+0000","last_peered":"2024-02-22T19:43:39.372575+0000","last_clean":"2024-02-22T19:43:39.372575+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:43:39.372575+0000","last_undegraded":"2024-02-22T19:43:39.372575+0000","last_fullsized":"2024-02-22T19:43:39.372575+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":348,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360133+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:43:39.360133+0000","last_peered":"2024-02-22T19:43:39.360133+0000","last_clean":"2024-02-22T19:43:39.360133+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:43:39.360133+0000","last_undegraded":"2024-02-22T19:43:39.360133+0000","last_fullsized":"2024-02-22T19:43:39.360133+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362402+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:43:39.362402+0000","last_peered":"2024-02-22T19:43:39.362402+0000","last_clean":"2024-02-22T19:43:39.362402+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:43:39.362402+0000","last_undegraded":"2024-02-22T19:43:39.362402+0000","last_fullsized":"2024-02-22T19:43:39.362402+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":436,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364056+0000","last_change":"2024-02-22T19:41:20.253383+0000","last_active":"2024-02-22T19:43:39.364056+0000","last_peered":"2024-02-22T19:43:39.364056+0000","last_clean":"2024-02-22T19:43:39.364056+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:39.364056+0000","last_undegraded":"2024-02-22T19:43:39.364056+0000","last_fullsized":"2024-02-22T19:43:39.364056+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:41:20.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:20.253277+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":379,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362683+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:43:39.362683+0000","last_peered":"2024-02-22T19:43:39.362683+0000","last_clean":"2024-02-22T19:43:39.362683+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:43:39.362683+0000","last_undegraded":"2024-02-22T19:43:39.362683+0000","last_fullsized":"2024-02-22T19:43:39.362683+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"360'16398","reported_seq":16483,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364244+0000","last_change":"2024-02-22T19:40:00.879698+0000","last_active":"2024-02-22T19:43:39.364244+0000","last_peered":"2024-02-22T19:43:39.364244+0000","last_clean":"2024-02-22T19:43:39.364244+0000","last_became_active":"2024-02-22T19:40:00.879269+0000","last_became_peered":"2024-02-22T19:40:00.879269+0000","last_unstale":"2024-02-22T19:43:39.364244+0000","last_undegraded":"2024-02-22T19:43:39.364244+0000","last_fullsized":"2024-02-22T19:43:39.364244+0000","mapping_epoch":285,"log_start":"346'13245","ondisk_log_start":"346'13245","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3153,"ondisk_log_size":3153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16398,"num_write_kb":524736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":168,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365489+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:43:39.365489+0000","last_peered":"2024-02-22T19:43:39.365489+0000","last_clean":"2024-02-22T19:43:39.365489+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:43:39.365489+0000","last_undegraded":"2024-02-22T19:43:39.365489+0000","last_fullsized":"2024-02-22T19:43:39.365489+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":385,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.370794+0000","last_change":"2024-02-22T19:41:52.104562+0000","last_active":"2024-02-22T19:43:39.370794+0000","last_peered":"2024-02-22T19:43:39.370794+0000","last_clean":"2024-02-22T19:43:39.370794+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:43:39.370794+0000","last_undegraded":"2024-02-22T19:43:39.370794+0000","last_fullsized":"2024-02-22T19:43:39.370794+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":353,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364799+0000","last_change":"2024-02-22T19:41:52.111795+0000","last_active":"2024-02-22T19:43:39.364799+0000","last_peered":"2024-02-22T19:43:39.364799+0000","last_clean":"2024-02-22T19:43:39.364799+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:43:39.364799+0000","last_undegraded":"2024-02-22T19:43:39.364799+0000","last_fullsized":"2024-02-22T19:43:39.364799+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":393,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371972+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:43:39.371972+0000","last_peered":"2024-02-22T19:43:39.371972+0000","last_clean":"2024-02-22T19:43:39.371972+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:43:39.371972+0000","last_undegraded":"2024-02-22T19:43:39.371972+0000","last_fullsized":"2024-02-22T19:43:39.371972+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":346,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.372134+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:43:39.372134+0000","last_peered":"2024-02-22T19:43:39.372134+0000","last_clean":"2024-02-22T19:43:39.372134+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:43:39.372134+0000","last_undegraded":"2024-02-22T19:43:39.372134+0000","last_fullsized":"2024-02-22T19:43:39.372134+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"360'16392","reported_seq":18953,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.391359+0000","last_change":"2024-02-22T19:43:40.391359+0000","last_active":"2024-02-22T19:43:40.391359+0000","last_peered":"2024-02-22T19:43:40.391359+0000","last_clean":"2024-02-22T19:43:40.391359+0000","last_became_active":"2024-02-22T19:43:40.374568+0000","last_became_peered":"2024-02-22T19:43:40.374568+0000","last_unstale":"2024-02-22T19:43:40.391359+0000","last_undegraded":"2024-02-22T19:43:40.391359+0000","last_fullsized":"2024-02-22T19:43:40.391359+0000","mapping_epoch":361,"log_start":"346'13245","ondisk_log_start":"346'13245","created":285,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3147,"ondisk_log_size":3147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16392,"num_write_kb":524544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1227,"num_bytes_recovered":80412672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":239,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363137+0000","last_change":"2024-02-22T19:41:52.114247+0000","last_active":"2024-02-22T19:43:39.363137+0000","last_peered":"2024-02-22T19:43:39.363137+0000","last_clean":"2024-02-22T19:43:39.363137+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:43:39.363137+0000","last_undegraded":"2024-02-22T19:43:39.363137+0000","last_fullsized":"2024-02-22T19:43:39.363137+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":351,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360566+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:43:39.360566+0000","last_peered":"2024-02-22T19:43:39.360566+0000","last_clean":"2024-02-22T19:43:39.360566+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:43:39.360566+0000","last_undegraded":"2024-02-22T19:43:39.360566+0000","last_fullsized":"2024-02-22T19:43:39.360566+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.369269+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:43:39.369269+0000","last_peered":"2024-02-22T19:43:39.369269+0000","last_clean":"2024-02-22T19:43:39.369269+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:43:39.369269+0000","last_undegraded":"2024-02-22T19:43:39.369269+0000","last_fullsized":"2024-02-22T19:43:39.369269+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1255,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:42.889117+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:43:42.889117+0000","last_peered":"2024-02-22T19:43:42.889117+0000","last_clean":"2024-02-22T19:43:42.889117+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:43:42.889117+0000","last_undegraded":"2024-02-22T19:43:42.889117+0000","last_fullsized":"2024-02-22T19:43:42.889117+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":866,"num_read_kb":861,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":168,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361681+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:43:39.361681+0000","last_peered":"2024-02-22T19:43:39.361681+0000","last_clean":"2024-02-22T19:43:39.361681+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:43:39.361681+0000","last_undegraded":"2024-02-22T19:43:39.361681+0000","last_fullsized":"2024-02-22T19:43:39.361681+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":213,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365490+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:43:39.365490+0000","last_peered":"2024-02-22T19:43:39.365490+0000","last_clean":"2024-02-22T19:43:39.365490+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:43:39.365490+0000","last_undegraded":"2024-02-22T19:43:39.365490+0000","last_fullsized":"2024-02-22T19:43:39.365490+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"360'16150","reported_seq":16235,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371958+0000","last_change":"2024-02-22T19:40:00.880926+0000","last_active":"2024-02-22T19:43:39.371958+0000","last_peered":"2024-02-22T19:43:39.371958+0000","last_clean":"2024-02-22T19:43:39.371958+0000","last_became_active":"2024-02-22T19:40:00.880593+0000","last_became_peered":"2024-02-22T19:40:00.880593+0000","last_unstale":"2024-02-22T19:43:39.371958+0000","last_undegraded":"2024-02-22T19:43:39.371958+0000","last_fullsized":"2024-02-22T19:43:39.371958+0000","mapping_epoch":285,"log_start":"344'12564","ondisk_log_start":"344'12564","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3586,"ondisk_log_size":3586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16150,"num_write_kb":516800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":215,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365734+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:43:39.365734+0000","last_peered":"2024-02-22T19:43:39.365734+0000","last_clean":"2024-02-22T19:43:39.365734+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:43:39.365734+0000","last_undegraded":"2024-02-22T19:43:39.365734+0000","last_fullsized":"2024-02-22T19:43:39.365734+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":337,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360499+0000","last_change":"2024-02-22T19:41:52.094945+0000","last_active":"2024-02-22T19:43:39.360499+0000","last_peered":"2024-02-22T19:43:39.360499+0000","last_clean":"2024-02-22T19:43:39.360499+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:43:39.360499+0000","last_undegraded":"2024-02-22T19:43:39.360499+0000","last_fullsized":"2024-02-22T19:43:39.360499+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":352,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360784+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:43:39.360784+0000","last_peered":"2024-02-22T19:43:39.360784+0000","last_clean":"2024-02-22T19:43:39.360784+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:43:39.360784+0000","last_undegraded":"2024-02-22T19:43:39.360784+0000","last_fullsized":"2024-02-22T19:43:39.360784+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":173,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.368300+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:43:39.368300+0000","last_peered":"2024-02-22T19:43:39.368300+0000","last_clean":"2024-02-22T19:43:39.368300+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:43:39.368300+0000","last_undegraded":"2024-02-22T19:43:39.368300+0000","last_fullsized":"2024-02-22T19:43:39.368300+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":322,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363962+0000","last_change":"2024-02-22T19:41:52.107671+0000","last_active":"2024-02-22T19:43:39.363962+0000","last_peered":"2024-02-22T19:43:39.363962+0000","last_clean":"2024-02-22T19:43:39.363962+0000","last_became_active":"2024-02-22T19:41:52.106767+0000","last_became_peered":"2024-02-22T19:41:52.106767+0000","last_unstale":"2024-02-22T19:43:39.363962+0000","last_undegraded":"2024-02-22T19:43:39.363962+0000","last_fullsized":"2024-02-22T19:43:39.363962+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"360'16420","reported_seq":16505,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364606+0000","last_change":"2024-02-22T19:40:00.879641+0000","last_active":"2024-02-22T19:43:39.364606+0000","last_peered":"2024-02-22T19:43:39.364606+0000","last_clean":"2024-02-22T19:43:39.364606+0000","last_became_active":"2024-02-22T19:40:00.879325+0000","last_became_peered":"2024-02-22T19:40:00.879325+0000","last_unstale":"2024-02-22T19:43:39.364606+0000","last_undegraded":"2024-02-22T19:43:39.364606+0000","last_fullsized":"2024-02-22T19:43:39.364606+0000","mapping_epoch":285,"log_start":"346'13290","ondisk_log_start":"346'13290","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3130,"ondisk_log_size":3130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16420,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.366033+0000","last_change":"2024-02-22T19:41:52.110027+0000","last_active":"2024-02-22T19:43:39.366033+0000","last_peered":"2024-02-22T19:43:39.366033+0000","last_clean":"2024-02-22T19:43:39.366033+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:43:39.366033+0000","last_undegraded":"2024-02-22T19:43:39.366033+0000","last_fullsized":"2024-02-22T19:43:39.366033+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.670303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":148,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364455+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:43:39.364455+0000","last_peered":"2024-02-22T19:43:39.364455+0000","last_clean":"2024-02-22T19:43:39.364455+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:43:39.364455+0000","last_undegraded":"2024-02-22T19:43:39.364455+0000","last_fullsized":"2024-02-22T19:43:39.364455+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":159,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365301+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:43:39.365301+0000","last_peered":"2024-02-22T19:43:39.365301+0000","last_clean":"2024-02-22T19:43:39.365301+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:43:39.365301+0000","last_undegraded":"2024-02-22T19:43:39.365301+0000","last_fullsized":"2024-02-22T19:43:39.365301+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":472,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360816+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:43:39.360816+0000","last_peered":"2024-02-22T19:43:39.360816+0000","last_clean":"2024-02-22T19:43:39.360816+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:43:39.360816+0000","last_undegraded":"2024-02-22T19:43:39.360816+0000","last_fullsized":"2024-02-22T19:43:39.360816+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"326'14","reported_seq":385,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.369371+0000","last_change":"2024-02-22T19:41:52.698274+0000","last_active":"2024-02-22T19:43:40.369371+0000","last_peered":"2024-02-22T19:43:40.369371+0000","last_clean":"2024-02-22T19:43:40.369371+0000","last_became_active":"2024-02-22T19:41:52.112897+0000","last_became_peered":"2024-02-22T19:41:52.112897+0000","last_unstale":"2024-02-22T19:43:40.369371+0000","last_undegraded":"2024-02-22T19:43:40.369371+0000","last_fullsized":"2024-02-22T19:43:40.369371+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"360'16296","reported_seq":18848,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.396246+0000","last_change":"2024-02-22T19:43:40.396246+0000","last_active":"2024-02-22T19:43:40.396246+0000","last_peered":"2024-02-22T19:43:40.396246+0000","last_clean":"2024-02-22T19:43:40.396246+0000","last_became_active":"2024-02-22T19:43:40.373388+0000","last_became_peered":"2024-02-22T19:43:40.373388+0000","last_unstale":"2024-02-22T19:43:40.396246+0000","last_undegraded":"2024-02-22T19:43:40.396246+0000","last_fullsized":"2024-02-22T19:43:40.396246+0000","mapping_epoch":361,"log_start":"346'13158","ondisk_log_start":"346'13158","created":285,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3138,"ondisk_log_size":3138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16296,"num_write_kb":521472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1222,"num_bytes_recovered":80084992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":239,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365218+0000","last_change":"2024-02-22T19:41:52.103880+0000","last_active":"2024-02-22T19:43:39.365218+0000","last_peered":"2024-02-22T19:43:39.365218+0000","last_clean":"2024-02-22T19:43:39.365218+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:43:39.365218+0000","last_undegraded":"2024-02-22T19:43:39.365218+0000","last_fullsized":"2024-02-22T19:43:39.365218+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:55.451193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:37:55.451193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":338,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371588+0000","last_change":"2024-02-22T19:41:52.103942+0000","last_active":"2024-02-22T19:43:39.371588+0000","last_peered":"2024-02-22T19:43:39.371588+0000","last_clean":"2024-02-22T19:43:39.371588+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:43:39.371588+0000","last_undegraded":"2024-02-22T19:43:39.371588+0000","last_fullsized":"2024-02-22T19:43:39.371588+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":157,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361518+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:43:39.361518+0000","last_peered":"2024-02-22T19:43:39.361518+0000","last_clean":"2024-02-22T19:43:39.361518+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:43:39.361518+0000","last_undegraded":"2024-02-22T19:43:39.361518+0000","last_fullsized":"2024-02-22T19:43:39.361518+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":474,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364210+0000","last_change":"2024-02-22T19:41:52.715987+0000","last_active":"2024-02-22T19:43:39.364210+0000","last_peered":"2024-02-22T19:43:39.364210+0000","last_clean":"2024-02-22T19:43:39.364210+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:43:39.364210+0000","last_undegraded":"2024-02-22T19:43:39.364210+0000","last_fullsized":"2024-02-22T19:43:39.364210+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":548,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.366385+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:43:40.366385+0000","last_peered":"2024-02-22T19:43:40.366385+0000","last_clean":"2024-02-22T19:43:40.366385+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:43:40.366385+0000","last_undegraded":"2024-02-22T19:43:40.366385+0000","last_fullsized":"2024-02-22T19:43:40.366385+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"360'16174","reported_seq":16260,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371516+0000","last_change":"2024-02-22T19:40:00.879748+0000","last_active":"2024-02-22T19:43:39.371516+0000","last_peered":"2024-02-22T19:43:39.371516+0000","last_clean":"2024-02-22T19:43:39.371516+0000","last_became_active":"2024-02-22T19:40:00.879466+0000","last_became_peered":"2024-02-22T19:40:00.879466+0000","last_unstale":"2024-02-22T19:43:39.371516+0000","last_undegraded":"2024-02-22T19:43:39.371516+0000","last_fullsized":"2024-02-22T19:43:39.371516+0000","mapping_epoch":285,"log_start":"345'12635","ondisk_log_start":"345'12635","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3539,"ondisk_log_size":3539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16174,"num_write_kb":517505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":141,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361196+0000","last_change":"2024-02-22T19:41:52.102626+0000","last_active":"2024-02-22T19:43:39.361196+0000","last_peered":"2024-02-22T19:43:39.361196+0000","last_clean":"2024-02-22T19:43:39.361196+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:43:39.361196+0000","last_undegraded":"2024-02-22T19:43:39.361196+0000","last_fullsized":"2024-02-22T19:43:39.361196+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:33.048434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.048434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":140,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365382+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:43:39.365382+0000","last_peered":"2024-02-22T19:43:39.365382+0000","last_clean":"2024-02-22T19:43:39.365382+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:43:39.365382+0000","last_undegraded":"2024-02-22T19:43:39.365382+0000","last_fullsized":"2024-02-22T19:43:39.365382+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":358,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.366220+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:43:39.366220+0000","last_peered":"2024-02-22T19:43:39.366220+0000","last_clean":"2024-02-22T19:43:39.366220+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:43:39.366220+0000","last_undegraded":"2024-02-22T19:43:39.366220+0000","last_fullsized":"2024-02-22T19:43:39.366220+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":462,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364724+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:43:39.364724+0000","last_peered":"2024-02-22T19:43:39.364724+0000","last_clean":"2024-02-22T19:43:39.364724+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:43:39.364724+0000","last_undegraded":"2024-02-22T19:43:39.364724+0000","last_fullsized":"2024-02-22T19:43:39.364724+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":149,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.369354+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:43:39.369354+0000","last_peered":"2024-02-22T19:43:39.369354+0000","last_clean":"2024-02-22T19:43:39.369354+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:43:39.369354+0000","last_undegraded":"2024-02-22T19:43:39.369354+0000","last_fullsized":"2024-02-22T19:43:39.369354+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"360'16176","reported_seq":16261,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364542+0000","last_change":"2024-02-22T19:40:01.098068+0000","last_active":"2024-02-22T19:43:39.364542+0000","last_peered":"2024-02-22T19:43:39.364542+0000","last_clean":"2024-02-22T19:43:39.364542+0000","last_became_active":"2024-02-22T19:40:01.097724+0000","last_became_peered":"2024-02-22T19:40:01.097724+0000","last_unstale":"2024-02-22T19:43:39.364542+0000","last_undegraded":"2024-02-22T19:43:39.364542+0000","last_fullsized":"2024-02-22T19:43:39.364542+0000","mapping_epoch":285,"log_start":"346'12958","ondisk_log_start":"346'12958","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3218,"ondisk_log_size":3218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16176,"num_write_kb":517632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":171,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364457+0000","last_change":"2024-02-22T19:41:52.107922+0000","last_active":"2024-02-22T19:43:39.364457+0000","last_peered":"2024-02-22T19:43:39.364457+0000","last_clean":"2024-02-22T19:43:39.364457+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:43:39.364457+0000","last_undegraded":"2024-02-22T19:43:39.364457+0000","last_fullsized":"2024-02-22T19:43:39.364457+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":186,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365138+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:43:39.365138+0000","last_peered":"2024-02-22T19:43:39.365138+0000","last_clean":"2024-02-22T19:43:39.365138+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:43:39.365138+0000","last_undegraded":"2024-02-22T19:43:39.365138+0000","last_fullsized":"2024-02-22T19:43:39.365138+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":663,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.370243+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:43:40.370243+0000","last_peered":"2024-02-22T19:43:40.370243+0000","last_clean":"2024-02-22T19:43:40.370243+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:43:40.370243+0000","last_undegraded":"2024-02-22T19:43:40.370243+0000","last_fullsized":"2024-02-22T19:43:40.370243+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":331,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.364677+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:43:39.364677+0000","last_peered":"2024-02-22T19:43:39.364677+0000","last_clean":"2024-02-22T19:43:39.364677+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:43:39.364677+0000","last_undegraded":"2024-02-22T19:43:39.364677+0000","last_fullsized":"2024-02-22T19:43:39.364677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":75,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365209+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:43:39.365209+0000","last_peered":"2024-02-22T19:43:39.365209+0000","last_clean":"2024-02-22T19:43:39.365209+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:43:39.365209+0000","last_undegraded":"2024-02-22T19:43:39.365209+0000","last_fullsized":"2024-02-22T19:43:39.365209+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"360'16430","reported_seq":16515,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.370705+0000","last_change":"2024-02-22T19:40:00.878514+0000","last_active":"2024-02-22T19:43:39.370705+0000","last_peered":"2024-02-22T19:43:39.370705+0000","last_clean":"2024-02-22T19:43:39.370705+0000","last_became_active":"2024-02-22T19:40:00.878173+0000","last_became_peered":"2024-02-22T19:40:00.878173+0000","last_unstale":"2024-02-22T19:43:39.370705+0000","last_undegraded":"2024-02-22T19:43:39.370705+0000","last_fullsized":"2024-02-22T19:43:39.370705+0000","mapping_epoch":285,"log_start":"345'12801","ondisk_log_start":"345'12801","created":285,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3629,"ondisk_log_size":3629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16430,"num_write_kb":525760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":347,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365412+0000","last_change":"2024-02-22T19:41:52.112553+0000","last_active":"2024-02-22T19:43:39.365412+0000","last_peered":"2024-02-22T19:43:39.365412+0000","last_clean":"2024-02-22T19:43:39.365412+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:43:39.365412+0000","last_undegraded":"2024-02-22T19:43:39.365412+0000","last_fullsized":"2024-02-22T19:43:39.365412+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"326'20","reported_seq":409,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.370170+0000","last_change":"2024-02-22T19:41:54.322898+0000","last_active":"2024-02-22T19:43:40.370170+0000","last_peered":"2024-02-22T19:43:40.370170+0000","last_clean":"2024-02-22T19:43:40.370170+0000","last_became_active":"2024-02-22T19:41:54.322531+0000","last_became_peered":"2024-02-22T19:41:54.322531+0000","last_unstale":"2024-02-22T19:43:40.370170+0000","last_undegraded":"2024-02-22T19:43:40.370170+0000","last_fullsized":"2024-02-22T19:43:40.370170+0000","mapping_epoch":325,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":136,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.362939+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:43:39.362939+0000","last_peered":"2024-02-22T19:43:39.362939+0000","last_clean":"2024-02-22T19:43:39.362939+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:43:39.362939+0000","last_undegraded":"2024-02-22T19:43:39.362939+0000","last_fullsized":"2024-02-22T19:43:39.362939+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":144,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361609+0000","last_change":"2024-02-22T19:41:52.358275+0000","last_active":"2024-02-22T19:43:39.361609+0000","last_peered":"2024-02-22T19:43:39.361609+0000","last_clean":"2024-02-22T19:43:39.361609+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:43:39.361609+0000","last_undegraded":"2024-02-22T19:43:39.361609+0000","last_fullsized":"2024-02-22T19:43:39.361609+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"360'16362","reported_seq":28497,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.359632+0000","last_change":"2024-02-22T19:42:34.754313+0000","last_active":"2024-02-22T19:43:39.359632+0000","last_peered":"2024-02-22T19:43:39.359632+0000","last_clean":"2024-02-22T19:43:39.359632+0000","last_became_active":"2024-02-22T19:42:34.737475+0000","last_became_peered":"2024-02-22T19:42:34.737475+0000","last_unstale":"2024-02-22T19:43:39.359632+0000","last_undegraded":"2024-02-22T19:43:39.359632+0000","last_fullsized":"2024-02-22T19:43:39.359632+0000","mapping_epoch":339,"log_start":"345'12870","ondisk_log_start":"345'12870","created":285,"last_epoch_clean":340,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3492,"ondisk_log_size":3492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16362,"num_write_kb":523584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6014,"num_bytes_recovered":394133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":159,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363764+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:43:39.363764+0000","last_peered":"2024-02-22T19:43:39.363764+0000","last_clean":"2024-02-22T19:43:39.363764+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:43:39.363764+0000","last_undegraded":"2024-02-22T19:43:39.363764+0000","last_fullsized":"2024-02-22T19:43:39.363764+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":351,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.363946+0000","last_change":"2024-02-22T19:41:52.107535+0000","last_active":"2024-02-22T19:43:39.363946+0000","last_peered":"2024-02-22T19:43:39.363946+0000","last_clean":"2024-02-22T19:43:39.363946+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:43:39.363946+0000","last_undegraded":"2024-02-22T19:43:39.363946+0000","last_fullsized":"2024-02-22T19:43:39.363946+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":393,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361007+0000","last_change":"2024-02-22T19:41:52.092333+0000","last_active":"2024-02-22T19:43:39.361007+0000","last_peered":"2024-02-22T19:43:39.361007+0000","last_clean":"2024-02-22T19:43:39.361007+0000","last_became_active":"2024-02-22T19:41:52.092045+0000","last_became_peered":"2024-02-22T19:41:52.092045+0000","last_unstale":"2024-02-22T19:43:39.361007+0000","last_undegraded":"2024-02-22T19:43:39.361007+0000","last_fullsized":"2024-02-22T19:43:39.361007+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":268,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.360449+0000","last_change":"2024-02-22T19:41:52.110141+0000","last_active":"2024-02-22T19:43:39.360449+0000","last_peered":"2024-02-22T19:43:39.360449+0000","last_clean":"2024-02-22T19:43:39.360449+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:43:39.360449+0000","last_undegraded":"2024-02-22T19:43:39.360449+0000","last_fullsized":"2024-02-22T19:43:39.360449+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":237,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361343+0000","last_change":"2024-02-22T19:41:52.810562+0000","last_active":"2024-02-22T19:43:39.361343+0000","last_peered":"2024-02-22T19:43:39.361343+0000","last_clean":"2024-02-22T19:43:39.361343+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:43:39.361343+0000","last_undegraded":"2024-02-22T19:43:39.361343+0000","last_fullsized":"2024-02-22T19:43:39.361343+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"360'16108","reported_seq":18662,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.387994+0000","last_change":"2024-02-22T19:43:40.387994+0000","last_active":"2024-02-22T19:43:40.387994+0000","last_peered":"2024-02-22T19:43:40.387994+0000","last_clean":"2024-02-22T19:43:40.387994+0000","last_became_active":"2024-02-22T19:43:40.371795+0000","last_became_peered":"2024-02-22T19:43:40.371795+0000","last_unstale":"2024-02-22T19:43:40.387994+0000","last_undegraded":"2024-02-22T19:43:40.387994+0000","last_fullsized":"2024-02-22T19:43:40.387994+0000","mapping_epoch":361,"log_start":"346'12917","ondisk_log_start":"346'12917","created":285,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:59.863953+0000","last_clean_scrub_stamp":"2024-02-22T19:39:59.863953+0000","log_size":3191,"ondisk_log_size":3191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16108,"num_write_kb":515456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":80150528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":349,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.371819+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:43:39.371819+0000","last_peered":"2024-02-22T19:43:39.371819+0000","last_clean":"2024-02-22T19:43:39.371819+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:43:39.371819+0000","last_undegraded":"2024-02-22T19:43:39.371819+0000","last_fullsized":"2024-02-22T19:43:39.371819+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":175,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.361376+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:43:39.361376+0000","last_peered":"2024-02-22T19:43:39.361376+0000","last_clean":"2024-02-22T19:43:39.361376+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:43:39.361376+0000","last_undegraded":"2024-02-22T19:43:39.361376+0000","last_fullsized":"2024-02-22T19:43:39.361376+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":587,"reported_epoch":362,"state":"active+clean","last_fresh":"2024-02-22T19:43:40.369536+0000","last_change":"2024-02-22T19:41:32.453253+0000","last_active":"2024-02-22T19:43:40.369536+0000","last_peered":"2024-02-22T19:43:40.369536+0000","last_clean":"2024-02-22T19:43:40.369536+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:43:40.369536+0000","last_undegraded":"2024-02-22T19:43:40.369536+0000","last_fullsized":"2024-02-22T19:43:40.369536+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":392,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365167+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:43:39.365167+0000","last_peered":"2024-02-22T19:43:39.365167+0000","last_clean":"2024-02-22T19:43:39.365167+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:43:39.365167+0000","last_undegraded":"2024-02-22T19:43:39.365167+0000","last_fullsized":"2024-02-22T19:43:39.365167+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":206,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-02-22T19:43:39.365687+0000","last_change":"2024-02-22T19:41:52.575055+0000","last_active":"2024-02-22T19:43:39.365687+0000","last_peered":"2024-02-22T19:43:39.365687+0000","last_clean":"2024-02-22T19:43:39.365687+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:43:39.365687+0000","last_undegraded":"2024-02-22T19:43:39.365687+0000","last_fullsized":"2024-02-22T19:43:39.365687+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260914,"num_write_kb":8349185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15577,"num_bytes_recovered":1020854272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":340656128,"data_stored":340656128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52729,"ondisk_log_size":52729,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":866,"num_read_kb":861,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266868,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":592284,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93152868,"statfs":{"total":95995035648,"available":95388536832,"internally_reserved":0,"allocated":2641920,"data_stored":1452013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.2490000000000001,"5min":1.532,"15min":1.5029999999999999},"min":{"1min":0.249,"5min":0.245,"15min":0.19900000000000001},"max":{"1min":7.9790000000000001,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.379,"15min":1.421},"min":{"1min":0.25600000000000001,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":3.25,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":2.4940000000000002,"15min":1.7529999999999999},"min":{"1min":0.28399999999999997,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":6.1200000000000001,"5min":54.951000000000001,"15min":54.951000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.218,"5min":2.5049999999999999,"15min":1.921},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":3.613,"5min":57.012,"15min":57.012},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.825,"15min":1.4370000000000001},"min":{"1min":0.27200000000000002,"5min":0.27000000000000002,"15min":0.23999999999999999},"max":{"1min":5.1200000000000001,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.3769999999999998,"15min":1.7310000000000001},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.23599999999999999},"max":{"1min":5.0960000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":2.6459999999999999,"15min":1.861},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":7.1289999999999996,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":3.1739999999999999,"5min":2.8090000000000002,"15min":1.9139999999999999},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":9.4039999999999999,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.443}]},{"osd":4,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.004,"15min":0.70599999999999996},"min":{"1min":0.31,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":4.4219999999999997,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.258,"15min":0.755},"min":{"1min":0.29899999999999999,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":5.0910000000000002,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.35299999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.419,"15min":0.86199999999999999},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.27000000000000002},"max":{"1min":6.625,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.093,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.28000000000000003},"max":{"1min":7.3239999999999998,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.42499999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.004,"15min":0.81000000000000005},"min":{"1min":0.32300000000000001,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":3.044,"5min":6.5,"15min":12.199999999999999},"last":0.502},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":0.90800000000000003},"min":{"1min":0.36899999999999999,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":3.6680000000000001,"5min":7.6390000000000002,"15min":12.067},"last":0.45700000000000002}]}]},{"osd":6,"up_from":0,"seq":193273528505,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575058,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":592296,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93152856,"statfs":{"total":95995035648,"available":95388524544,"internally_reserved":0,"allocated":2654208,"data_stored":1451989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":5.7480000000000002,"5min":2.391,"15min":1.8580000000000001},"min":{"1min":0.38900000000000001,"5min":0.185,"15min":0.185},"max":{"1min":51.091000000000001,"5min":51.091000000000001,"15min":51.091000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":5.1799999999999997,"5min":2.3860000000000001,"15min":1.952},"min":{"1min":0.38300000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":51.289999999999999,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.41799999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":5.4669999999999996,"5min":2.4630000000000001,"15min":1.9710000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":51.027000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":4.7220000000000004,"5min":2.4950000000000001,"15min":2.077},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":51.369999999999997,"5min":51.369999999999997,"15min":51.369999999999997},"last":0.499}]},{"osd":3,"last update":"Thu Feb 22 19:42:53 2024","interfaces":[{"interface":"back","average":{"1min":5.6390000000000002,"5min":2.5960000000000001,"15min":2.0630000000000002},"min":{"1min":0.39800000000000002,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":51.755000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":5.1509999999999998,"5min":2.6339999999999999,"15min":2.1360000000000001},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":51.741,"5min":51.741,"15min":51.741},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:43:10 2024","interfaces":[{"interface":"back","average":{"1min":6.1319999999999997,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.32600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":51.158999999999999,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":3.843,"5min":1.853,"15min":1.385},"min":{"1min":0.311,"5min":0.311,"15min":0.245},"max":{"1min":51.276000000000003,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:43:32 2024","interfaces":[{"interface":"back","average":{"1min":5.1959999999999997,"5min":2.7109999999999999,"15min":1.8009999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":51.183999999999997,"5min":51.183999999999997,"15min":51.183999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":4.1459999999999999,"5min":2.2629999999999999,"15min":1.6100000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":51.212000000000003,"5min":51.212000000000003,"15min":51.212000000000003},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:42:55 2024","interfaces":[{"interface":"back","average":{"1min":4.1929999999999996,"5min":1.9059999999999999,"15min":1.4370000000000001},"min":{"1min":0.433,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":51.079999999999998,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":6.1859999999999999,"5min":2.4590000000000001,"15min":1.6899999999999999},"min":{"1min":0.44,"5min":0.379,"15min":0.34999999999999998},"max":{"1min":51.119999999999997,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":4.7080000000000002,"5min":2.5830000000000002,"15min":1.784},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":51.244999999999997,"5min":51.244999999999997,"15min":51.244999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":4.4050000000000002,"5min":1.8540000000000001,"15min":1.454},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":51.317999999999998,"5min":51.317999999999998,"15min":51.317999999999998},"last":0.77900000000000003}]}]},{"osd":0,"up_from":0,"seq":34359738583,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411532,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303892,"kb_used_data":2572,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441260,"statfs":{"total":95995035648,"available":95683850240,"internally_reserved":0,"allocated":2633728,"data_stored":1450280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":2.0699999999999998,"15min":1.706},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.18099999999999999},"max":{"1min":4.7000000000000002,"5min":16.763999999999999,"15min":22.338999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":2.2789999999999999,"5min":2.5640000000000001,"15min":2.1120000000000001},"min":{"1min":0.26700000000000002,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":14.603999999999999,"5min":30.427,"15min":30.427},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.8560000000000001,"15min":1.679},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.0819999999999999,"5min":12.349,"15min":12.349},"last":0.34300000000000003},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.889,"15min":1.585},"min":{"1min":0.215,"5min":0.215,"15min":0.193},"max":{"1min":5.1900000000000004,"5min":13.782,"15min":13.782},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:42:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":2.3740000000000001,"15min":1.9410000000000001},"min":{"1min":0.28299999999999997,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":12.151999999999999,"5min":16.076000000000001,"15min":30.071000000000002},"last":0.432},{"interface":"front","average":{"1min":4.5179999999999998,"5min":3.0430000000000001,"15min":2.258},"min":{"1min":0.312,"5min":0.29399999999999998,"15min":0.184},"max":{"1min":28.263999999999999,"5min":30.282,"15min":32.119},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:43:06 2024","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.585,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.0140000000000002,"5min":12.43,"15min":12.43},"last":0.747},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.861,"15min":1.2529999999999999},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":4.6420000000000003,"5min":10.34,"15min":10.34},"last":0.82999999999999996}]},{"osd":5,"last update":"Thu Feb 22 19:43:29 2024","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":2.2879999999999998,"15min":1.4690000000000001},"min":{"1min":0.28499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.867,"5min":40.633000000000003,"15min":40.633000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.964,"5min":2.3980000000000001,"15min":1.6859999999999999},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":6.2999999999999998,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:43:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2669999999999999,"5min":1.8140000000000001,"15min":1.4990000000000001},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.311},"max":{"1min":7.1550000000000002,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.698,"5min":1.9530000000000001,"15min":1.304},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":4.5940000000000003,"5min":10.369999999999999,"15min":10.369999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.736,"15min":1.5049999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.1580000000000004,"5min":12.484,"15min":12.484},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.9299999999999999,"15min":1.407},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":4.5999999999999996,"5min":10.539,"15min":10.539},"last":0.81200000000000006}]}]},{"osd":3,"up_from":26,"seq":111669149894,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898344,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":895744,"kb_avail":92846808,"statfs":{"total":95995035648,"available":95075131392,"internally_reserved":0,"allocated":2654208,"data_stored":1451882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":917241856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":2.544,"5min":1.964,"15min":1.8169999999999999},"min":{"1min":0.22,"5min":0.191,"15min":0.191},"max":{"1min":7.2560000000000002,"5min":9.9789999999999992,"15min":23.48},"last":0.30399999999999999},{"interface":"front","average":{"1min":2.7469999999999999,"5min":2.4209999999999998,"15min":2.1139999999999999},"min":{"1min":0.254,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":7.1980000000000004,"5min":18.172000000000001,"15min":23.411999999999999},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":3.395,"15min":2.427},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.749000000000001,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":2.7189999999999999,"5min":2.851,"15min":1.974},"min":{"1min":0.38400000000000001,"5min":0.23000000000000001,"15min":0.214},"max":{"1min":8.0869999999999997,"5min":22.187000000000001,"15min":22.187000000000001},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:42:48 2024","interfaces":[{"interface":"back","average":{"1min":2.5139999999999998,"5min":2.367,"15min":1.994},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.2800000000000002,"5min":20.105,"15min":20.105},"last":0.311},{"interface":"front","average":{"1min":2.823,"5min":2.5270000000000001,"15min":1.978},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.217},"max":{"1min":9.1479999999999997,"5min":35.939999999999998,"15min":35.939999999999998},"last":0.26000000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":2.3740000000000001,"5min":2.1699999999999999,"15min":1.73},"min":{"1min":0.30599999999999999,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":9.968,"5min":12.416,"15min":13.680999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.9890000000000001,"15min":1.5429999999999999},"min":{"1min":0.40400000000000003,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":3.6019999999999999,"5min":13.548,"15min":13.548},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Feb 22 19:43:35 2024","interfaces":[{"interface":"back","average":{"1min":2.1099999999999999,"5min":1.9570000000000001,"15min":1.5800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1159999999999997,"5min":11.161,"15min":12.879},"last":0.63600000000000001},{"interface":"front","average":{"1min":2.7080000000000002,"5min":2.2029999999999998,"15min":1.7829999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":10.041,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:42:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3119999999999998,"5min":1.9019999999999999,"15min":1.514},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":7.1440000000000001,"5min":11.263,"15min":12.898999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":2.573,"5min":2.1230000000000002,"15min":1.772},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":10.02,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:43:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":2.0920000000000001,"15min":1.595},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":3.6259999999999999,"5min":13.483000000000001,"15min":13.483000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":2.2770000000000001,"5min":2.0659999999999998,"15min":1.621},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.2240000000000002,"5min":11.539,"15min":12.846},"last":0.70999999999999996}]}]},{"osd":4,"up_from":0,"seq":137438953668,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724734,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":588204,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":585600,"kb_avail":93156948,"statfs":{"total":95995035648,"available":95392714752,"internally_reserved":0,"allocated":2658304,"data_stored":1451314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":599654400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":8.4299999999999997,"5min":17.684000000000001,"15min":17.684000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.339,"5min":1.7629999999999999,"15min":1.5129999999999999},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":4.7060000000000004,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.4059999999999999,"15min":1.375},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":3.0070000000000001,"5min":11.202,"15min":29.390999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":2.052,"5min":1.8089999999999999,"15min":1.5169999999999999},"min":{"1min":0.309,"5min":0.248,"15min":0.218},"max":{"1min":6.5129999999999999,"5min":8.7490000000000006,"15min":29.367000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":2.044,"15min":1.534},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.7550000000000008,"5min":11.367000000000001,"15min":14.613},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.595,"5min":1.8520000000000001,"15min":1.534},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":6.3979999999999997,"5min":16.099,"15min":16.099},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":4.0640000000000001,"5min":2.0329999999999999,"15min":1.778},"min":{"1min":0.41499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":16.370999999999999,"5min":16.370999999999999,"15min":16.370999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.6599999999999999,"15min":1.609},"min":{"1min":0.41499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.9199999999999999,"5min":8.6069999999999993,"15min":10.16},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.1379999999999999,"15min":0.73999999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.219},"max":{"1min":7.5419999999999998,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":2.234,"5min":1.496,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.23499999999999999},"max":{"1min":5.577,"5min":10.407,"15min":10.407},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":0.64600000000000002},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":2.0169999999999999,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.442},{"interface":"front","average":{"1min":1.756,"5min":1.238,"15min":0.78800000000000003},"min":{"1min":0.36699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.436,"5min":10.436,"15min":10.436},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Feb 22 19:43:26 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1830000000000001,"15min":0.79200000000000004},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.8410000000000002,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.496},{"interface":"front","average":{"1min":1.901,"5min":1.3899999999999999,"15min":0.876},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.27100000000000002},"max":{"1min":7.5069999999999997,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":340656128,"data_stored":340656128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:43:50.719 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:50.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:50.721 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:50.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:50.853 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:51.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:50.348521+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:43:51.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:50.348598+0000 mon.a (mon.0) 1473 : cluster [DBG] osdmap e364: 8 total, 8 up, 5 in 2024-02-22T19:43:51.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:50.348521+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:43:51.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:50.348598+0000 mon.a (mon.0) 1473 : cluster [DBG] osdmap e364: 8 total, 8 up, 5 in 2024-02-22T19:43:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:50.348521+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:43:51.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:50.348598+0000 mon.a (mon.0) 1473 : cluster [DBG] osdmap e364: 8 total, 8 up, 5 in 2024-02-22T19:43:52.305 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:43:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:51.673936+0000 mgr.y (mgr.14535) 569 : cluster [DBG] pgmap v736: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:52.582 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:51.673936+0000 mgr.y (mgr.14535) 569 : cluster [DBG] pgmap v736: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:52.600 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-02-22T19:43:52.600 INFO:tasks.radosbench:Beginning radosbench... 2024-02-22T19:43:52.600 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-02-22T19:43:52.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool create unique_pool_3 16 2024-02-22T19:43:52.603 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:52.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:52.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:52.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:43:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:51.673936+0000 mgr.y (mgr.14535) 569 : cluster [DBG] pgmap v736: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:43:52.864 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:52.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:43:52.880 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:43:52.880 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197527 8.b 8:d5caae7e:::benchmark_data_smithi053_130774_object85793:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.099502+0000", 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 306.77964564899997, 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119527755, 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197527 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099502+0000", 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099502+0000", 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999952 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099497+0000", 2024-02-22T19:43:52.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6509999999999996e-06 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099507+0000", 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099517+0000", 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5379e-05 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099532+0000", 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000157899 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099690+0000", 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0009e-05 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099700+0000", 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98788994600000002 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087590+0000", 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0902e-05 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087601+0000", 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1114758579999999 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.199077+0000", 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013723000000000001 2024-02-22T19:43:52.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.199214+0000", 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041806900000000003 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.199632+0000", 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017562853999999999 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217195+0000", 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010780099999999999 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217303+0000", 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023422699999999999 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217537+0000", 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013003050000000001 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218837+0000", 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8440999999999998e-05 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218896+0000", 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134117 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.219030+0000", 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119527755 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197529 8.b 8:d1af294b:::benchmark_data_smithi053_130774_object85795:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 306.77960713099998, 2024-02-22T19:43:52.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1239954820000002, 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197529 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:43:52.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099540+0000", 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5370000000000002e-06 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099543+0000", 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.905e-06 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099550+0000", 2024-02-22T19:43:52.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4659999999999998e-06 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099556+0000", 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175448 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099731+0000", 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0879999999999997e-06 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.099738+0000", 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98789419499999997 2024-02-22T19:43:52.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087632+0000", 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8289999999999997e-06 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087641+0000", 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.121857726 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.209499+0000", 2024-02-22T19:43:52.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133626 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.209633+0000", 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016687000000000001 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.209800+0000", 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074392160000000002 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217239+0000", 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030646299999999998 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217545+0000", 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001521435 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.219067+0000", 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042951250000000003 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223362+0000", 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3224999999999999e-05 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223415+0000", 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000120957 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223536+0000", 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1239954820000002 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197563 8.b 8:d21bad24:::benchmark_data_smithi053_130774_object85829:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.165645+0000", 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 306.713503022, 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0580848550000002, 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197563 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165645+0000", 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165645+0000", 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165643+0000", 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0870000000000003e-06 2024-02-22T19:43:52.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165650+0000", 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0628999999999999e-05 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165660+0000", 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9530000000000002e-06 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165667+0000", 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8199999999999999e-05 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165715+0000", 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0727000000000001e-05 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.165726+0000", 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92193606900000002 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087662+0000", 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087669+0000", 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.123577021 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.211246+0000", 2024-02-22T19:43:52.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112305 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.211358+0000", 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001133886 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.212492+0000", 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050974360000000003 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:43:52.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217590+0000", 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011748539999999999 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218765+0000", 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015207249999999999 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.220285+0000", 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032889820000000002 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223574+0000", 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2823e-05 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223617+0000", 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011241 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.223729+0000", 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0580848550000002 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197570 8.b 8:ddea507a:::benchmark_data_smithi053_130774_object85836:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:46.187988+0000", 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 306.69116005900003, 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0375089649999998, 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197570 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187988+0000", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187988+0000", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187986+0000", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1079999999999998e-06 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.187993+0000", 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8490000000000001e-06 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188002+0000", 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8790000000000002e-06 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188007+0000", 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7867000000000001e-05 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188055+0000", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.37e-06 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:46.188062+0000", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89962829 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087690+0000", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7170000000000008e-06 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.087698+0000", 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.126183054 2024-02-22T19:43:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.213881+0000", 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011198599999999999 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.213993+0000", 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016729099999999999 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.214160+0000", 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034367669999999999 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217597+0000", 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012083549999999999 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.218805+0000", 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001519357 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.220324+0000", 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050053090000000003 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.225330+0000", 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2014e-05 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.225382+0000", 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114812 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.225497+0000", 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0375089649999998 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:197603 8.b 8:d4577564:::benchmark_data_smithi053_130774_object85869:head [delete] snapc 0=[] ondisk+write+known_if_redirected e248)", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:38:47.359648+0000", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 305.51949975999997, 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87032219899999996, 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 197603 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359648+0000", 2024-02-22T19:43:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359648+0000", 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359646+0000", 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.126e-06 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359653+0000", 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2813000000000001e-05 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359666+0000", 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3090000000000001e-06 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359673+0000", 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5130999999999997e-05 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359738+0000", 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0944999999999999e-05 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:47.359749+0000", 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85562269999999996 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.215371+0000", 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102002 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.215473+0000", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000138956 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.215612+0000", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019917979999999999 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.217604+0000", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039613310000000002 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.221565+0000", 2024-02-22T19:43:52.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037830200000000001 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.221944+0000", 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078577630000000002 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.229801+0000", 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7669e-05 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.229849+0000", 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012089 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:38:48.229970+0000", 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.87032219899999996 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15324.0:180365 9.8 9:185704f7:::benchmark_data_smithi053_157429_object49906:head [delete] snapc 0=[] ondisk+write+known_if_redirected e336)", 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:42:27.548737+0000", 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.330410740000005, 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.058054812999999997, 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15324", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 180365 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548737+0000", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548737+0000", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548736+0000", 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9830000000000001e-06 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548742+0000", 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044e-05 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548752+0000", 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7409999999999998e-06 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548758+0000", 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3812000000000005e-05 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548822+0000", 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012638499999999999 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.548948+0000", 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042999800000000001 2024-02-22T19:43:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5,7", 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.549378+0000", 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024329669999999999 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.551811+0000", 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015248799999999999 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.551964+0000", 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.054707574000000002 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.606671+0000", 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6115000000000001e-05 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.606727+0000", 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4482999999999999e-05 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:42:27.606792+0000", 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.058054812999999997 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:2172 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 94129597097088 gen 3] snapc 0=[] ondisk+write+known_if_redirected e362)", 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:43:49.356286+0000", 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.522861684, 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047218100000000001, 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2172 2024-02-22T19:43:52.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356286+0000", 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356286+0000", 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356284+0000", 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0260000000000007e-06 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356292+0000", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8621999999999999e-05 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356311+0000", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356319+0000", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1747000000000001e-05 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356410+0000", 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000231499 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356642+0000", 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000116325 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356758+0000", 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047218100000000001 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.24332.0:2173 5.14 5:2b04a3e9:::notify.6:head [watch ping cookie 94129597102848] snapc 0=[] ondisk+write+known_if_redirected e362)", 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:43:49.356350+0000", 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5227972159999998, 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027663699999999999, 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.24332", 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2173 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356350+0000", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356350+0000", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356350+0000", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9579999999999999e-06 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:43:52.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356353+0000", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4210000000000003e-06 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356358+0000", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7369999999999997e-06 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356363+0000", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4973999999999999e-05 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356418+0000", 2024-02-22T19:43:52.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8585000000000006e-05 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356517+0000", 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110436 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:43:49.356627+0000", 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027663699999999999 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:43:52.930 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:43:52.931 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:52.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:53.020 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:53.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:53.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:53.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:53.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:43:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:53.181 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:53.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:53.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:53.313 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:53.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:53.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:53.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:53.466 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:53.540 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:53.543 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:53.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:53.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:43:53.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:53.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:43:53.822 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:53.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:53.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:53.841 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:43:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:53.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:53.957 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:54.156 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:54.159 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:54.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:54.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:54.289 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:54.844 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:43:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:43:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:53.674276+0000 mgr.y (mgr.14535) 570 : cluster [DBG] pgmap v737: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 511 B/s rd, 0 op/s 2024-02-22T19:43:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:54.044364+0000 mon.a (mon.0) 1474 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:43:55.296 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:53.674276+0000 mgr.y (mgr.14535) 570 : cluster [DBG] pgmap v737: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 511 B/s rd, 0 op/s 2024-02-22T19:43:55.296 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:54.044364+0000 mon.a (mon.0) 1474 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:43:55.296 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:53.674276+0000 mgr.y (mgr.14535) 570 : cluster [DBG] pgmap v737: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 511 B/s rd, 0 op/s 2024-02-22T19:43:55.296 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:54.044364+0000 mon.a (mon.0) 1474 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:43:55.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:43:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:43:55.638 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:55.720 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:55.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:55.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:55.758 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:43:55.758 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:43:55.758 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:43:55.759 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:43:55.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:55.864 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:55.886 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:55.886 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:55.987 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:43:56.027 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:56.232 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:43:56.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:56.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:43:56.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:55.140363+0000 mgr.y (mgr.14535) 571 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:55.140363+0000 mgr.y (mgr.14535) 571 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:56.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:43:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:55.140363+0000 mgr.y (mgr.14535) 571 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:43:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:56.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:43:56.993 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:43:57.094 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:43:57.144 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_3' created 2024-02-22T19:43:57.144 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:43:57.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:55.674871+0000 mgr.y (mgr.14535) 572 : cluster [DBG] pgmap v738: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:57.229 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:55.674871+0000 mgr.y (mgr.14535) 572 : cluster [DBG] pgmap v738: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:57.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:55.674871+0000 mgr.y (mgr.14535) 572 : cluster [DBG] pgmap v738: 132 pgs: 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:43:57.429 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:43:57.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:57.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:43:57.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:43:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:57.764 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:43:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:57.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:43:57.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-02-22T19:43:57.868 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:43:57.931 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:56.646503+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? 172.21.15.53:0/934746207' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-02-22T19:43:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:57.065586+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.53:0/1603783287' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:43:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:57.082346+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.132126+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:57.136700+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? 172.21.15.53:0/934746207' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:43:57.136775+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.136841+0000 mon.a (mon.0) 1480 : cluster [DBG] osdmap e365: 8 total, 8 up, 5 in 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.339442+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e366: 8 total, 8 up, 5 in 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.404159+0000 osd.7 (osd.7) 220 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.440213+0000 osd.7 (osd.7) 221 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:56.646503+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? 172.21.15.53:0/934746207' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:57.065586+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.53:0/1603783287' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:57.082346+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.132126+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:57.136700+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? 172.21.15.53:0/934746207' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-02-22T19:43:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:43:57.136775+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:43:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.136841+0000 mon.a (mon.0) 1480 : cluster [DBG] osdmap e365: 8 total, 8 up, 5 in 2024-02-22T19:43:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.339442+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e366: 8 total, 8 up, 5 in 2024-02-22T19:43:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.404159+0000 osd.7 (osd.7) 220 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:43:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.440213+0000 osd.7 (osd.7) 221 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:56.646503+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? 172.21.15.53:0/934746207' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:57.065586+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.53:0/1603783287' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:57.082346+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.132126+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:57.136700+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? 172.21.15.53:0/934746207' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:43:57.136775+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:43:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.136841+0000 mon.a (mon.0) 1480 : cluster [DBG] osdmap e365: 8 total, 8 up, 5 in 2024-02-22T19:43:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.339442+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e366: 8 total, 8 up, 5 in 2024-02-22T19:43:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.404159+0000 osd.7 (osd.7) 220 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:43:58.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.440213+0000 osd.7 (osd.7) 221 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:43:58.888 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:43:58.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:58.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:43:58.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:59.001 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:43:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:43:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:43:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:43:59.002 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:43:59.021 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:59.042 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:59.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:59.045 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:59.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:59.173 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:59.190 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.675285+0000 mgr.y (mgr.14535) 573 : cluster [DBG] pgmap v741: 148 pgs: 16 unknown, 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 698 B/s rd, 0 op/s 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.719334+0000 osd.5 (osd.5) 114 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.722223+0000 osd.5 (osd.5) 115 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:57.997222+0000 osd.3 (osd.3) 229 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:58.000154+0000 osd.3 (osd.3) 230 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:58.003318+0000 osd.2 (osd.2) 192 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:58.006214+0000 osd.2 (osd.2) 193 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:58.340772+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e367: 8 total, 8 up, 5 in 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.675285+0000 mgr.y (mgr.14535) 573 : cluster [DBG] pgmap v741: 148 pgs: 16 unknown, 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 698 B/s rd, 0 op/s 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.719334+0000 osd.5 (osd.5) 114 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:43:59.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.722223+0000 osd.5 (osd.5) 115 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:43:59.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:57.997222+0000 osd.3 (osd.3) 229 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:43:59.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:58.000154+0000 osd.3 (osd.3) 230 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:43:59.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:58.003318+0000 osd.2 (osd.2) 192 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:43:59.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:58.006214+0000 osd.2 (osd.2) 193 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:43:59.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:58.340772+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e367: 8 total, 8 up, 5 in 2024-02-22T19:43:59.226 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:59.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:59.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:43:59.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.675285+0000 mgr.y (mgr.14535) 573 : cluster [DBG] pgmap v741: 148 pgs: 16 unknown, 132 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 698 B/s rd, 0 op/s 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.719334+0000 osd.5 (osd.5) 114 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.722223+0000 osd.5 (osd.5) 115 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:57.997222+0000 osd.3 (osd.3) 229 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:58.000154+0000 osd.3 (osd.3) 230 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:58.003318+0000 osd.2 (osd.2) 192 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:58.006214+0000 osd.2 (osd.2) 193 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:43:59.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:43:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:58.340772+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e367: 8 total, 8 up, 5 in 2024-02-22T19:43:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:59.460 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:43:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:43:59] "GET /metrics HTTP/1.1" 200 173832 "" "Prometheus/2.33.4" 2024-02-22T19:43:59.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:43:59.807 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:43:59.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:59.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:43:59.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:43:59.939 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:43:59.958 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:43:59.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:43:59.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:00.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:00.115 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:00.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:00.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:00.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:00.344 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:00.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:58.776639+0000 osd.5 (osd.5) 116 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:44:00.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:58.780096+0000 osd.5 (osd.5) 117 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:44:00.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:59.016923+0000 osd.3 (osd.3) 231 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:44:00.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:59.062907+0000 osd.3 (osd.3) 232 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:44:00.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:59.331426+0000 osd.7 (osd.7) 222 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:44:00.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:59.334416+0000 osd.7 (osd.7) 223 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:44:00.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:00.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:00.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:00.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:00.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:00.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:00.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:00.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:00.417 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:00.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:44:00.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:58.776639+0000 osd.5 (osd.5) 116 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:44:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:58.780096+0000 osd.5 (osd.5) 117 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:44:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:59.016923+0000 osd.3 (osd.3) 231 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:44:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:59.062907+0000 osd.3 (osd.3) 232 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:44:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:59.331426+0000 osd.7 (osd.7) 222 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:44:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:59.334416+0000 osd.7 (osd.7) 223 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:44:00.459 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:58.776639+0000 osd.5 (osd.5) 116 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:44:00.459 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:58.780096+0000 osd.5 (osd.5) 117 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:44:00.459 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:59.016923+0000 osd.3 (osd.3) 231 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:44:00.459 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:59.062907+0000 osd.3 (osd.3) 232 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:44:00.459 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:59.331426+0000 osd.7 (osd.7) 222 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:44:00.459 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:59.334416+0000 osd.7 (osd.7) 223 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:44:00.478 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:00.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:00.581 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:00.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:00.740 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:00.741 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:00.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:00.872 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:00.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:44:00.892 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:00.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:00.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:00.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:01.023 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:01.078 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:01.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:01.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:01.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:01.211 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:01.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:43:59.676099+0000 mgr.y (mgr.14535) 574 : cluster [DBG] pgmap v743: 148 pgs: 4 unknown, 144 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:44:01.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:00.296529+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? 172.21.15.53:0/3881603975' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:01.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:43:59.676099+0000 mgr.y (mgr.14535) 574 : cluster [DBG] pgmap v743: 148 pgs: 4 unknown, 144 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:44:01.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:00.296529+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? 172.21.15.53:0/3881603975' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:01.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:43:59.676099+0000 mgr.y (mgr.14535) 574 : cluster [DBG] pgmap v743: 148 pgs: 4 unknown, 144 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:44:01.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:00.296529+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? 172.21.15.53:0/3881603975' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:01.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:01.393 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:01.494 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:01.524 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:01.627 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:01.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:01.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:01.759 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:01.830 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:01.830 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:01.833 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:01.836 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:01.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:01.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:01.967 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:02.064 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:02.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:02.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:02.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:02.197 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:44:02.282 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:44:02.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:44:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:02.750 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:02.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:02.771 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:02.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:03.024 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:03.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:03.125 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:03.151 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:01.661422+0000 osd.3 (osd.3) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:03.152 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:01.661498+0000 osd.3 (osd.3) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:03.152 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:01.676508+0000 mgr.y (mgr.14535) 575 : cluster [DBG] pgmap v744: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:44:03.152 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:02.097659+0000 mon.a (mon.0) 1483 : cluster [DBG] osdmap e368: 8 total, 8 up, 5 in 2024-02-22T19:44:03.152 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:02.264243+0000 mgr.y (mgr.14535) 576 : audit [DBG] from='client.25433 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:03.257 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:03.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:03.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:01.661422+0000 osd.3 (osd.3) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:01.661498+0000 osd.3 (osd.3) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:01.676508+0000 mgr.y (mgr.14535) 575 : cluster [DBG] pgmap v744: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:02.097659+0000 mon.a (mon.0) 1483 : cluster [DBG] osdmap e368: 8 total, 8 up, 5 in 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:02.264243+0000 mgr.y (mgr.14535) 576 : audit [DBG] from='client.25433 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:01.661422+0000 osd.3 (osd.3) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:01.661498+0000 osd.3 (osd.3) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:01.676508+0000 mgr.y (mgr.14535) 575 : cluster [DBG] pgmap v744: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:02.097659+0000 mon.a (mon.0) 1483 : cluster [DBG] osdmap e368: 8 total, 8 up, 5 in 2024-02-22T19:44:03.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:02.264243+0000 mgr.y (mgr.14535) 576 : audit [DBG] from='client.25433 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:03.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:44:03.910 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":745,"stamp":"2024-02-22T19:44:02.095461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":2001,"num_read_kb":1753,"num_write":714,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":15766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":422,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":2979436,"kb_used_data":13252,"kb_used_omap":0,"kb_used_meta":2966144,"kb_avail":465746324,"statfs":{"total":479975178240,"available":476924235776,"internally_reserved":0,"allocated":13570048,"data_stored":7433318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3037331456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.961588"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":324,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344797+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:43:57.344797+0000","last_peered":"2024-02-22T19:43:57.344797+0000","last_clean":"2024-02-22T19:43:57.344797+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:43:57.344797+0000","last_undegraded":"2024-02-22T19:43:57.344797+0000","last_fullsized":"2024-02-22T19:43:57.344797+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":359,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.780165+0000","last_change":"2024-02-22T19:43:58.780165+0000","last_active":"2024-02-22T19:43:58.780165+0000","last_peered":"2024-02-22T19:43:58.780165+0000","last_clean":"2024-02-22T19:43:58.780165+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:43:58.780165+0000","last_undegraded":"2024-02-22T19:43:58.780165+0000","last_fullsized":"2024-02-22T19:43:58.780165+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.780100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.780100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":138,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339039+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:43:57.339039+0000","last_peered":"2024-02-22T19:43:57.339039+0000","last_clean":"2024-02-22T19:43:57.339039+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:43:57.339039+0000","last_undegraded":"2024-02-22T19:43:57.339039+0000","last_fullsized":"2024-02-22T19:43:57.339039+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":339,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346294+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:43:57.346294+0000","last_peered":"2024-02-22T19:43:57.346294+0000","last_clean":"2024-02-22T19:43:57.346294+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:43:57.346294+0000","last_undegraded":"2024-02-22T19:43:57.346294+0000","last_fullsized":"2024-02-22T19:43:57.346294+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":340,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346710+0000","last_change":"2024-02-22T19:41:52.112817+0000","last_active":"2024-02-22T19:43:57.346710+0000","last_peered":"2024-02-22T19:43:57.346710+0000","last_clean":"2024-02-22T19:43:57.346710+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:43:57.346710+0000","last_undegraded":"2024-02-22T19:43:57.346710+0000","last_fullsized":"2024-02-22T19:43:57.346710+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":447,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344385+0000","last_change":"2024-02-22T19:39:41.797009+0000","last_active":"2024-02-22T19:43:57.344385+0000","last_peered":"2024-02-22T19:43:57.344385+0000","last_clean":"2024-02-22T19:43:57.344385+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:43:57.344385+0000","last_undegraded":"2024-02-22T19:43:57.344385+0000","last_fullsized":"2024-02-22T19:43:57.344385+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:39:41.796940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.796940+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":168,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338220+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:43:57.338220+0000","last_peered":"2024-02-22T19:43:57.338220+0000","last_clean":"2024-02-22T19:43:57.338220+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:43:57.338220+0000","last_undegraded":"2024-02-22T19:43:57.338220+0000","last_fullsized":"2024-02-22T19:43:57.338220+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":177,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344127+0000","last_change":"2024-02-22T19:41:52.113562+0000","last_active":"2024-02-22T19:43:57.344127+0000","last_peered":"2024-02-22T19:43:57.344127+0000","last_clean":"2024-02-22T19:43:57.344127+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:43:57.344127+0000","last_undegraded":"2024-02-22T19:43:57.344127+0000","last_fullsized":"2024-02-22T19:43:57.344127+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:36:58.219819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:36:58.219819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":244,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346037+0000","last_change":"2024-02-22T19:41:52.752324+0000","last_active":"2024-02-22T19:43:57.346037+0000","last_peered":"2024-02-22T19:43:57.346037+0000","last_clean":"2024-02-22T19:43:57.346037+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:43:57.346037+0000","last_undegraded":"2024-02-22T19:43:57.346037+0000","last_fullsized":"2024-02-22T19:43:57.346037+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":164,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343178+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:43:57.343178+0000","last_peered":"2024-02-22T19:43:57.343178+0000","last_clean":"2024-02-22T19:43:57.343178+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:43:57.343178+0000","last_undegraded":"2024-02-22T19:43:57.343178+0000","last_fullsized":"2024-02-22T19:43:57.343178+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":194,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343643+0000","last_change":"2024-02-22T19:41:52.116983+0000","last_active":"2024-02-22T19:43:57.343643+0000","last_peered":"2024-02-22T19:43:57.343643+0000","last_clean":"2024-02-22T19:43:57.343643+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:43:57.343643+0000","last_undegraded":"2024-02-22T19:43:57.343643+0000","last_fullsized":"2024-02-22T19:43:57.343643+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":167,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345879+0000","last_change":"2024-02-22T19:39:34.813311+0000","last_active":"2024-02-22T19:43:57.345879+0000","last_peered":"2024-02-22T19:43:57.345879+0000","last_clean":"2024-02-22T19:43:57.345879+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:43:57.345879+0000","last_undegraded":"2024-02-22T19:43:57.345879+0000","last_fullsized":"2024-02-22T19:43:57.345879+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:39:34.813225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:34.813225+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":242,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345212+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:43:57.345212+0000","last_peered":"2024-02-22T19:43:57.345212+0000","last_clean":"2024-02-22T19:43:57.345212+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:43:57.345212+0000","last_undegraded":"2024-02-22T19:43:57.345212+0000","last_fullsized":"2024-02-22T19:43:57.345212+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":141,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348414+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:43:57.348414+0000","last_peered":"2024-02-22T19:43:57.348414+0000","last_clean":"2024-02-22T19:43:57.348414+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:43:57.348414+0000","last_undegraded":"2024-02-22T19:43:57.348414+0000","last_fullsized":"2024-02-22T19:43:57.348414+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":394,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339983+0000","last_change":"2024-02-22T19:40:54.192769+0000","last_active":"2024-02-22T19:43:57.339983+0000","last_peered":"2024-02-22T19:43:57.339983+0000","last_clean":"2024-02-22T19:43:57.339983+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:43:57.339983+0000","last_undegraded":"2024-02-22T19:43:57.339983+0000","last_fullsized":"2024-02-22T19:43:57.339983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:40:54.192643+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.192643+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":357,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347879+0000","last_change":"2024-02-22T19:41:52.121546+0000","last_active":"2024-02-22T19:43:57.347879+0000","last_peered":"2024-02-22T19:43:57.347879+0000","last_clean":"2024-02-22T19:43:57.347879+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:43:57.347879+0000","last_undegraded":"2024-02-22T19:43:57.347879+0000","last_fullsized":"2024-02-22T19:43:57.347879+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":154,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339214+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:43:57.339214+0000","last_peered":"2024-02-22T19:43:57.339214+0000","last_clean":"2024-02-22T19:43:57.339214+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:43:57.339214+0000","last_undegraded":"2024-02-22T19:43:57.339214+0000","last_fullsized":"2024-02-22T19:43:57.339214+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":408,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347026+0000","last_change":"2024-02-22T19:41:53.187026+0000","last_active":"2024-02-22T19:43:57.347026+0000","last_peered":"2024-02-22T19:43:57.347026+0000","last_clean":"2024-02-22T19:43:57.347026+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:43:57.347026+0000","last_undegraded":"2024-02-22T19:43:57.347026+0000","last_fullsized":"2024-02-22T19:43:57.347026+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":373,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348580+0000","last_change":"2024-02-22T19:39:33.737170+0000","last_active":"2024-02-22T19:43:57.348580+0000","last_peered":"2024-02-22T19:43:57.348580+0000","last_clean":"2024-02-22T19:43:57.348580+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:43:57.348580+0000","last_undegraded":"2024-02-22T19:43:57.348580+0000","last_fullsized":"2024-02-22T19:43:57.348580+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:39:33.737067+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:39:33.737067+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":362,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338527+0000","last_change":"2024-02-22T19:41:52.110049+0000","last_active":"2024-02-22T19:43:57.338527+0000","last_peered":"2024-02-22T19:43:57.338527+0000","last_clean":"2024-02-22T19:43:57.338527+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:43:57.338527+0000","last_undegraded":"2024-02-22T19:43:57.338527+0000","last_fullsized":"2024-02-22T19:43:57.338527+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":174,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347940+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:43:57.347940+0000","last_peered":"2024-02-22T19:43:57.347940+0000","last_clean":"2024-02-22T19:43:57.347940+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:43:57.347940+0000","last_undegraded":"2024-02-22T19:43:57.347940+0000","last_fullsized":"2024-02-22T19:43:57.347940+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":362,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339568+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:43:57.339568+0000","last_peered":"2024-02-22T19:43:57.339568+0000","last_clean":"2024-02-22T19:43:57.339568+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:43:57.339568+0000","last_undegraded":"2024-02-22T19:43:57.339568+0000","last_fullsized":"2024-02-22T19:43:57.339568+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":360,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.342937+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:43:57.342937+0000","last_peered":"2024-02-22T19:43:57.342937+0000","last_clean":"2024-02-22T19:43:57.342937+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:43:57.342937+0000","last_undegraded":"2024-02-22T19:43:57.342937+0000","last_fullsized":"2024-02-22T19:43:57.342937+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":402,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344835+0000","last_change":"2024-02-22T19:41:52.743206+0000","last_active":"2024-02-22T19:43:57.344835+0000","last_peered":"2024-02-22T19:43:57.344835+0000","last_clean":"2024-02-22T19:43:57.344835+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:43:57.344835+0000","last_undegraded":"2024-02-22T19:43:57.344835+0000","last_fullsized":"2024-02-22T19:43:57.344835+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":347,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:59.334492+0000","last_change":"2024-02-22T19:43:59.334492+0000","last_active":"2024-02-22T19:43:59.334492+0000","last_peered":"2024-02-22T19:43:59.334492+0000","last_clean":"2024-02-22T19:43:59.334492+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:43:59.334492+0000","last_undegraded":"2024-02-22T19:43:59.334492+0000","last_fullsized":"2024-02-22T19:43:59.334492+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.334420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.334420+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":307,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345034+0000","last_change":"2024-02-22T19:41:52.111808+0000","last_active":"2024-02-22T19:43:57.345034+0000","last_peered":"2024-02-22T19:43:57.345034+0000","last_clean":"2024-02-22T19:43:57.345034+0000","last_became_active":"2024-02-22T19:41:52.110758+0000","last_became_peered":"2024-02-22T19:41:52.110758+0000","last_unstale":"2024-02-22T19:43:57.345034+0000","last_undegraded":"2024-02-22T19:43:57.345034+0000","last_fullsized":"2024-02-22T19:43:57.345034+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":354,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343654+0000","last_change":"2024-02-22T19:41:52.114607+0000","last_active":"2024-02-22T19:43:57.343654+0000","last_peered":"2024-02-22T19:43:57.343654+0000","last_clean":"2024-02-22T19:43:57.343654+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:43:57.343654+0000","last_undegraded":"2024-02-22T19:43:57.343654+0000","last_fullsized":"2024-02-22T19:43:57.343654+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":361,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343047+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:43:57.343047+0000","last_peered":"2024-02-22T19:43:57.343047+0000","last_clean":"2024-02-22T19:43:57.343047+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:43:57.343047+0000","last_undegraded":"2024-02-22T19:43:57.343047+0000","last_fullsized":"2024-02-22T19:43:57.343047+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":368,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:59.063011+0000","last_change":"2024-02-22T19:43:59.063011+0000","last_active":"2024-02-22T19:43:59.063011+0000","last_peered":"2024-02-22T19:43:59.063011+0000","last_clean":"2024-02-22T19:43:59.063011+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:43:59.063011+0000","last_undegraded":"2024-02-22T19:43:59.063011+0000","last_fullsized":"2024-02-22T19:43:59.063011+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.062912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.062912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":374,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345134+0000","last_change":"2024-02-22T19:41:52.104771+0000","last_active":"2024-02-22T19:43:57.345134+0000","last_peered":"2024-02-22T19:43:57.345134+0000","last_clean":"2024-02-22T19:43:57.345134+0000","last_became_active":"2024-02-22T19:41:52.104234+0000","last_became_peered":"2024-02-22T19:41:52.104234+0000","last_unstale":"2024-02-22T19:43:57.345134+0000","last_undegraded":"2024-02-22T19:43:57.345134+0000","last_fullsized":"2024-02-22T19:43:57.345134+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.702590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.702590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":159,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345108+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:43:57.345108+0000","last_peered":"2024-02-22T19:43:57.345108+0000","last_clean":"2024-02-22T19:43:57.345108+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:43:57.345108+0000","last_undegraded":"2024-02-22T19:43:57.345108+0000","last_fullsized":"2024-02-22T19:43:57.345108+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":381,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344180+0000","last_change":"2024-02-22T19:41:53.508908+0000","last_active":"2024-02-22T19:43:57.344180+0000","last_peered":"2024-02-22T19:43:57.344180+0000","last_clean":"2024-02-22T19:43:57.344180+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:43:57.344180+0000","last_undegraded":"2024-02-22T19:43:57.344180+0000","last_fullsized":"2024-02-22T19:43:57.344180+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":345,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344166+0000","last_change":"2024-02-22T19:41:52.112028+0000","last_active":"2024-02-22T19:43:57.344166+0000","last_peered":"2024-02-22T19:43:57.344166+0000","last_clean":"2024-02-22T19:43:57.344166+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:43:57.344166+0000","last_undegraded":"2024-02-22T19:43:57.344166+0000","last_fullsized":"2024-02-22T19:43:57.344166+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":354,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346045+0000","last_change":"2024-02-22T19:39:31.771315+0000","last_active":"2024-02-22T19:43:57.346045+0000","last_peered":"2024-02-22T19:43:57.346045+0000","last_clean":"2024-02-22T19:43:57.346045+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:43:57.346045+0000","last_undegraded":"2024-02-22T19:43:57.346045+0000","last_fullsized":"2024-02-22T19:43:57.346045+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:31.771244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:31.771244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":382,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348649+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:43:57.348649+0000","last_peered":"2024-02-22T19:43:57.348649+0000","last_clean":"2024-02-22T19:43:57.348649+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:43:57.348649+0000","last_undegraded":"2024-02-22T19:43:57.348649+0000","last_fullsized":"2024-02-22T19:43:57.348649+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":386,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339752+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:43:57.339752+0000","last_peered":"2024-02-22T19:43:57.339752+0000","last_clean":"2024-02-22T19:43:57.339752+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:43:57.339752+0000","last_undegraded":"2024-02-22T19:43:57.339752+0000","last_fullsized":"2024-02-22T19:43:57.339752+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":375,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343762+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:43:57.343762+0000","last_peered":"2024-02-22T19:43:57.343762+0000","last_clean":"2024-02-22T19:43:57.343762+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:43:57.343762+0000","last_undegraded":"2024-02-22T19:43:57.343762+0000","last_fullsized":"2024-02-22T19:43:57.343762+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":353,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339569+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:43:57.339569+0000","last_peered":"2024-02-22T19:43:57.339569+0000","last_clean":"2024-02-22T19:43:57.339569+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:43:57.339569+0000","last_undegraded":"2024-02-22T19:43:57.339569+0000","last_fullsized":"2024-02-22T19:43:57.339569+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":145,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348346+0000","last_change":"2024-02-22T19:41:52.104077+0000","last_active":"2024-02-22T19:43:57.348346+0000","last_peered":"2024-02-22T19:43:57.348346+0000","last_clean":"2024-02-22T19:43:57.348346+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:43:57.348346+0000","last_undegraded":"2024-02-22T19:43:57.348346+0000","last_fullsized":"2024-02-22T19:43:57.348346+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":149,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340451+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:43:57.340451+0000","last_peered":"2024-02-22T19:43:57.340451+0000","last_clean":"2024-02-22T19:43:57.340451+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:43:57.340451+0000","last_undegraded":"2024-02-22T19:43:57.340451+0000","last_fullsized":"2024-02-22T19:43:57.340451+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":162,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347736+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:43:57.347736+0000","last_peered":"2024-02-22T19:43:57.347736+0000","last_clean":"2024-02-22T19:43:57.347736+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:43:57.347736+0000","last_undegraded":"2024-02-22T19:43:57.347736+0000","last_fullsized":"2024-02-22T19:43:57.347736+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":278,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.779067+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:43:58.779067+0000","last_peered":"2024-02-22T19:43:58.779067+0000","last_clean":"2024-02-22T19:43:58.779067+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:43:58.779067+0000","last_undegraded":"2024-02-22T19:43:58.779067+0000","last_fullsized":"2024-02-22T19:43:58.779067+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":355,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348278+0000","last_change":"2024-02-22T19:41:52.103816+0000","last_active":"2024-02-22T19:43:57.348278+0000","last_peered":"2024-02-22T19:43:57.348278+0000","last_clean":"2024-02-22T19:43:57.348278+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:43:57.348278+0000","last_undegraded":"2024-02-22T19:43:57.348278+0000","last_fullsized":"2024-02-22T19:43:57.348278+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":369,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346831+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:43:57.346831+0000","last_peered":"2024-02-22T19:43:57.346831+0000","last_clean":"2024-02-22T19:43:57.346831+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:43:57.346831+0000","last_undegraded":"2024-02-22T19:43:57.346831+0000","last_fullsized":"2024-02-22T19:43:57.346831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":347,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338854+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:43:57.338854+0000","last_peered":"2024-02-22T19:43:57.338854+0000","last_clean":"2024-02-22T19:43:57.338854+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:43:57.338854+0000","last_undegraded":"2024-02-22T19:43:57.338854+0000","last_fullsized":"2024-02-22T19:43:57.338854+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":515,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.751220+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:43:58.751220+0000","last_peered":"2024-02-22T19:43:58.751220+0000","last_clean":"2024-02-22T19:43:58.751220+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:43:58.751220+0000","last_undegraded":"2024-02-22T19:43:58.751220+0000","last_fullsized":"2024-02-22T19:43:58.751220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":153,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347107+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:43:57.347107+0000","last_peered":"2024-02-22T19:43:57.347107+0000","last_clean":"2024-02-22T19:43:57.347107+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:43:57.347107+0000","last_undegraded":"2024-02-22T19:43:57.347107+0000","last_fullsized":"2024-02-22T19:43:57.347107+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":183,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344299+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:43:57.344299+0000","last_peered":"2024-02-22T19:43:57.344299+0000","last_clean":"2024-02-22T19:43:57.344299+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:43:57.344299+0000","last_undegraded":"2024-02-22T19:43:57.344299+0000","last_fullsized":"2024-02-22T19:43:57.344299+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":370,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340291+0000","last_change":"2024-02-22T19:41:38.526359+0000","last_active":"2024-02-22T19:43:57.340291+0000","last_peered":"2024-02-22T19:43:57.340291+0000","last_clean":"2024-02-22T19:43:57.340291+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:57.340291+0000","last_undegraded":"2024-02-22T19:43:57.340291+0000","last_fullsized":"2024-02-22T19:43:57.340291+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.526259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.526259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":378,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345226+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:43:57.345226+0000","last_peered":"2024-02-22T19:43:57.345226+0000","last_clean":"2024-02-22T19:43:57.345226+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:43:57.345226+0000","last_undegraded":"2024-02-22T19:43:57.345226+0000","last_fullsized":"2024-02-22T19:43:57.345226+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":155,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346859+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:43:57.346859+0000","last_peered":"2024-02-22T19:43:57.346859+0000","last_clean":"2024-02-22T19:43:57.346859+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:43:57.346859+0000","last_undegraded":"2024-02-22T19:43:57.346859+0000","last_fullsized":"2024-02-22T19:43:57.346859+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":380,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.440328+0000","last_change":"2024-02-22T19:43:57.440300+0000","last_active":"2024-02-22T19:43:57.440328+0000","last_peered":"2024-02-22T19:43:57.440328+0000","last_clean":"2024-02-22T19:43:57.440328+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:43:57.440328+0000","last_undegraded":"2024-02-22T19:43:57.440328+0000","last_fullsized":"2024-02-22T19:43:57.440328+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.440217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.440217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":331,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344041+0000","last_change":"2024-02-22T19:41:52.112840+0000","last_active":"2024-02-22T19:43:57.344041+0000","last_peered":"2024-02-22T19:43:57.344041+0000","last_clean":"2024-02-22T19:43:57.344041+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:43:57.344041+0000","last_undegraded":"2024-02-22T19:43:57.344041+0000","last_fullsized":"2024-02-22T19:43:57.344041+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":369,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347969+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:43:57.347969+0000","last_peered":"2024-02-22T19:43:57.347969+0000","last_clean":"2024-02-22T19:43:57.347969+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:43:57.347969+0000","last_undegraded":"2024-02-22T19:43:57.347969+0000","last_fullsized":"2024-02-22T19:43:57.347969+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.349741+0000","last_change":"2024-02-22T19:43:57.347864+0000","last_active":"2024-02-22T19:43:57.349741+0000","last_peered":"2024-02-22T19:43:57.349741+0000","last_clean":"2024-02-22T19:43:57.349741+0000","last_became_active":"2024-02-22T19:43:57.347537+0000","last_became_peered":"2024-02-22T19:43:57.347537+0000","last_unstale":"2024-02-22T19:43:57.349741+0000","last_undegraded":"2024-02-22T19:43:57.349741+0000","last_fullsized":"2024-02-22T19:43:57.349741+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":200,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344604+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:43:57.344604+0000","last_peered":"2024-02-22T19:43:57.344604+0000","last_clean":"2024-02-22T19:43:57.344604+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:43:57.344604+0000","last_undegraded":"2024-02-22T19:43:57.344604+0000","last_fullsized":"2024-02-22T19:43:57.344604+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":427,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348349+0000","last_change":"2024-02-22T19:43:26.953378+0000","last_active":"2024-02-22T19:43:57.348349+0000","last_peered":"2024-02-22T19:43:57.348349+0000","last_clean":"2024-02-22T19:43:57.348349+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:43:57.348349+0000","last_undegraded":"2024-02-22T19:43:57.348349+0000","last_fullsized":"2024-02-22T19:43:57.348349+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:39:41.753990+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:39:41.753990+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":223,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343957+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:43:57.343957+0000","last_peered":"2024-02-22T19:43:57.343957+0000","last_clean":"2024-02-22T19:43:57.343957+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:43:57.343957+0000","last_undegraded":"2024-02-22T19:43:57.343957+0000","last_fullsized":"2024-02-22T19:43:57.343957+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":597,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.753906+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:43:58.753906+0000","last_peered":"2024-02-22T19:43:58.753906+0000","last_clean":"2024-02-22T19:43:58.753906+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:43:58.753906+0000","last_undegraded":"2024-02-22T19:43:58.753906+0000","last_fullsized":"2024-02-22T19:43:58.753906+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.351188+0000","last_change":"2024-02-22T19:43:57.350818+0000","last_active":"2024-02-22T19:43:57.351188+0000","last_peered":"2024-02-22T19:43:57.351188+0000","last_clean":"2024-02-22T19:43:57.351188+0000","last_became_active":"2024-02-22T19:43:57.350467+0000","last_became_peered":"2024-02-22T19:43:57.350467+0000","last_unstale":"2024-02-22T19:43:57.351188+0000","last_undegraded":"2024-02-22T19:43:57.351188+0000","last_fullsized":"2024-02-22T19:43:57.351188+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":325,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346387+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:43:57.346387+0000","last_peered":"2024-02-22T19:43:57.346387+0000","last_clean":"2024-02-22T19:43:57.346387+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:43:57.346387+0000","last_undegraded":"2024-02-22T19:43:57.346387+0000","last_fullsized":"2024-02-22T19:43:57.346387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":69,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.341258+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:43:57.341258+0000","last_peered":"2024-02-22T19:43:57.341258+0000","last_clean":"2024-02-22T19:43:57.341258+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:43:57.341258+0000","last_undegraded":"2024-02-22T19:43:57.341258+0000","last_fullsized":"2024-02-22T19:43:57.341258+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":213,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338315+0000","last_change":"2024-02-22T19:41:52.597069+0000","last_active":"2024-02-22T19:43:57.338315+0000","last_peered":"2024-02-22T19:43:57.338315+0000","last_clean":"2024-02-22T19:43:57.338315+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:43:57.338315+0000","last_undegraded":"2024-02-22T19:43:57.338315+0000","last_fullsized":"2024-02-22T19:43:57.338315+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":385,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.342183+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:43:57.342183+0000","last_peered":"2024-02-22T19:43:57.342183+0000","last_clean":"2024-02-22T19:43:57.342183+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:43:57.342183+0000","last_undegraded":"2024-02-22T19:43:57.342183+0000","last_fullsized":"2024-02-22T19:43:57.342183+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.350943+0000","last_change":"2024-02-22T19:43:57.349932+0000","last_active":"2024-02-22T19:43:57.350943+0000","last_peered":"2024-02-22T19:43:57.350943+0000","last_clean":"2024-02-22T19:43:57.350943+0000","last_became_active":"2024-02-22T19:43:57.349636+0000","last_became_peered":"2024-02-22T19:43:57.349636+0000","last_unstale":"2024-02-22T19:43:57.350943+0000","last_undegraded":"2024-02-22T19:43:57.350943+0000","last_fullsized":"2024-02-22T19:43:57.350943+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":331,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344817+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:43:57.344817+0000","last_peered":"2024-02-22T19:43:57.344817+0000","last_clean":"2024-02-22T19:43:57.344817+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:43:57.344817+0000","last_undegraded":"2024-02-22T19:43:57.344817+0000","last_fullsized":"2024-02-22T19:43:57.344817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":411,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340716+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:43:57.340716+0000","last_peered":"2024-02-22T19:43:57.340716+0000","last_clean":"2024-02-22T19:43:57.340716+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:43:57.340716+0000","last_undegraded":"2024-02-22T19:43:57.340716+0000","last_fullsized":"2024-02-22T19:43:57.340716+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":369,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348782+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:43:57.348782+0000","last_peered":"2024-02-22T19:43:57.348782+0000","last_clean":"2024-02-22T19:43:57.348782+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:43:57.348782+0000","last_undegraded":"2024-02-22T19:43:57.348782+0000","last_fullsized":"2024-02-22T19:43:57.348782+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":360,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339485+0000","last_change":"2024-02-22T19:41:52.135450+0000","last_active":"2024-02-22T19:43:57.339485+0000","last_peered":"2024-02-22T19:43:57.339485+0000","last_clean":"2024-02-22T19:43:57.339485+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:43:57.339485+0000","last_undegraded":"2024-02-22T19:43:57.339485+0000","last_fullsized":"2024-02-22T19:43:57.339485+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.349015+0000","last_change":"2024-02-22T19:43:57.347961+0000","last_active":"2024-02-22T19:43:57.349015+0000","last_peered":"2024-02-22T19:43:57.349015+0000","last_clean":"2024-02-22T19:43:57.349015+0000","last_became_active":"2024-02-22T19:43:57.347657+0000","last_became_peered":"2024-02-22T19:43:57.347657+0000","last_unstale":"2024-02-22T19:43:57.349015+0000","last_undegraded":"2024-02-22T19:43:57.349015+0000","last_fullsized":"2024-02-22T19:43:57.349015+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":153,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345378+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:43:57.345378+0000","last_peered":"2024-02-22T19:43:57.345378+0000","last_clean":"2024-02-22T19:43:57.345378+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:43:57.345378+0000","last_undegraded":"2024-02-22T19:43:57.345378+0000","last_fullsized":"2024-02-22T19:43:57.345378+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":390,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340889+0000","last_change":"2024-02-22T19:41:16.236762+0000","last_active":"2024-02-22T19:43:57.340889+0000","last_peered":"2024-02-22T19:43:57.340889+0000","last_clean":"2024-02-22T19:43:57.340889+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:43:57.340889+0000","last_undegraded":"2024-02-22T19:43:57.340889+0000","last_fullsized":"2024-02-22T19:43:57.340889+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.236664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.236664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":173,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343540+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:43:57.343540+0000","last_peered":"2024-02-22T19:43:57.343540+0000","last_clean":"2024-02-22T19:43:57.343540+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:43:57.343540+0000","last_undegraded":"2024-02-22T19:43:57.343540+0000","last_fullsized":"2024-02-22T19:43:57.343540+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":165,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340923+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:43:57.340923+0000","last_peered":"2024-02-22T19:43:57.340923+0000","last_clean":"2024-02-22T19:43:57.340923+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:43:57.340923+0000","last_undegraded":"2024-02-22T19:43:57.340923+0000","last_fullsized":"2024-02-22T19:43:57.340923+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.352236+0000","last_change":"2024-02-22T19:43:57.351947+0000","last_active":"2024-02-22T19:43:57.352236+0000","last_peered":"2024-02-22T19:43:57.352236+0000","last_clean":"2024-02-22T19:43:57.352236+0000","last_became_active":"2024-02-22T19:43:57.351647+0000","last_became_peered":"2024-02-22T19:43:57.351647+0000","last_unstale":"2024-02-22T19:43:57.352236+0000","last_undegraded":"2024-02-22T19:43:57.352236+0000","last_fullsized":"2024-02-22T19:43:57.352236+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":200,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343308+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:43:57.343308+0000","last_peered":"2024-02-22T19:43:57.343308+0000","last_clean":"2024-02-22T19:43:57.343308+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:43:57.343308+0000","last_undegraded":"2024-02-22T19:43:57.343308+0000","last_fullsized":"2024-02-22T19:43:57.343308+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":348,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346944+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:43:57.346944+0000","last_peered":"2024-02-22T19:43:57.346944+0000","last_clean":"2024-02-22T19:43:57.346944+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:43:57.346944+0000","last_undegraded":"2024-02-22T19:43:57.346944+0000","last_fullsized":"2024-02-22T19:43:57.346944+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.722311+0000","last_change":"2024-02-22T19:43:57.722289+0000","last_active":"2024-02-22T19:43:57.722311+0000","last_peered":"2024-02-22T19:43:57.722311+0000","last_clean":"2024-02-22T19:43:57.722311+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:43:57.722311+0000","last_undegraded":"2024-02-22T19:43:57.722311+0000","last_fullsized":"2024-02-22T19:43:57.722311+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.722226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.722226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":283,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340014+0000","last_change":"2024-02-22T19:41:52.113125+0000","last_active":"2024-02-22T19:43:57.340014+0000","last_peered":"2024-02-22T19:43:57.340014+0000","last_clean":"2024-02-22T19:43:57.340014+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:43:57.340014+0000","last_undegraded":"2024-02-22T19:43:57.340014+0000","last_fullsized":"2024-02-22T19:43:57.340014+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":596,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.342154+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:43:58.342154+0000","last_peered":"2024-02-22T19:43:58.342154+0000","last_clean":"2024-02-22T19:43:58.342154+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:43:58.342154+0000","last_undegraded":"2024-02-22T19:43:58.342154+0000","last_fullsized":"2024-02-22T19:43:58.342154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":171,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339479+0000","last_change":"2024-02-22T19:41:52.101375+0000","last_active":"2024-02-22T19:43:57.339479+0000","last_peered":"2024-02-22T19:43:57.339479+0000","last_clean":"2024-02-22T19:43:57.339479+0000","last_became_active":"2024-02-22T19:41:52.101126+0000","last_became_peered":"2024-02-22T19:41:52.101126+0000","last_unstale":"2024-02-22T19:43:57.339479+0000","last_undegraded":"2024-02-22T19:43:57.339479+0000","last_fullsized":"2024-02-22T19:43:57.339479+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347523+0000","last_change":"2024-02-22T19:43:57.347092+0000","last_active":"2024-02-22T19:43:57.347523+0000","last_peered":"2024-02-22T19:43:57.347523+0000","last_clean":"2024-02-22T19:43:57.347523+0000","last_became_active":"2024-02-22T19:43:57.346743+0000","last_became_peered":"2024-02-22T19:43:57.346743+0000","last_unstale":"2024-02-22T19:43:57.347523+0000","last_undegraded":"2024-02-22T19:43:57.347523+0000","last_fullsized":"2024-02-22T19:43:57.347523+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":390,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347278+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:43:57.347278+0000","last_peered":"2024-02-22T19:43:57.347278+0000","last_clean":"2024-02-22T19:43:57.347278+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:43:57.347278+0000","last_undegraded":"2024-02-22T19:43:57.347278+0000","last_fullsized":"2024-02-22T19:43:57.347278+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":340,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348945+0000","last_change":"2024-02-22T19:41:52.109613+0000","last_active":"2024-02-22T19:43:57.348945+0000","last_peered":"2024-02-22T19:43:57.348945+0000","last_clean":"2024-02-22T19:43:57.348945+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:43:57.348945+0000","last_undegraded":"2024-02-22T19:43:57.348945+0000","last_fullsized":"2024-02-22T19:43:57.348945+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":353,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338832+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:43:57.338832+0000","last_peered":"2024-02-22T19:43:57.338832+0000","last_clean":"2024-02-22T19:43:57.338832+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:43:57.338832+0000","last_undegraded":"2024-02-22T19:43:57.338832+0000","last_fullsized":"2024-02-22T19:43:57.338832+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339982+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:43:57.339982+0000","last_peered":"2024-02-22T19:43:57.339982+0000","last_clean":"2024-02-22T19:43:57.339982+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:43:57.339982+0000","last_undegraded":"2024-02-22T19:43:57.339982+0000","last_fullsized":"2024-02-22T19:43:57.339982+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":441,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338937+0000","last_change":"2024-02-22T19:41:20.253383+0000","last_active":"2024-02-22T19:43:57.338937+0000","last_peered":"2024-02-22T19:43:57.338937+0000","last_clean":"2024-02-22T19:43:57.338937+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:43:57.338937+0000","last_undegraded":"2024-02-22T19:43:57.338937+0000","last_fullsized":"2024-02-22T19:43:57.338937+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:41:20.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:20.253277+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":384,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.341588+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:43:57.341588+0000","last_peered":"2024-02-22T19:43:57.341588+0000","last_clean":"2024-02-22T19:43:57.341588+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:43:57.341588+0000","last_undegraded":"2024-02-22T19:43:57.341588+0000","last_fullsized":"2024-02-22T19:43:57.341588+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344888+0000","last_change":"2024-02-22T19:43:57.344606+0000","last_active":"2024-02-22T19:43:57.344888+0000","last_peered":"2024-02-22T19:43:57.344888+0000","last_clean":"2024-02-22T19:43:57.344888+0000","last_became_active":"2024-02-22T19:43:57.344303+0000","last_became_peered":"2024-02-22T19:43:57.344303+0000","last_unstale":"2024-02-22T19:43:57.344888+0000","last_undegraded":"2024-02-22T19:43:57.344888+0000","last_fullsized":"2024-02-22T19:43:57.344888+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":173,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345044+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:43:57.345044+0000","last_peered":"2024-02-22T19:43:57.345044+0000","last_clean":"2024-02-22T19:43:57.345044+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:43:57.345044+0000","last_undegraded":"2024-02-22T19:43:57.345044+0000","last_fullsized":"2024-02-22T19:43:57.345044+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":390,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344956+0000","last_change":"2024-02-22T19:41:52.104562+0000","last_active":"2024-02-22T19:43:57.344956+0000","last_peered":"2024-02-22T19:43:57.344956+0000","last_clean":"2024-02-22T19:43:57.344956+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:43:57.344956+0000","last_undegraded":"2024-02-22T19:43:57.344956+0000","last_fullsized":"2024-02-22T19:43:57.344956+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":358,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345863+0000","last_change":"2024-02-22T19:41:52.111795+0000","last_active":"2024-02-22T19:43:57.345863+0000","last_peered":"2024-02-22T19:43:57.345863+0000","last_clean":"2024-02-22T19:43:57.345863+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:43:57.345863+0000","last_undegraded":"2024-02-22T19:43:57.345863+0000","last_fullsized":"2024-02-22T19:43:57.345863+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.705738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.705738+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":398,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348709+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:43:57.348709+0000","last_peered":"2024-02-22T19:43:57.348709+0000","last_clean":"2024-02-22T19:43:57.348709+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:43:57.348709+0000","last_undegraded":"2024-02-22T19:43:57.348709+0000","last_fullsized":"2024-02-22T19:43:57.348709+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":351,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348668+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:43:57.348668+0000","last_peered":"2024-02-22T19:43:57.348668+0000","last_clean":"2024-02-22T19:43:57.348668+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:43:57.348668+0000","last_undegraded":"2024-02-22T19:43:57.348668+0000","last_fullsized":"2024-02-22T19:43:57.348668+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347324+0000","last_change":"2024-02-22T19:43:57.346858+0000","last_active":"2024-02-22T19:43:57.347324+0000","last_peered":"2024-02-22T19:43:57.347324+0000","last_clean":"2024-02-22T19:43:57.347324+0000","last_became_active":"2024-02-22T19:43:57.346554+0000","last_became_peered":"2024-02-22T19:43:57.346554+0000","last_unstale":"2024-02-22T19:43:57.347324+0000","last_undegraded":"2024-02-22T19:43:57.347324+0000","last_fullsized":"2024-02-22T19:43:57.347324+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":244,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340235+0000","last_change":"2024-02-22T19:41:52.114247+0000","last_active":"2024-02-22T19:43:57.340235+0000","last_peered":"2024-02-22T19:43:57.340235+0000","last_clean":"2024-02-22T19:43:57.340235+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:43:57.340235+0000","last_undegraded":"2024-02-22T19:43:57.340235+0000","last_fullsized":"2024-02-22T19:43:57.340235+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":356,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339058+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:43:57.339058+0000","last_peered":"2024-02-22T19:43:57.339058+0000","last_clean":"2024-02-22T19:43:57.339058+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:43:57.339058+0000","last_undegraded":"2024-02-22T19:43:57.339058+0000","last_fullsized":"2024-02-22T19:43:57.339058+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":141,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343888+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:43:57.343888+0000","last_peered":"2024-02-22T19:43:57.343888+0000","last_clean":"2024-02-22T19:43:57.343888+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:43:57.343888+0000","last_undegraded":"2024-02-22T19:43:57.343888+0000","last_fullsized":"2024-02-22T19:43:57.343888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1275,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.753668+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:43:58.753668+0000","last_peered":"2024-02-22T19:43:58.753668+0000","last_clean":"2024-02-22T19:43:58.753668+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:43:58.753668+0000","last_undegraded":"2024-02-22T19:43:58.753668+0000","last_fullsized":"2024-02-22T19:43:58.753668+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":881,"num_read_kb":876,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":173,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.341238+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:43:57.341238+0000","last_peered":"2024-02-22T19:43:57.341238+0000","last_clean":"2024-02-22T19:43:57.341238+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:43:57.341238+0000","last_undegraded":"2024-02-22T19:43:57.341238+0000","last_fullsized":"2024-02-22T19:43:57.341238+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":218,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345167+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:43:57.345167+0000","last_peered":"2024-02-22T19:43:57.345167+0000","last_clean":"2024-02-22T19:43:57.345167+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:43:57.345167+0000","last_undegraded":"2024-02-22T19:43:57.345167+0000","last_fullsized":"2024-02-22T19:43:57.345167+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348333+0000","last_change":"2024-02-22T19:43:57.347743+0000","last_active":"2024-02-22T19:43:57.348333+0000","last_peered":"2024-02-22T19:43:57.348333+0000","last_clean":"2024-02-22T19:43:57.348333+0000","last_became_active":"2024-02-22T19:43:57.347455+0000","last_became_peered":"2024-02-22T19:43:57.347455+0000","last_unstale":"2024-02-22T19:43:57.348333+0000","last_undegraded":"2024-02-22T19:43:57.348333+0000","last_fullsized":"2024-02-22T19:43:57.348333+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":220,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347306+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:43:57.347306+0000","last_peered":"2024-02-22T19:43:57.347306+0000","last_clean":"2024-02-22T19:43:57.347306+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:43:57.347306+0000","last_undegraded":"2024-02-22T19:43:57.347306+0000","last_fullsized":"2024-02-22T19:43:57.347306+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":342,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338402+0000","last_change":"2024-02-22T19:41:52.094945+0000","last_active":"2024-02-22T19:43:57.338402+0000","last_peered":"2024-02-22T19:43:57.338402+0000","last_clean":"2024-02-22T19:43:57.338402+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:43:57.338402+0000","last_undegraded":"2024-02-22T19:43:57.338402+0000","last_fullsized":"2024-02-22T19:43:57.338402+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":357,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339205+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:43:57.339205+0000","last_peered":"2024-02-22T19:43:57.339205+0000","last_clean":"2024-02-22T19:43:57.339205+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:43:57.339205+0000","last_undegraded":"2024-02-22T19:43:57.339205+0000","last_fullsized":"2024-02-22T19:43:57.339205+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":178,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343485+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:43:57.343485+0000","last_peered":"2024-02-22T19:43:57.343485+0000","last_clean":"2024-02-22T19:43:57.343485+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:43:57.343485+0000","last_undegraded":"2024-02-22T19:43:57.343485+0000","last_fullsized":"2024-02-22T19:43:57.343485+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":327,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344480+0000","last_change":"2024-02-22T19:41:52.107671+0000","last_active":"2024-02-22T19:43:57.344480+0000","last_peered":"2024-02-22T19:43:57.344480+0000","last_clean":"2024-02-22T19:43:57.344480+0000","last_became_active":"2024-02-22T19:41:52.106767+0000","last_became_peered":"2024-02-22T19:41:52.106767+0000","last_unstale":"2024-02-22T19:43:57.344480+0000","last_undegraded":"2024-02-22T19:43:57.344480+0000","last_fullsized":"2024-02-22T19:43:57.344480+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.352101+0000","last_change":"2024-02-22T19:43:57.351727+0000","last_active":"2024-02-22T19:43:57.352101+0000","last_peered":"2024-02-22T19:43:57.352101+0000","last_clean":"2024-02-22T19:43:57.352101+0000","last_became_active":"2024-02-22T19:43:57.351385+0000","last_became_peered":"2024-02-22T19:43:57.351385+0000","last_unstale":"2024-02-22T19:43:57.352101+0000","last_undegraded":"2024-02-22T19:43:57.352101+0000","last_fullsized":"2024-02-22T19:43:57.352101+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347428+0000","last_change":"2024-02-22T19:41:52.110027+0000","last_active":"2024-02-22T19:43:57.347428+0000","last_peered":"2024-02-22T19:43:57.347428+0000","last_clean":"2024-02-22T19:43:57.347428+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:43:57.347428+0000","last_undegraded":"2024-02-22T19:43:57.347428+0000","last_fullsized":"2024-02-22T19:43:57.347428+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.670303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":153,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.342862+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:43:57.342862+0000","last_peered":"2024-02-22T19:43:57.342862+0000","last_clean":"2024-02-22T19:43:57.342862+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:43:57.342862+0000","last_undegraded":"2024-02-22T19:43:57.342862+0000","last_fullsized":"2024-02-22T19:43:57.342862+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":164,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344206+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:43:57.344206+0000","last_peered":"2024-02-22T19:43:57.344206+0000","last_clean":"2024-02-22T19:43:57.344206+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:43:57.344206+0000","last_undegraded":"2024-02-22T19:43:57.344206+0000","last_fullsized":"2024-02-22T19:43:57.344206+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":477,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339661+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:43:57.339661+0000","last_peered":"2024-02-22T19:43:57.339661+0000","last_clean":"2024-02-22T19:43:57.339661+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:43:57.339661+0000","last_undegraded":"2024-02-22T19:43:57.339661+0000","last_fullsized":"2024-02-22T19:43:57.339661+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"326'14","reported_seq":393,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.777390+0000","last_change":"2024-02-22T19:41:52.698274+0000","last_active":"2024-02-22T19:43:58.777390+0000","last_peered":"2024-02-22T19:43:58.777390+0000","last_clean":"2024-02-22T19:43:58.777390+0000","last_became_active":"2024-02-22T19:41:52.112897+0000","last_became_peered":"2024-02-22T19:41:52.112897+0000","last_unstale":"2024-02-22T19:43:58.777390+0000","last_undegraded":"2024-02-22T19:43:58.777390+0000","last_fullsized":"2024-02-22T19:43:58.777390+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348788+0000","last_change":"2024-02-22T19:43:57.348552+0000","last_active":"2024-02-22T19:43:57.348788+0000","last_peered":"2024-02-22T19:43:57.348788+0000","last_clean":"2024-02-22T19:43:57.348788+0000","last_became_active":"2024-02-22T19:43:57.348228+0000","last_became_peered":"2024-02-22T19:43:57.348228+0000","last_unstale":"2024-02-22T19:43:57.348788+0000","last_undegraded":"2024-02-22T19:43:57.348788+0000","last_fullsized":"2024-02-22T19:43:57.348788+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":249,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.000311+0000","last_change":"2024-02-22T19:43:58.000275+0000","last_active":"2024-02-22T19:43:58.000311+0000","last_peered":"2024-02-22T19:43:58.000311+0000","last_clean":"2024-02-22T19:43:58.000311+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:43:58.000311+0000","last_undegraded":"2024-02-22T19:43:58.000311+0000","last_fullsized":"2024-02-22T19:43:58.000311+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.000163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.000163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":343,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348570+0000","last_change":"2024-02-22T19:41:52.103942+0000","last_active":"2024-02-22T19:43:57.348570+0000","last_peered":"2024-02-22T19:43:57.348570+0000","last_clean":"2024-02-22T19:43:57.348570+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:43:57.348570+0000","last_undegraded":"2024-02-22T19:43:57.348570+0000","last_fullsized":"2024-02-22T19:43:57.348570+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":162,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.341003+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:43:57.341003+0000","last_peered":"2024-02-22T19:43:57.341003+0000","last_clean":"2024-02-22T19:43:57.341003+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:43:57.341003+0000","last_undegraded":"2024-02-22T19:43:57.341003+0000","last_fullsized":"2024-02-22T19:43:57.341003+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":479,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343344+0000","last_change":"2024-02-22T19:41:52.715987+0000","last_active":"2024-02-22T19:43:57.343344+0000","last_peered":"2024-02-22T19:43:57.343344+0000","last_clean":"2024-02-22T19:43:57.343344+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:43:57.343344+0000","last_undegraded":"2024-02-22T19:43:57.343344+0000","last_fullsized":"2024-02-22T19:43:57.343344+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":556,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.750385+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:43:58.750385+0000","last_peered":"2024-02-22T19:43:58.750385+0000","last_clean":"2024-02-22T19:43:58.750385+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:43:58.750385+0000","last_undegraded":"2024-02-22T19:43:58.750385+0000","last_fullsized":"2024-02-22T19:43:58.750385+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.349094+0000","last_change":"2024-02-22T19:43:57.348310+0000","last_active":"2024-02-22T19:43:57.349094+0000","last_peered":"2024-02-22T19:43:57.349094+0000","last_clean":"2024-02-22T19:43:57.349094+0000","last_became_active":"2024-02-22T19:43:57.348032+0000","last_became_peered":"2024-02-22T19:43:57.348032+0000","last_unstale":"2024-02-22T19:43:57.349094+0000","last_undegraded":"2024-02-22T19:43:57.349094+0000","last_fullsized":"2024-02-22T19:43:57.349094+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":151,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.006346+0000","last_change":"2024-02-22T19:43:58.006321+0000","last_active":"2024-02-22T19:43:58.006346+0000","last_peered":"2024-02-22T19:43:58.006346+0000","last_clean":"2024-02-22T19:43:58.006346+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:43:58.006346+0000","last_undegraded":"2024-02-22T19:43:58.006346+0000","last_fullsized":"2024-02-22T19:43:58.006346+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.006221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.006221+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":145,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346180+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:43:57.346180+0000","last_peered":"2024-02-22T19:43:57.346180+0000","last_clean":"2024-02-22T19:43:57.346180+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:43:57.346180+0000","last_undegraded":"2024-02-22T19:43:57.346180+0000","last_fullsized":"2024-02-22T19:43:57.346180+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":363,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347617+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:43:57.347617+0000","last_peered":"2024-02-22T19:43:57.347617+0000","last_clean":"2024-02-22T19:43:57.347617+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:43:57.347617+0000","last_undegraded":"2024-02-22T19:43:57.347617+0000","last_fullsized":"2024-02-22T19:43:57.347617+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":467,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343302+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:43:57.343302+0000","last_peered":"2024-02-22T19:43:57.343302+0000","last_clean":"2024-02-22T19:43:57.343302+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:43:57.343302+0000","last_undegraded":"2024-02-22T19:43:57.343302+0000","last_fullsized":"2024-02-22T19:43:57.343302+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":154,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344093+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:43:57.344093+0000","last_peered":"2024-02-22T19:43:57.344093+0000","last_clean":"2024-02-22T19:43:57.344093+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:43:57.344093+0000","last_undegraded":"2024-02-22T19:43:57.344093+0000","last_fullsized":"2024-02-22T19:43:57.344093+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.348853+0000","last_change":"2024-02-22T19:43:57.347525+0000","last_active":"2024-02-22T19:43:57.348853+0000","last_peered":"2024-02-22T19:43:57.348853+0000","last_clean":"2024-02-22T19:43:57.348853+0000","last_became_active":"2024-02-22T19:43:57.347224+0000","last_became_peered":"2024-02-22T19:43:57.347224+0000","last_unstale":"2024-02-22T19:43:57.348853+0000","last_undegraded":"2024-02-22T19:43:57.348853+0000","last_fullsized":"2024-02-22T19:43:57.348853+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":176,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344709+0000","last_change":"2024-02-22T19:41:52.107922+0000","last_active":"2024-02-22T19:43:57.344709+0000","last_peered":"2024-02-22T19:43:57.344709+0000","last_clean":"2024-02-22T19:43:57.344709+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:43:57.344709+0000","last_undegraded":"2024-02-22T19:43:57.344709+0000","last_fullsized":"2024-02-22T19:43:57.344709+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":191,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343947+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:43:57.343947+0000","last_peered":"2024-02-22T19:43:57.343947+0000","last_clean":"2024-02-22T19:43:57.343947+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:43:57.343947+0000","last_undegraded":"2024-02-22T19:43:57.343947+0000","last_fullsized":"2024-02-22T19:43:57.343947+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":671,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.779120+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:43:58.779120+0000","last_peered":"2024-02-22T19:43:58.779120+0000","last_clean":"2024-02-22T19:43:58.779120+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:43:58.779120+0000","last_undegraded":"2024-02-22T19:43:58.779120+0000","last_fullsized":"2024-02-22T19:43:58.779120+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":336,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.343205+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:43:57.343205+0000","last_peered":"2024-02-22T19:43:57.343205+0000","last_clean":"2024-02-22T19:43:57.343205+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:43:57.343205+0000","last_undegraded":"2024-02-22T19:43:57.343205+0000","last_fullsized":"2024-02-22T19:43:57.343205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":80,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344430+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:43:57.344430+0000","last_peered":"2024-02-22T19:43:57.344430+0000","last_clean":"2024-02-22T19:43:57.344430+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:43:57.344430+0000","last_undegraded":"2024-02-22T19:43:57.344430+0000","last_fullsized":"2024-02-22T19:43:57.344430+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347048+0000","last_change":"2024-02-22T19:43:57.346250+0000","last_active":"2024-02-22T19:43:57.347048+0000","last_peered":"2024-02-22T19:43:57.347048+0000","last_clean":"2024-02-22T19:43:57.347048+0000","last_became_active":"2024-02-22T19:43:57.345992+0000","last_became_peered":"2024-02-22T19:43:57.345992+0000","last_unstale":"2024-02-22T19:43:57.347048+0000","last_undegraded":"2024-02-22T19:43:57.347048+0000","last_fullsized":"2024-02-22T19:43:57.347048+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":352,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.346744+0000","last_change":"2024-02-22T19:41:52.112553+0000","last_active":"2024-02-22T19:43:57.346744+0000","last_peered":"2024-02-22T19:43:57.346744+0000","last_clean":"2024-02-22T19:43:57.346744+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:43:57.346744+0000","last_undegraded":"2024-02-22T19:43:57.346744+0000","last_fullsized":"2024-02-22T19:43:57.346744+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"326'20","reported_seq":417,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.779225+0000","last_change":"2024-02-22T19:41:54.322898+0000","last_active":"2024-02-22T19:43:58.779225+0000","last_peered":"2024-02-22T19:43:58.779225+0000","last_clean":"2024-02-22T19:43:58.779225+0000","last_became_active":"2024-02-22T19:41:54.322531+0000","last_became_peered":"2024-02-22T19:41:54.322531+0000","last_unstale":"2024-02-22T19:43:58.779225+0000","last_undegraded":"2024-02-22T19:43:58.779225+0000","last_fullsized":"2024-02-22T19:43:58.779225+0000","mapping_epoch":325,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":141,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340262+0000","last_change":"2024-02-22T19:40:52.130146+0000","last_active":"2024-02-22T19:43:57.340262+0000","last_peered":"2024-02-22T19:43:57.340262+0000","last_clean":"2024-02-22T19:43:57.340262+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:43:57.340262+0000","last_undegraded":"2024-02-22T19:43:57.340262+0000","last_fullsized":"2024-02-22T19:43:57.340262+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.130022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.130022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":149,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.341150+0000","last_change":"2024-02-22T19:41:52.358275+0000","last_active":"2024-02-22T19:43:57.341150+0000","last_peered":"2024-02-22T19:43:57.341150+0000","last_clean":"2024-02-22T19:43:57.341150+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:43:57.341150+0000","last_undegraded":"2024-02-22T19:43:57.341150+0000","last_fullsized":"2024-02-22T19:43:57.341150+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.349058+0000","last_change":"2024-02-22T19:43:57.348166+0000","last_active":"2024-02-22T19:43:57.349058+0000","last_peered":"2024-02-22T19:43:57.349058+0000","last_clean":"2024-02-22T19:43:57.349058+0000","last_became_active":"2024-02-22T19:43:57.347832+0000","last_became_peered":"2024-02-22T19:43:57.347832+0000","last_unstale":"2024-02-22T19:43:57.349058+0000","last_undegraded":"2024-02-22T19:43:57.349058+0000","last_fullsized":"2024-02-22T19:43:57.349058+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":164,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.344113+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:43:57.344113+0000","last_peered":"2024-02-22T19:43:57.344113+0000","last_clean":"2024-02-22T19:43:57.344113+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:43:57.344113+0000","last_undegraded":"2024-02-22T19:43:57.344113+0000","last_fullsized":"2024-02-22T19:43:57.344113+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":356,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.338773+0000","last_change":"2024-02-22T19:41:52.107535+0000","last_active":"2024-02-22T19:43:57.338773+0000","last_peered":"2024-02-22T19:43:57.338773+0000","last_clean":"2024-02-22T19:43:57.338773+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:43:57.338773+0000","last_undegraded":"2024-02-22T19:43:57.338773+0000","last_fullsized":"2024-02-22T19:43:57.338773+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.092018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.092018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":400,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.752166+0000","last_change":"2024-02-22T19:43:58.752699+0000","last_active":"2024-02-22T19:43:58.752166+0000","last_peered":"2024-02-22T19:43:58.752166+0000","last_clean":"2024-02-22T19:43:58.752166+0000","last_became_active":"2024-02-22T19:41:52.092045+0000","last_became_peered":"2024-02-22T19:41:52.092045+0000","last_unstale":"2024-02-22T19:43:58.752166+0000","last_undegraded":"2024-02-22T19:43:58.752166+0000","last_fullsized":"2024-02-22T19:43:58.752166+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.402940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.402940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":273,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.339295+0000","last_change":"2024-02-22T19:41:52.110141+0000","last_active":"2024-02-22T19:43:57.339295+0000","last_peered":"2024-02-22T19:43:57.339295+0000","last_clean":"2024-02-22T19:43:57.339295+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:43:57.339295+0000","last_undegraded":"2024-02-22T19:43:57.339295+0000","last_fullsized":"2024-02-22T19:43:57.339295+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":242,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340658+0000","last_change":"2024-02-22T19:41:52.810562+0000","last_active":"2024-02-22T19:43:57.340658+0000","last_peered":"2024-02-22T19:43:57.340658+0000","last_clean":"2024-02-22T19:43:57.340658+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:43:57.340658+0000","last_undegraded":"2024-02-22T19:43:57.340658+0000","last_fullsized":"2024-02-22T19:43:57.340658+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:40:35.038166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.038166+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.352340+0000","last_change":"2024-02-22T19:43:57.351793+0000","last_active":"2024-02-22T19:43:57.352340+0000","last_peered":"2024-02-22T19:43:57.352340+0000","last_clean":"2024-02-22T19:43:57.352340+0000","last_became_active":"2024-02-22T19:43:57.351420+0000","last_became_peered":"2024-02-22T19:43:57.351420+0000","last_unstale":"2024-02-22T19:43:57.352340+0000","last_undegraded":"2024-02-22T19:43:57.352340+0000","last_fullsized":"2024-02-22T19:43:57.352340+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":354,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345446+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:43:57.345446+0000","last_peered":"2024-02-22T19:43:57.345446+0000","last_clean":"2024-02-22T19:43:57.345446+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:43:57.345446+0000","last_undegraded":"2024-02-22T19:43:57.345446+0000","last_fullsized":"2024-02-22T19:43:57.345446+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":180,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.340868+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:43:57.340868+0000","last_peered":"2024-02-22T19:43:57.340868+0000","last_clean":"2024-02-22T19:43:57.340868+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:43:57.340868+0000","last_undegraded":"2024-02-22T19:43:57.340868+0000","last_fullsized":"2024-02-22T19:43:57.340868+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":595,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-02-22T19:43:58.342156+0000","last_change":"2024-02-22T19:41:32.453253+0000","last_active":"2024-02-22T19:43:58.342156+0000","last_peered":"2024-02-22T19:43:58.342156+0000","last_clean":"2024-02-22T19:43:58.342156+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:43:58.342156+0000","last_undegraded":"2024-02-22T19:43:58.342156+0000","last_fullsized":"2024-02-22T19:43:58.342156+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":397,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.345956+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:43:57.345956+0000","last_peered":"2024-02-22T19:43:57.345956+0000","last_clean":"2024-02-22T19:43:57.345956+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:43:57.345956+0000","last_undegraded":"2024-02-22T19:43:57.345956+0000","last_fullsized":"2024-02-22T19:43:57.345956+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":211,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-02-22T19:43:57.347005+0000","last_change":"2024-02-22T19:41:52.575055+0000","last_active":"2024-02-22T19:43:57.347005+0000","last_peered":"2024-02-22T19:43:57.347005+0000","last_clean":"2024-02-22T19:43:57.347005+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:43:57.347005+0000","last_undegraded":"2024-02-22T19:43:57.347005+0000","last_fullsized":"2024-02-22T19:43:57.347005+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":5},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":881,"num_read_kb":876,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266871,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":592348,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93152804,"statfs":{"total":95995035648,"available":95388471296,"internally_reserved":0,"allocated":2707456,"data_stored":1487181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.2490000000000001,"5min":1.532,"15min":1.5029999999999999},"min":{"1min":0.249,"5min":0.245,"15min":0.19900000000000001},"max":{"1min":7.9790000000000001,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.497},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.379,"15min":1.421},"min":{"1min":0.25600000000000001,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":3.25,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":2.4940000000000002,"15min":1.7529999999999999},"min":{"1min":0.28399999999999997,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":6.1200000000000001,"5min":54.951000000000001,"15min":54.951000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.218,"5min":2.5049999999999999,"15min":1.921},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":3.613,"5min":57.012,"15min":57.012},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.825,"15min":1.4370000000000001},"min":{"1min":0.27200000000000002,"5min":0.27000000000000002,"15min":0.23999999999999999},"max":{"1min":5.1200000000000001,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.3769999999999998,"15min":1.7310000000000001},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.23599999999999999},"max":{"1min":5.0960000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.434}]},{"osd":3,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":2.6459999999999999,"15min":1.861},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":7.1289999999999996,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.628},{"interface":"front","average":{"1min":3.1739999999999999,"5min":2.8090000000000002,"15min":1.9139999999999999},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":9.4039999999999999,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.004,"15min":0.70599999999999996},"min":{"1min":0.31,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":4.4219999999999997,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.258,"15min":0.755},"min":{"1min":0.29899999999999999,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":5.0910000000000002,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.419,"15min":0.86199999999999999},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.27000000000000002},"max":{"1min":6.625,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.034,"5min":1.093,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.28000000000000003},"max":{"1min":7.3239999999999998,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.004,"15min":0.81000000000000005},"min":{"1min":0.32300000000000001,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":3.044,"5min":6.5,"15min":12.199999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":0.90800000000000003},"min":{"1min":0.36899999999999999,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":3.6680000000000001,"5min":7.6390000000000002,"15min":12.067},"last":0.52300000000000002}]}]},{"osd":6,"up_from":0,"seq":193273528508,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575061,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":592360,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93152792,"statfs":{"total":95995035648,"available":95388459008,"internally_reserved":0,"allocated":2719744,"data_stored":1487157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":5.7480000000000002,"5min":2.391,"15min":1.8580000000000001},"min":{"1min":0.38900000000000001,"5min":0.185,"15min":0.185},"max":{"1min":51.091000000000001,"5min":51.091000000000001,"15min":51.091000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":5.1799999999999997,"5min":2.3860000000000001,"15min":1.952},"min":{"1min":0.38300000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":51.289999999999999,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":5.4669999999999996,"5min":2.4630000000000001,"15min":1.9710000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":51.027000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":4.7220000000000004,"5min":2.4950000000000001,"15min":2.077},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":51.369999999999997,"5min":51.369999999999997,"15min":51.369999999999997},"last":0.25600000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:43:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":2.4119999999999999,"15min":2.145},"min":{"1min":0.28999999999999998,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":5.8040000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":0.23400000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":2.5419999999999998,"15min":2.206},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":6.6449999999999996,"5min":51.741,"15min":51.741},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:43:10 2024","interfaces":[{"interface":"back","average":{"1min":6.1319999999999997,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.32600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":51.158999999999999,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":3.843,"5min":1.853,"15min":1.385},"min":{"1min":0.311,"5min":0.311,"15min":0.245},"max":{"1min":51.276000000000003,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:43:32 2024","interfaces":[{"interface":"back","average":{"1min":5.1959999999999997,"5min":2.7109999999999999,"15min":1.8009999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":51.183999999999997,"5min":51.183999999999997,"15min":51.183999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":4.1459999999999999,"5min":2.2629999999999999,"15min":1.6100000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":51.212000000000003,"5min":51.212000000000003,"15min":51.212000000000003},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:43:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.8069999999999999,"15min":1.4770000000000001},"min":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":5.1879999999999997,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.954,"5min":2.544,"15min":1.78},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.34999999999999998},"max":{"1min":8.4269999999999996,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.63}]},{"osd":7,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":4.7080000000000002,"5min":2.5830000000000002,"15min":1.784},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":51.244999999999997,"5min":51.244999999999997,"15min":51.244999999999997},"last":0.62},{"interface":"front","average":{"1min":4.4050000000000002,"5min":1.8540000000000001,"15min":1.454},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":51.317999999999998,"5min":51.317999999999998,"15min":51.317999999999998},"last":0.55700000000000005}]}]},{"osd":0,"up_from":0,"seq":34359738586,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411535,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303956,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441196,"statfs":{"total":95995035648,"available":95683784704,"internally_reserved":0,"allocated":2699264,"data_stored":1485448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":2.0699999999999998,"15min":1.706},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.18099999999999999},"max":{"1min":4.7000000000000002,"5min":16.763999999999999,"15min":22.338999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":2.2789999999999999,"5min":2.5640000000000001,"15min":2.1120000000000001},"min":{"1min":0.26700000000000002,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":14.603999999999999,"5min":30.427,"15min":30.427},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.8560000000000001,"15min":1.679},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.0819999999999999,"5min":12.349,"15min":12.349},"last":0.504},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.889,"15min":1.585},"min":{"1min":0.215,"5min":0.215,"15min":0.193},"max":{"1min":5.1900000000000004,"5min":13.782,"15min":13.782},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":2.0379999999999998,"15min":2.0379999999999998},"min":{"1min":0.27000000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":8.8599999999999994,"5min":16.076000000000001,"15min":30.071000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.4940000000000002,"15min":2.3239999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.184},"max":{"1min":9.3149999999999995,"5min":28.263999999999999,"15min":32.119},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:43:06 2024","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.585,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":7.0140000000000002,"5min":12.43,"15min":12.43},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.861,"15min":1.2529999999999999},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":4.6420000000000003,"5min":10.34,"15min":10.34},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:43:29 2024","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":2.2879999999999998,"15min":1.4690000000000001},"min":{"1min":0.28499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.867,"5min":40.633000000000003,"15min":40.633000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.964,"5min":2.3980000000000001,"15min":1.6859999999999999},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":6.2999999999999998,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:43:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2669999999999999,"5min":1.8140000000000001,"15min":1.4990000000000001},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.311},"max":{"1min":7.1550000000000002,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.698,"5min":1.9530000000000001,"15min":1.304},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":4.5940000000000003,"5min":10.369999999999999,"15min":10.369999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.736,"15min":1.5049999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.1580000000000004,"5min":12.484,"15min":12.484},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.9299999999999999,"15min":1.407},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":4.5999999999999996,"5min":10.539,"15min":10.539},"last":0.44600000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149897,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898408,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":895744,"kb_avail":92846744,"statfs":{"total":95995035648,"available":95075065856,"internally_reserved":0,"allocated":2719744,"data_stored":1487050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":917241856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.9410000000000001,"15min":1.901},"min":{"1min":0.26800000000000002,"5min":0.191,"15min":0.191},"max":{"1min":6.3220000000000001,"5min":9.9789999999999992,"15min":23.48},"last":0.311},{"interface":"front","average":{"1min":2.2000000000000002,"5min":2.2370000000000001,"15min":2.2280000000000002},"min":{"1min":0.24199999999999999,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":10.297000000000001,"5min":18.172000000000001,"15min":23.411999999999999},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":2.4169999999999998,"5min":2.8530000000000002,"15min":2.5569999999999999},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":11.385999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.51,"5min":2.2639999999999998,"15min":2.044},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":6.8040000000000003,"5min":22.187000000000001,"15min":22.187000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":2.2050000000000001,"5min":2.2650000000000001,"15min":2.1080000000000001},"min":{"1min":0.28199999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.577,"5min":20.105,"15min":20.105},"last":0.37},{"interface":"front","average":{"1min":1.9770000000000001,"5min":2.4660000000000002,"15min":2.0779999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.217},"max":{"1min":6.8390000000000004,"5min":35.939999999999998,"15min":35.939999999999998},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:43:03 2024","interfaces":[{"interface":"back","average":{"1min":2.3740000000000001,"5min":2.1699999999999999,"15min":1.73},"min":{"1min":0.30599999999999999,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":9.968,"5min":12.416,"15min":13.680999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.9890000000000001,"15min":1.5429999999999999},"min":{"1min":0.40400000000000003,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":3.6019999999999999,"5min":13.548,"15min":13.548},"last":0.60399999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:43:35 2024","interfaces":[{"interface":"back","average":{"1min":2.1099999999999999,"5min":1.9570000000000001,"15min":1.5800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1159999999999997,"5min":11.161,"15min":12.879},"last":0.58899999999999997},{"interface":"front","average":{"1min":2.7080000000000002,"5min":2.2029999999999998,"15min":1.7829999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":10.041,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:43:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.879,"15min":1.573},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":6.3109999999999999,"5min":11.263,"15min":12.898999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":2.0840000000000001,"5min":2.1320000000000001,"15min":1.867},"min":{"1min":0.41799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.2859999999999996,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Feb 22 19:43:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":2.0920000000000001,"15min":1.595},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":3.6259999999999999,"5min":13.483000000000001,"15min":13.483000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":2.2770000000000001,"5min":2.0659999999999998,"15min":1.621},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.2240000000000002,"5min":11.539,"15min":12.846},"last":0.622}]}]},{"osd":4,"up_from":0,"seq":137438953671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724737,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":592364,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93152788,"statfs":{"total":95995035648,"available":95388454912,"internally_reserved":0,"allocated":2723840,"data_stored":1486482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":8.4299999999999997,"5min":17.684000000000001,"15min":17.684000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.7629999999999999,"15min":1.5129999999999999},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":4.7060000000000004,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.4059999999999999,"15min":1.375},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":3.0070000000000001,"5min":11.202,"15min":29.390999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":2.052,"5min":1.8089999999999999,"15min":1.5169999999999999},"min":{"1min":0.309,"5min":0.248,"15min":0.218},"max":{"1min":6.5129999999999999,"5min":8.7490000000000006,"15min":29.367000000000001},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":2.044,"15min":1.534},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.7550000000000008,"5min":11.367000000000001,"15min":14.613},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.595,"5min":1.8520000000000001,"15min":1.534},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":6.3979999999999997,"5min":16.099,"15min":16.099},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":4.0640000000000001,"5min":2.0329999999999999,"15min":1.778},"min":{"1min":0.41499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":16.370999999999999,"5min":16.370999999999999,"15min":16.370999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.6599999999999999,"15min":1.609},"min":{"1min":0.41499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.9199999999999999,"5min":8.6069999999999993,"15min":10.16},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.1379999999999999,"15min":0.73999999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.219},"max":{"1min":7.5419999999999998,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.376},{"interface":"front","average":{"1min":2.234,"5min":1.496,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.23499999999999999},"max":{"1min":5.577,"5min":10.407,"15min":10.407},"last":0.36099999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:43:58 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.873,"15min":0.66900000000000004},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":2.7509999999999999,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.337,"15min":0.85599999999999998},"min":{"1min":0.36799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":4.2160000000000002,"5min":10.436,"15min":10.436},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:43:26 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1830000000000001,"15min":0.79200000000000004},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.8410000000000002,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":1.901,"5min":1.3899999999999999,"15min":0.876},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.27100000000000002},"max":{"1min":7.5069999999999997,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.307}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:44:03.916 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-22T19:44:03.916 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:44:04.147 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:04.151 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:04.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:04.252 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:04.285 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:04.887 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:44:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:44:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-02-22T19:44:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:03.677055+0000 mgr.y (mgr.14535) 577 : cluster [DBG] pgmap v746: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 807 B/s rd, 0 op/s 2024-02-22T19:44:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:04.311794+0000 osd.7 (osd.7) 224 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:44:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:04.315261+0000 osd.7 (osd.7) 225 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:44:05.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:04.321581+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? 172.21.15.53:0/3783798201' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:44:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:03.677055+0000 mgr.y (mgr.14535) 577 : cluster [DBG] pgmap v746: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 807 B/s rd, 0 op/s 2024-02-22T19:44:05.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:04.311794+0000 osd.7 (osd.7) 224 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:44:05.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:04.315261+0000 osd.7 (osd.7) 225 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:44:05.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:04.321581+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? 172.21.15.53:0/3783798201' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:44:05.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:44:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:44:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:03.677055+0000 mgr.y (mgr.14535) 577 : cluster [DBG] pgmap v746: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 807 B/s rd, 0 op/s 2024-02-22T19:44:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:04.311794+0000 osd.7 (osd.7) 224 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:44:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:04.315261+0000 osd.7 (osd.7) 225 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:44:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:04.321581+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? 172.21.15.53:0/3783798201' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:44:06.115 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:06.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:06.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:06.160 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:06.160 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:06.220 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:04.946096+0000 osd.3 (osd.3) 235 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:04.949554+0000 osd.3 (osd.3) 236 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:05.102124+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? 172.21.15.53:0/3783798201' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:05.102195+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e369: 8 total, 8 up, 5 in 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:05.126354+0000 osd.2 (osd.2) 194 : cluster [DBG] 6.f scrub starts 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:05.146733+0000 mgr.y (mgr.14535) 578 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:04.946096+0000 osd.3 (osd.3) 235 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:04.949554+0000 osd.3 (osd.3) 236 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:05.102124+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? 172.21.15.53:0/3783798201' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:44:06.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:05.102195+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e369: 8 total, 8 up, 5 in 2024-02-22T19:44:06.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:05.126354+0000 osd.2 (osd.2) 194 : cluster [DBG] 6.f scrub starts 2024-02-22T19:44:06.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:05.146733+0000 mgr.y (mgr.14535) 578 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:06.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:04.946096+0000 osd.3 (osd.3) 235 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:44:06.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:04.949554+0000 osd.3 (osd.3) 236 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:44:06.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:05.102124+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? 172.21.15.53:0/3783798201' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:44:06.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:05.102195+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e369: 8 total, 8 up, 5 in 2024-02-22T19:44:06.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:05.126354+0000 osd.2 (osd.2) 194 : cluster [DBG] 6.f scrub starts 2024-02-22T19:44:06.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:05.146733+0000 mgr.y (mgr.14535) 578 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:06.499 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:06.503 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:06.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:06.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:06.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:44:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:06.834 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:06.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:06.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:06.962 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:06.982 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:06.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:06.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:07.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:07.114 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:07.141 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:07.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:07.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:07.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:05.228866+0000 osd.2 (osd.2) 195 : cluster [DBG] 6.f scrub ok 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:05.677371+0000 mgr.y (mgr.14535) 579 : cluster [DBG] pgmap v748: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:05.905488+0000 osd.3 (osd.3) 237 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:05.908376+0000 osd.3 (osd.3) 238 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:06.169446+0000 osd.2 (osd.2) 196 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:06.172355+0000 osd.2 (osd.2) 197 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:05.228866+0000 osd.2 (osd.2) 195 : cluster [DBG] 6.f scrub ok 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:05.677371+0000 mgr.y (mgr.14535) 579 : cluster [DBG] pgmap v748: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:05.905488+0000 osd.3 (osd.3) 237 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:05.908376+0000 osd.3 (osd.3) 238 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:44:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:06.169446+0000 osd.2 (osd.2) 196 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:44:07.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:06.172355+0000 osd.2 (osd.2) 197 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:44:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:07.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:05.228866+0000 osd.2 (osd.2) 195 : cluster [DBG] 6.f scrub ok 2024-02-22T19:44:07.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:05.677371+0000 mgr.y (mgr.14535) 579 : cluster [DBG] pgmap v748: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-22T19:44:07.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:05.905488+0000 osd.3 (osd.3) 237 : cluster [DBG] 5.13 scrub starts 2024-02-22T19:44:07.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:05.908376+0000 osd.3 (osd.3) 238 : cluster [DBG] 5.13 scrub ok 2024-02-22T19:44:07.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:06.169446+0000 osd.2 (osd.2) 196 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:44:07.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:06.172355+0000 osd.2 (osd.2) 197 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:44:07.828 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:07.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:07.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:07.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:07.963 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:08.018 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:08.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:08.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:08.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:08.182 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:08.184 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:08.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:08.226 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:06.752837+0000 osd.1 (osd.1) 210 : cluster [DBG] 5.c scrub starts 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:06.753738+0000 osd.1 (osd.1) 211 : cluster [DBG] 5.c scrub ok 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:06.757807+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:06.757835+0000 osd.6 (osd.6) 157 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.112894+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e370: 8 total, 8 up, 5 in 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.169090+0000 osd.2 (osd.2) 198 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.172557+0000 osd.2 (osd.2) 199 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:07.352737+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.53:0/4210994181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:08.227 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:07.353299+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:08.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:08.311 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:08.315 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:08.328 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:08.348 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:08.349 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:44:08.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:08.374 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:06.752837+0000 osd.1 (osd.1) 210 : cluster [DBG] 5.c scrub starts 2024-02-22T19:44:08.374 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:06.753738+0000 osd.1 (osd.1) 211 : cluster [DBG] 5.c scrub ok 2024-02-22T19:44:08.374 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:06.757807+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:08.374 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:06.757835+0000 osd.6 (osd.6) 157 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:08.374 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.112894+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e370: 8 total, 8 up, 5 in 2024-02-22T19:44:08.374 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.169090+0000 osd.2 (osd.2) 198 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.172557+0000 osd.2 (osd.2) 199 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:07.352737+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.53:0/4210994181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:07.353299+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:06.752837+0000 osd.1 (osd.1) 210 : cluster [DBG] 5.c scrub starts 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:06.753738+0000 osd.1 (osd.1) 211 : cluster [DBG] 5.c scrub ok 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:06.757807+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:06.757835+0000 osd.6 (osd.6) 157 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.112894+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e370: 8 total, 8 up, 5 in 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.169090+0000 osd.2 (osd.2) 198 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.172557+0000 osd.2 (osd.2) 199 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:07.352737+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.53:0/4210994181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:08.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:07.353299+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:08.385 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:08.404 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:08.423 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:08.441 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:08.442 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:44:08.566 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:08.568 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:08.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:08.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:08.682 DEBUG:teuthology.orchestra.run.smithi053:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-02-22T19:44:08.682 INFO:tasks.thrashosds.thrasher:Shrinking pool default.rgw.control 2024-02-22T19:44:08.683 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool default.rgw.control by 10 min 8 2024-02-22T19:44:08.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:44:08.685 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-02-22T19:44:08.685 INFO:teuthology.orchestra.run:waiting for 3000 2024-02-22T19:44:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:08.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:hints = 1 2024-02-22T19:44:08.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-02-22T19:44:08.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object prefix: benchmark_data_smithi053_180506 2024-02-22T19:44:08.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:44:08.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 0 0 0 0 0 0 - 0 2024-02-22T19:44:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:08.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:08.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:09.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:09.097 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:09.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:09.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:09.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:09.240 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.677699+0000 mgr.y (mgr.14535) 580 : cluster [DBG] pgmap v750: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:44:09.240 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.792962+0000 osd.5 (osd.5) 118 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.794939+0000 osd.5 (osd.5) 119 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.981184+0000 osd.3 (osd.3) 239 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:07.984704+0000 osd.3 (osd.3) 240 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:08.269328+0000 osd.7 (osd.7) 226 : cluster [DBG] 3.f scrub starts 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:08.271000+0000 osd.7 (osd.7) 227 : cluster [DBG] 3.f scrub ok 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:08.293225+0000 mon.c (mon.2) 71 : audit [DBG] from='client.? 172.21.15.53:0/1503379196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:08.339450+0000 mon.a (mon.0) 1489 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:08.339472+0000 mon.a (mon.0) 1490 : cluster [INF] Cluster is now healthy 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:08.345671+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:08.345741+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e371: 8 total, 8 up, 5 in 2024-02-22T19:44:09.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:08.375897+0000 mon.a (mon.0) 1493 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:09.247 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:09.275 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:09.278 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:44:09.278 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:44:09.278 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:44:09.278 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:44:09.278 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:44:09.280 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:09.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:09.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:09.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.677699+0000 mgr.y (mgr.14535) 580 : cluster [DBG] pgmap v750: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:44:09.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.792962+0000 osd.5 (osd.5) 118 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:44:09.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.794939+0000 osd.5 (osd.5) 119 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:44:09.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.981184+0000 osd.3 (osd.3) 239 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:44:09.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:07.984704+0000 osd.3 (osd.3) 240 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:08.269328+0000 osd.7 (osd.7) 226 : cluster [DBG] 3.f scrub starts 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:08.271000+0000 osd.7 (osd.7) 227 : cluster [DBG] 3.f scrub ok 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:08.293225+0000 mon.c (mon.2) 71 : audit [DBG] from='client.? 172.21.15.53:0/1503379196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:08.339450+0000 mon.a (mon.0) 1489 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:08.339472+0000 mon.a (mon.0) 1490 : cluster [INF] Cluster is now healthy 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:08.345671+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:08.345741+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e371: 8 total, 8 up, 5 in 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:08.375897+0000 mon.a (mon.0) 1493 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:09.391 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:09] "GET /metrics HTTP/1.1" 200 176309 "" "Prometheus/2.33.4" 2024-02-22T19:44:09.392 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.677699+0000 mgr.y (mgr.14535) 580 : cluster [DBG] pgmap v750: 148 pgs: 148 active+clean; 5.7 KiB data, 2.8 GiB used, 444 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:44:09.392 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.792962+0000 osd.5 (osd.5) 118 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:44:09.392 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.794939+0000 osd.5 (osd.5) 119 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:44:09.392 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.981184+0000 osd.3 (osd.3) 239 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:07.984704+0000 osd.3 (osd.3) 240 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:08.269328+0000 osd.7 (osd.7) 226 : cluster [DBG] 3.f scrub starts 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:08.271000+0000 osd.7 (osd.7) 227 : cluster [DBG] 3.f scrub ok 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:08.293225+0000 mon.c (mon.2) 71 : audit [DBG] from='client.? 172.21.15.53:0/1503379196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:08.339450+0000 mon.a (mon.0) 1489 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:08.339472+0000 mon.a (mon.0) 1490 : cluster [INF] Cluster is now healthy 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:08.345671+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:08.345741+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e371: 8 total, 8 up, 5 in 2024-02-22T19:44:09.393 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:08.375897+0000 mon.a (mon.0) 1493 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:09.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 1 16 2951 2935 183.427 183.438 0.00649066 0.00542933 2024-02-22T19:44:09.918 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:09.973 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:09.977 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:09.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:09.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:44:10.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:10.136 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.023545+0000 osd.3 (osd.3) 241 : cluster [DBG] 3.2 deep-scrub starts 2024-02-22T19:44:10.136 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.027311+0000 osd.3 (osd.3) 242 : cluster [DBG] 3.2 deep-scrub ok 2024-02-22T19:44:10.136 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.127266+0000 osd.2 (osd.2) 200 : cluster [DBG] 4.d scrub starts 2024-02-22T19:44:10.137 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.132433+0000 osd.2 (osd.2) 201 : cluster [DBG] 4.d scrub ok 2024-02-22T19:44:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:10.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:10.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:10.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:10.308 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:10.343 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:10.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:10.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.023545+0000 osd.3 (osd.3) 241 : cluster [DBG] 3.2 deep-scrub starts 2024-02-22T19:44:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.027311+0000 osd.3 (osd.3) 242 : cluster [DBG] 3.2 deep-scrub ok 2024-02-22T19:44:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.127266+0000 osd.2 (osd.2) 200 : cluster [DBG] 4.d scrub starts 2024-02-22T19:44:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.132433+0000 osd.2 (osd.2) 201 : cluster [DBG] 4.d scrub ok 2024-02-22T19:44:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:10.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:10.496 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.023545+0000 osd.3 (osd.3) 241 : cluster [DBG] 3.2 deep-scrub starts 2024-02-22T19:44:10.496 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.027311+0000 osd.3 (osd.3) 242 : cluster [DBG] 3.2 deep-scrub ok 2024-02-22T19:44:10.496 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.127266+0000 osd.2 (osd.2) 200 : cluster [DBG] 4.d scrub starts 2024-02-22T19:44:10.496 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.132433+0000 osd.2 (osd.2) 201 : cluster [DBG] 4.d scrub ok 2024-02-22T19:44:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:10.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 2 16 5615 5599 174.956 166.5 0.00677247 0.00569857 2024-02-22T19:44:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.678363+0000 mgr.y (mgr.14535) 581 : cluster [DBG] pgmap v752: 148 pgs: 148 active+clean; 107 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 285 op/s 2024-02-22T19:44:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.744287+0000 osd.5 (osd.5) 120 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:44:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:09.747290+0000 osd.5 (osd.5) 121 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:44:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:10.013346+0000 osd.3 (osd.3) 243 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:44:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:10.017662+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:44:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.678363+0000 mgr.y (mgr.14535) 581 : cluster [DBG] pgmap v752: 148 pgs: 148 active+clean; 107 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 285 op/s 2024-02-22T19:44:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.744287+0000 osd.5 (osd.5) 120 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:44:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:09.747290+0000 osd.5 (osd.5) 121 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:44:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:10.013346+0000 osd.3 (osd.3) 243 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:44:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:10.017662+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:44:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.678363+0000 mgr.y (mgr.14535) 581 : cluster [DBG] pgmap v752: 148 pgs: 148 active+clean; 107 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 285 op/s 2024-02-22T19:44:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.744287+0000 osd.5 (osd.5) 120 : cluster [DBG] 5.15 scrub starts 2024-02-22T19:44:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:09.747290+0000 osd.5 (osd.5) 121 : cluster [DBG] 5.15 scrub ok 2024-02-22T19:44:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:10.013346+0000 osd.3 (osd.3) 243 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:44:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:10.017662+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:44:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:11.371 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:11.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:11.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:44:11.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:11.735 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 3 16 8142 8126 169.278 157.938 0.00642161 0.00589674 2024-02-22T19:44:11.862 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:11.868 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:11.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:11.897 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:44:11.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:12.031 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:12.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:12.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:12.098 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:12.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.036852+0000 osd.3 (osd.3) 245 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.041449+0000 osd.3 (osd.3) 246 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.172057+0000 osd.2 (osd.2) 202 : cluster [DBG] 4.b scrub starts 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.175775+0000 osd.2 (osd.2) 203 : cluster [DBG] 4.b scrub ok 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.036852+0000 osd.3 (osd.3) 245 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.041449+0000 osd.3 (osd.3) 246 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.172057+0000 osd.2 (osd.2) 202 : cluster [DBG] 4.b scrub starts 2024-02-22T19:44:12.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.175775+0000 osd.2 (osd.2) 203 : cluster [DBG] 4.b scrub ok 2024-02-22T19:44:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:44:12.377 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:12.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:12.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:12.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.036852+0000 osd.3 (osd.3) 245 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:44:12.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.041449+0000 osd.3 (osd.3) 246 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:44:12.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.172057+0000 osd.2 (osd.2) 202 : cluster [DBG] 4.b scrub starts 2024-02-22T19:44:12.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.175775+0000 osd.2 (osd.2) 203 : cluster [DBG] 4.b scrub ok 2024-02-22T19:44:12.392 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:44:12.415 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165019 8.8 8.f9eb30f8 (undecoded) ondisk+retry+write+known_if_redirected e217)", 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.671025+0000", 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.7427624, 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49627743200000002, 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165019 2024-02-22T19:44:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671025+0000", 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671025+0000", 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671023+0000", 2024-02-22T19:44:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2129999999999997e-06 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671032+0000", 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3607e-05 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671046+0000", 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0129999999999999e-06 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671055+0000", 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1451000000000002e-05 2024-02-22T19:44:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671146+0000", 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4880000000000007e-06 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671155+0000", 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49613517299999998 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167290+0000", 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2617e-05 2024-02-22T19:44:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167302+0000", 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49627743200000002 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165021 8.8 8.8bb662f8 (undecoded) ondisk+retry+write+known_if_redirected e217)", 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.671071+0000", 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.742716481, 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49624754100000001, 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165021 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671071+0000", 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671071+0000", 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671070+0000", 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1300000000000001e-06 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671074+0000", 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1552999999999999e-05 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671085+0000", 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7370000000000003e-06 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671091+0000", 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9857999999999993e-05 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671171+0000", 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3150000000000002e-06 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671175+0000", 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49614006300000002 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167315+0000", 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4120000000000001e-06 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167318+0000", 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49624754100000001 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1369 5.15 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 2] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672397+0000", 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.74139020600001, 2024-02-22T19:44:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.52036051100000003, 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1369 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672397+0000", 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672397+0000", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672396+0000", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672402+0000", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3174e-05 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672416+0000", 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.498e-06 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672423+0000", 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7455999999999997e-05 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672511+0000", 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.537e-06 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672518+0000", 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.51161114399999996 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.184129+0000", 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014382899999999999 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for missing object", 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.184273+0000", 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048510130000000004 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.189124+0000", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0124000000000002e-05 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for rw locks", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.189184+0000", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016101819999999999 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.190794+0000", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1518000000000002e-05 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.190866+0000", 2024-02-22T19:44:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001891799 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.192758+0000", 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.52036051100000003 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1371 5.f 5.4322fa9f (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.74135124600002, 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49085674200000001, 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1371 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672436+0000", 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0910000000000004e-06 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672443+0000", 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7689999999999996e-06 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672450+0000", 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5933000000000001e-05 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672466+0000", 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014442799999999999 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672611+0000", 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2309999999999996e-06 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672619+0000", 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49066390399999998 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.163283+0000", 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9820000000000003e-06 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.163293+0000", 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.49085674200000001 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165028 8.8 8.bf9c1768 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.688837+0000", 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.724950713, 2024-02-22T19:44:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47849444299999999, 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165028 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688837+0000", 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688837+0000", 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688835+0000", 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7429999999999999e-06 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688841+0000", 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2629999999999992e-06 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688850+0000", 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6480000000000002e-06 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688855+0000", 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3702000000000002e-05 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688909+0000", 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0879999999999995e-06 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688917+0000", 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478410891 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167328+0000", 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5750000000000001e-06 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167331+0000", 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47849444299999999 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165030 8.8 8.d08a68 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.72491319300002, 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478491316, 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165030 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688874+0000", 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4669999999999998e-06 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688881+0000", 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7190000000000004e-06 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688888+0000", 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3460000000000004e-06 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688892+0000", 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3278000000000001e-05 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688936+0000", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-06 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.688939+0000", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478417957 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167357+0000", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289999999999996e-06 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167366+0000", 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.478491316 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165036 8.4 8.ff588524 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.692461+0000", 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.72132621899999, 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47865222200000002, 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:44:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165036 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692461+0000", 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692461+0000", 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692460+0000", 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0480000000000004e-06 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692466+0000", 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3577999999999999e-05 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692480+0000", 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.413e-06 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692487+0000", 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7818000000000002e-05 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692535+0000", 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.692542+0000", 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47856282 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171105+0000", 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8910000000000001e-06 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171113+0000", 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47865222200000002 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165046 8.8 8.5f50e908 (undecoded) ondisk+write+known_if_redirected e217)", 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.716885+0000", 2024-02-22T19:44:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 375.69690229499997, 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.450508415, 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165046 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716885+0000", 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716885+0000", 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716883+0000", 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.289e-06 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716891+0000", 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3481e-05 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716904+0000", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.827e-06 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716911+0000", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6832000000000001e-05 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716968+0000", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4290000000000001e-06 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.716976+0000", 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.45041040799999998 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167386+0000", 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0600000000000002e-06 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167394+0000", 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.450508415 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1385 5.f 5:f95f44c2:::notify.0:head [watch reconnect cookie 94129597095936 gen 6] snapc 0=[] ondisk+write+known_if_redirected e220)", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.267450+0000", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 373.14633786600001, 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94290151300000002, 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1385 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267450+0000", 2024-02-22T19:44:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267450+0000", 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267448+0000", 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3709999999999999e-06 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267453+0000", 2024-02-22T19:44:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1401e-05 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267465+0000", 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0000000000000002e-06 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267471+0000", 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.774e-05 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267518+0000", 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7289999999999997e-06 2024-02-22T19:44:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.267524+0000", 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93230813000000001 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.199832+0000", 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000136332 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.199968+0000", 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010382590000000001 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.210351+0000", 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94290151300000002 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1562 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129603190784 gen 4] snapc 0=[] ondisk+write+known_if_redirected e263)", 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:39:09.377123+0000", 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 303.036664358, 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.079811426, 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:44:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1562 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377123+0000", 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377123+0000", 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:44:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377121+0000", 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2539999999999998e-06 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377128+0000", 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2010000000000005e-06 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377137+0000", 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377143+0000", 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0889999999999994e-05 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377214+0000", 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9229999999999999e-06 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:09.377220+0000", 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0677836199999999 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:10.445004+0000", 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000121175 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:10.445125+0000", 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011809748 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:10.456934+0000", 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.079811426 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67443 9.7 9:e92be9d4:::benchmark_data_smithi053_157429_object67442:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 204.03658021800001, 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082204634999999998, 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67443 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377207+0000", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3791999999999998e-05 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377240+0000", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1e-06 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377247+0000", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2690000000000001e-06 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377251+0000", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025018979999999998 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.379753+0000", 2024-02-22T19:44:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000130087 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.379883+0000", 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000156577 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.380039+0000", 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004379371 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384419+0000", 2024-02-22T19:44:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017064900000000001 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384589+0000", 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.074719061000000003 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459308+0000", 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4391000000000002e-05 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459363+0000", 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9033999999999998e-05 2024-02-22T19:44:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459412+0000", 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082204634999999998 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67445 9.6 9:67a14049:::benchmark_data_smithi053_157429_object67444:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 204.03653219099999, 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085338631999999998, 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67445 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377255+0000", 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1210000000000001e-05 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377286+0000", 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4e-06 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377291+0000", 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8519999999999997e-06 2024-02-22T19:44:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377295+0000", 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066316279999999997 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.383927+0000", 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118863 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384046+0000", 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014849399999999999 2024-02-22T19:44:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384194+0000", 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075635162000000006 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459829+0000", 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5648000000000002e-05 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459865+0000", 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026404129999999999 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462505+0000", 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0389999999999997e-05 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462556+0000", 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8062000000000003e-05 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462594+0000", 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085338631999999998 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67446 9.f 9:fd8446b1:::benchmark_data_smithi053_157429_object67445:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 204.03648192099999, 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082320247999999999, 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67446 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377305+0000", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3516000000000002e-05 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377339+0000", 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7749999999999998e-06 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377344+0000", 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9770000000000002e-06 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377348+0000", 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046235599999999996 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.381972+0000", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000132559 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382104+0000", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018340000000000001 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382288+0000", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002577415 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384865+0000", 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.074599304000000005 2024-02-22T19:44:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459465+0000", 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3739999999999995e-05 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459538+0000", 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8214999999999998e-05 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459587+0000", 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9246999999999999e-05 2024-02-22T19:44:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459626+0000", 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.082320247999999999 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67448 9.6 9:6112e6a4:::benchmark_data_smithi053_157429_object67447:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.377353+0000", 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 204.03643464500001, 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085341474, 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67448 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377353+0000", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377353+0000", 2024-02-22T19:44:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.377352+0000", 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051044819999999996 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382457+0000", 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0871e-05 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382468+0000", 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1190000000000002e-06 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382474+0000", 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034600680000000002 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.385934+0000", 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120294 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386054+0000", 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8580999999999997e-05 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.386153+0000", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.073719253999999998 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459872+0000", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1220999999999997e-05 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459913+0000", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00271478 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462628+0000", 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6109999999999998e-05 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462664+0000", 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0173000000000001e-05 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.462694+0000", 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.085341474 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:67450 9.7 9:ee1c16d7:::benchmark_data_smithi053_157429_object67449:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:40:48.382481+0000", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 204.031306202, 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077561816000000006, 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67450 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382481+0000", 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382481+0000", 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382480+0000", 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3385e-05 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382513+0000", 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6250000000000004e-06 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382519+0000", 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1400000000000002e-06 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382523+0000", 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8408e-05 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382552+0000", 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6744999999999998e-05 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382628+0000", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014082299999999999 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.382769+0000", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021010009999999999 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.384870+0000", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.074629951 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459500+0000", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00044630100000000002 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.459946+0000", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8202999999999998e-05 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.460005+0000", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8498000000000003e-05 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:40:48.460043+0000", 2024-02-22T19:44:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077561816000000006 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:83018 9.f 9:f9322158:::benchmark_data_smithi053_157429_object83017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e305)", 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:01.032138+0000", 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 191.38164978899999, 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077364552000000003, 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 83018 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032138+0000", 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032138+0000", 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032137+0000", 2024-02-22T19:44:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0630000000000003e-05 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032168+0000", 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.182e-06 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032175+0000", 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5600000000000004e-06 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.032179+0000", 2024-02-22T19:44:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025713250000000002 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.034751+0000", 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123466 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.034874+0000", 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000185787 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.035060+0000", 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025580569999999999 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.037618+0000", 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064692199999999997 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.038265+0000", 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071143933000000006 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109409+0000", 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4432000000000003e-05 2024-02-22T19:44:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109463+0000", 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8856999999999997e-05 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:01.109502+0000", 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077364552000000003 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:103373 9.f 9:fda213f0:::benchmark_data_smithi053_157429_object103372:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e310)", 2024-02-22T19:44:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:18.096563+0000", 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 174.31722398799999, 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.080430362000000005, 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 103373 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.096563+0000", 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.096563+0000", 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.096563+0000", 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.853e-05 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.096591+0000", 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0379999999999996e-06 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.096597+0000", 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0640000000000004e-06 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.096601+0000", 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.067636476000000001 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.164238+0000", 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000154957 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.164393+0000", 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017596800000000001 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.164569+0000", 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046190479999999997 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.169188+0000", 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003276143 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.172464+0000", 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044416810000000003 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.176906+0000", 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7460000000000003e-05 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.176953+0000", 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0657000000000001e-05 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.176994+0000", 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.080430362000000005 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:104307 9.f 9:f036d773:::benchmark_data_smithi053_157429_object104306:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e310)", 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:18.887996+0000", 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 173.525791884, 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.084928749999999997, 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 104307 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.887996+0000", 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.887996+0000", 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.887995+0000", 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5913000000000001e-05 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.888021+0000", 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5809999999999996e-06 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.888026+0000", 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0990000000000001e-06 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.888031+0000", 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020569659999999999 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.890087+0000", 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8709999999999997e-05 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.890166+0000", 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3906000000000005e-05 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.890260+0000", 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018566120000000001 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.892117+0000", 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055215999999999998 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.897638+0000", 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.075186665999999999 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.972825+0000", 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6063999999999997e-05 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.972881+0000", 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3239999999999999e-05 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.972924+0000", 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.084928749999999997 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:104309 9.5 9:a07c7d76:::benchmark_data_smithi053_157429_object104308:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e310)", 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:18.893287+0000", 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 173.520500459, 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077443528999999997, 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 104309 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893287+0000", 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893287+0000", 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893285+0000", 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.773e-05 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893323+0000", 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.066e-05 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893333+0000", 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2149999999999998e-06 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893340+0000", 2024-02-22T19:44:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5882e-05 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893376+0000", 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012212099999999999 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893498+0000", 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115078 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,7", 2024-02-22T19:44:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.893613+0000", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034352160000000001 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.897048+0000", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018600600000000001 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.897234+0000", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.073417726000000003 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.970652+0000", 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1060000000000003e-05 2024-02-22T19:44:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.970693+0000", 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7713000000000003e-05 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.970730+0000", 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077443528999999997 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15324.0:104314 9.5 9:a26ac49d:::benchmark_data_smithi053_157429_object104313:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e310)", 2024-02-22T19:44:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:41:18.902425+0000", 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 173.51136212399999, 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077742998999999993, 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15324", 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2204631535", 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 104314 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902425+0000", 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902425+0000", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902424+0000", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1938000000000002e-05 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902486+0000", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0498e-05 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:44:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902496+0000", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5459999999999997e-06 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902503+0000", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7879000000000001e-05 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902541+0000", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010881499999999999 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902649+0000", 2024-02-22T19:44:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104368 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,7", 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.902754+0000", 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.068015202999999996 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.970769+0000", 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070953759999999996 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.977864+0000", 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022093350000000002 2024-02-22T19:44:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.980074+0000", 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2308999999999997e-05 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.980126+0000", 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2209000000000002e-05 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:41:18.980168+0000", 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.077742998999999993 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:44:12.516 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:44:12.516 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:44:12.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:12.568 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:12.633 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:12.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:12.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:12.736 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 4 16 10013 9997 156.165 116.938 0.00827951 0.00639084 2024-02-22T19:44:12.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:12.773 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:13.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:44:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:13.134 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:13.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:13.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:13.273 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:13.303 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:13.305 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:13.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.679126+0000 mgr.y (mgr.14535) 582 : cluster [DBG] pgmap v753: 148 pgs: 148 active+clean; 153 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 778 B/s rd, 23 MiB/s wr, 371 op/s 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.710726+0000 osd.1 (osd.1) 212 : cluster [DBG] 4.e scrub starts 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:11.716214+0000 osd.1 (osd.1) 213 : cluster [DBG] 4.e scrub ok 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.038408+0000 osd.3 (osd.3) 247 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.048060+0000 osd.3 (osd.3) 248 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.248166+0000 osd.7 (osd.7) 228 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.295041+0000 osd.7 (osd.7) 229 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:44:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.349190+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e372: 8 total, 8 up, 5 in 2024-02-22T19:44:13.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:13.449 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:13.480 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:13.483 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:13.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:13.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.679126+0000 mgr.y (mgr.14535) 582 : cluster [DBG] pgmap v753: 148 pgs: 148 active+clean; 153 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 778 B/s rd, 23 MiB/s wr, 371 op/s 2024-02-22T19:44:13.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.710726+0000 osd.1 (osd.1) 212 : cluster [DBG] 4.e scrub starts 2024-02-22T19:44:13.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:11.716214+0000 osd.1 (osd.1) 213 : cluster [DBG] 4.e scrub ok 2024-02-22T19:44:13.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.038408+0000 osd.3 (osd.3) 247 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:44:13.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.048060+0000 osd.3 (osd.3) 248 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:44:13.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.248166+0000 osd.7 (osd.7) 228 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.295041+0000 osd.7 (osd.7) 229 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.349190+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e372: 8 total, 8 up, 5 in 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.679126+0000 mgr.y (mgr.14535) 582 : cluster [DBG] pgmap v753: 148 pgs: 148 active+clean; 153 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 778 B/s rd, 23 MiB/s wr, 371 op/s 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.710726+0000 osd.1 (osd.1) 212 : cluster [DBG] 4.e scrub starts 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:11.716214+0000 osd.1 (osd.1) 213 : cluster [DBG] 4.e scrub ok 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.038408+0000 osd.3 (osd.3) 247 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.048060+0000 osd.3 (osd.3) 248 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.248166+0000 osd.7 (osd.7) 228 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.295041+0000 osd.7 (osd.7) 229 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:44:13.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.349190+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e372: 8 total, 8 up, 5 in 2024-02-22T19:44:13.527 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:13.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:13.621 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:13.736 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 5 16 12473 12457 155.679 153.75 0.00698757 0.00641539 2024-02-22T19:44:13.827 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:13.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:13.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:13.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:13.975 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:12.349900+0000 mgr.y (mgr.14535) 583 : audit [DBG] from='client.25466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.673907+0000 osd.5 (osd.5) 122 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:44:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.674848+0000 osd.5 (osd.5) 123 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:44:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.717170+0000 osd.1 (osd.1) 214 : cluster [DBG] 3.a scrub starts 2024-02-22T19:44:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:12.719490+0000 osd.1 (osd.1) 215 : cluster [DBG] 3.a scrub ok 2024-02-22T19:44:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:14.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:14.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:12.349900+0000 mgr.y (mgr.14535) 583 : audit [DBG] from='client.25466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.673907+0000 osd.5 (osd.5) 122 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.674848+0000 osd.5 (osd.5) 123 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.717170+0000 osd.1 (osd.1) 214 : cluster [DBG] 3.a scrub starts 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:12.719490+0000 osd.1 (osd.1) 215 : cluster [DBG] 3.a scrub ok 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:12.349900+0000 mgr.y (mgr.14535) 583 : audit [DBG] from='client.25466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.673907+0000 osd.5 (osd.5) 122 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:44:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.674848+0000 osd.5 (osd.5) 123 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:44:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.717170+0000 osd.1 (osd.1) 214 : cluster [DBG] 3.a scrub starts 2024-02-22T19:44:14.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:12.719490+0000 osd.1 (osd.1) 215 : cluster [DBG] 3.a scrub ok 2024-02-22T19:44:14.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:14.560 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:14.736 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 6 16 14610 14594 151.991 133.562 0.00447139 0.00653535 2024-02-22T19:44:14.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:44:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:44:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:14.918 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:14.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:15.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:15.057 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:15.190 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:15.194 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:15.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:15.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:15.347 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:15.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:13.679477+0000 mgr.y (mgr.14535) 584 : cluster [DBG] pgmap v755: 148 pgs: 148 active+clean; 153 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 779 B/s rd, 23 MiB/s wr, 372 op/s 2024-02-22T19:44:15.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:13.692699+0000 osd.5 (osd.5) 124 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:44:15.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:13.694549+0000 osd.5 (osd.5) 125 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:44:15.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:13.729564+0000 osd.1 (osd.1) 216 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:44:15.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:13.730967+0000 osd.1 (osd.1) 217 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:44:15.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:13.679477+0000 mgr.y (mgr.14535) 584 : cluster [DBG] pgmap v755: 148 pgs: 148 active+clean; 153 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 779 B/s rd, 23 MiB/s wr, 372 op/s 2024-02-22T19:44:15.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:13.692699+0000 osd.5 (osd.5) 124 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:44:15.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:13.694549+0000 osd.5 (osd.5) 125 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:44:15.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:13.729564+0000 osd.1 (osd.1) 216 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:44:15.458 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:13.730967+0000 osd.1 (osd.1) 217 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:44:15.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:15.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:15.492 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:44:15.492 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:44:15.492 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:44:15.492 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:44:15.517 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:44:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:44:15.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:13.679477+0000 mgr.y (mgr.14535) 584 : cluster [DBG] pgmap v755: 148 pgs: 148 active+clean; 153 MiB data, 3.0 GiB used, 444 GiB / 447 GiB avail; 779 B/s rd, 23 MiB/s wr, 372 op/s 2024-02-22T19:44:15.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:13.692699+0000 osd.5 (osd.5) 124 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:44:15.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:13.694549+0000 osd.5 (osd.5) 125 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:44:15.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:13.729564+0000 osd.1 (osd.1) 216 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:44:15.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:13.730967+0000 osd.1 (osd.1) 217 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:44:15.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:15.612 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:15.736 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 7 16 16790 16774 149.74 136.25 0.00464201 0.00667205 2024-02-22T19:44:15.875 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:15.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:15.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:16.036 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:16.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:14.657542+0000 osd.5 (osd.5) 126 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:44:16.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:14.661318+0000 osd.5 (osd.5) 127 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:44:16.188 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:15.157239+0000 mgr.y (mgr.14535) 585 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:16.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:44:16.351 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":754,"stamp":"2024-02-22T19:44:12.349465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":160044708,"num_objects":2634,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":2011,"num_read_kb":1763,"num_write":3156,"num_write_kb":156329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":15766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3264,"ondisk_log_size":3264,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":3129628,"kb_used_data":159348,"kb_used_omap":0,"kb_used_meta":2970240,"kb_avail":465596132,"statfs":{"total":479975178240,"available":476770439168,"internally_reserved":0,"allocated":163172352,"data_stored":156929637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3041525760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160038912,"num_objects":2442,"num_object_clones":0,"num_object_copies":7326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2442,"num_write_kb":156288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.672262"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":328,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.793482+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:44:07.793482+0000","last_peered":"2024-02-22T19:44:07.793482+0000","last_clean":"2024-02-22T19:44:07.793482+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:44:07.793482+0000","last_undegraded":"2024-02-22T19:44:07.793482+0000","last_fullsized":"2024-02-22T19:44:07.793482+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":363,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794948+0000","last_change":"2024-02-22T19:43:58.780165+0000","last_active":"2024-02-22T19:44:07.794948+0000","last_peered":"2024-02-22T19:44:07.794948+0000","last_clean":"2024-02-22T19:44:07.794948+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:44:07.794948+0000","last_undegraded":"2024-02-22T19:44:07.794948+0000","last_fullsized":"2024-02-22T19:44:07.794948+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.780100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.780100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":142,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114451+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:44:07.114451+0000","last_peered":"2024-02-22T19:44:07.114451+0000","last_clean":"2024-02-22T19:44:07.114451+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:44:07.114451+0000","last_undegraded":"2024-02-22T19:44:07.114451+0000","last_fullsized":"2024-02-22T19:44:07.114451+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":343,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.169023+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:44:07.169023+0000","last_peered":"2024-02-22T19:44:07.169023+0000","last_clean":"2024-02-22T19:44:07.169023+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:44:07.169023+0000","last_undegraded":"2024-02-22T19:44:07.169023+0000","last_fullsized":"2024-02-22T19:44:07.169023+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":344,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167773+0000","last_change":"2024-02-22T19:41:52.112817+0000","last_active":"2024-02-22T19:44:07.167773+0000","last_peered":"2024-02-22T19:44:07.167773+0000","last_clean":"2024-02-22T19:44:07.167773+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:44:07.167773+0000","last_undegraded":"2024-02-22T19:44:07.167773+0000","last_fullsized":"2024-02-22T19:44:07.167773+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":456,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.984847+0000","last_change":"2024-02-22T19:44:07.984802+0000","last_active":"2024-02-22T19:44:07.984847+0000","last_peered":"2024-02-22T19:44:07.984847+0000","last_clean":"2024-02-22T19:44:07.984847+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:44:07.984847+0000","last_undegraded":"2024-02-22T19:44:07.984847+0000","last_fullsized":"2024-02-22T19:44:07.984847+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:44:07.984710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.984710+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":172,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.111673+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:44:07.111673+0000","last_peered":"2024-02-22T19:44:07.111673+0000","last_clean":"2024-02-22T19:44:07.111673+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:44:07.111673+0000","last_undegraded":"2024-02-22T19:44:07.111673+0000","last_fullsized":"2024-02-22T19:44:07.111673+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":186,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.795072+0000","last_change":"2024-02-22T19:44:07.795043+0000","last_active":"2024-02-22T19:44:07.795072+0000","last_peered":"2024-02-22T19:44:07.795072+0000","last_clean":"2024-02-22T19:44:07.795072+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:44:07.795072+0000","last_undegraded":"2024-02-22T19:44:07.795072+0000","last_fullsized":"2024-02-22T19:44:07.795072+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:07.794943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.794943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":248,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167752+0000","last_change":"2024-02-22T19:41:52.752324+0000","last_active":"2024-02-22T19:44:07.167752+0000","last_peered":"2024-02-22T19:44:07.167752+0000","last_clean":"2024-02-22T19:44:07.167752+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:44:07.167752+0000","last_undegraded":"2024-02-22T19:44:07.167752+0000","last_fullsized":"2024-02-22T19:44:07.167752+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":168,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794021+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:44:07.794021+0000","last_peered":"2024-02-22T19:44:07.794021+0000","last_clean":"2024-02-22T19:44:07.794021+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:44:07.794021+0000","last_undegraded":"2024-02-22T19:44:07.794021+0000","last_fullsized":"2024-02-22T19:44:07.794021+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":198,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.795056+0000","last_change":"2024-02-22T19:41:52.116983+0000","last_active":"2024-02-22T19:44:07.795056+0000","last_peered":"2024-02-22T19:44:07.795056+0000","last_clean":"2024-02-22T19:44:07.795056+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:44:07.795056+0000","last_undegraded":"2024-02-22T19:44:07.795056+0000","last_fullsized":"2024-02-22T19:44:07.795056+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":176,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166432+0000","last_change":"2024-02-22T19:44:04.949624+0000","last_active":"2024-02-22T19:44:07.166432+0000","last_peered":"2024-02-22T19:44:07.166432+0000","last_clean":"2024-02-22T19:44:07.166432+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:44:07.166432+0000","last_undegraded":"2024-02-22T19:44:07.166432+0000","last_fullsized":"2024-02-22T19:44:07.166432+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:44:04.949558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.949558+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794776+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:44:07.794776+0000","last_peered":"2024-02-22T19:44:07.794776+0000","last_clean":"2024-02-22T19:44:07.794776+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:44:07.794776+0000","last_undegraded":"2024-02-22T19:44:07.794776+0000","last_fullsized":"2024-02-22T19:44:07.794776+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":145,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303049+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:44:07.303049+0000","last_peered":"2024-02-22T19:44:07.303049+0000","last_clean":"2024-02-22T19:44:07.303049+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:44:07.303049+0000","last_undegraded":"2024-02-22T19:44:07.303049+0000","last_fullsized":"2024-02-22T19:44:07.303049+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":403,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.172658+0000","last_change":"2024-02-22T19:44:07.172636+0000","last_active":"2024-02-22T19:44:07.172658+0000","last_peered":"2024-02-22T19:44:07.172658+0000","last_clean":"2024-02-22T19:44:07.172658+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:44:07.172658+0000","last_undegraded":"2024-02-22T19:44:07.172658+0000","last_fullsized":"2024-02-22T19:44:07.172658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:44:07.172562+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.172562+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":361,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168421+0000","last_change":"2024-02-22T19:41:52.121546+0000","last_active":"2024-02-22T19:44:07.168421+0000","last_peered":"2024-02-22T19:44:07.168421+0000","last_clean":"2024-02-22T19:44:07.168421+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:44:07.168421+0000","last_undegraded":"2024-02-22T19:44:07.168421+0000","last_fullsized":"2024-02-22T19:44:07.168421+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":158,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112292+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:44:07.112292+0000","last_peered":"2024-02-22T19:44:07.112292+0000","last_clean":"2024-02-22T19:44:07.112292+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:44:07.112292+0000","last_undegraded":"2024-02-22T19:44:07.112292+0000","last_fullsized":"2024-02-22T19:44:07.112292+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":412,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167952+0000","last_change":"2024-02-22T19:41:53.187026+0000","last_active":"2024-02-22T19:44:07.167952+0000","last_peered":"2024-02-22T19:44:07.167952+0000","last_clean":"2024-02-22T19:44:07.167952+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:44:07.167952+0000","last_undegraded":"2024-02-22T19:44:07.167952+0000","last_fullsized":"2024-02-22T19:44:07.167952+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":382,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303335+0000","last_change":"2024-02-22T19:44:04.315337+0000","last_active":"2024-02-22T19:44:07.303335+0000","last_peered":"2024-02-22T19:44:07.303335+0000","last_clean":"2024-02-22T19:44:07.303335+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:44:07.303335+0000","last_undegraded":"2024-02-22T19:44:07.303335+0000","last_fullsized":"2024-02-22T19:44:07.303335+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:44:04.315265+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.315265+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":366,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112270+0000","last_change":"2024-02-22T19:41:52.110049+0000","last_active":"2024-02-22T19:44:07.112270+0000","last_peered":"2024-02-22T19:44:07.112270+0000","last_clean":"2024-02-22T19:44:07.112270+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:44:07.112270+0000","last_undegraded":"2024-02-22T19:44:07.112270+0000","last_fullsized":"2024-02-22T19:44:07.112270+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":178,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.304082+0000","last_change":"2024-02-22T19:39:32.702632+0000","last_active":"2024-02-22T19:44:07.304082+0000","last_peered":"2024-02-22T19:44:07.304082+0000","last_clean":"2024-02-22T19:44:07.304082+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:44:07.304082+0000","last_undegraded":"2024-02-22T19:44:07.304082+0000","last_fullsized":"2024-02-22T19:44:07.304082+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:32.702536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:32.702536+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":366,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114190+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:44:07.114190+0000","last_peered":"2024-02-22T19:44:07.114190+0000","last_clean":"2024-02-22T19:44:07.114190+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:44:07.114190+0000","last_undegraded":"2024-02-22T19:44:07.114190+0000","last_fullsized":"2024-02-22T19:44:07.114190+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":364,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166747+0000","last_change":"2024-02-22T19:39:50.696741+0000","last_active":"2024-02-22T19:44:07.166747+0000","last_peered":"2024-02-22T19:44:07.166747+0000","last_clean":"2024-02-22T19:44:07.166747+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:44:07.166747+0000","last_undegraded":"2024-02-22T19:44:07.166747+0000","last_fullsized":"2024-02-22T19:44:07.166747+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:50.696676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:39:50.696676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":406,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.795107+0000","last_change":"2024-02-22T19:41:52.743206+0000","last_active":"2024-02-22T19:44:07.795107+0000","last_peered":"2024-02-22T19:44:07.795107+0000","last_clean":"2024-02-22T19:44:07.795107+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:44:07.795107+0000","last_undegraded":"2024-02-22T19:44:07.795107+0000","last_fullsized":"2024-02-22T19:44:07.795107+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":351,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302391+0000","last_change":"2024-02-22T19:43:59.334492+0000","last_active":"2024-02-22T19:44:07.302391+0000","last_peered":"2024-02-22T19:44:07.302391+0000","last_clean":"2024-02-22T19:44:07.302391+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:44:07.302391+0000","last_undegraded":"2024-02-22T19:44:07.302391+0000","last_fullsized":"2024-02-22T19:44:07.302391+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.334420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.334420+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":311,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302610+0000","last_change":"2024-02-22T19:41:52.111808+0000","last_active":"2024-02-22T19:44:07.302610+0000","last_peered":"2024-02-22T19:44:07.302610+0000","last_clean":"2024-02-22T19:44:07.302610+0000","last_became_active":"2024-02-22T19:41:52.110758+0000","last_became_peered":"2024-02-22T19:41:52.110758+0000","last_unstale":"2024-02-22T19:44:07.302610+0000","last_undegraded":"2024-02-22T19:44:07.302610+0000","last_fullsized":"2024-02-22T19:44:07.302610+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":358,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794835+0000","last_change":"2024-02-22T19:41:52.114607+0000","last_active":"2024-02-22T19:44:07.794835+0000","last_peered":"2024-02-22T19:44:07.794835+0000","last_clean":"2024-02-22T19:44:07.794835+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:44:07.794835+0000","last_undegraded":"2024-02-22T19:44:07.794835+0000","last_fullsized":"2024-02-22T19:44:07.794835+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:18.925508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:18.925508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":365,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166830+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:44:07.166830+0000","last_peered":"2024-02-22T19:44:07.166830+0000","last_clean":"2024-02-22T19:44:07.166830+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:44:07.166830+0000","last_undegraded":"2024-02-22T19:44:07.166830+0000","last_fullsized":"2024-02-22T19:44:07.166830+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":372,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.169123+0000","last_change":"2024-02-22T19:43:59.063011+0000","last_active":"2024-02-22T19:44:07.169123+0000","last_peered":"2024-02-22T19:44:07.169123+0000","last_clean":"2024-02-22T19:44:07.169123+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:44:07.169123+0000","last_undegraded":"2024-02-22T19:44:07.169123+0000","last_fullsized":"2024-02-22T19:44:07.169123+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.062912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.062912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":378,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302915+0000","last_change":"2024-02-22T19:41:52.104771+0000","last_active":"2024-02-22T19:44:07.302915+0000","last_peered":"2024-02-22T19:44:07.302915+0000","last_clean":"2024-02-22T19:44:07.302915+0000","last_became_active":"2024-02-22T19:41:52.104234+0000","last_became_peered":"2024-02-22T19:41:52.104234+0000","last_unstale":"2024-02-22T19:44:07.302915+0000","last_undegraded":"2024-02-22T19:44:07.302915+0000","last_fullsized":"2024-02-22T19:44:07.302915+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.702590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.702590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":163,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794695+0000","last_change":"2024-02-22T19:37:57.183391+0000","last_active":"2024-02-22T19:44:07.794695+0000","last_peered":"2024-02-22T19:44:07.794695+0000","last_clean":"2024-02-22T19:44:07.794695+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:44:07.794695+0000","last_undegraded":"2024-02-22T19:44:07.794695+0000","last_fullsized":"2024-02-22T19:44:07.794695+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:08.454846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:37:08.454846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":385,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167061+0000","last_change":"2024-02-22T19:41:53.508908+0000","last_active":"2024-02-22T19:44:07.167061+0000","last_peered":"2024-02-22T19:44:07.167061+0000","last_clean":"2024-02-22T19:44:07.167061+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:44:07.167061+0000","last_undegraded":"2024-02-22T19:44:07.167061+0000","last_fullsized":"2024-02-22T19:44:07.167061+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:37.816661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.816661+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":349,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794011+0000","last_change":"2024-02-22T19:41:52.112028+0000","last_active":"2024-02-22T19:44:07.794011+0000","last_peered":"2024-02-22T19:44:07.794011+0000","last_clean":"2024-02-22T19:44:07.794011+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:44:07.794011+0000","last_undegraded":"2024-02-22T19:44:07.794011+0000","last_fullsized":"2024-02-22T19:44:07.794011+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:37:04.719706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.719706+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":363,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168965+0000","last_change":"2024-02-22T19:44:05.908463+0000","last_active":"2024-02-22T19:44:07.168965+0000","last_peered":"2024-02-22T19:44:07.168965+0000","last_clean":"2024-02-22T19:44:07.168965+0000","last_became_active":"2024-02-22T19:39:22.244098+0000","last_became_peered":"2024-02-22T19:39:22.244098+0000","last_unstale":"2024-02-22T19:44:07.168965+0000","last_undegraded":"2024-02-22T19:44:07.168965+0000","last_fullsized":"2024-02-22T19:44:07.168965+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:05.908382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:05.908382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":386,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303450+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:44:07.303450+0000","last_peered":"2024-02-22T19:44:07.303450+0000","last_clean":"2024-02-22T19:44:07.303450+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:44:07.303450+0000","last_undegraded":"2024-02-22T19:44:07.303450+0000","last_fullsized":"2024-02-22T19:44:07.303450+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":390,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112824+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:44:07.112824+0000","last_peered":"2024-02-22T19:44:07.112824+0000","last_clean":"2024-02-22T19:44:07.112824+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:44:07.112824+0000","last_undegraded":"2024-02-22T19:44:07.112824+0000","last_fullsized":"2024-02-22T19:44:07.112824+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":379,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166756+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:44:07.166756+0000","last_peered":"2024-02-22T19:44:07.166756+0000","last_clean":"2024-02-22T19:44:07.166756+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:44:07.166756+0000","last_undegraded":"2024-02-22T19:44:07.166756+0000","last_fullsized":"2024-02-22T19:44:07.166756+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":357,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112654+0000","last_change":"2024-02-22T19:40:45.398799+0000","last_active":"2024-02-22T19:44:07.112654+0000","last_peered":"2024-02-22T19:44:07.112654+0000","last_clean":"2024-02-22T19:44:07.112654+0000","last_became_active":"2024-02-22T19:37:57.187161+0000","last_became_peered":"2024-02-22T19:37:57.187161+0000","last_unstale":"2024-02-22T19:44:07.112654+0000","last_undegraded":"2024-02-22T19:44:07.112654+0000","last_fullsized":"2024-02-22T19:44:07.112654+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":149,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303253+0000","last_change":"2024-02-22T19:41:52.104077+0000","last_active":"2024-02-22T19:44:07.303253+0000","last_peered":"2024-02-22T19:44:07.303253+0000","last_clean":"2024-02-22T19:44:07.303253+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:44:07.303253+0000","last_undegraded":"2024-02-22T19:44:07.303253+0000","last_fullsized":"2024-02-22T19:44:07.303253+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":153,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112433+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:44:07.112433+0000","last_peered":"2024-02-22T19:44:07.112433+0000","last_clean":"2024-02-22T19:44:07.112433+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:44:07.112433+0000","last_undegraded":"2024-02-22T19:44:07.112433+0000","last_fullsized":"2024-02-22T19:44:07.112433+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":166,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.304025+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:44:07.304025+0000","last_peered":"2024-02-22T19:44:07.304025+0000","last_clean":"2024-02-22T19:44:07.304025+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:44:07.304025+0000","last_undegraded":"2024-02-22T19:44:07.304025+0000","last_fullsized":"2024-02-22T19:44:07.304025+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"268'14","reported_seq":284,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.740338+0000","last_change":"2024-02-22T19:39:22.455660+0000","last_active":"2024-02-22T19:44:08.740338+0000","last_peered":"2024-02-22T19:44:08.740338+0000","last_clean":"2024-02-22T19:44:08.740338+0000","last_became_active":"2024-02-22T19:39:22.249801+0000","last_became_peered":"2024-02-22T19:39:22.249801+0000","last_unstale":"2024-02-22T19:44:08.740338+0000","last_undegraded":"2024-02-22T19:44:08.740338+0000","last_fullsized":"2024-02-22T19:44:08.740338+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"183'10","last_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_deep_scrub":"183'10","last_deep_scrub_stamp":"2024-02-22T19:37:04.973730+0000","last_clean_scrub_stamp":"2024-02-22T19:37:04.973730+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":359,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302948+0000","last_change":"2024-02-22T19:41:52.103816+0000","last_active":"2024-02-22T19:44:07.302948+0000","last_peered":"2024-02-22T19:44:07.302948+0000","last_clean":"2024-02-22T19:44:07.302948+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:44:07.302948+0000","last_undegraded":"2024-02-22T19:44:07.302948+0000","last_fullsized":"2024-02-22T19:44:07.302948+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":373,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168573+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:44:07.168573+0000","last_peered":"2024-02-22T19:44:07.168573+0000","last_clean":"2024-02-22T19:44:07.168573+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:44:07.168573+0000","last_undegraded":"2024-02-22T19:44:07.168573+0000","last_fullsized":"2024-02-22T19:44:07.168573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":351,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115019+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:44:07.115019+0000","last_peered":"2024-02-22T19:44:07.115019+0000","last_clean":"2024-02-22T19:44:07.115019+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:44:07.115019+0000","last_undegraded":"2024-02-22T19:44:07.115019+0000","last_fullsized":"2024-02-22T19:44:07.115019+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":521,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.747540+0000","last_change":"2024-02-22T19:40:38.671802+0000","last_active":"2024-02-22T19:44:08.747540+0000","last_peered":"2024-02-22T19:44:08.747540+0000","last_clean":"2024-02-22T19:44:08.747540+0000","last_became_active":"2024-02-22T19:29:05.095560+0000","last_became_peered":"2024-02-22T19:29:05.095560+0000","last_unstale":"2024-02-22T19:44:08.747540+0000","last_undegraded":"2024-02-22T19:44:08.747540+0000","last_fullsized":"2024-02-22T19:44:08.747540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-22T19:40:38.671691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:38.671691+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":157,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302831+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:44:07.302831+0000","last_peered":"2024-02-22T19:44:07.302831+0000","last_clean":"2024-02-22T19:44:07.302831+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:44:07.302831+0000","last_undegraded":"2024-02-22T19:44:07.302831+0000","last_fullsized":"2024-02-22T19:44:07.302831+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":187,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794722+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:44:07.794722+0000","last_peered":"2024-02-22T19:44:07.794722+0000","last_clean":"2024-02-22T19:44:07.794722+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:44:07.794722+0000","last_undegraded":"2024-02-22T19:44:07.794722+0000","last_fullsized":"2024-02-22T19:44:07.794722+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":374,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115626+0000","last_change":"2024-02-22T19:41:38.526359+0000","last_active":"2024-02-22T19:44:07.115626+0000","last_peered":"2024-02-22T19:44:07.115626+0000","last_clean":"2024-02-22T19:44:07.115626+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:44:07.115626+0000","last_undegraded":"2024-02-22T19:44:07.115626+0000","last_fullsized":"2024-02-22T19:44:07.115626+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.526259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.526259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":382,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302722+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:44:07.302722+0000","last_peered":"2024-02-22T19:44:07.302722+0000","last_clean":"2024-02-22T19:44:07.302722+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:44:07.302722+0000","last_undegraded":"2024-02-22T19:44:07.302722+0000","last_fullsized":"2024-02-22T19:44:07.302722+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":159,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167930+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:44:07.167930+0000","last_peered":"2024-02-22T19:44:07.167930+0000","last_clean":"2024-02-22T19:44:07.167930+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:44:07.167930+0000","last_undegraded":"2024-02-22T19:44:07.167930+0000","last_fullsized":"2024-02-22T19:44:07.167930+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":384,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303369+0000","last_change":"2024-02-22T19:43:57.440300+0000","last_active":"2024-02-22T19:44:07.303369+0000","last_peered":"2024-02-22T19:44:07.303369+0000","last_clean":"2024-02-22T19:44:07.303369+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:44:07.303369+0000","last_undegraded":"2024-02-22T19:44:07.303369+0000","last_fullsized":"2024-02-22T19:44:07.303369+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.440217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.440217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":335,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.793904+0000","last_change":"2024-02-22T19:41:52.112840+0000","last_active":"2024-02-22T19:44:07.793904+0000","last_peered":"2024-02-22T19:44:07.793904+0000","last_clean":"2024-02-22T19:44:07.793904+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:44:07.793904+0000","last_undegraded":"2024-02-22T19:44:07.793904+0000","last_fullsized":"2024-02-22T19:44:07.793904+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":373,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168445+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:44:07.168445+0000","last_peered":"2024-02-22T19:44:07.168445+0000","last_clean":"2024-02-22T19:44:07.168445+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:44:07.168445+0000","last_undegraded":"2024-02-22T19:44:07.168445+0000","last_fullsized":"2024-02-22T19:44:07.168445+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"371'137","reported_seq":151,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.445330+0000","last_change":"2024-02-22T19:43:57.347864+0000","last_active":"2024-02-22T19:44:09.445330+0000","last_peered":"2024-02-22T19:44:09.445330+0000","last_clean":"2024-02-22T19:44:09.445330+0000","last_became_active":"2024-02-22T19:43:57.347537+0000","last_became_peered":"2024-02-22T19:43:57.347537+0000","last_unstale":"2024-02-22T19:44:09.445330+0000","last_undegraded":"2024-02-22T19:44:09.445330+0000","last_fullsized":"2024-02-22T19:44:09.445330+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":137,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8978432,"num_objects":137,"num_object_clones":0,"num_object_copies":411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":137,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":204,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794737+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:44:07.794737+0000","last_peered":"2024-02-22T19:44:07.794737+0000","last_clean":"2024-02-22T19:44:07.794737+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:44:07.794737+0000","last_undegraded":"2024-02-22T19:44:07.794737+0000","last_fullsized":"2024-02-22T19:44:07.794737+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":436,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.271111+0000","last_change":"2024-02-22T19:44:08.271084+0000","last_active":"2024-02-22T19:44:08.271111+0000","last_peered":"2024-02-22T19:44:08.271111+0000","last_clean":"2024-02-22T19:44:08.271111+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:44:08.271111+0000","last_undegraded":"2024-02-22T19:44:08.271111+0000","last_fullsized":"2024-02-22T19:44:08.271111+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:44:08.271004+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:44:08.271004+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":227,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.793786+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:44:07.793786+0000","last_peered":"2024-02-22T19:44:07.793786+0000","last_clean":"2024-02-22T19:44:07.793786+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:44:07.793786+0000","last_undegraded":"2024-02-22T19:44:07.793786+0000","last_fullsized":"2024-02-22T19:44:07.793786+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":603,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.655723+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:44:08.655723+0000","last_peered":"2024-02-22T19:44:08.655723+0000","last_clean":"2024-02-22T19:44:08.655723+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:44:08.655723+0000","last_undegraded":"2024-02-22T19:44:08.655723+0000","last_fullsized":"2024-02-22T19:44:08.655723+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"371'181","reported_seq":194,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.729518+0000","last_change":"2024-02-22T19:43:57.350818+0000","last_active":"2024-02-22T19:44:09.729518+0000","last_peered":"2024-02-22T19:44:09.729518+0000","last_clean":"2024-02-22T19:44:09.729518+0000","last_became_active":"2024-02-22T19:43:57.350467+0000","last_became_peered":"2024-02-22T19:43:57.350467+0000","last_unstale":"2024-02-22T19:44:09.729518+0000","last_undegraded":"2024-02-22T19:44:09.729518+0000","last_fullsized":"2024-02-22T19:44:09.729518+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":181,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11862016,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":329,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.169065+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:44:07.169065+0000","last_peered":"2024-02-22T19:44:07.169065+0000","last_clean":"2024-02-22T19:44:07.169065+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:44:07.169065+0000","last_undegraded":"2024-02-22T19:44:07.169065+0000","last_fullsized":"2024-02-22T19:44:07.169065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":73,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115135+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:44:07.115135+0000","last_peered":"2024-02-22T19:44:07.115135+0000","last_clean":"2024-02-22T19:44:07.115135+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:44:07.115135+0000","last_undegraded":"2024-02-22T19:44:07.115135+0000","last_fullsized":"2024-02-22T19:44:07.115135+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":217,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112070+0000","last_change":"2024-02-22T19:41:52.597069+0000","last_active":"2024-02-22T19:44:07.112070+0000","last_peered":"2024-02-22T19:44:07.112070+0000","last_clean":"2024-02-22T19:44:07.112070+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:44:07.112070+0000","last_undegraded":"2024-02-22T19:44:07.112070+0000","last_fullsized":"2024-02-22T19:44:07.112070+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":389,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115020+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:44:07.115020+0000","last_peered":"2024-02-22T19:44:07.115020+0000","last_clean":"2024-02-22T19:44:07.115020+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:44:07.115020+0000","last_undegraded":"2024-02-22T19:44:07.115020+0000","last_fullsized":"2024-02-22T19:44:07.115020+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"371'186","reported_seq":200,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.724550+0000","last_change":"2024-02-22T19:43:57.349932+0000","last_active":"2024-02-22T19:44:09.724550+0000","last_peered":"2024-02-22T19:44:09.724550+0000","last_clean":"2024-02-22T19:44:09.724550+0000","last_became_active":"2024-02-22T19:43:57.349636+0000","last_became_peered":"2024-02-22T19:43:57.349636+0000","last_unstale":"2024-02-22T19:44:09.724550+0000","last_undegraded":"2024-02-22T19:44:09.724550+0000","last_fullsized":"2024-02-22T19:44:09.724550+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":186,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12189696,"num_objects":186,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":11904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":335,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302509+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:44:07.302509+0000","last_peered":"2024-02-22T19:44:07.302509+0000","last_clean":"2024-02-22T19:44:07.302509+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:44:07.302509+0000","last_undegraded":"2024-02-22T19:44:07.302509+0000","last_fullsized":"2024-02-22T19:44:07.302509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":415,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.113592+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:44:07.113592+0000","last_peered":"2024-02-22T19:44:07.113592+0000","last_clean":"2024-02-22T19:44:07.113592+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:44:07.113592+0000","last_undegraded":"2024-02-22T19:44:07.113592+0000","last_fullsized":"2024-02-22T19:44:07.113592+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":373,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303676+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:44:07.303676+0000","last_peered":"2024-02-22T19:44:07.303676+0000","last_clean":"2024-02-22T19:44:07.303676+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:44:07.303676+0000","last_undegraded":"2024-02-22T19:44:07.303676+0000","last_fullsized":"2024-02-22T19:44:07.303676+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":364,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112578+0000","last_change":"2024-02-22T19:41:52.135450+0000","last_active":"2024-02-22T19:44:07.112578+0000","last_peered":"2024-02-22T19:44:07.112578+0000","last_clean":"2024-02-22T19:44:07.112578+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:44:07.112578+0000","last_undegraded":"2024-02-22T19:44:07.112578+0000","last_fullsized":"2024-02-22T19:44:07.112578+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:40:31.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.504365+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"371'162","reported_seq":176,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.588633+0000","last_change":"2024-02-22T19:43:57.347961+0000","last_active":"2024-02-22T19:44:09.588633+0000","last_peered":"2024-02-22T19:44:09.588633+0000","last_clean":"2024-02-22T19:44:09.588633+0000","last_became_active":"2024-02-22T19:43:57.347657+0000","last_became_peered":"2024-02-22T19:43:57.347657+0000","last_unstale":"2024-02-22T19:44:09.588633+0000","last_undegraded":"2024-02-22T19:44:09.588633+0000","last_fullsized":"2024-02-22T19:44:09.588633+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":162,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10616832,"num_objects":162,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":157,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302869+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:44:07.302869+0000","last_peered":"2024-02-22T19:44:07.302869+0000","last_clean":"2024-02-22T19:44:07.302869+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:44:07.302869+0000","last_undegraded":"2024-02-22T19:44:07.302869+0000","last_fullsized":"2024-02-22T19:44:07.302869+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":394,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112945+0000","last_change":"2024-02-22T19:41:16.236762+0000","last_active":"2024-02-22T19:44:07.112945+0000","last_peered":"2024-02-22T19:44:07.112945+0000","last_clean":"2024-02-22T19:44:07.112945+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:44:07.112945+0000","last_undegraded":"2024-02-22T19:44:07.112945+0000","last_fullsized":"2024-02-22T19:44:07.112945+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.236664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.236664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":177,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.795003+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:44:07.795003+0000","last_peered":"2024-02-22T19:44:07.795003+0000","last_clean":"2024-02-22T19:44:07.795003+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:44:07.795003+0000","last_undegraded":"2024-02-22T19:44:07.795003+0000","last_fullsized":"2024-02-22T19:44:07.795003+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":169,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.113800+0000","last_change":"2024-02-22T19:40:48.125201+0000","last_active":"2024-02-22T19:44:07.113800+0000","last_peered":"2024-02-22T19:44:07.113800+0000","last_clean":"2024-02-22T19:44:07.113800+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:44:07.113800+0000","last_undegraded":"2024-02-22T19:44:07.113800+0000","last_fullsized":"2024-02-22T19:44:07.113800+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.125098+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"371'123","reported_seq":137,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.446801+0000","last_change":"2024-02-22T19:43:57.351947+0000","last_active":"2024-02-22T19:44:09.446801+0000","last_peered":"2024-02-22T19:44:09.446801+0000","last_clean":"2024-02-22T19:44:09.446801+0000","last_became_active":"2024-02-22T19:43:57.351647+0000","last_became_peered":"2024-02-22T19:43:57.351647+0000","last_unstale":"2024-02-22T19:44:09.446801+0000","last_undegraded":"2024-02-22T19:44:09.446801+0000","last_fullsized":"2024-02-22T19:44:09.446801+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":123,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8060928,"num_objects":123,"num_object_clones":0,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.793943+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:44:07.793943+0000","last_peered":"2024-02-22T19:44:07.793943+0000","last_clean":"2024-02-22T19:44:07.793943+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:44:07.793943+0000","last_undegraded":"2024-02-22T19:44:07.793943+0000","last_fullsized":"2024-02-22T19:44:07.793943+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":352,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168019+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:44:07.168019+0000","last_peered":"2024-02-22T19:44:07.168019+0000","last_clean":"2024-02-22T19:44:07.168019+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:44:07.168019+0000","last_undegraded":"2024-02-22T19:44:07.168019+0000","last_fullsized":"2024-02-22T19:44:07.168019+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":177,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794613+0000","last_change":"2024-02-22T19:43:57.722289+0000","last_active":"2024-02-22T19:44:07.794613+0000","last_peered":"2024-02-22T19:44:07.794613+0000","last_clean":"2024-02-22T19:44:07.794613+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:44:07.794613+0000","last_undegraded":"2024-02-22T19:44:07.794613+0000","last_fullsized":"2024-02-22T19:44:07.794613+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.722226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.722226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":287,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112060+0000","last_change":"2024-02-22T19:41:52.113125+0000","last_active":"2024-02-22T19:44:07.112060+0000","last_peered":"2024-02-22T19:44:07.112060+0000","last_clean":"2024-02-22T19:44:07.112060+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:44:07.112060+0000","last_undegraded":"2024-02-22T19:44:07.112060+0000","last_fullsized":"2024-02-22T19:44:07.112060+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.569155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.569155+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":602,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.345187+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:44:08.345187+0000","last_peered":"2024-02-22T19:44:08.345187+0000","last_clean":"2024-02-22T19:44:08.345187+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:44:08.345187+0000","last_undegraded":"2024-02-22T19:44:08.345187+0000","last_fullsized":"2024-02-22T19:44:08.345187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":175,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114078+0000","last_change":"2024-02-22T19:41:52.101375+0000","last_active":"2024-02-22T19:44:07.114078+0000","last_peered":"2024-02-22T19:44:07.114078+0000","last_clean":"2024-02-22T19:44:07.114078+0000","last_became_active":"2024-02-22T19:41:52.101126+0000","last_became_peered":"2024-02-22T19:41:52.101126+0000","last_unstale":"2024-02-22T19:44:07.114078+0000","last_undegraded":"2024-02-22T19:44:07.114078+0000","last_fullsized":"2024-02-22T19:44:07.114078+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:46.299686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.299686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"371'143","reported_seq":157,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.426957+0000","last_change":"2024-02-22T19:43:57.347092+0000","last_active":"2024-02-22T19:44:09.426957+0000","last_peered":"2024-02-22T19:44:09.426957+0000","last_clean":"2024-02-22T19:44:09.426957+0000","last_became_active":"2024-02-22T19:43:57.346743+0000","last_became_peered":"2024-02-22T19:43:57.346743+0000","last_unstale":"2024-02-22T19:44:09.426957+0000","last_undegraded":"2024-02-22T19:44:09.426957+0000","last_fullsized":"2024-02-22T19:44:09.426957+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":143,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9371648,"num_objects":143,"num_object_clones":0,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":394,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302938+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:44:07.302938+0000","last_peered":"2024-02-22T19:44:07.302938+0000","last_clean":"2024-02-22T19:44:07.302938+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:44:07.302938+0000","last_undegraded":"2024-02-22T19:44:07.302938+0000","last_fullsized":"2024-02-22T19:44:07.302938+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":344,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303865+0000","last_change":"2024-02-22T19:41:52.109613+0000","last_active":"2024-02-22T19:44:07.303865+0000","last_peered":"2024-02-22T19:44:07.303865+0000","last_clean":"2024-02-22T19:44:07.303865+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:44:07.303865+0000","last_undegraded":"2024-02-22T19:44:07.303865+0000","last_fullsized":"2024-02-22T19:44:07.303865+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:47.644913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:39:47.644913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":357,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114213+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:44:07.114213+0000","last_peered":"2024-02-22T19:44:07.114213+0000","last_clean":"2024-02-22T19:44:07.114213+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:44:07.114213+0000","last_undegraded":"2024-02-22T19:44:07.114213+0000","last_fullsized":"2024-02-22T19:44:07.114213+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114889+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:44:07.114889+0000","last_peered":"2024-02-22T19:44:07.114889+0000","last_clean":"2024-02-22T19:44:07.114889+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:44:07.114889+0000","last_undegraded":"2024-02-22T19:44:07.114889+0000","last_fullsized":"2024-02-22T19:44:07.114889+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":445,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115155+0000","last_change":"2024-02-22T19:41:20.253383+0000","last_active":"2024-02-22T19:44:07.115155+0000","last_peered":"2024-02-22T19:44:07.115155+0000","last_clean":"2024-02-22T19:44:07.115155+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:44:07.115155+0000","last_undegraded":"2024-02-22T19:44:07.115155+0000","last_fullsized":"2024-02-22T19:44:07.115155+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:41:20.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:20.253277+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":388,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115372+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:44:07.115372+0000","last_peered":"2024-02-22T19:44:07.115372+0000","last_clean":"2024-02-22T19:44:07.115372+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:44:07.115372+0000","last_undegraded":"2024-02-22T19:44:07.115372+0000","last_fullsized":"2024-02-22T19:44:07.115372+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"371'139","reported_seq":153,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.553432+0000","last_change":"2024-02-22T19:43:57.344606+0000","last_active":"2024-02-22T19:44:09.553432+0000","last_peered":"2024-02-22T19:44:09.553432+0000","last_clean":"2024-02-22T19:44:09.553432+0000","last_became_active":"2024-02-22T19:43:57.344303+0000","last_became_peered":"2024-02-22T19:43:57.344303+0000","last_unstale":"2024-02-22T19:44:09.553432+0000","last_undegraded":"2024-02-22T19:44:09.553432+0000","last_fullsized":"2024-02-22T19:44:09.553432+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":139,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9109504,"num_objects":139,"num_object_clones":0,"num_object_copies":417,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":8896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":177,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.793557+0000","last_change":"2024-02-22T19:39:48.479212+0000","last_active":"2024-02-22T19:44:07.793557+0000","last_peered":"2024-02-22T19:44:07.793557+0000","last_clean":"2024-02-22T19:44:07.793557+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:44:07.793557+0000","last_undegraded":"2024-02-22T19:44:07.793557+0000","last_fullsized":"2024-02-22T19:44:07.793557+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.479132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.479132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":394,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302612+0000","last_change":"2024-02-22T19:41:52.104562+0000","last_active":"2024-02-22T19:44:07.302612+0000","last_peered":"2024-02-22T19:44:07.302612+0000","last_clean":"2024-02-22T19:44:07.302612+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:44:07.302612+0000","last_undegraded":"2024-02-22T19:44:07.302612+0000","last_fullsized":"2024-02-22T19:44:07.302612+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":367,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.027439+0000","last_change":"2024-02-22T19:44:09.027439+0000","last_active":"2024-02-22T19:44:09.027439+0000","last_peered":"2024-02-22T19:44:09.027439+0000","last_clean":"2024-02-22T19:44:09.027439+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:44:09.027439+0000","last_undegraded":"2024-02-22T19:44:09.027439+0000","last_fullsized":"2024-02-22T19:44:09.027439+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.027320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":402,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303538+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:44:07.303538+0000","last_peered":"2024-02-22T19:44:07.303538+0000","last_clean":"2024-02-22T19:44:07.303538+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:44:07.303538+0000","last_undegraded":"2024-02-22T19:44:07.303538+0000","last_fullsized":"2024-02-22T19:44:07.303538+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":355,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303331+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:44:07.303331+0000","last_peered":"2024-02-22T19:44:07.303331+0000","last_clean":"2024-02-22T19:44:07.303331+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:44:07.303331+0000","last_undegraded":"2024-02-22T19:44:07.303331+0000","last_fullsized":"2024-02-22T19:44:07.303331+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"371'129","reported_seq":143,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.586959+0000","last_change":"2024-02-22T19:43:57.346858+0000","last_active":"2024-02-22T19:44:09.586959+0000","last_peered":"2024-02-22T19:44:09.586959+0000","last_clean":"2024-02-22T19:44:09.586959+0000","last_became_active":"2024-02-22T19:43:57.346554+0000","last_became_peered":"2024-02-22T19:43:57.346554+0000","last_unstale":"2024-02-22T19:44:09.586959+0000","last_undegraded":"2024-02-22T19:44:09.586959+0000","last_fullsized":"2024-02-22T19:44:09.586959+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":129,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":248,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112289+0000","last_change":"2024-02-22T19:41:52.114247+0000","last_active":"2024-02-22T19:44:07.112289+0000","last_peered":"2024-02-22T19:44:07.112289+0000","last_clean":"2024-02-22T19:44:07.112289+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:44:07.112289+0000","last_undegraded":"2024-02-22T19:44:07.112289+0000","last_fullsized":"2024-02-22T19:44:07.112289+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":360,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.113293+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:44:07.113293+0000","last_peered":"2024-02-22T19:44:07.113293+0000","last_clean":"2024-02-22T19:44:07.113293+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:44:07.113293+0000","last_undegraded":"2024-02-22T19:44:07.113293+0000","last_fullsized":"2024-02-22T19:44:07.113293+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":145,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168490+0000","last_change":"2024-02-22T19:39:54.620621+0000","last_active":"2024-02-22T19:44:07.168490+0000","last_peered":"2024-02-22T19:44:07.168490+0000","last_clean":"2024-02-22T19:44:07.168490+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:44:07.168490+0000","last_undegraded":"2024-02-22T19:44:07.168490+0000","last_fullsized":"2024-02-22T19:44:07.168490+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.620512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.620512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1289,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.655492+0000","last_change":"2024-02-22T19:39:45.659239+0000","last_active":"2024-02-22T19:44:08.655492+0000","last_peered":"2024-02-22T19:44:08.655492+0000","last_clean":"2024-02-22T19:44:08.655492+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:44:08.655492+0000","last_undegraded":"2024-02-22T19:44:08.655492+0000","last_fullsized":"2024-02-22T19:44:08.655492+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:39:45.659143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.659143+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":891,"num_read_kb":886,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":177,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114477+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:44:07.114477+0000","last_peered":"2024-02-22T19:44:07.114477+0000","last_clean":"2024-02-22T19:44:07.114477+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:44:07.114477+0000","last_undegraded":"2024-02-22T19:44:07.114477+0000","last_fullsized":"2024-02-22T19:44:07.114477+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":222,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.795555+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:44:07.795555+0000","last_peered":"2024-02-22T19:44:07.795555+0000","last_clean":"2024-02-22T19:44:07.795555+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:44:07.795555+0000","last_undegraded":"2024-02-22T19:44:07.795555+0000","last_fullsized":"2024-02-22T19:44:07.795555+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"371'175","reported_seq":189,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.720806+0000","last_change":"2024-02-22T19:43:57.347743+0000","last_active":"2024-02-22T19:44:09.720806+0000","last_peered":"2024-02-22T19:44:09.720806+0000","last_clean":"2024-02-22T19:44:09.720806+0000","last_became_active":"2024-02-22T19:43:57.347455+0000","last_became_peered":"2024-02-22T19:43:57.347455+0000","last_unstale":"2024-02-22T19:44:09.720806+0000","last_undegraded":"2024-02-22T19:44:09.720806+0000","last_fullsized":"2024-02-22T19:44:09.720806+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":175,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11468800,"num_objects":175,"num_object_clones":0,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":11200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":224,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167692+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:44:07.167692+0000","last_peered":"2024-02-22T19:44:07.167692+0000","last_clean":"2024-02-22T19:44:07.167692+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:44:07.167692+0000","last_undegraded":"2024-02-22T19:44:07.167692+0000","last_fullsized":"2024-02-22T19:44:07.167692+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":346,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112153+0000","last_change":"2024-02-22T19:41:52.094945+0000","last_active":"2024-02-22T19:44:07.112153+0000","last_peered":"2024-02-22T19:44:07.112153+0000","last_clean":"2024-02-22T19:44:07.112153+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:44:07.112153+0000","last_undegraded":"2024-02-22T19:44:07.112153+0000","last_fullsized":"2024-02-22T19:44:07.112153+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":361,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.113559+0000","last_change":"2024-02-22T19:39:48.361494+0000","last_active":"2024-02-22T19:44:07.113559+0000","last_peered":"2024-02-22T19:44:07.113559+0000","last_clean":"2024-02-22T19:44:07.113559+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:44:07.113559+0000","last_undegraded":"2024-02-22T19:44:07.113559+0000","last_fullsized":"2024-02-22T19:44:07.113559+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:39:48.361427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":182,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.795009+0000","last_change":"2024-02-22T19:39:42.355299+0000","last_active":"2024-02-22T19:44:07.795009+0000","last_peered":"2024-02-22T19:44:07.795009+0000","last_clean":"2024-02-22T19:44:07.795009+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:44:07.795009+0000","last_undegraded":"2024-02-22T19:44:07.795009+0000","last_fullsized":"2024-02-22T19:44:07.795009+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:39:42.355182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:42.355182+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":331,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794332+0000","last_change":"2024-02-22T19:41:52.107671+0000","last_active":"2024-02-22T19:44:07.794332+0000","last_peered":"2024-02-22T19:44:07.794332+0000","last_clean":"2024-02-22T19:44:07.794332+0000","last_became_active":"2024-02-22T19:41:52.106767+0000","last_became_peered":"2024-02-22T19:41:52.106767+0000","last_unstale":"2024-02-22T19:44:07.794332+0000","last_undegraded":"2024-02-22T19:44:07.794332+0000","last_fullsized":"2024-02-22T19:44:07.794332+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"371'129","reported_seq":143,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.400984+0000","last_change":"2024-02-22T19:43:57.351727+0000","last_active":"2024-02-22T19:44:09.400984+0000","last_peered":"2024-02-22T19:44:09.400984+0000","last_clean":"2024-02-22T19:44:09.400984+0000","last_became_active":"2024-02-22T19:43:57.351385+0000","last_became_peered":"2024-02-22T19:43:57.351385+0000","last_unstale":"2024-02-22T19:44:09.400984+0000","last_undegraded":"2024-02-22T19:44:09.400984+0000","last_fullsized":"2024-02-22T19:44:09.400984+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":129,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168181+0000","last_change":"2024-02-22T19:41:52.110027+0000","last_active":"2024-02-22T19:44:07.168181+0000","last_peered":"2024-02-22T19:44:07.168181+0000","last_clean":"2024-02-22T19:44:07.168181+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:44:07.168181+0000","last_undegraded":"2024-02-22T19:44:07.168181+0000","last_fullsized":"2024-02-22T19:44:07.168181+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.670303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":157,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166671+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:44:07.166671+0000","last_peered":"2024-02-22T19:44:07.166671+0000","last_clean":"2024-02-22T19:44:07.166671+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:44:07.166671+0000","last_undegraded":"2024-02-22T19:44:07.166671+0000","last_fullsized":"2024-02-22T19:44:07.166671+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":168,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794612+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:44:07.794612+0000","last_peered":"2024-02-22T19:44:07.794612+0000","last_clean":"2024-02-22T19:44:07.794612+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:44:07.794612+0000","last_undegraded":"2024-02-22T19:44:07.794612+0000","last_fullsized":"2024-02-22T19:44:07.794612+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":481,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112736+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:44:07.112736+0000","last_peered":"2024-02-22T19:44:07.112736+0000","last_clean":"2024-02-22T19:44:07.112736+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:44:07.112736+0000","last_undegraded":"2024-02-22T19:44:07.112736+0000","last_fullsized":"2024-02-22T19:44:07.112736+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"326'14","reported_seq":399,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.739468+0000","last_change":"2024-02-22T19:41:52.698274+0000","last_active":"2024-02-22T19:44:08.739468+0000","last_peered":"2024-02-22T19:44:08.739468+0000","last_clean":"2024-02-22T19:44:08.739468+0000","last_became_active":"2024-02-22T19:41:52.112897+0000","last_became_peered":"2024-02-22T19:41:52.112897+0000","last_unstale":"2024-02-22T19:44:08.739468+0000","last_undegraded":"2024-02-22T19:44:08.739468+0000","last_fullsized":"2024-02-22T19:44:08.739468+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"371'195","reported_seq":209,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.729975+0000","last_change":"2024-02-22T19:43:57.348552+0000","last_active":"2024-02-22T19:44:09.729975+0000","last_peered":"2024-02-22T19:44:09.729975+0000","last_clean":"2024-02-22T19:44:09.729975+0000","last_became_active":"2024-02-22T19:43:57.348228+0000","last_became_peered":"2024-02-22T19:43:57.348228+0000","last_unstale":"2024-02-22T19:44:09.729975+0000","last_undegraded":"2024-02-22T19:44:09.729975+0000","last_fullsized":"2024-02-22T19:44:09.729975+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":195,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12779520,"num_objects":195,"num_object_clones":0,"num_object_copies":585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":195,"num_write_kb":12480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":253,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168353+0000","last_change":"2024-02-22T19:43:58.000275+0000","last_active":"2024-02-22T19:44:07.168353+0000","last_peered":"2024-02-22T19:44:07.168353+0000","last_clean":"2024-02-22T19:44:07.168353+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:44:07.168353+0000","last_undegraded":"2024-02-22T19:44:07.168353+0000","last_fullsized":"2024-02-22T19:44:07.168353+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.000163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.000163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":347,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.303143+0000","last_change":"2024-02-22T19:41:52.103942+0000","last_active":"2024-02-22T19:44:07.303143+0000","last_peered":"2024-02-22T19:44:07.303143+0000","last_clean":"2024-02-22T19:44:07.303143+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:44:07.303143+0000","last_undegraded":"2024-02-22T19:44:07.303143+0000","last_fullsized":"2024-02-22T19:44:07.303143+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":166,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114382+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:44:07.114382+0000","last_peered":"2024-02-22T19:44:07.114382+0000","last_clean":"2024-02-22T19:44:07.114382+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:44:07.114382+0000","last_undegraded":"2024-02-22T19:44:07.114382+0000","last_fullsized":"2024-02-22T19:44:07.114382+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":483,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794114+0000","last_change":"2024-02-22T19:41:52.715987+0000","last_active":"2024-02-22T19:44:07.794114+0000","last_peered":"2024-02-22T19:44:07.794114+0000","last_clean":"2024-02-22T19:44:07.794114+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:44:07.794114+0000","last_undegraded":"2024-02-22T19:44:07.794114+0000","last_fullsized":"2024-02-22T19:44:07.794114+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":562,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.742959+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:44:08.742959+0000","last_peered":"2024-02-22T19:44:08.742959+0000","last_clean":"2024-02-22T19:44:08.742959+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:44:08.742959+0000","last_undegraded":"2024-02-22T19:44:08.742959+0000","last_fullsized":"2024-02-22T19:44:08.742959+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"371'160","reported_seq":174,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.580158+0000","last_change":"2024-02-22T19:43:57.348310+0000","last_active":"2024-02-22T19:44:09.580158+0000","last_peered":"2024-02-22T19:44:09.580158+0000","last_clean":"2024-02-22T19:44:09.580158+0000","last_became_active":"2024-02-22T19:43:57.348032+0000","last_became_peered":"2024-02-22T19:43:57.348032+0000","last_unstale":"2024-02-22T19:44:09.580158+0000","last_undegraded":"2024-02-22T19:44:09.580158+0000","last_fullsized":"2024-02-22T19:44:09.580158+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":160,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10485760,"num_objects":160,"num_object_clones":0,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":155,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114127+0000","last_change":"2024-02-22T19:43:58.006321+0000","last_active":"2024-02-22T19:44:07.114127+0000","last_peered":"2024-02-22T19:44:07.114127+0000","last_clean":"2024-02-22T19:44:07.114127+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:44:07.114127+0000","last_undegraded":"2024-02-22T19:44:07.114127+0000","last_fullsized":"2024-02-22T19:44:07.114127+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.006221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.006221+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":149,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167891+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:44:07.167891+0000","last_peered":"2024-02-22T19:44:07.167891+0000","last_clean":"2024-02-22T19:44:07.167891+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:44:07.167891+0000","last_undegraded":"2024-02-22T19:44:07.167891+0000","last_fullsized":"2024-02-22T19:44:07.167891+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":367,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168275+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:44:07.168275+0000","last_peered":"2024-02-22T19:44:07.168275+0000","last_clean":"2024-02-22T19:44:07.168275+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:44:07.168275+0000","last_undegraded":"2024-02-22T19:44:07.168275+0000","last_fullsized":"2024-02-22T19:44:07.168275+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":471,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.167055+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:44:07.167055+0000","last_peered":"2024-02-22T19:44:07.167055+0000","last_clean":"2024-02-22T19:44:07.167055+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:44:07.167055+0000","last_undegraded":"2024-02-22T19:44:07.167055+0000","last_fullsized":"2024-02-22T19:44:07.167055+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":158,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166977+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:44:07.166977+0000","last_peered":"2024-02-22T19:44:07.166977+0000","last_clean":"2024-02-22T19:44:07.166977+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:44:07.166977+0000","last_undegraded":"2024-02-22T19:44:07.166977+0000","last_fullsized":"2024-02-22T19:44:07.166977+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"371'116","reported_seq":130,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.395470+0000","last_change":"2024-02-22T19:43:57.347525+0000","last_active":"2024-02-22T19:44:09.395470+0000","last_peered":"2024-02-22T19:44:09.395470+0000","last_clean":"2024-02-22T19:44:09.395470+0000","last_became_active":"2024-02-22T19:43:57.347224+0000","last_became_peered":"2024-02-22T19:43:57.347224+0000","last_unstale":"2024-02-22T19:44:09.395470+0000","last_undegraded":"2024-02-22T19:44:09.395470+0000","last_fullsized":"2024-02-22T19:44:09.395470+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":116,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":180,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794839+0000","last_change":"2024-02-22T19:41:52.107922+0000","last_active":"2024-02-22T19:44:07.794839+0000","last_peered":"2024-02-22T19:44:07.794839+0000","last_clean":"2024-02-22T19:44:07.794839+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:44:07.794839+0000","last_undegraded":"2024-02-22T19:44:07.794839+0000","last_fullsized":"2024-02-22T19:44:07.794839+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":195,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794489+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:44:07.794489+0000","last_peered":"2024-02-22T19:44:07.794489+0000","last_clean":"2024-02-22T19:44:07.794489+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:44:07.794489+0000","last_undegraded":"2024-02-22T19:44:07.794489+0000","last_fullsized":"2024-02-22T19:44:07.794489+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"294'26","reported_seq":677,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.738519+0000","last_change":"2024-02-22T19:40:34.372311+0000","last_active":"2024-02-22T19:44:08.738519+0000","last_peered":"2024-02-22T19:44:08.738519+0000","last_clean":"2024-02-22T19:44:08.738519+0000","last_became_active":"2024-02-22T19:40:24.667075+0000","last_became_peered":"2024-02-22T19:40:24.667075+0000","last_unstale":"2024-02-22T19:44:08.738519+0000","last_undegraded":"2024-02-22T19:44:08.738519+0000","last_fullsized":"2024-02-22T19:44:08.738519+0000","mapping_epoch":293,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":340,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.166971+0000","last_change":"2024-02-22T19:39:51.680888+0000","last_active":"2024-02-22T19:44:07.166971+0000","last_peered":"2024-02-22T19:44:07.166971+0000","last_clean":"2024-02-22T19:44:07.166971+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:44:07.166971+0000","last_undegraded":"2024-02-22T19:44:07.166971+0000","last_fullsized":"2024-02-22T19:44:07.166971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:51.680817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:51.680817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":84,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.794918+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:44:07.794918+0000","last_peered":"2024-02-22T19:44:07.794918+0000","last_clean":"2024-02-22T19:44:07.794918+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:44:07.794918+0000","last_undegraded":"2024-02-22T19:44:07.794918+0000","last_fullsized":"2024-02-22T19:44:07.794918+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"371'159","reported_seq":173,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.552222+0000","last_change":"2024-02-22T19:43:57.346250+0000","last_active":"2024-02-22T19:44:09.552222+0000","last_peered":"2024-02-22T19:44:09.552222+0000","last_clean":"2024-02-22T19:44:09.552222+0000","last_became_active":"2024-02-22T19:43:57.345992+0000","last_became_peered":"2024-02-22T19:43:57.345992+0000","last_unstale":"2024-02-22T19:44:09.552222+0000","last_undegraded":"2024-02-22T19:44:09.552222+0000","last_fullsized":"2024-02-22T19:44:09.552222+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":159,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10420224,"num_objects":159,"num_object_clones":0,"num_object_copies":477,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":10176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":356,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168489+0000","last_change":"2024-02-22T19:41:52.112553+0000","last_active":"2024-02-22T19:44:07.168489+0000","last_peered":"2024-02-22T19:44:07.168489+0000","last_clean":"2024-02-22T19:44:07.168489+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:44:07.168489+0000","last_undegraded":"2024-02-22T19:44:07.168489+0000","last_fullsized":"2024-02-22T19:44:07.168489+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"326'20","reported_seq":423,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.740738+0000","last_change":"2024-02-22T19:41:54.322898+0000","last_active":"2024-02-22T19:44:08.740738+0000","last_peered":"2024-02-22T19:44:08.740738+0000","last_clean":"2024-02-22T19:44:08.740738+0000","last_became_active":"2024-02-22T19:41:54.322531+0000","last_became_peered":"2024-02-22T19:41:54.322531+0000","last_unstale":"2024-02-22T19:44:08.740738+0000","last_undegraded":"2024-02-22T19:44:08.740738+0000","last_fullsized":"2024-02-22T19:44:08.740738+0000","mapping_epoch":325,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":150,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.115069+0000","last_change":"2024-02-22T19:44:06.172481+0000","last_active":"2024-02-22T19:44:07.115069+0000","last_peered":"2024-02-22T19:44:07.115069+0000","last_clean":"2024-02-22T19:44:07.115069+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:44:07.115069+0000","last_undegraded":"2024-02-22T19:44:07.115069+0000","last_fullsized":"2024-02-22T19:44:07.115069+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:06.172363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:06.172363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":153,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114468+0000","last_change":"2024-02-22T19:41:52.358275+0000","last_active":"2024-02-22T19:44:07.114468+0000","last_peered":"2024-02-22T19:44:07.114468+0000","last_clean":"2024-02-22T19:44:07.114468+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:44:07.114468+0000","last_undegraded":"2024-02-22T19:44:07.114468+0000","last_fullsized":"2024-02-22T19:44:07.114468+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:39:43.589477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:43.589477+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3","version":"371'153","reported_seq":167,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.577689+0000","last_change":"2024-02-22T19:43:57.348166+0000","last_active":"2024-02-22T19:44:09.577689+0000","last_peered":"2024-02-22T19:44:09.577689+0000","last_clean":"2024-02-22T19:44:09.577689+0000","last_became_active":"2024-02-22T19:43:57.347832+0000","last_became_peered":"2024-02-22T19:43:57.347832+0000","last_unstale":"2024-02-22T19:44:09.577689+0000","last_undegraded":"2024-02-22T19:44:09.577689+0000","last_fullsized":"2024-02-22T19:44:09.577689+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":153,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10027008,"num_objects":153,"num_object_clones":0,"num_object_copies":459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":9792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":168,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.793906+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:44:07.793906+0000","last_peered":"2024-02-22T19:44:07.793906+0000","last_clean":"2024-02-22T19:44:07.793906+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:44:07.793906+0000","last_undegraded":"2024-02-22T19:44:07.793906+0000","last_fullsized":"2024-02-22T19:44:07.793906+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":365,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114934+0000","last_change":"2024-02-22T19:44:05.228957+0000","last_active":"2024-02-22T19:44:07.114934+0000","last_peered":"2024-02-22T19:44:07.114934+0000","last_clean":"2024-02-22T19:44:07.114934+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:44:07.114934+0000","last_undegraded":"2024-02-22T19:44:07.114934+0000","last_fullsized":"2024-02-22T19:44:07.114934+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:05.228870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:05.228870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":409,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.113847+0000","last_change":"2024-02-22T19:44:06.753861+0000","last_active":"2024-02-22T19:44:07.113847+0000","last_peered":"2024-02-22T19:44:07.113847+0000","last_clean":"2024-02-22T19:44:07.113847+0000","last_became_active":"2024-02-22T19:41:52.092045+0000","last_became_peered":"2024-02-22T19:41:52.092045+0000","last_unstale":"2024-02-22T19:44:07.113847+0000","last_undegraded":"2024-02-22T19:44:07.113847+0000","last_fullsized":"2024-02-22T19:44:07.113847+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:06.753748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:06.753748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":277,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.112374+0000","last_change":"2024-02-22T19:41:52.110141+0000","last_active":"2024-02-22T19:44:07.112374+0000","last_peered":"2024-02-22T19:44:07.112374+0000","last_clean":"2024-02-22T19:44:07.112374+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:44:07.112374+0000","last_undegraded":"2024-02-22T19:44:07.112374+0000","last_fullsized":"2024-02-22T19:44:07.112374+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:45.615346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:45.615346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":251,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.132549+0000","last_change":"2024-02-22T19:44:09.132549+0000","last_active":"2024-02-22T19:44:09.132549+0000","last_peered":"2024-02-22T19:44:09.132549+0000","last_clean":"2024-02-22T19:44:09.132549+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:44:09.132549+0000","last_undegraded":"2024-02-22T19:44:09.132549+0000","last_fullsized":"2024-02-22T19:44:09.132549+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:44:09.132441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.132441+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"371'155","reported_seq":169,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:09.556868+0000","last_change":"2024-02-22T19:43:57.351793+0000","last_active":"2024-02-22T19:44:09.556868+0000","last_peered":"2024-02-22T19:44:09.556868+0000","last_clean":"2024-02-22T19:44:09.556868+0000","last_became_active":"2024-02-22T19:43:57.351420+0000","last_became_peered":"2024-02-22T19:43:57.351420+0000","last_unstale":"2024-02-22T19:44:09.556868+0000","last_undegraded":"2024-02-22T19:44:09.556868+0000","last_fullsized":"2024-02-22T19:44:09.556868+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":155,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10158080,"num_objects":155,"num_object_clones":0,"num_object_copies":465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":358,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.302986+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:44:07.302986+0000","last_peered":"2024-02-22T19:44:07.302986+0000","last_clean":"2024-02-22T19:44:07.302986+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:44:07.302986+0000","last_undegraded":"2024-02-22T19:44:07.302986+0000","last_fullsized":"2024-02-22T19:44:07.302986+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":184,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.114297+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:44:07.114297+0000","last_peered":"2024-02-22T19:44:07.114297+0000","last_clean":"2024-02-22T19:44:07.114297+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:44:07.114297+0000","last_undegraded":"2024-02-22T19:44:07.114297+0000","last_fullsized":"2024-02-22T19:44:07.114297+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"221'14","reported_seq":601,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-02-22T19:44:08.347461+0000","last_change":"2024-02-22T19:41:32.453253+0000","last_active":"2024-02-22T19:44:08.347461+0000","last_peered":"2024-02-22T19:44:08.347461+0000","last_clean":"2024-02-22T19:44:08.347461+0000","last_became_active":"2024-02-22T19:38:00.220168+0000","last_became_peered":"2024-02-22T19:38:00.220168+0000","last_unstale":"2024-02-22T19:44:08.347461+0000","last_undegraded":"2024-02-22T19:44:08.347461+0000","last_fullsized":"2024-02-22T19:44:08.347461+0000","mapping_epoch":220,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":401,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168909+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:44:07.168909+0000","last_peered":"2024-02-22T19:44:07.168909+0000","last_clean":"2024-02-22T19:44:07.168909+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:44:07.168909+0000","last_undegraded":"2024-02-22T19:44:07.168909+0000","last_fullsized":"2024-02-22T19:44:07.168909+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":215,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-02-22T19:44:07.168636+0000","last_change":"2024-02-22T19:41:52.575055+0000","last_active":"2024-02-22T19:44:07.168636+0000","last_peered":"2024-02-22T19:44:07.168636+0000","last_clean":"2024-02-22T19:44:07.168636+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:44:07.168636+0000","last_undegraded":"2024-02-22T19:44:07.168636+0000","last_fullsized":"2024-02-22T19:44:07.168636+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":160038912,"num_objects":2442,"num_object_clones":0,"num_object_copies":7326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":156288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":470286336,"data_stored":470286336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2442,"ondisk_log_size":2442,"up":48,"acting":48,"num_store_stats":5},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":891,"num_read_kb":886,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":22,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":66,"acting":66,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266873,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":651212,"kb_used_data":57412,"kb_used_omap":0,"kb_used_meta":593792,"kb_avail":93093940,"statfs":{"total":95995035648,"available":95328194560,"internally_reserved":0,"allocated":58789888,"data_stored":57548416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":608043008},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.2490000000000001,"5min":1.532,"15min":1.5029999999999999},"min":{"1min":0.249,"5min":0.245,"15min":0.19900000000000001},"max":{"1min":7.9790000000000001,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.379,"15min":1.421},"min":{"1min":0.25600000000000001,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":3.25,"5min":8.0489999999999995,"15min":42.155999999999999},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":2.4940000000000002,"15min":1.7529999999999999},"min":{"1min":0.28399999999999997,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":6.1200000000000001,"5min":54.951000000000001,"15min":54.951000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.218,"5min":2.5049999999999999,"15min":1.921},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":3.613,"5min":57.012,"15min":57.012},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.825,"15min":1.4370000000000001},"min":{"1min":0.27200000000000002,"5min":0.27000000000000002,"15min":0.23999999999999999},"max":{"1min":5.1200000000000001,"5min":40.606000000000002,"15min":41.640999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.3769999999999998,"15min":1.7310000000000001},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.23599999999999999},"max":{"1min":5.0960000000000001,"5min":38.411000000000001,"15min":41.722000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":2.6459999999999999,"15min":1.861},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":7.1289999999999996,"5min":38.692999999999998,"15min":41.457000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":3.1739999999999999,"5min":2.8090000000000002,"15min":1.9139999999999999},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":9.4039999999999999,"5min":38.219000000000001,"15min":41.311999999999998},"last":0.437}]},{"osd":4,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.004,"15min":0.70599999999999996},"min":{"1min":0.31,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":4.4219999999999997,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.258,"15min":0.755},"min":{"1min":0.29899999999999999,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":5.0910000000000002,"5min":7.6219999999999999,"15min":7.6219999999999999},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.419,"15min":0.86199999999999999},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.27000000000000002},"max":{"1min":6.625,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.093,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.28000000000000003},"max":{"1min":7.3239999999999998,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.004,"15min":0.81000000000000005},"min":{"1min":0.32300000000000001,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":3.044,"5min":6.5,"15min":12.199999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":0.90800000000000003},"min":{"1min":0.36899999999999999,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":3.6680000000000001,"5min":7.6390000000000002,"15min":12.067},"last":0.51400000000000001}]}]},{"osd":6,"up_from":0,"seq":193273528510,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575063,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":592408,"kb_used_data":2704,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93152744,"statfs":{"total":95995035648,"available":95388409856,"internally_reserved":0,"allocated":2768896,"data_stored":1515112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:44:05 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":2.331,"15min":1.921},"min":{"1min":0.245,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":6.6760000000000002,"5min":51.091000000000001,"15min":51.091000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":1.081,"5min":2.2269999999999999,"15min":1.9930000000000001},"min":{"1min":0.215,"5min":0.20999999999999999,"15min":0.17499999999999999},"max":{"1min":4.9370000000000003,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":5.4669999999999996,"5min":2.4630000000000001,"15min":1.9710000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":51.027000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":4.7220000000000004,"5min":2.4950000000000001,"15min":2.077},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":51.369999999999997,"5min":51.369999999999997,"15min":51.369999999999997},"last":1.345}]},{"osd":3,"last update":"Thu Feb 22 19:43:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":2.4119999999999999,"15min":2.145},"min":{"1min":0.28999999999999998,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":5.8040000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":0.375},{"interface":"front","average":{"1min":1.5700000000000001,"5min":2.5419999999999998,"15min":2.206},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":6.6449999999999996,"5min":51.741,"15min":51.741},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:43:10 2024","interfaces":[{"interface":"back","average":{"1min":6.1319999999999997,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.32600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":51.158999999999999,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.753},{"interface":"front","average":{"1min":3.843,"5min":1.853,"15min":1.385},"min":{"1min":0.311,"5min":0.311,"15min":0.245},"max":{"1min":51.276000000000003,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Feb 22 19:43:32 2024","interfaces":[{"interface":"back","average":{"1min":5.1959999999999997,"5min":2.7109999999999999,"15min":1.8009999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":51.183999999999997,"5min":51.183999999999997,"15min":51.183999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":4.1459999999999999,"5min":2.2629999999999999,"15min":1.6100000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":51.212000000000003,"5min":51.212000000000003,"15min":51.212000000000003},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:43:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.8069999999999999,"15min":1.4770000000000001},"min":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":5.1879999999999997,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.954,"5min":2.544,"15min":1.78},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.34999999999999998},"max":{"1min":8.4269999999999996,"5min":51.119999999999997,"15min":51.119999999999997},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:43:16 2024","interfaces":[{"interface":"back","average":{"1min":4.7080000000000002,"5min":2.5830000000000002,"15min":1.784},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":51.244999999999997,"5min":51.244999999999997,"15min":51.244999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":4.4050000000000002,"5min":1.8540000000000001,"15min":1.454},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":51.317999999999998,"5min":51.317999999999998,"15min":51.317999999999998},"last":1.4079999999999999}]}]},{"osd":0,"up_from":0,"seq":34359738588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411537,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":359748,"kb_used_data":58428,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93385404,"statfs":{"total":95995035648,"available":95626653696,"internally_reserved":0,"allocated":59830272,"data_stored":58595259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":2.0699999999999998,"15min":1.706},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.18099999999999999},"max":{"1min":4.7000000000000002,"5min":16.763999999999999,"15min":22.338999999999999},"last":0.217},{"interface":"front","average":{"1min":2.2789999999999999,"5min":2.5640000000000001,"15min":2.1120000000000001},"min":{"1min":0.26700000000000002,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":14.603999999999999,"5min":30.427,"15min":30.427},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.8560000000000001,"15min":1.679},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.0819999999999999,"5min":12.349,"15min":12.349},"last":0.27400000000000002},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.889,"15min":1.585},"min":{"1min":0.215,"5min":0.215,"15min":0.193},"max":{"1min":5.1900000000000004,"5min":13.782,"15min":13.782},"last":0.249}]},{"osd":3,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":2.0379999999999998,"15min":2.0379999999999998},"min":{"1min":0.27000000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":8.8599999999999994,"5min":16.076000000000001,"15min":30.071000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.4940000000000002,"15min":2.3239999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.184},"max":{"1min":9.3149999999999995,"5min":28.263999999999999,"15min":32.119},"last":0.31}]},{"osd":4,"last update":"Thu Feb 22 19:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.357,"15min":1.3899999999999999},"min":{"1min":0.41299999999999998,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":5.3390000000000004,"5min":12.43,"15min":12.43},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.4890000000000001,"15min":1.27},"min":{"1min":0.38600000000000001,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":2.117,"5min":10.34,"15min":10.34},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:43:29 2024","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":2.2879999999999998,"15min":1.4690000000000001},"min":{"1min":0.28499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.867,"5min":40.633000000000003,"15min":40.633000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.964,"5min":2.3980000000000001,"15min":1.6859999999999999},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":6.2999999999999998,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:44:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.655,"15min":1.5309999999999999},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.311},"max":{"1min":5.3730000000000002,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.505},{"interface":"front","average":{"1min":1.099,"5min":1.6339999999999999,"15min":1.335},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.29899999999999999},"max":{"1min":2.6440000000000001,"5min":10.369999999999999,"15min":10.369999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Feb 22 19:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.736,"15min":1.5049999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.1580000000000004,"5min":12.484,"15min":12.484},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.9299999999999999,"15min":1.407},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":4.5999999999999996,"5min":10.539,"15min":10.539},"last":0.497}]}]},{"osd":3,"up_from":26,"seq":111669149899,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":930136,"kb_used_data":34384,"kb_used_omap":0,"kb_used_meta":895744,"kb_avail":92815016,"statfs":{"total":95995035648,"available":95042576384,"internally_reserved":0,"allocated":35209216,"data_stored":33955325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":917241856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.9410000000000001,"15min":1.901},"min":{"1min":0.26800000000000002,"5min":0.191,"15min":0.191},"max":{"1min":6.3220000000000001,"5min":9.9789999999999992,"15min":23.48},"last":0.36399999999999999},{"interface":"front","average":{"1min":2.2000000000000002,"5min":2.2370000000000001,"15min":2.2280000000000002},"min":{"1min":0.24199999999999999,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":10.297000000000001,"5min":18.172000000000001,"15min":23.411999999999999},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":2.4169999999999998,"5min":2.8530000000000002,"15min":2.5569999999999999},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":11.385999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":1.51,"5min":2.2639999999999998,"15min":2.044},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":6.8040000000000003,"5min":22.187000000000001,"15min":22.187000000000001},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":2.2050000000000001,"5min":2.2650000000000001,"15min":2.1080000000000001},"min":{"1min":0.28199999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.577,"5min":20.105,"15min":20.105},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":2.4660000000000002,"15min":2.0779999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.217},"max":{"1min":6.8390000000000004,"5min":35.939999999999998,"15min":35.939999999999998},"last":0.443}]},{"osd":4,"last update":"Thu Feb 22 19:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.9450000000000001,"15min":1.804},"min":{"1min":0.38100000000000001,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":5.3760000000000003,"5min":12.416,"15min":13.680999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.508,"15min":1.621},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.27400000000000002},"max":{"1min":7.6479999999999997,"5min":12.273,"15min":13.548},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Feb 22 19:43:35 2024","interfaces":[{"interface":"back","average":{"1min":2.1099999999999999,"5min":1.9570000000000001,"15min":1.5800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1159999999999997,"5min":11.161,"15min":12.879},"last":0.79100000000000004},{"interface":"front","average":{"1min":2.7080000000000002,"5min":2.2029999999999998,"15min":1.7829999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":10.041,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Feb 22 19:43:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.879,"15min":1.573},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":6.3109999999999999,"5min":11.263,"15min":12.898999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":2.0840000000000001,"5min":2.1320000000000001,"15min":1.867},"min":{"1min":0.41799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.2859999999999996,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:43:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":2.0920000000000001,"15min":1.595},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":3.6259999999999999,"5min":13.483000000000001,"15min":13.483000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":2.2770000000000001,"5min":2.0659999999999998,"15min":1.621},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.2240000000000002,"5min":11.539,"15min":12.846},"last":0.82599999999999996}]}]},{"osd":4,"up_from":0,"seq":137438953673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724739,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":596124,"kb_used_data":6420,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":93149028,"statfs":{"total":95995035648,"available":95384604672,"internally_reserved":0,"allocated":6574080,"data_stored":5315525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":8.4299999999999997,"5min":17.684000000000001,"15min":17.684000000000001},"last":0.499},{"interface":"front","average":{"1min":1.339,"5min":1.7629999999999999,"15min":1.5129999999999999},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":4.7060000000000004,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.4059999999999999,"15min":1.375},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":3.0070000000000001,"5min":11.202,"15min":29.390999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":2.052,"5min":1.8089999999999999,"15min":1.5169999999999999},"min":{"1min":0.309,"5min":0.248,"15min":0.218},"max":{"1min":6.5129999999999999,"5min":8.7490000000000006,"15min":29.367000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":2.044,"15min":1.534},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.7550000000000008,"5min":11.367000000000001,"15min":14.613},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.8520000000000001,"15min":1.534},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":6.3979999999999997,"5min":16.099,"15min":16.099},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":4.0640000000000001,"5min":2.0329999999999999,"15min":1.778},"min":{"1min":0.41499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":16.370999999999999,"5min":16.370999999999999,"15min":16.370999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.6599999999999999,"15min":1.609},"min":{"1min":0.41499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.9199999999999999,"5min":8.6069999999999993,"15min":10.16},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.1379999999999999,"15min":0.73999999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.219},"max":{"1min":7.5419999999999998,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.436},{"interface":"front","average":{"1min":2.234,"5min":1.496,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.23499999999999999},"max":{"1min":5.577,"5min":10.407,"15min":10.407},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:43:58 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.873,"15min":0.66900000000000004},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":2.7509999999999999,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.49,"5min":1.337,"15min":0.85599999999999998},"min":{"1min":0.36799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":4.2160000000000002,"5min":10.436,"15min":10.436},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:43:26 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1830000000000001,"15min":0.79200000000000004},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.8410000000000002,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":1.901,"5min":1.3899999999999999,"15min":0.876},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.27100000000000002},"max":{"1min":7.5069999999999997,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79953920,"data_stored":79953920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132841472,"data_stored":132841472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":119734272,"data_stored":119734272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":68354048,"data_stored":68354048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69402624,"data_stored":69402624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:44:16.356 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-02-22T19:44:16.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set default.rgw.control pg_num 12 2024-02-22T19:44:16.357 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:16.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:14.657542+0000 osd.5 (osd.5) 126 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:44:16.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:14.661318+0000 osd.5 (osd.5) 127 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:44:16.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:15.157239+0000 mgr.y (mgr.14535) 585 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:16.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:14.657542+0000 osd.5 (osd.5) 126 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:44:16.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:14.661318+0000 osd.5 (osd.5) 127 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:44:16.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:15.157239+0000 mgr.y (mgr.14535) 585 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:16.511 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:16.517 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:16.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:16.590 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:16.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:16.650 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:16.736 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 8 16 18628 18612 145.379 114.875 0.00683994 0.00687053 2024-02-22T19:44:16.857 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:16.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:16.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:16.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:16.996 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:17.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:17.075 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:17.095 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:17.131 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:17.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:17.180 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:17.210 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:17.215 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:17.248 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.635082+0000 osd.5 (osd.5) 128 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.640264+0000 osd.5 (osd.5) 129 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.680321+0000 mgr.y (mgr.14535) 586 : cluster [DBG] pgmap v756: 148 pgs: 148 active+clean; 895 MiB data, 5.5 GiB used, 441 GiB / 447 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.780140+0000 osd.1 (osd.1) 218 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.782325+0000 osd.1 (osd.1) 219 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.986466+0000 osd.3 (osd.3) 249 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:44:17.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:15.989928+0000 osd.3 (osd.3) 250 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:17.063866+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? 172.21.15.53:0/2929244014' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.635082+0000 osd.5 (osd.5) 128 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.640264+0000 osd.5 (osd.5) 129 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.680321+0000 mgr.y (mgr.14535) 586 : cluster [DBG] pgmap v756: 148 pgs: 148 active+clean; 895 MiB data, 5.5 GiB used, 441 GiB / 447 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.780140+0000 osd.1 (osd.1) 218 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.782325+0000 osd.1 (osd.1) 219 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.986466+0000 osd.3 (osd.3) 249 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:15.989928+0000 osd.3 (osd.3) 250 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:44:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:17.063866+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? 172.21.15.53:0/2929244014' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:17.286 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:17.316 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:17.347 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.635082+0000 osd.5 (osd.5) 128 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.640264+0000 osd.5 (osd.5) 129 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.680321+0000 mgr.y (mgr.14535) 586 : cluster [DBG] pgmap v756: 148 pgs: 148 active+clean; 895 MiB data, 5.5 GiB used, 441 GiB / 447 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.780140+0000 osd.1 (osd.1) 218 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.782325+0000 osd.1 (osd.1) 219 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.986466+0000 osd.3 (osd.3) 249 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:15.989928+0000 osd.3 (osd.3) 250 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:44:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:17.063866+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? 172.21.15.53:0/2929244014' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:44:17.465 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:17.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:17.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:17.569 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:17.629 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:17.740 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 9 16 20413 20397 141.573 111.562 0.00380744 0.00705738 2024-02-22T19:44:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:17.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:17.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:17.884 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:17.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:18.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:16.731295+0000 osd.1 (osd.1) 220 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:44:18.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:16.734308+0000 osd.1 (osd.1) 221 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:44:18.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.118107+0000 osd.2 (osd.2) 204 : cluster [DBG] 5.a scrub starts 2024-02-22T19:44:18.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.122555+0000 osd.2 (osd.2) 205 : cluster [DBG] 5.a scrub ok 2024-02-22T19:44:18.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.415787+0000 osd.7 (osd.7) 230 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.415839+0000 osd.7 (osd.7) 231 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:16.731295+0000 osd.1 (osd.1) 220 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:16.734308+0000 osd.1 (osd.1) 221 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.118107+0000 osd.2 (osd.2) 204 : cluster [DBG] 5.a scrub starts 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.122555+0000 osd.2 (osd.2) 205 : cluster [DBG] 5.a scrub ok 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.415787+0000 osd.7 (osd.7) 230 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:18.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.415839+0000 osd.7 (osd.7) 231 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:18.495 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:18.495 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:18.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:16.731295+0000 osd.1 (osd.1) 220 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:44:18.517 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:16.734308+0000 osd.1 (osd.1) 221 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:44:18.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.118107+0000 osd.2 (osd.2) 204 : cluster [DBG] 5.a scrub starts 2024-02-22T19:44:18.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.122555+0000 osd.2 (osd.2) 205 : cluster [DBG] 5.a scrub ok 2024-02-22T19:44:18.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.415787+0000 osd.7 (osd.7) 230 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:18.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.415839+0000 osd.7 (osd.7) 231 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:18.740 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 10 16 22323 22307 139.353 119.375 0.0051834 0.00717006 2024-02-22T19:44:18.880 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:18.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:18.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:18.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:19.171 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:19.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:19.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:19.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:44:19.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:19.349 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:19.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:19.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:19.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.574202+0000 osd.5 (osd.5) 130 : cluster [DBG] 6.7 deep-scrub starts 2024-02-22T19:44:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.656513+0000 osd.5 (osd.5) 131 : cluster [DBG] 6.7 deep-scrub ok 2024-02-22T19:44:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:17.682429+0000 mgr.y (mgr.14535) 587 : cluster [DBG] pgmap v757: 148 pgs: 148 active+clean; 895 MiB data, 5.5 GiB used, 441 GiB / 447 GiB avail; 1.1 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-02-22T19:44:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:18.213505+0000 mon.a (mon.0) 1495 : cluster [DBG] osdmap e373: 8 total, 8 up, 5 in 2024-02-22T19:44:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:18.293841+0000 osd.7 (osd.7) 232 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:44:19.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:18.299824+0000 osd.7 (osd.7) 233 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.574202+0000 osd.5 (osd.5) 130 : cluster [DBG] 6.7 deep-scrub starts 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.656513+0000 osd.5 (osd.5) 131 : cluster [DBG] 6.7 deep-scrub ok 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:17.682429+0000 mgr.y (mgr.14535) 587 : cluster [DBG] pgmap v757: 148 pgs: 148 active+clean; 895 MiB data, 5.5 GiB used, 441 GiB / 447 GiB avail; 1.1 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:18.213505+0000 mon.a (mon.0) 1495 : cluster [DBG] osdmap e373: 8 total, 8 up, 5 in 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:18.293841+0000 osd.7 (osd.7) 232 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:18.299824+0000 osd.7 (osd.7) 233 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:44:19.505 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:44:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:19] "GET /metrics HTTP/1.1" 200 176309 "" "Prometheus/2.33.4" 2024-02-22T19:44:19.523 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:19.588 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.574202+0000 osd.5 (osd.5) 130 : cluster [DBG] 6.7 deep-scrub starts 2024-02-22T19:44:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.656513+0000 osd.5 (osd.5) 131 : cluster [DBG] 6.7 deep-scrub ok 2024-02-22T19:44:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:17.682429+0000 mgr.y (mgr.14535) 587 : cluster [DBG] pgmap v757: 148 pgs: 148 active+clean; 895 MiB data, 5.5 GiB used, 441 GiB / 447 GiB avail; 1.1 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-02-22T19:44:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:18.213505+0000 mon.a (mon.0) 1495 : cluster [DBG] osdmap e373: 8 total, 8 up, 5 in 2024-02-22T19:44:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:18.293841+0000 osd.7 (osd.7) 232 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:44:19.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:18.299824+0000 osd.7 (osd.7) 233 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:44:19.740 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 11 15 24082 24067 136.685 110 0.00503936 0.00730972 2024-02-22T19:44:19.788 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:19.794 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:19.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:19.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:19.946 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:20.012 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:20.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:20.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:20.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:20.241 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:44:20.362 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:20.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:20.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:20.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:20.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:19.267205+0000 osd.7 (osd.7) 234 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:44:20.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:19.273183+0000 osd.7 (osd.7) 235 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:44:20.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:19.878603+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.53:0/1340012054' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:44:20.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:19.886798+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:44:20.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:20.198946+0000 osd.2 (osd.2) 206 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:44:20.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:20.206067+0000 osd.2 (osd.2) 207 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:44:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:19.267205+0000 osd.7 (osd.7) 234 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:44:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:19.273183+0000 osd.7 (osd.7) 235 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:44:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:19.878603+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.53:0/1340012054' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:44:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:19.886798+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:44:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:20.198946+0000 osd.2 (osd.2) 206 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:20.206067+0000 osd.2 (osd.2) 207 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:19.267205+0000 osd.7 (osd.7) 234 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:19.273183+0000 osd.7 (osd.7) 235 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:19.878603+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.53:0/1340012054' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:19.886798+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:20.198946+0000 osd.2 (osd.2) 206 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:44:20.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:20.206067+0000 osd.2 (osd.2) 207 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:44:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:20.740 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 12 16 25808 25792 134.279 107.812 0.0111173 0.00743938 2024-02-22T19:44:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:20.865 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:20.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:20.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:44:20.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:21.035 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:21.098 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:21.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:21.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:21.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:21.239 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:19.684378+0000 mgr.y (mgr.14535) 588 : cluster [DBG] pgmap v759: 148 pgs: 148 active+clean; 1.3 GiB data, 6.9 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.38k op/s 2024-02-22T19:44:21.239 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:19.740781+0000 osd.1 (osd.1) 222 : cluster [DBG] 4.a scrub starts 2024-02-22T19:44:21.239 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:19.749425+0000 osd.1 (osd.1) 223 : cluster [DBG] 4.a scrub ok 2024-02-22T19:44:21.239 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:20.212244+0000 mon.a (mon.0) 1497 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:21.240 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:20.229782+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:44:21.240 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:20.229832+0000 mon.a (mon.0) 1499 : cluster [DBG] osdmap e374: 8 total, 8 up, 5 in 2024-02-22T19:44:21.240 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:20.440277+0000 osd.4 (osd.4) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:21.240 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:20.440336+0000 osd.4 (osd.4) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:19.684378+0000 mgr.y (mgr.14535) 588 : cluster [DBG] pgmap v759: 148 pgs: 148 active+clean; 1.3 GiB data, 6.9 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.38k op/s 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:19.740781+0000 osd.1 (osd.1) 222 : cluster [DBG] 4.a scrub starts 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:19.749425+0000 osd.1 (osd.1) 223 : cluster [DBG] 4.a scrub ok 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:20.212244+0000 mon.a (mon.0) 1497 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:20.229782+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:20.229832+0000 mon.a (mon.0) 1499 : cluster [DBG] osdmap e374: 8 total, 8 up, 5 in 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:20.440277+0000 osd.4 (osd.4) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:20.440336+0000 osd.4 (osd.4) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:19.684378+0000 mgr.y (mgr.14535) 588 : cluster [DBG] pgmap v759: 148 pgs: 148 active+clean; 1.3 GiB data, 6.9 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.38k op/s 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:19.740781+0000 osd.1 (osd.1) 222 : cluster [DBG] 4.a scrub starts 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:19.749425+0000 osd.1 (osd.1) 223 : cluster [DBG] 4.a scrub ok 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:20.212244+0000 mon.a (mon.0) 1497 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:20.229782+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:20.229832+0000 mon.a (mon.0) 1499 : cluster [DBG] osdmap e374: 8 total, 8 up, 5 in 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:20.440277+0000 osd.4 (osd.4) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:20.440336+0000 osd.4 (osd.4) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:21.583 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:21.583 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:21.598 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:21.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:21.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:21.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:21.740 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 13 16 27492 27476 132.041 105.25 0.00487092 0.00756717 2024-02-22T19:44:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:21.869 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:21.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:21.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:21.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:22.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:21.246593+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e375: 8 total, 8 up, 5 in 2024-02-22T19:44:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:22.366 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:22.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:44:22.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:22.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:22.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:21.246593+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e375: 8 total, 8 up, 5 in 2024-02-22T19:44:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:22.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:21.246593+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e375: 8 total, 8 up, 5 in 2024-02-22T19:44:22.537 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:22.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:22.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:22.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:22.698 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:22.747 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 14 16 28953 28937 129.073 91.3125 0.0124715 0.00774178 2024-02-22T19:44:22.753 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:22.757 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:22.759 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:22.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:22.860 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:22.891 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:22.923 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:22.929 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:22.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:23.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:23.086 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:23.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:44:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:23.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:23.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:23.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:21.685294+0000 mgr.y (mgr.14535) 589 : cluster [DBG] pgmap v762: 148 pgs: 148 active+clean; 1.5 GiB data, 7.3 GiB used, 440 GiB / 447 GiB avail; 852 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-02-22T19:44:23.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:22.357006+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e376: 8 total, 8 up, 5 in 2024-02-22T19:44:23.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:22.716784+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.53:0/2002256937' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]: dispatch 2024-02-22T19:44:23.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:22.717314+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]: dispatch 2024-02-22T19:44:23.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:23.402 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 5 pg_num to 12 2024-02-22T19:44:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:21.685294+0000 mgr.y (mgr.14535) 589 : cluster [DBG] pgmap v762: 148 pgs: 148 active+clean; 1.5 GiB data, 7.3 GiB used, 440 GiB / 447 GiB avail; 852 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:22.357006+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e376: 8 total, 8 up, 5 in 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:22.716784+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.53:0/2002256937' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]: dispatch 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:22.717314+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]: dispatch 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:21.685294+0000 mgr.y (mgr.14535) 589 : cluster [DBG] pgmap v762: 148 pgs: 148 active+clean; 1.5 GiB data, 7.3 GiB used, 440 GiB / 447 GiB avail; 852 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:22.357006+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e376: 8 total, 8 up, 5 in 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:22.716784+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.53:0/2002256937' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]: dispatch 2024-02-22T19:44:23.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:22.717314+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]: dispatch 2024-02-22T19:44:23.748 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 15 16 30475 30459 126.806 95.125 0.00627005 0.00788046 2024-02-22T19:44:24.034 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:24.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:24.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:24.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:24.208 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:23.370828+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]': finished 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:23.370891+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e377: 8 total, 8 up, 5 in 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:23.379277+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:23.685758+0000 mgr.y (mgr.14535) 590 : cluster [DBG] pgmap v765: 148 pgs: 148 active+clean; 1.5 GiB data, 7.3 GiB used, 440 GiB / 447 GiB avail; 36 MiB/s wr, 583 op/s 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:23.687032+0000 mon.a (mon.0) 1506 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "12"}]: dispatch 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:23.370828+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]': finished 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:23.370891+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e377: 8 total, 8 up, 5 in 2024-02-22T19:44:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:23.379277+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:24.428 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:23.685758+0000 mgr.y (mgr.14535) 590 : cluster [DBG] pgmap v765: 148 pgs: 148 active+clean; 1.5 GiB data, 7.3 GiB used, 440 GiB / 447 GiB avail; 36 MiB/s wr, 583 op/s 2024-02-22T19:44:24.428 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:23.687032+0000 mon.a (mon.0) 1506 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "12"}]: dispatch 2024-02-22T19:44:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:23.370828+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "12"}]': finished 2024-02-22T19:44:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:23.370891+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e377: 8 total, 8 up, 5 in 2024-02-22T19:44:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:23.379277+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:23.685758+0000 mgr.y (mgr.14535) 590 : cluster [DBG] pgmap v765: 148 pgs: 148 active+clean; 1.5 GiB data, 7.3 GiB used, 440 GiB / 447 GiB avail; 36 MiB/s wr, 583 op/s 2024-02-22T19:44:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:23.687032+0000 mon.a (mon.0) 1506 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "12"}]: dispatch 2024-02-22T19:44:24.661 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:44:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:44:24.684 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:24.684 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:24.749 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 16 16 32034 32018 124.971 97.4375 0.00534601 0.00799675 2024-02-22T19:44:24.900 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:24.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:24.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:25.006 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:25.083 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:25.124 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:25.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:25.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:25.156 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:25.189 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:25.230 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:25.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:25.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:25.297 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:25.331 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:25.351 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:44:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:44:25.353 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:25.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:25.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:25.363 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:44:25.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:24.395714+0000 mon.a (mon.0) 1507 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "12"}]': finished 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:24.395778+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e378: 8 total, 8 up, 5 in 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:25.080383+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? 172.21.15.53:0/2306517146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:25.175333+0000 mgr.y (mgr.14535) 591 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:24.395714+0000 mon.a (mon.0) 1507 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "12"}]': finished 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:24.395778+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e378: 8 total, 8 up, 5 in 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:25.080383+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? 172.21.15.53:0/2306517146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:25.400 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:25.175333+0000 mgr.y (mgr.14535) 591 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:25.458 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:25.503 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:25.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:25.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:24.395714+0000 mon.a (mon.0) 1507 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "12"}]': finished 2024-02-22T19:44:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:24.395778+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e378: 8 total, 8 up, 5 in 2024-02-22T19:44:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:25.080383+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? 172.21.15.53:0/2306517146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:25.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:25.175333+0000 mgr.y (mgr.14535) 591 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:25.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:25.688 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:25.751 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 17 16 33443 33427 122.801 88.0625 0.00724345 0.0081387 2024-02-22T19:44:25.785 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:26.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:25.410111+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e379: 8 total, 8 up, 5 in 2024-02-22T19:44:26.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:25.686300+0000 mgr.y (mgr.14535) 592 : cluster [DBG] pgmap v768: 148 pgs: 8 peering, 140 active+clean; 1.9 GiB data, 8.8 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-02-22T19:44:26.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:25.410111+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e379: 8 total, 8 up, 5 in 2024-02-22T19:44:26.430 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:25.686300+0000 mgr.y (mgr.14535) 592 : cluster [DBG] pgmap v768: 148 pgs: 8 peering, 140 active+clean; 1.9 GiB data, 8.8 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-02-22T19:44:26.565 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:26.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:26.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:26.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:26.706 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:25.410111+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e379: 8 total, 8 up, 5 in 2024-02-22T19:44:26.706 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:25.686300+0000 mgr.y (mgr.14535) 592 : cluster [DBG] pgmap v768: 148 pgs: 8 peering, 140 active+clean; 1.9 GiB data, 8.8 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-02-22T19:44:26.710 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:26.748 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 18 16 34945 34929 121.194 93.875 0.0111308 0.00824592 2024-02-22T19:44:27.433 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:26.406926+0000 mon.a (mon.0) 1510 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-02-22T19:44:27.433 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:26.427422+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e380: 8 total, 8 up, 5 in 2024-02-22T19:44:27.433 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:26.434306+0000 osd.5 (osd.5) 132 : cluster [DBG] 5.f starting backfill to osd.7 from (0'0,0'0] MAX to 326'20 2024-02-22T19:44:27.434 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:26.435826+0000 osd.2 (osd.2) 208 : cluster [DBG] 5.d starting backfill to osd.1 from (0'0,0'0] MAX to 221'14 2024-02-22T19:44:27.434 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:26.435913+0000 osd.2 (osd.2) 209 : cluster [DBG] 5.d starting backfill to osd.3 from (0'0,0'0] MAX to 221'14 2024-02-22T19:44:27.434 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:26.439017+0000 osd.2 (osd.2) 210 : cluster [DBG] 5.e starting backfill to osd.7 from (0'0,0'0] MAX to 294'26 2024-02-22T19:44:27.434 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:27.357136+0000 mon.a (mon.0) 1512 : cluster [DBG] osdmap e381: 8 total, 8 up, 5 in 2024-02-22T19:44:27.748 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 19 16 36399 36383 119.599 90.875 0.00540392 0.00835677 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:26.406926+0000 mon.a (mon.0) 1510 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:26.427422+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e380: 8 total, 8 up, 5 in 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:26.434306+0000 osd.5 (osd.5) 132 : cluster [DBG] 5.f starting backfill to osd.7 from (0'0,0'0] MAX to 326'20 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:26.435826+0000 osd.2 (osd.2) 208 : cluster [DBG] 5.d starting backfill to osd.1 from (0'0,0'0] MAX to 221'14 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:26.435913+0000 osd.2 (osd.2) 209 : cluster [DBG] 5.d starting backfill to osd.3 from (0'0,0'0] MAX to 221'14 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:26.439017+0000 osd.2 (osd.2) 210 : cluster [DBG] 5.e starting backfill to osd.7 from (0'0,0'0] MAX to 294'26 2024-02-22T19:44:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:27.357136+0000 mon.a (mon.0) 1512 : cluster [DBG] osdmap e381: 8 total, 8 up, 5 in 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:26.406926+0000 mon.a (mon.0) 1510 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:26.427422+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e380: 8 total, 8 up, 5 in 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:26.434306+0000 osd.5 (osd.5) 132 : cluster [DBG] 5.f starting backfill to osd.7 from (0'0,0'0] MAX to 326'20 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:26.435826+0000 osd.2 (osd.2) 208 : cluster [DBG] 5.d starting backfill to osd.1 from (0'0,0'0] MAX to 221'14 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:26.435913+0000 osd.2 (osd.2) 209 : cluster [DBG] 5.d starting backfill to osd.3 from (0'0,0'0] MAX to 221'14 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:26.439017+0000 osd.2 (osd.2) 210 : cluster [DBG] 5.e starting backfill to osd.7 from (0'0,0'0] MAX to 294'26 2024-02-22T19:44:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:27.357136+0000 mon.a (mon.0) 1512 : cluster [DBG] osdmap e381: 8 total, 8 up, 5 in 2024-02-22T19:44:28.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:44:28.749 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:44:28.750092+0000 min lat: 0.00135253 max lat: 0.0822911 avg lat: 0.00848427 2024-02-22T19:44:28.749 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:44:28.749 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 20 16 37734 37718 117.786 83.4375 0.0189608 0.00848427 2024-02-22T19:44:28.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:27.686902+0000 mgr.y (mgr.14535) 593 : cluster [DBG] pgmap v771: 148 pgs: 8 peering, 140 active+clean; 1.9 GiB data, 8.8 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-02-22T19:44:28.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:28.361392+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e382: 8 total, 8 up, 5 in 2024-02-22T19:44:28.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:27.686902+0000 mgr.y (mgr.14535) 593 : cluster [DBG] pgmap v771: 148 pgs: 8 peering, 140 active+clean; 1.9 GiB data, 8.8 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-02-22T19:44:28.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:28.361392+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e382: 8 total, 8 up, 5 in 2024-02-22T19:44:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:27.686902+0000 mgr.y (mgr.14535) 593 : cluster [DBG] pgmap v771: 148 pgs: 8 peering, 140 active+clean; 1.9 GiB data, 8.8 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-02-22T19:44:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:28.361392+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e382: 8 total, 8 up, 5 in 2024-02-22T19:44:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:29.475 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:44:29.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:28.495885+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.53:0/4140599008' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:29.630 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:28.498040+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:29.631 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:44:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:29] "GET /metrics HTTP/1.1" 200 176383 "" "Prometheus/2.33.4" 2024-02-22T19:44:29.631 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:28.495885+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.53:0/4140599008' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:29.631 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:28.498040+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:29.749 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 21 16 38949 38933 115.794 75.9375 0.0146861 0.00863022 2024-02-22T19:44:29.828 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:44:29.828 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:44:29.828 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.control 2024-02-22T19:44:29.829 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:44:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:28.495885+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.53:0/4140599008' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:29.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:28.498040+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:30.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:30.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:30.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:30.527 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:30.751 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 22 16 40263 40247 114.257 82.125 0.0109826 0.00874615 2024-02-22T19:44:30.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:29.442735+0000 mon.a (mon.0) 1515 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:30.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:29.449096+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:30.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:29.449149+0000 mon.a (mon.0) 1517 : cluster [DBG] osdmap e383: 8 total, 8 up, 5 in 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:29.687448+0000 mgr.y (mgr.14535) 594 : cluster [DBG] pgmap v774: 148 pgs: 2 peering, 146 active+clean; 2.3 GiB data, 9.8 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:29.688325+0000 mon.a (mon.0) 1518 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:29.442735+0000 mon.a (mon.0) 1515 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:29.449096+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:29.449149+0000 mon.a (mon.0) 1517 : cluster [DBG] osdmap e383: 8 total, 8 up, 5 in 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:29.687448+0000 mgr.y (mgr.14535) 594 : cluster [DBG] pgmap v774: 148 pgs: 2 peering, 146 active+clean; 2.3 GiB data, 9.8 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:30.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:29.688325+0000 mon.a (mon.0) 1518 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-22T19:44:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:29.442735+0000 mon.a (mon.0) 1515 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:29.449096+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:29.449149+0000 mon.a (mon.0) 1517 : cluster [DBG] osdmap e383: 8 total, 8 up, 5 in 2024-02-22T19:44:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:29.687448+0000 mgr.y (mgr.14535) 594 : cluster [DBG] pgmap v774: 148 pgs: 2 peering, 146 active+clean; 2.3 GiB data, 9.8 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:29.688325+0000 mon.a (mon.0) 1518 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-22T19:44:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:31.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:31.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:31.230 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:31.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:31.479 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:30.468269+0000 mon.a (mon.0) 1519 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-22T19:44:31.479 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:30.468332+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e384: 8 total, 8 up, 5 in 2024-02-22T19:44:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:30.468269+0000 mon.a (mon.0) 1519 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-22T19:44:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:30.468332+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e384: 8 total, 8 up, 5 in 2024-02-22T19:44:31.629 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:44:31.629 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:44:31.629 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:44:31.629 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:44:31.629 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:44:31.751 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 23 16 41662 41646 113.091 87.4375 0.0108578 0.00883844 2024-02-22T19:44:31.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:30.468269+0000 mon.a (mon.0) 1519 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-22T19:44:31.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:30.468332+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e384: 8 total, 8 up, 5 in 2024-02-22T19:44:32.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:44:32.402 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:44:32.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:31.488971+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e385: 8 total, 8 up, 5 in 2024-02-22T19:44:32.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:31.687915+0000 mgr.y (mgr.14535) 595 : cluster [DBG] pgmap v777: 148 pgs: 148 active+clean; 2.4 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.75k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:32.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:32.352683+0000 mon.a (mon.0) 1522 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-22T19:44:32.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:32.359647+0000 mon.a (mon.0) 1523 : cluster [DBG] osdmap e386: 8 total, 8 up, 5 in 2024-02-22T19:44:32.674 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:32.751 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 24 16 42940 42924 111.708 79.875 0.0158599 0.00894649 2024-02-22T19:44:32.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:31.488971+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e385: 8 total, 8 up, 5 in 2024-02-22T19:44:32.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:31.687915+0000 mgr.y (mgr.14535) 595 : cluster [DBG] pgmap v777: 148 pgs: 148 active+clean; 2.4 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.75k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:32.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:32.352683+0000 mon.a (mon.0) 1522 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-22T19:44:32.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:32.359647+0000 mon.a (mon.0) 1523 : cluster [DBG] osdmap e386: 8 total, 8 up, 5 in 2024-02-22T19:44:32.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:31.488971+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e385: 8 total, 8 up, 5 in 2024-02-22T19:44:32.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:31.687915+0000 mgr.y (mgr.14535) 595 : cluster [DBG] pgmap v777: 148 pgs: 148 active+clean; 2.4 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.75k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:32.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:32.352683+0000 mon.a (mon.0) 1522 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-22T19:44:32.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:32.359647+0000 mon.a (mon.0) 1523 : cluster [DBG] osdmap e386: 8 total, 8 up, 5 in 2024-02-22T19:44:33.755 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 25 16 44244 44228 110.497 81.5 0.0167327 0.00904384 2024-02-22T19:44:33.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:32.485832+0000 mon.a (mon.0) 1524 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-22T19:44:33.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:33.370412+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e387: 8 total, 8 up, 5 in 2024-02-22T19:44:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:32.485832+0000 mon.a (mon.0) 1524 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-22T19:44:33.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:33.370412+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e387: 8 total, 8 up, 5 in 2024-02-22T19:44:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:33.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:33.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:33.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:32.485832+0000 mon.a (mon.0) 1524 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-22T19:44:33.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:33.370412+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e387: 8 total, 8 up, 5 in 2024-02-22T19:44:33.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:34.000 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:34.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:34.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:34.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:34.208 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:34.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:34.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:34.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:34.351 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:34.399 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:34.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:34.441 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:34.455 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:34.482 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:34.516 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:34.550 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:34.583 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:34.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:33.692182+0000 mgr.y (mgr.14535) 596 : cluster [DBG] pgmap v780: 147 pgs: 147 active+clean; 2.4 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 27 MiB/s wr, 426 op/s 2024-02-22T19:44:34.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:33.695692+0000 mon.a (mon.0) 1526 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-22T19:44:34.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:34.241077+0000 mon.a (mon.0) 1527 : audit [DBG] from='client.? 172.21.15.53:0/2319427415' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:34.587 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:34.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:44:34.608 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:34.608 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:34.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:34.749 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:33.692182+0000 mgr.y (mgr.14535) 596 : cluster [DBG] pgmap v780: 147 pgs: 147 active+clean; 2.4 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 27 MiB/s wr, 426 op/s 2024-02-22T19:44:34.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:33.695692+0000 mon.a (mon.0) 1526 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-22T19:44:34.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:34.241077+0000 mon.a (mon.0) 1527 : audit [DBG] from='client.? 172.21.15.53:0/2319427415' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:34.750 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:33.692182+0000 mgr.y (mgr.14535) 596 : cluster [DBG] pgmap v780: 147 pgs: 147 active+clean; 2.4 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 27 MiB/s wr, 426 op/s 2024-02-22T19:44:34.750 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:33.695692+0000 mon.a (mon.0) 1526 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-22T19:44:34.750 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:34.241077+0000 mon.a (mon.0) 1527 : audit [DBG] from='client.? 172.21.15.53:0/2319427415' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:34.752 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 26 16 45457 45441 109.16 75.8125 0.019552 0.00915652 2024-02-22T19:44:34.760 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:34.808 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:34.811 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:34.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:34.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:44:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:44:34.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:34.951 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:35.273 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:44:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:44:35.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:44:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:35.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:35.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:35.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:35.446 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:35.469 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:35.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:35.474 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:35.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:35.649 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:35.753 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 27 16 46696 46680 107.985 77.4375 0.0113105 0.00925642 2024-02-22T19:44:35.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:34.521062+0000 mon.a (mon.0) 1528 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-22T19:44:35.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:34.521126+0000 mon.a (mon.0) 1529 : cluster [DBG] osdmap e388: 8 total, 8 up, 5 in 2024-02-22T19:44:35.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:35.185208+0000 mgr.y (mgr.14535) 597 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:34.521062+0000 mon.a (mon.0) 1528 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-22T19:44:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:34.521126+0000 mon.a (mon.0) 1529 : cluster [DBG] osdmap e388: 8 total, 8 up, 5 in 2024-02-22T19:44:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:35.185208+0000 mgr.y (mgr.14535) 597 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:34.521062+0000 mon.a (mon.0) 1528 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-22T19:44:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:34.521126+0000 mon.a (mon.0) 1529 : cluster [DBG] osdmap e388: 8 total, 8 up, 5 in 2024-02-22T19:44:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:35.185208+0000 mgr.y (mgr.14535) 597 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:36.106 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:36.127 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:36.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:36.229 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:36.295 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:36.422 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:36.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:36.529 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:36.606 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:36.683 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:36.687 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:36.688 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:35.529713+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e389: 8 total, 8 up, 5 in 2024-02-22T19:44:36.688 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:35.692771+0000 mgr.y (mgr.14535) 598 : cluster [DBG] pgmap v783: 147 pgs: 147 active+clean; 2.8 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-02-22T19:44:36.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:35.529713+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e389: 8 total, 8 up, 5 in 2024-02-22T19:44:36.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:35.692771+0000 mgr.y (mgr.14535) 598 : cluster [DBG] pgmap v783: 147 pgs: 147 active+clean; 2.8 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-02-22T19:44:36.689 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:36.753 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 28 16 47906 47890 106.83 75.625 0.0146987 0.00935422 2024-02-22T19:44:37.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:37.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:35.529713+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e389: 8 total, 8 up, 5 in 2024-02-22T19:44:37.172 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:35.692771+0000 mgr.y (mgr.14535) 598 : cluster [DBG] pgmap v783: 147 pgs: 147 active+clean; 2.8 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-02-22T19:44:37.204 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:37.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:37.232 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:37.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:44:37.405 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:44:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:37.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:44:37.755 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 29 16 49022 49006 105.551 69.75 0.0142883 0.00946887 2024-02-22T19:44:37.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:36.569527+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e390: 8 total, 8 up, 5 in 2024-02-22T19:44:37.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:36.569527+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e390: 8 total, 8 up, 5 in 2024-02-22T19:44:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:36.569527+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e390: 8 total, 8 up, 5 in 2024-02-22T19:44:37.893 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:37.897 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:37.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:37.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:38.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:38.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:38.146 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:38.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:38.228 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:38.697 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:38.701 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:38.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:38.719 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":784,"stamp":"2024-02-22T19:44:36.565266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2958956196,"num_objects":45341,"num_object_clones":0,"num_object_copies":136023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45341,"num_whiteouts":0,"num_read":2036,"num_read_kb":1788,"num_write":45864,"num_write_kb":2889641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":451,"num_bytes_recovered":15766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45959,"ondisk_log_size":45959,"up":438,"acting":438,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":443,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":11829084,"kb_used_data":8570420,"kb_used_omap":0,"kb_used_meta":3258624,"kb_avail":456896676,"statfs":{"total":479975178240,"available":467862196224,"internally_reserved":0,"allocated":8776110080,"data_stored":8769444545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3336830976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,6,4,13],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":428015616,"num_objects":6531,"num_object_clones":0,"num_object_copies":19593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6531,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6531,"num_write_kb":417984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.203583"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":344,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369735+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:44:32.369735+0000","last_peered":"2024-02-22T19:44:32.369735+0000","last_clean":"2024-02-22T19:44:32.369735+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:44:32.369735+0000","last_undegraded":"2024-02-22T19:44:32.369735+0000","last_fullsized":"2024-02-22T19:44:32.369735+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":379,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.371020+0000","last_change":"2024-02-22T19:43:58.780165+0000","last_active":"2024-02-22T19:44:32.371020+0000","last_peered":"2024-02-22T19:44:32.371020+0000","last_clean":"2024-02-22T19:44:32.371020+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:44:32.371020+0000","last_undegraded":"2024-02-22T19:44:32.371020+0000","last_fullsized":"2024-02-22T19:44:32.371020+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.780100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.780100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":158,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.364364+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:44:32.364364+0000","last_peered":"2024-02-22T19:44:32.364364+0000","last_clean":"2024-02-22T19:44:32.364364+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:44:32.364364+0000","last_undegraded":"2024-02-22T19:44:32.364364+0000","last_fullsized":"2024-02-22T19:44:32.364364+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":360,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381348+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:44:33.381348+0000","last_peered":"2024-02-22T19:44:33.381348+0000","last_clean":"2024-02-22T19:44:33.381348+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:44:33.381348+0000","last_undegraded":"2024-02-22T19:44:33.381348+0000","last_fullsized":"2024-02-22T19:44:33.381348+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":361,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379245+0000","last_change":"2024-02-22T19:41:52.112817+0000","last_active":"2024-02-22T19:44:33.379245+0000","last_peered":"2024-02-22T19:44:33.379245+0000","last_clean":"2024-02-22T19:44:33.379245+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:44:33.379245+0000","last_undegraded":"2024-02-22T19:44:33.379245+0000","last_fullsized":"2024-02-22T19:44:33.379245+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.661044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.661044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":473,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379783+0000","last_change":"2024-02-22T19:44:07.984802+0000","last_active":"2024-02-22T19:44:33.379783+0000","last_peered":"2024-02-22T19:44:33.379783+0000","last_clean":"2024-02-22T19:44:33.379783+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:44:33.379783+0000","last_undegraded":"2024-02-22T19:44:33.379783+0000","last_fullsized":"2024-02-22T19:44:33.379783+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:44:07.984710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.984710+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":188,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363041+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:44:32.363041+0000","last_peered":"2024-02-22T19:44:32.363041+0000","last_clean":"2024-02-22T19:44:32.363041+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:44:32.363041+0000","last_undegraded":"2024-02-22T19:44:32.363041+0000","last_fullsized":"2024-02-22T19:44:32.363041+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":202,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370685+0000","last_change":"2024-02-22T19:44:07.795043+0000","last_active":"2024-02-22T19:44:32.370685+0000","last_peered":"2024-02-22T19:44:32.370685+0000","last_clean":"2024-02-22T19:44:32.370685+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:44:32.370685+0000","last_undegraded":"2024-02-22T19:44:32.370685+0000","last_fullsized":"2024-02-22T19:44:32.370685+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:07.794943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.794943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":265,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379783+0000","last_change":"2024-02-22T19:41:52.752324+0000","last_active":"2024-02-22T19:44:33.379783+0000","last_peered":"2024-02-22T19:44:33.379783+0000","last_clean":"2024-02-22T19:44:33.379783+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:44:33.379783+0000","last_undegraded":"2024-02-22T19:44:33.379783+0000","last_fullsized":"2024-02-22T19:44:33.379783+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.683110+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":184,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369444+0000","last_change":"2024-02-22T19:40:48.494721+0000","last_active":"2024-02-22T19:44:32.369444+0000","last_peered":"2024-02-22T19:44:32.369444+0000","last_clean":"2024-02-22T19:44:32.369444+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:44:32.369444+0000","last_undegraded":"2024-02-22T19:44:32.369444+0000","last_fullsized":"2024-02-22T19:44:32.369444+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:48.494661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:02.783771+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.494661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":214,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370667+0000","last_change":"2024-02-22T19:41:52.116983+0000","last_active":"2024-02-22T19:44:32.370667+0000","last_peered":"2024-02-22T19:44:32.370667+0000","last_clean":"2024-02-22T19:44:32.370667+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:44:32.370667+0000","last_undegraded":"2024-02-22T19:44:32.370667+0000","last_fullsized":"2024-02-22T19:44:32.370667+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:39:53.438293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.438293+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":193,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381136+0000","last_change":"2024-02-22T19:44:04.949624+0000","last_active":"2024-02-22T19:44:33.381136+0000","last_peered":"2024-02-22T19:44:33.381136+0000","last_clean":"2024-02-22T19:44:33.381136+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:44:33.381136+0000","last_undegraded":"2024-02-22T19:44:33.381136+0000","last_fullsized":"2024-02-22T19:44:33.381136+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:44:04.949558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.949558+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":262,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.371178+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:44:32.371178+0000","last_peered":"2024-02-22T19:44:32.371178+0000","last_clean":"2024-02-22T19:44:32.371178+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:44:32.371178+0000","last_undegraded":"2024-02-22T19:44:32.371178+0000","last_fullsized":"2024-02-22T19:44:32.371178+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":161,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367028+0000","last_change":"2024-02-22T19:41:22.650383+0000","last_active":"2024-02-22T19:44:32.367028+0000","last_peered":"2024-02-22T19:44:32.367028+0000","last_clean":"2024-02-22T19:44:32.367028+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:44:32.367028+0000","last_undegraded":"2024-02-22T19:44:32.367028+0000","last_fullsized":"2024-02-22T19:44:32.367028+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":420,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.391833+0000","last_change":"2024-02-22T19:44:07.172636+0000","last_active":"2024-02-22T19:44:33.391833+0000","last_peered":"2024-02-22T19:44:33.391833+0000","last_clean":"2024-02-22T19:44:33.391833+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:44:33.391833+0000","last_undegraded":"2024-02-22T19:44:33.391833+0000","last_fullsized":"2024-02-22T19:44:33.391833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:44:07.172562+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.172562+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":378,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.383685+0000","last_change":"2024-02-22T19:41:52.121546+0000","last_active":"2024-02-22T19:44:33.383685+0000","last_peered":"2024-02-22T19:44:33.383685+0000","last_clean":"2024-02-22T19:44:33.383685+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:44:33.383685+0000","last_undegraded":"2024-02-22T19:44:33.383685+0000","last_fullsized":"2024-02-22T19:44:33.383685+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.844327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.844327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":174,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363933+0000","last_change":"2024-02-22T19:40:46.441667+0000","last_active":"2024-02-22T19:44:32.363933+0000","last_peered":"2024-02-22T19:44:32.363933+0000","last_clean":"2024-02-22T19:44:32.363933+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:44:32.363933+0000","last_undegraded":"2024-02-22T19:44:32.363933+0000","last_fullsized":"2024-02-22T19:44:32.363933+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:46.441567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.441567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":429,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380029+0000","last_change":"2024-02-22T19:41:53.187026+0000","last_active":"2024-02-22T19:44:33.380029+0000","last_peered":"2024-02-22T19:44:33.380029+0000","last_clean":"2024-02-22T19:44:33.380029+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:44:33.380029+0000","last_undegraded":"2024-02-22T19:44:33.380029+0000","last_fullsized":"2024-02-22T19:44:33.380029+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:40:39.681788+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.681788+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":398,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367860+0000","last_change":"2024-02-22T19:44:04.315337+0000","last_active":"2024-02-22T19:44:32.367860+0000","last_peered":"2024-02-22T19:44:32.367860+0000","last_clean":"2024-02-22T19:44:32.367860+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:44:32.367860+0000","last_undegraded":"2024-02-22T19:44:32.367860+0000","last_fullsized":"2024-02-22T19:44:32.367860+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:44:04.315265+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.315265+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":382,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.365905+0000","last_change":"2024-02-22T19:41:52.110049+0000","last_active":"2024-02-22T19:44:32.365905+0000","last_peered":"2024-02-22T19:44:32.365905+0000","last_clean":"2024-02-22T19:44:32.365905+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:44:32.365905+0000","last_undegraded":"2024-02-22T19:44:32.365905+0000","last_fullsized":"2024-02-22T19:44:32.365905+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.388459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.388459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":199,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367887+0000","last_change":"2024-02-22T19:44:18.299947+0000","last_active":"2024-02-22T19:44:32.367887+0000","last_peered":"2024-02-22T19:44:32.367887+0000","last_clean":"2024-02-22T19:44:32.367887+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:44:32.367887+0000","last_undegraded":"2024-02-22T19:44:32.367887+0000","last_fullsized":"2024-02-22T19:44:32.367887+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:44:18.299831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:18.299831+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":382,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.364666+0000","last_change":"2024-02-22T19:40:39.441590+0000","last_active":"2024-02-22T19:44:32.364666+0000","last_peered":"2024-02-22T19:44:32.364666+0000","last_clean":"2024-02-22T19:44:32.364666+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:44:32.364666+0000","last_undegraded":"2024-02-22T19:44:32.364666+0000","last_fullsized":"2024-02-22T19:44:32.364666+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:39.441486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:39.441486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":386,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380214+0000","last_change":"2024-02-22T19:44:10.017728+0000","last_active":"2024-02-22T19:44:33.380214+0000","last_peered":"2024-02-22T19:44:33.380214+0000","last_clean":"2024-02-22T19:44:33.380214+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:44:33.380214+0000","last_undegraded":"2024-02-22T19:44:33.380214+0000","last_fullsized":"2024-02-22T19:44:33.380214+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:10.017666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:44:10.017666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":422,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.371226+0000","last_change":"2024-02-22T19:41:52.743206+0000","last_active":"2024-02-22T19:44:32.371226+0000","last_peered":"2024-02-22T19:44:32.371226+0000","last_clean":"2024-02-22T19:44:32.371226+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:44:32.371226+0000","last_undegraded":"2024-02-22T19:44:32.371226+0000","last_fullsized":"2024-02-22T19:44:32.371226+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:40:49.582859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.582859+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":367,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366277+0000","last_change":"2024-02-22T19:43:59.334492+0000","last_active":"2024-02-22T19:44:32.366277+0000","last_peered":"2024-02-22T19:44:32.366277+0000","last_clean":"2024-02-22T19:44:32.366277+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:44:32.366277+0000","last_undegraded":"2024-02-22T19:44:32.366277+0000","last_fullsized":"2024-02-22T19:44:32.366277+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.334420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.334420+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":275,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379401+0000","last_change":"2024-02-22T19:44:25.424198+0000","last_active":"2024-02-22T19:44:33.379401+0000","last_peered":"2024-02-22T19:44:33.379401+0000","last_clean":"2024-02-22T19:44:33.379401+0000","last_became_active":"2024-02-22T19:44:25.423887+0000","last_became_peered":"2024-02-22T19:44:25.423887+0000","last_unstale":"2024-02-22T19:44:33.379401+0000","last_undegraded":"2024-02-22T19:44:33.379401+0000","last_fullsized":"2024-02-22T19:44:33.379401+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:19.866827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:19.866827+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":379,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370834+0000","last_change":"2024-02-22T19:44:13.694618+0000","last_active":"2024-02-22T19:44:32.370834+0000","last_peered":"2024-02-22T19:44:32.370834+0000","last_clean":"2024-02-22T19:44:32.370834+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:44:32.370834+0000","last_undegraded":"2024-02-22T19:44:32.370834+0000","last_fullsized":"2024-02-22T19:44:32.370834+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.694552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.694552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":382,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380287+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:44:33.380287+0000","last_peered":"2024-02-22T19:44:33.380287+0000","last_clean":"2024-02-22T19:44:33.380287+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:44:33.380287+0000","last_undegraded":"2024-02-22T19:44:33.380287+0000","last_fullsized":"2024-02-22T19:44:33.380287+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":389,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.383143+0000","last_change":"2024-02-22T19:43:59.063011+0000","last_active":"2024-02-22T19:44:33.383143+0000","last_peered":"2024-02-22T19:44:33.383143+0000","last_clean":"2024-02-22T19:44:33.383143+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:44:33.383143+0000","last_undegraded":"2024-02-22T19:44:33.383143+0000","last_fullsized":"2024-02-22T19:44:33.383143+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.062912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.062912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":19,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380500+0000","last_change":"2024-02-22T19:44:25.431411+0000","last_active":"2024-02-22T19:44:33.380500+0000","last_peered":"2024-02-22T19:44:33.380500+0000","last_clean":"2024-02-22T19:44:33.380500+0000","last_became_active":"2024-02-22T19:44:25.431068+0000","last_became_peered":"2024-02-22T19:44:25.431068+0000","last_unstale":"2024-02-22T19:44:33.380500+0000","last_undegraded":"2024-02-22T19:44:33.380500+0000","last_fullsized":"2024-02-22T19:44:33.380500+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.702590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:15.602101+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.702590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":184,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.371094+0000","last_change":"2024-02-22T19:44:12.674961+0000","last_active":"2024-02-22T19:44:32.371094+0000","last_peered":"2024-02-22T19:44:32.371094+0000","last_clean":"2024-02-22T19:44:32.371094+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:44:32.371094+0000","last_undegraded":"2024-02-22T19:44:32.371094+0000","last_fullsized":"2024-02-22T19:44:32.371094+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.674856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.674856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":407,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379639+0000","last_change":"2024-02-22T19:44:11.041545+0000","last_active":"2024-02-22T19:44:33.379639+0000","last_peered":"2024-02-22T19:44:33.379639+0000","last_clean":"2024-02-22T19:44:33.379639+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:44:33.379639+0000","last_undegraded":"2024-02-22T19:44:33.379639+0000","last_fullsized":"2024-02-22T19:44:33.379639+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:44:11.041458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.041458+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":370,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369513+0000","last_change":"2024-02-22T19:44:14.661384+0000","last_active":"2024-02-22T19:44:32.369513+0000","last_peered":"2024-02-22T19:44:32.369513+0000","last_clean":"2024-02-22T19:44:32.369513+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:44:32.369513+0000","last_undegraded":"2024-02-22T19:44:32.369513+0000","last_fullsized":"2024-02-22T19:44:32.369513+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:14.661322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:14.661322+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":261,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369922+0000","last_change":"2024-02-22T19:44:25.425238+0000","last_active":"2024-02-22T19:44:32.369922+0000","last_peered":"2024-02-22T19:44:32.369922+0000","last_clean":"2024-02-22T19:44:32.369922+0000","last_became_active":"2024-02-22T19:44:25.425001+0000","last_became_peered":"2024-02-22T19:44:25.425001+0000","last_unstale":"2024-02-22T19:44:32.369922+0000","last_undegraded":"2024-02-22T19:44:32.369922+0000","last_fullsized":"2024-02-22T19:44:32.369922+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:05.908382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:05.908382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":402,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367442+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:44:32.367442+0000","last_peered":"2024-02-22T19:44:32.367442+0000","last_clean":"2024-02-22T19:44:32.367442+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:44:32.367442+0000","last_undegraded":"2024-02-22T19:44:32.367442+0000","last_fullsized":"2024-02-22T19:44:32.367442+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":406,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.364201+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:44:32.364201+0000","last_peered":"2024-02-22T19:44:32.364201+0000","last_clean":"2024-02-22T19:44:32.364201+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:44:32.364201+0000","last_undegraded":"2024-02-22T19:44:32.364201+0000","last_fullsized":"2024-02-22T19:44:32.364201+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":396,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379175+0000","last_change":"2024-02-22T19:40:50.680522+0000","last_active":"2024-02-22T19:44:33.379175+0000","last_peered":"2024-02-22T19:44:33.379175+0000","last_clean":"2024-02-22T19:44:33.379175+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:44:33.379175+0000","last_undegraded":"2024-02-22T19:44:33.379175+0000","last_fullsized":"2024-02-22T19:44:33.379175+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:50.680434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:50.680434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":210,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369543+0000","last_change":"2024-02-22T19:44:25.423750+0000","last_active":"2024-02-22T19:44:32.369543+0000","last_peered":"2024-02-22T19:44:32.369543+0000","last_clean":"2024-02-22T19:44:32.369543+0000","last_became_active":"2024-02-22T19:44:25.423395+0000","last_became_peered":"2024-02-22T19:44:25.423395+0000","last_unstale":"2024-02-22T19:44:32.369543+0000","last_undegraded":"2024-02-22T19:44:32.369543+0000","last_fullsized":"2024-02-22T19:44:32.369543+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:45.398705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:45.398705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":165,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367571+0000","last_change":"2024-02-22T19:41:52.104077+0000","last_active":"2024-02-22T19:44:32.367571+0000","last_peered":"2024-02-22T19:44:32.367571+0000","last_clean":"2024-02-22T19:44:32.367571+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:44:32.367571+0000","last_undegraded":"2024-02-22T19:44:32.367571+0000","last_fullsized":"2024-02-22T19:44:32.367571+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.559984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.559984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":169,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363330+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:44:32.363330+0000","last_peered":"2024-02-22T19:44:32.363330+0000","last_clean":"2024-02-22T19:44:32.363330+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:44:32.363330+0000","last_undegraded":"2024-02-22T19:44:32.363330+0000","last_fullsized":"2024-02-22T19:44:32.363330+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":182,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367670+0000","last_change":"2024-02-22T19:39:37.803588+0000","last_active":"2024-02-22T19:44:32.367670+0000","last_peered":"2024-02-22T19:44:32.367670+0000","last_clean":"2024-02-22T19:44:32.367670+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:44:32.367670+0000","last_undegraded":"2024-02-22T19:44:32.367670+0000","last_fullsized":"2024-02-22T19:44:32.367670+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:37.803478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:39:37.803478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":375,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366948+0000","last_change":"2024-02-22T19:41:52.103816+0000","last_active":"2024-02-22T19:44:32.366948+0000","last_peered":"2024-02-22T19:44:32.366948+0000","last_clean":"2024-02-22T19:44:32.366948+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:44:32.366948+0000","last_undegraded":"2024-02-22T19:44:32.366948+0000","last_fullsized":"2024-02-22T19:44:32.366948+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:24.569394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.569394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":390,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381268+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:44:33.381268+0000","last_peered":"2024-02-22T19:44:33.381268+0000","last_clean":"2024-02-22T19:44:33.381268+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:44:33.381268+0000","last_undegraded":"2024-02-22T19:44:33.381268+0000","last_fullsized":"2024-02-22T19:44:33.381268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":368,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392747+0000","last_change":"2024-02-22T19:41:23.338446+0000","last_active":"2024-02-22T19:44:33.392747+0000","last_peered":"2024-02-22T19:44:33.392747+0000","last_clean":"2024-02-22T19:44:33.392747+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:44:33.392747+0000","last_undegraded":"2024-02-22T19:44:33.392747+0000","last_fullsized":"2024-02-22T19:44:33.392747+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.338373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.338373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":173,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366162+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:44:32.366162+0000","last_peered":"2024-02-22T19:44:32.366162+0000","last_clean":"2024-02-22T19:44:32.366162+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:44:32.366162+0000","last_undegraded":"2024-02-22T19:44:32.366162+0000","last_fullsized":"2024-02-22T19:44:32.366162+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":203,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370856+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:44:32.370856+0000","last_peered":"2024-02-22T19:44:32.370856+0000","last_clean":"2024-02-22T19:44:32.370856+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:44:32.370856+0000","last_undegraded":"2024-02-22T19:44:32.370856+0000","last_fullsized":"2024-02-22T19:44:32.370856+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":391,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392057+0000","last_change":"2024-02-22T19:41:38.526359+0000","last_active":"2024-02-22T19:44:33.392057+0000","last_peered":"2024-02-22T19:44:33.392057+0000","last_clean":"2024-02-22T19:44:33.392057+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:44:33.392057+0000","last_undegraded":"2024-02-22T19:44:33.392057+0000","last_fullsized":"2024-02-22T19:44:33.392057+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:38.526259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:41:38.526259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":398,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366860+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:44:32.366860+0000","last_peered":"2024-02-22T19:44:32.366860+0000","last_clean":"2024-02-22T19:44:32.366860+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:44:32.366860+0000","last_undegraded":"2024-02-22T19:44:32.366860+0000","last_fullsized":"2024-02-22T19:44:32.366860+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":176,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379395+0000","last_change":"2024-02-22T19:40:36.677571+0000","last_active":"2024-02-22T19:44:33.379395+0000","last_peered":"2024-02-22T19:44:33.379395+0000","last_clean":"2024-02-22T19:44:33.379395+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:44:33.379395+0000","last_undegraded":"2024-02-22T19:44:33.379395+0000","last_fullsized":"2024-02-22T19:44:33.379395+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:40:36.677475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.677475+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":400,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367349+0000","last_change":"2024-02-22T19:43:57.440300+0000","last_active":"2024-02-22T19:44:32.367349+0000","last_peered":"2024-02-22T19:44:32.367349+0000","last_clean":"2024-02-22T19:44:32.367349+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:44:32.367349+0000","last_undegraded":"2024-02-22T19:44:32.367349+0000","last_fullsized":"2024-02-22T19:44:32.367349+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.440217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.440217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":351,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369341+0000","last_change":"2024-02-22T19:41:52.112840+0000","last_active":"2024-02-22T19:44:32.369341+0000","last_peered":"2024-02-22T19:44:32.369341+0000","last_clean":"2024-02-22T19:44:32.369341+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:44:32.369341+0000","last_undegraded":"2024-02-22T19:44:32.369341+0000","last_fullsized":"2024-02-22T19:44:32.369341+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.559793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.559793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":390,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.383729+0000","last_change":"2024-02-22T19:40:47.668144+0000","last_active":"2024-02-22T19:44:33.383729+0000","last_peered":"2024-02-22T19:44:33.383729+0000","last_clean":"2024-02-22T19:44:33.383729+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:44:33.383729+0000","last_undegraded":"2024-02-22T19:44:33.383729+0000","last_fullsized":"2024-02-22T19:44:33.383729+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:40:47.668065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.668065+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"387'2865","reported_seq":2895,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.434453+0000","last_change":"2024-02-22T19:43:57.347864+0000","last_active":"2024-02-22T19:44:34.434453+0000","last_peered":"2024-02-22T19:44:34.434453+0000","last_clean":"2024-02-22T19:44:34.434453+0000","last_became_active":"2024-02-22T19:43:57.347537+0000","last_became_peered":"2024-02-22T19:43:57.347537+0000","last_unstale":"2024-02-22T19:44:34.434453+0000","last_undegraded":"2024-02-22T19:44:34.434453+0000","last_fullsized":"2024-02-22T19:44:34.434453+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2865,"ondisk_log_size":2865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":187760640,"num_objects":2865,"num_object_clones":0,"num_object_copies":8595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":183360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":220,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369689+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:44:32.369689+0000","last_peered":"2024-02-22T19:44:32.369689+0000","last_clean":"2024-02-22T19:44:32.369689+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:44:32.369689+0000","last_undegraded":"2024-02-22T19:44:32.369689+0000","last_fullsized":"2024-02-22T19:44:32.369689+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":452,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.368353+0000","last_change":"2024-02-22T19:44:08.271084+0000","last_active":"2024-02-22T19:44:32.368353+0000","last_peered":"2024-02-22T19:44:32.368353+0000","last_clean":"2024-02-22T19:44:32.368353+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:44:32.368353+0000","last_undegraded":"2024-02-22T19:44:32.368353+0000","last_fullsized":"2024-02-22T19:44:32.368353+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:44:08.271004+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:44:08.271004+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":243,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369351+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:44:32.369351+0000","last_peered":"2024-02-22T19:44:32.369351+0000","last_clean":"2024-02-22T19:44:32.369351+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:44:32.369351+0000","last_undegraded":"2024-02-22T19:44:32.369351+0000","last_fullsized":"2024-02-22T19:44:32.369351+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":624,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.382120+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:44:33.382120+0000","last_peered":"2024-02-22T19:44:33.382120+0000","last_clean":"2024-02-22T19:44:33.382120+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:44:33.382120+0000","last_undegraded":"2024-02-22T19:44:33.382120+0000","last_fullsized":"2024-02-22T19:44:33.382120+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"388'2865","reported_seq":2896,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.745582+0000","last_change":"2024-02-22T19:43:57.350818+0000","last_active":"2024-02-22T19:44:34.745582+0000","last_peered":"2024-02-22T19:44:34.745582+0000","last_clean":"2024-02-22T19:44:34.745582+0000","last_became_active":"2024-02-22T19:43:57.350467+0000","last_became_peered":"2024-02-22T19:43:57.350467+0000","last_unstale":"2024-02-22T19:44:34.745582+0000","last_undegraded":"2024-02-22T19:44:34.745582+0000","last_fullsized":"2024-02-22T19:44:34.745582+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2865,"ondisk_log_size":2865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":187760640,"num_objects":2865,"num_object_clones":0,"num_object_copies":8595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":183360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":346,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381421+0000","last_change":"2024-02-22T19:40:43.701880+0000","last_active":"2024-02-22T19:44:33.381421+0000","last_peered":"2024-02-22T19:44:33.381421+0000","last_clean":"2024-02-22T19:44:33.381421+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:44:33.381421+0000","last_undegraded":"2024-02-22T19:44:33.381421+0000","last_fullsized":"2024-02-22T19:44:33.381421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:43.701757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.701757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":90,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392429+0000","last_change":"2024-02-22T19:41:28.385529+0000","last_active":"2024-02-22T19:44:33.392429+0000","last_peered":"2024-02-22T19:44:33.392429+0000","last_clean":"2024-02-22T19:44:33.392429+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:44:33.392429+0000","last_undegraded":"2024-02-22T19:44:33.392429+0000","last_fullsized":"2024-02-22T19:44:33.392429+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:28.385428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.385428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":233,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.365707+0000","last_change":"2024-02-22T19:41:52.597069+0000","last_active":"2024-02-22T19:44:32.365707+0000","last_peered":"2024-02-22T19:44:32.365707+0000","last_clean":"2024-02-22T19:44:32.365707+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:44:32.365707+0000","last_undegraded":"2024-02-22T19:44:32.365707+0000","last_fullsized":"2024-02-22T19:44:32.365707+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:40:48.455224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:48.455224+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":406,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.386867+0000","last_change":"2024-02-22T19:41:25.263322+0000","last_active":"2024-02-22T19:44:33.386867+0000","last_peered":"2024-02-22T19:44:33.386867+0000","last_clean":"2024-02-22T19:44:33.386867+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:44:33.386867+0000","last_undegraded":"2024-02-22T19:44:33.386867+0000","last_fullsized":"2024-02-22T19:44:33.386867+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.263219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"388'2762","reported_seq":2792,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.748734+0000","last_change":"2024-02-22T19:43:57.349932+0000","last_active":"2024-02-22T19:44:34.748734+0000","last_peered":"2024-02-22T19:44:34.748734+0000","last_clean":"2024-02-22T19:44:34.748734+0000","last_became_active":"2024-02-22T19:43:57.349636+0000","last_became_peered":"2024-02-22T19:43:57.349636+0000","last_unstale":"2024-02-22T19:44:34.748734+0000","last_undegraded":"2024-02-22T19:44:34.748734+0000","last_fullsized":"2024-02-22T19:44:34.748734+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2762,"ondisk_log_size":2762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":181010432,"num_objects":2762,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2762,"num_write_kb":176768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":351,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366376+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:44:32.366376+0000","last_peered":"2024-02-22T19:44:32.366376+0000","last_clean":"2024-02-22T19:44:32.366376+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:44:32.366376+0000","last_undegraded":"2024-02-22T19:44:32.366376+0000","last_fullsized":"2024-02-22T19:44:32.366376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":432,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392089+0000","last_change":"2024-02-22T19:41:26.310118+0000","last_active":"2024-02-22T19:44:33.392089+0000","last_peered":"2024-02-22T19:44:33.392089+0000","last_clean":"2024-02-22T19:44:33.392089+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:44:33.392089+0000","last_undegraded":"2024-02-22T19:44:33.392089+0000","last_fullsized":"2024-02-22T19:44:33.392089+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.310003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.310003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":389,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367705+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:44:32.367705+0000","last_peered":"2024-02-22T19:44:32.367705+0000","last_clean":"2024-02-22T19:44:32.367705+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:44:32.367705+0000","last_undegraded":"2024-02-22T19:44:32.367705+0000","last_fullsized":"2024-02-22T19:44:32.367705+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":385,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.364088+0000","last_change":"2024-02-22T19:44:19.749503+0000","last_active":"2024-02-22T19:44:32.364088+0000","last_peered":"2024-02-22T19:44:32.364088+0000","last_clean":"2024-02-22T19:44:32.364088+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:44:32.364088+0000","last_undegraded":"2024-02-22T19:44:32.364088+0000","last_fullsized":"2024-02-22T19:44:32.364088+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:19.749430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.749430+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"388'2777","reported_seq":2808,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.568919+0000","last_change":"2024-02-22T19:43:57.347961+0000","last_active":"2024-02-22T19:44:34.568919+0000","last_peered":"2024-02-22T19:44:34.568919+0000","last_clean":"2024-02-22T19:44:34.568919+0000","last_became_active":"2024-02-22T19:43:57.347657+0000","last_became_peered":"2024-02-22T19:43:57.347657+0000","last_unstale":"2024-02-22T19:44:34.568919+0000","last_undegraded":"2024-02-22T19:44:34.568919+0000","last_fullsized":"2024-02-22T19:44:34.568919+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2777,"ondisk_log_size":2777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":181993472,"num_objects":2777,"num_object_clones":0,"num_object_copies":8331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2777,"num_write_kb":177728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":173,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366975+0000","last_change":"2024-02-22T19:40:51.594081+0000","last_active":"2024-02-22T19:44:32.366975+0000","last_peered":"2024-02-22T19:44:32.366975+0000","last_clean":"2024-02-22T19:44:32.366975+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:44:32.366975+0000","last_undegraded":"2024-02-22T19:44:32.366975+0000","last_fullsized":"2024-02-22T19:44:32.366975+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:51.593979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:51.593979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":416,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.391328+0000","last_change":"2024-02-22T19:44:17.122626+0000","last_active":"2024-02-22T19:44:33.391328+0000","last_peered":"2024-02-22T19:44:33.391328+0000","last_clean":"2024-02-22T19:44:33.391328+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:44:33.391328+0000","last_undegraded":"2024-02-22T19:44:33.391328+0000","last_fullsized":"2024-02-22T19:44:33.391328+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.122559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.122559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":193,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370583+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:44:32.370583+0000","last_peered":"2024-02-22T19:44:32.370583+0000","last_clean":"2024-02-22T19:44:32.370583+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:44:32.370583+0000","last_undegraded":"2024-02-22T19:44:32.370583+0000","last_fullsized":"2024-02-22T19:44:32.370583+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":191,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392285+0000","last_change":"2024-02-22T19:44:11.175908+0000","last_active":"2024-02-22T19:44:33.392285+0000","last_peered":"2024-02-22T19:44:33.392285+0000","last_clean":"2024-02-22T19:44:33.392285+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:44:33.392285+0000","last_undegraded":"2024-02-22T19:44:33.392285+0000","last_fullsized":"2024-02-22T19:44:33.392285+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:44:11.175784+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.175784+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"387'2872","reported_seq":2902,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.439139+0000","last_change":"2024-02-22T19:43:57.351947+0000","last_active":"2024-02-22T19:44:34.439139+0000","last_peered":"2024-02-22T19:44:34.439139+0000","last_clean":"2024-02-22T19:44:34.439139+0000","last_became_active":"2024-02-22T19:43:57.351647+0000","last_became_peered":"2024-02-22T19:43:57.351647+0000","last_unstale":"2024-02-22T19:44:34.439139+0000","last_undegraded":"2024-02-22T19:44:34.439139+0000","last_fullsized":"2024-02-22T19:44:34.439139+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2872,"ondisk_log_size":2872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":188219392,"num_objects":2872,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2872,"num_write_kb":183808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":220,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369403+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:44:32.369403+0000","last_peered":"2024-02-22T19:44:32.369403+0000","last_clean":"2024-02-22T19:44:32.369403+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:44:32.369403+0000","last_undegraded":"2024-02-22T19:44:32.369403+0000","last_fullsized":"2024-02-22T19:44:32.369403+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":369,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379486+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:44:33.379486+0000","last_peered":"2024-02-22T19:44:33.379486+0000","last_clean":"2024-02-22T19:44:33.379486+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:44:33.379486+0000","last_undegraded":"2024-02-22T19:44:33.379486+0000","last_fullsized":"2024-02-22T19:44:33.379486+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":193,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369698+0000","last_change":"2024-02-22T19:43:57.722289+0000","last_active":"2024-02-22T19:44:32.369698+0000","last_peered":"2024-02-22T19:44:32.369698+0000","last_clean":"2024-02-22T19:44:32.369698+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:44:32.369698+0000","last_undegraded":"2024-02-22T19:44:32.369698+0000","last_fullsized":"2024-02-22T19:44:32.369698+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.722226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.722226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":308,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.362828+0000","last_change":"2024-02-22T19:44:16.734407+0000","last_active":"2024-02-22T19:44:32.362828+0000","last_peered":"2024-02-22T19:44:32.362828+0000","last_clean":"2024-02-22T19:44:32.362828+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:44:32.362828+0000","last_undegraded":"2024-02-22T19:44:32.362828+0000","last_fullsized":"2024-02-22T19:44:32.362828+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:16.734314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:16.734314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":622,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363767+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:44:32.363767+0000","last_peered":"2024-02-22T19:44:32.363767+0000","last_clean":"2024-02-22T19:44:32.363767+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:44:32.363767+0000","last_undegraded":"2024-02-22T19:44:32.363767+0000","last_fullsized":"2024-02-22T19:44:32.363767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"382'15","reported_seq":214,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.390295+0000","last_change":"2024-02-22T19:44:33.390295+0000","last_active":"2024-02-22T19:44:33.390295+0000","last_peered":"2024-02-22T19:44:33.390295+0000","last_clean":"2024-02-22T19:44:33.390295+0000","last_became_active":"2024-02-22T19:44:33.389965+0000","last_became_peered":"2024-02-22T19:44:33.389965+0000","last_unstale":"2024-02-22T19:44:33.390295+0000","last_undegraded":"2024-02-22T19:44:33.390295+0000","last_fullsized":"2024-02-22T19:44:33.390295+0000","mapping_epoch":322,"log_start":"382'15","ondisk_log_start":"382'15","created":60,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.782332+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"387'2862","reported_seq":2892,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.403338+0000","last_change":"2024-02-22T19:43:57.347092+0000","last_active":"2024-02-22T19:44:34.403338+0000","last_peered":"2024-02-22T19:44:34.403338+0000","last_clean":"2024-02-22T19:44:34.403338+0000","last_became_active":"2024-02-22T19:43:57.346743+0000","last_became_peered":"2024-02-22T19:43:57.346743+0000","last_unstale":"2024-02-22T19:44:34.403338+0000","last_undegraded":"2024-02-22T19:44:34.403338+0000","last_fullsized":"2024-02-22T19:44:34.403338+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2862,"ondisk_log_size":2862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":187564032,"num_objects":2862,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2862,"num_write_kb":183168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":410,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366269+0000","last_change":"2024-02-22T19:40:34.605333+0000","last_active":"2024-02-22T19:44:32.366269+0000","last_peered":"2024-02-22T19:44:32.366269+0000","last_clean":"2024-02-22T19:44:32.366269+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:44:32.366269+0000","last_undegraded":"2024-02-22T19:44:32.366269+0000","last_fullsized":"2024-02-22T19:44:32.366269+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:34.605236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.605236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":365,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367892+0000","last_change":"2024-02-22T19:44:12.295154+0000","last_active":"2024-02-22T19:44:32.367892+0000","last_peered":"2024-02-22T19:44:32.367892+0000","last_clean":"2024-02-22T19:44:32.367892+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:44:32.367892+0000","last_undegraded":"2024-02-22T19:44:32.367892+0000","last_fullsized":"2024-02-22T19:44:32.367892+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.295048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.295048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":373,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.364171+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:44:32.364171+0000","last_peered":"2024-02-22T19:44:32.364171+0000","last_clean":"2024-02-22T19:44:32.364171+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:44:32.364171+0000","last_undegraded":"2024-02-22T19:44:32.364171+0000","last_fullsized":"2024-02-22T19:44:32.364171+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.385828+0000","last_change":"2024-02-22T19:41:31.395953+0000","last_active":"2024-02-22T19:44:33.385828+0000","last_peered":"2024-02-22T19:44:33.385828+0000","last_clean":"2024-02-22T19:44:33.385828+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:44:33.385828+0000","last_undegraded":"2024-02-22T19:44:33.385828+0000","last_fullsized":"2024-02-22T19:44:33.385828+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.395856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.395856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":467,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392798+0000","last_change":"2024-02-22T19:44:20.206193+0000","last_active":"2024-02-22T19:44:33.392798+0000","last_peered":"2024-02-22T19:44:33.392798+0000","last_clean":"2024-02-22T19:44:33.392798+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:44:33.392798+0000","last_undegraded":"2024-02-22T19:44:33.392798+0000","last_fullsized":"2024-02-22T19:44:33.392798+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:44:20.206076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:20.206076+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":405,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392915+0000","last_change":"2024-02-22T19:41:34.438109+0000","last_active":"2024-02-22T19:44:33.392915+0000","last_peered":"2024-02-22T19:44:33.392915+0000","last_clean":"2024-02-22T19:44:33.392915+0000","last_became_active":"2024-02-22T19:38:09.218708+0000","last_became_peered":"2024-02-22T19:38:09.218708+0000","last_unstale":"2024-02-22T19:44:33.392915+0000","last_undegraded":"2024-02-22T19:44:33.392915+0000","last_fullsized":"2024-02-22T19:44:33.392915+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.438008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.438008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"387'2913","reported_seq":2943,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.145685+0000","last_change":"2024-02-22T19:43:57.344606+0000","last_active":"2024-02-22T19:44:34.145685+0000","last_peered":"2024-02-22T19:44:34.145685+0000","last_clean":"2024-02-22T19:44:34.145685+0000","last_became_active":"2024-02-22T19:43:57.344303+0000","last_became_peered":"2024-02-22T19:43:57.344303+0000","last_unstale":"2024-02-22T19:44:34.145685+0000","last_undegraded":"2024-02-22T19:44:34.145685+0000","last_fullsized":"2024-02-22T19:44:34.145685+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2913,"ondisk_log_size":2913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":190906368,"num_objects":2913,"num_object_clones":0,"num_object_copies":8739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2913,"num_write_kb":186432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":198,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.371399+0000","last_change":"2024-02-22T19:44:17.656581+0000","last_active":"2024-02-22T19:44:32.371399+0000","last_peered":"2024-02-22T19:44:32.371399+0000","last_clean":"2024-02-22T19:44:32.371399+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:44:32.371399+0000","last_undegraded":"2024-02-22T19:44:32.371399+0000","last_fullsized":"2024-02-22T19:44:32.371399+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.656518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":410,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.366501+0000","last_change":"2024-02-22T19:41:52.104562+0000","last_active":"2024-02-22T19:44:32.366501+0000","last_peered":"2024-02-22T19:44:32.366501+0000","last_clean":"2024-02-22T19:44:32.366501+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:44:32.366501+0000","last_undegraded":"2024-02-22T19:44:32.366501+0000","last_fullsized":"2024-02-22T19:44:32.366501+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:47.591625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:47.591625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":384,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379695+0000","last_change":"2024-02-22T19:44:09.027439+0000","last_active":"2024-02-22T19:44:33.379695+0000","last_peered":"2024-02-22T19:44:33.379695+0000","last_clean":"2024-02-22T19:44:33.379695+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:44:33.379695+0000","last_undegraded":"2024-02-22T19:44:33.379695+0000","last_fullsized":"2024-02-22T19:44:33.379695+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.027320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":418,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367538+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:44:32.367538+0000","last_peered":"2024-02-22T19:44:32.367538+0000","last_clean":"2024-02-22T19:44:32.367538+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:44:32.367538+0000","last_undegraded":"2024-02-22T19:44:32.367538+0000","last_fullsized":"2024-02-22T19:44:32.367538+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":371,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367963+0000","last_change":"2024-02-22T19:40:42.672768+0000","last_active":"2024-02-22T19:44:32.367963+0000","last_peered":"2024-02-22T19:44:32.367963+0000","last_clean":"2024-02-22T19:44:32.367963+0000","last_became_active":"2024-02-22T19:37:57.175635+0000","last_became_peered":"2024-02-22T19:37:57.175635+0000","last_unstale":"2024-02-22T19:44:32.367963+0000","last_undegraded":"2024-02-22T19:44:32.367963+0000","last_fullsized":"2024-02-22T19:44:32.367963+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:42.672670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:42.672670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"388'2740","reported_seq":2771,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.545964+0000","last_change":"2024-02-22T19:43:57.346858+0000","last_active":"2024-02-22T19:44:34.545964+0000","last_peered":"2024-02-22T19:44:34.545964+0000","last_clean":"2024-02-22T19:44:34.545964+0000","last_became_active":"2024-02-22T19:43:57.346554+0000","last_became_peered":"2024-02-22T19:43:57.346554+0000","last_unstale":"2024-02-22T19:44:34.545964+0000","last_undegraded":"2024-02-22T19:44:34.545964+0000","last_fullsized":"2024-02-22T19:44:34.545964+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2740,"ondisk_log_size":2740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":179568640,"num_objects":2740,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2740,"num_write_kb":175360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":264,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363061+0000","last_change":"2024-02-22T19:41:52.114247+0000","last_active":"2024-02-22T19:44:32.363061+0000","last_peered":"2024-02-22T19:44:32.363061+0000","last_clean":"2024-02-22T19:44:32.363061+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:44:32.363061+0000","last_undegraded":"2024-02-22T19:44:32.363061+0000","last_fullsized":"2024-02-22T19:44:32.363061+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:41.405409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:41.405409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":376,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363371+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:44:32.363371+0000","last_peered":"2024-02-22T19:44:32.363371+0000","last_clean":"2024-02-22T19:44:32.363371+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:44:32.363371+0000","last_undegraded":"2024-02-22T19:44:32.363371+0000","last_fullsized":"2024-02-22T19:44:32.363371+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":167,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379320+0000","last_change":"2024-02-22T19:44:15.990043+0000","last_active":"2024-02-22T19:44:33.379320+0000","last_peered":"2024-02-22T19:44:33.379320+0000","last_clean":"2024-02-22T19:44:33.379320+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:44:33.379320+0000","last_undegraded":"2024-02-22T19:44:33.379320+0000","last_fullsized":"2024-02-22T19:44:33.379320+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.989937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.989937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1335,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381874+0000","last_change":"2024-02-22T19:44:19.273347+0000","last_active":"2024-02-22T19:44:33.381874+0000","last_peered":"2024-02-22T19:44:33.381874+0000","last_clean":"2024-02-22T19:44:33.381874+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:44:33.381874+0000","last_undegraded":"2024-02-22T19:44:33.381874+0000","last_fullsized":"2024-02-22T19:44:33.381874+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:44:19.273191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.273191+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":916,"num_read_kb":911,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":193,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363548+0000","last_change":"2024-02-22T19:40:43.362570+0000","last_active":"2024-02-22T19:44:32.363548+0000","last_peered":"2024-02-22T19:44:32.363548+0000","last_clean":"2024-02-22T19:44:32.363548+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:44:32.363548+0000","last_undegraded":"2024-02-22T19:44:32.363548+0000","last_fullsized":"2024-02-22T19:44:32.363548+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:40:43.362469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:43.362469+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":238,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.371478+0000","last_change":"2024-02-22T19:40:52.541524+0000","last_active":"2024-02-22T19:44:32.371478+0000","last_peered":"2024-02-22T19:44:32.371478+0000","last_clean":"2024-02-22T19:44:32.371478+0000","last_became_active":"2024-02-22T19:39:22.247868+0000","last_became_peered":"2024-02-22T19:39:22.247868+0000","last_unstale":"2024-02-22T19:44:32.371478+0000","last_undegraded":"2024-02-22T19:44:32.371478+0000","last_fullsized":"2024-02-22T19:44:32.371478+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:40:52.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"388'2798","reported_seq":2829,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.744583+0000","last_change":"2024-02-22T19:43:57.347743+0000","last_active":"2024-02-22T19:44:34.744583+0000","last_peered":"2024-02-22T19:44:34.744583+0000","last_clean":"2024-02-22T19:44:34.744583+0000","last_became_active":"2024-02-22T19:43:57.347455+0000","last_became_peered":"2024-02-22T19:43:57.347455+0000","last_unstale":"2024-02-22T19:44:34.744583+0000","last_undegraded":"2024-02-22T19:44:34.744583+0000","last_fullsized":"2024-02-22T19:44:34.744583+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2798,"ondisk_log_size":2798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":183369728,"num_objects":2798,"num_object_clones":0,"num_object_copies":8394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2798,"num_write_kb":179072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":241,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379163+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:44:33.379163+0000","last_peered":"2024-02-22T19:44:33.379163+0000","last_clean":"2024-02-22T19:44:33.379163+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:44:33.379163+0000","last_undegraded":"2024-02-22T19:44:33.379163+0000","last_fullsized":"2024-02-22T19:44:33.379163+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":362,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.365789+0000","last_change":"2024-02-22T19:41:52.094945+0000","last_active":"2024-02-22T19:44:32.365789+0000","last_peered":"2024-02-22T19:44:32.365789+0000","last_clean":"2024-02-22T19:44:32.365789+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:44:32.365789+0000","last_undegraded":"2024-02-22T19:44:32.365789+0000","last_fullsized":"2024-02-22T19:44:32.365789+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:40.450459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:40:40.450459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":382,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363676+0000","last_change":"2024-02-22T19:44:13.731072+0000","last_active":"2024-02-22T19:44:32.363676+0000","last_peered":"2024-02-22T19:44:32.363676+0000","last_clean":"2024-02-22T19:44:32.363676+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:44:32.363676+0000","last_undegraded":"2024-02-22T19:44:32.363676+0000","last_fullsized":"2024-02-22T19:44:32.363676+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.730975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.730975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":203,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370035+0000","last_change":"2024-02-22T19:44:15.640361+0000","last_active":"2024-02-22T19:44:32.370035+0000","last_peered":"2024-02-22T19:44:32.370035+0000","last_clean":"2024-02-22T19:44:32.370035+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:44:32.370035+0000","last_undegraded":"2024-02-22T19:44:32.370035+0000","last_fullsized":"2024-02-22T19:44:32.370035+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:44:15.640271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.640271+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":347,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369573+0000","last_change":"2024-02-22T19:41:52.107671+0000","last_active":"2024-02-22T19:44:32.369573+0000","last_peered":"2024-02-22T19:44:32.369573+0000","last_clean":"2024-02-22T19:44:32.369573+0000","last_became_active":"2024-02-22T19:41:52.106767+0000","last_became_peered":"2024-02-22T19:41:52.106767+0000","last_unstale":"2024-02-22T19:44:32.369573+0000","last_undegraded":"2024-02-22T19:44:32.369573+0000","last_fullsized":"2024-02-22T19:44:32.369573+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:49.629120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:39:49.629120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"387'2821","reported_seq":2851,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.365472+0000","last_change":"2024-02-22T19:43:57.351727+0000","last_active":"2024-02-22T19:44:34.365472+0000","last_peered":"2024-02-22T19:44:34.365472+0000","last_clean":"2024-02-22T19:44:34.365472+0000","last_became_active":"2024-02-22T19:43:57.351385+0000","last_became_peered":"2024-02-22T19:43:57.351385+0000","last_unstale":"2024-02-22T19:44:34.365472+0000","last_undegraded":"2024-02-22T19:44:34.365472+0000","last_fullsized":"2024-02-22T19:44:34.365472+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2821,"ondisk_log_size":2821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":184877056,"num_objects":2821,"num_object_clones":0,"num_object_copies":8463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2821,"num_write_kb":180544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.383460+0000","last_change":"2024-02-22T19:41:52.110027+0000","last_active":"2024-02-22T19:44:33.383460+0000","last_peered":"2024-02-22T19:44:33.383460+0000","last_clean":"2024-02-22T19:44:33.383460+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:44:33.383460+0000","last_undegraded":"2024-02-22T19:44:33.383460+0000","last_fullsized":"2024-02-22T19:44:33.383460+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.670303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":174,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380131+0000","last_change":"2024-02-22T19:40:35.695758+0000","last_active":"2024-02-22T19:44:33.380131+0000","last_peered":"2024-02-22T19:44:33.380131+0000","last_clean":"2024-02-22T19:44:33.380131+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:44:33.380131+0000","last_undegraded":"2024-02-22T19:44:33.380131+0000","last_fullsized":"2024-02-22T19:44:33.380131+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:35.695655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:40:35.695655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":184,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370765+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:44:32.370765+0000","last_peered":"2024-02-22T19:44:32.370765+0000","last_clean":"2024-02-22T19:44:32.370765+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:44:32.370765+0000","last_undegraded":"2024-02-22T19:44:32.370765+0000","last_fullsized":"2024-02-22T19:44:32.370765+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":497,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.364152+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:44:32.364152+0000","last_peered":"2024-02-22T19:44:32.364152+0000","last_clean":"2024-02-22T19:44:32.364152+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:44:32.364152+0000","last_undegraded":"2024-02-22T19:44:32.364152+0000","last_fullsized":"2024-02-22T19:44:32.364152+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"326'14","reported_seq":420,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.386368+0000","last_change":"2024-02-22T19:41:52.698274+0000","last_active":"2024-02-22T19:44:33.386368+0000","last_peered":"2024-02-22T19:44:33.386368+0000","last_clean":"2024-02-22T19:44:33.386368+0000","last_became_active":"2024-02-22T19:41:52.112897+0000","last_became_peered":"2024-02-22T19:41:52.112897+0000","last_unstale":"2024-02-22T19:44:33.386368+0000","last_undegraded":"2024-02-22T19:44:33.386368+0000","last_fullsized":"2024-02-22T19:44:33.386368+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"388'2906","reported_seq":2937,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.712585+0000","last_change":"2024-02-22T19:43:57.348552+0000","last_active":"2024-02-22T19:44:34.712585+0000","last_peered":"2024-02-22T19:44:34.712585+0000","last_clean":"2024-02-22T19:44:34.712585+0000","last_became_active":"2024-02-22T19:43:57.348228+0000","last_became_peered":"2024-02-22T19:43:57.348228+0000","last_unstale":"2024-02-22T19:44:34.712585+0000","last_undegraded":"2024-02-22T19:44:34.712585+0000","last_fullsized":"2024-02-22T19:44:34.712585+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2906,"ondisk_log_size":2906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":190447616,"num_objects":2906,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2906,"num_write_kb":185984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":270,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381055+0000","last_change":"2024-02-22T19:43:58.000275+0000","last_active":"2024-02-22T19:44:33.381055+0000","last_peered":"2024-02-22T19:44:33.381055+0000","last_clean":"2024-02-22T19:44:33.381055+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:44:33.381055+0000","last_undegraded":"2024-02-22T19:44:33.381055+0000","last_fullsized":"2024-02-22T19:44:33.381055+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.000163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.000163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":363,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.368455+0000","last_change":"2024-02-22T19:41:52.103942+0000","last_active":"2024-02-22T19:44:32.368455+0000","last_peered":"2024-02-22T19:44:32.368455+0000","last_clean":"2024-02-22T19:44:32.368455+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:44:32.368455+0000","last_undegraded":"2024-02-22T19:44:32.368455+0000","last_fullsized":"2024-02-22T19:44:32.368455+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:31.663769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:40:31.663769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":182,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363312+0000","last_change":"2024-02-22T19:40:36.491280+0000","last_active":"2024-02-22T19:44:32.363312+0000","last_peered":"2024-02-22T19:44:32.363312+0000","last_clean":"2024-02-22T19:44:32.363312+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:44:32.363312+0000","last_undegraded":"2024-02-22T19:44:32.363312+0000","last_fullsized":"2024-02-22T19:44:32.363312+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.491186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.491186+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":499,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369542+0000","last_change":"2024-02-22T19:41:52.715987+0000","last_active":"2024-02-22T19:44:32.369542+0000","last_peered":"2024-02-22T19:44:32.369542+0000","last_clean":"2024-02-22T19:44:32.369542+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:44:32.369542+0000","last_undegraded":"2024-02-22T19:44:32.369542+0000","last_fullsized":"2024-02-22T19:44:32.369542+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:39:46.430010+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:39:46.430010+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"268'11","reported_seq":584,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.369687+0000","last_change":"2024-02-22T19:40:33.643448+0000","last_active":"2024-02-22T19:44:34.369687+0000","last_peered":"2024-02-22T19:44:34.369687+0000","last_clean":"2024-02-22T19:44:34.369687+0000","last_became_active":"2024-02-22T19:39:22.270952+0000","last_became_peered":"2024-02-22T19:39:22.270952+0000","last_unstale":"2024-02-22T19:44:34.369687+0000","last_undegraded":"2024-02-22T19:44:34.369687+0000","last_fullsized":"2024-02-22T19:44:34.369687+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:40:33.643338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:33.643338+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"388'2800","reported_seq":2831,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.581467+0000","last_change":"2024-02-22T19:43:57.348310+0000","last_active":"2024-02-22T19:44:34.581467+0000","last_peered":"2024-02-22T19:44:34.581467+0000","last_clean":"2024-02-22T19:44:34.581467+0000","last_became_active":"2024-02-22T19:43:57.348032+0000","last_became_peered":"2024-02-22T19:43:57.348032+0000","last_unstale":"2024-02-22T19:44:34.581467+0000","last_undegraded":"2024-02-22T19:44:34.581467+0000","last_fullsized":"2024-02-22T19:44:34.581467+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2800,"ondisk_log_size":2800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":183500800,"num_objects":2800,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2800,"num_write_kb":179200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":172,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.393166+0000","last_change":"2024-02-22T19:43:58.006321+0000","last_active":"2024-02-22T19:44:33.393166+0000","last_peered":"2024-02-22T19:44:33.393166+0000","last_clean":"2024-02-22T19:44:33.393166+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:44:33.393166+0000","last_undegraded":"2024-02-22T19:44:33.393166+0000","last_fullsized":"2024-02-22T19:44:33.393166+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.006221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.006221+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":166,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379806+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:44:33.379806+0000","last_peered":"2024-02-22T19:44:33.379806+0000","last_clean":"2024-02-22T19:44:33.379806+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:44:33.379806+0000","last_undegraded":"2024-02-22T19:44:33.379806+0000","last_fullsized":"2024-02-22T19:44:33.379806+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":384,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.383550+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:44:33.383550+0000","last_peered":"2024-02-22T19:44:33.383550+0000","last_clean":"2024-02-22T19:44:33.383550+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:44:33.383550+0000","last_undegraded":"2024-02-22T19:44:33.383550+0000","last_fullsized":"2024-02-22T19:44:33.383550+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":488,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380579+0000","last_change":"2024-02-22T19:40:54.790830+0000","last_active":"2024-02-22T19:44:33.380579+0000","last_peered":"2024-02-22T19:44:33.380579+0000","last_clean":"2024-02-22T19:44:33.380579+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:44:33.380579+0000","last_undegraded":"2024-02-22T19:44:33.380579+0000","last_fullsized":"2024-02-22T19:44:33.380579+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:40:54.790693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.790693+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":175,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.379550+0000","last_change":"2024-02-22T19:40:37.721500+0000","last_active":"2024-02-22T19:44:33.379550+0000","last_peered":"2024-02-22T19:44:33.379550+0000","last_clean":"2024-02-22T19:44:33.379550+0000","last_became_active":"2024-02-22T19:39:22.247338+0000","last_became_peered":"2024-02-22T19:39:22.247338+0000","last_unstale":"2024-02-22T19:44:33.379550+0000","last_undegraded":"2024-02-22T19:44:33.379550+0000","last_fullsized":"2024-02-22T19:44:33.379550+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:37.721407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:37.721407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"387'2815","reported_seq":2845,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.400043+0000","last_change":"2024-02-22T19:43:57.347525+0000","last_active":"2024-02-22T19:44:34.400043+0000","last_peered":"2024-02-22T19:44:34.400043+0000","last_clean":"2024-02-22T19:44:34.400043+0000","last_became_active":"2024-02-22T19:43:57.347224+0000","last_became_peered":"2024-02-22T19:43:57.347224+0000","last_unstale":"2024-02-22T19:44:34.400043+0000","last_undegraded":"2024-02-22T19:44:34.400043+0000","last_fullsized":"2024-02-22T19:44:34.400043+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2815,"ondisk_log_size":2815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":184483840,"num_objects":2815,"num_object_clones":0,"num_object_copies":8445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":180160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":196,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369778+0000","last_change":"2024-02-22T19:41:52.107922+0000","last_active":"2024-02-22T19:44:32.369778+0000","last_peered":"2024-02-22T19:44:32.369778+0000","last_clean":"2024-02-22T19:44:32.369778+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:44:32.369778+0000","last_undegraded":"2024-02-22T19:44:32.369778+0000","last_fullsized":"2024-02-22T19:44:32.369778+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:36.433281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:36.433281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":211,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370602+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:44:32.370602+0000","last_peered":"2024-02-22T19:44:32.370602+0000","last_clean":"2024-02-22T19:44:32.370602+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:44:32.370602+0000","last_undegraded":"2024-02-22T19:44:32.370602+0000","last_fullsized":"2024-02-22T19:44:32.370602+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"382'29","reported_seq":448,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.371955+0000","last_change":"2024-02-22T19:44:28.376815+0000","last_active":"2024-02-22T19:44:34.371955+0000","last_peered":"2024-02-22T19:44:34.371955+0000","last_clean":"2024-02-22T19:44:34.371955+0000","last_became_active":"2024-02-22T19:44:28.376449+0000","last_became_peered":"2024-02-22T19:44:28.376449+0000","last_unstale":"2024-02-22T19:44:34.371955+0000","last_undegraded":"2024-02-22T19:44:34.371955+0000","last_fullsized":"2024-02-22T19:44:34.371955+0000","mapping_epoch":381,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'26","last_scrub_stamp":"2024-02-22T19:40:34.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:40:34.372237+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":362,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.380421+0000","last_change":"2024-02-22T19:44:12.048204+0000","last_active":"2024-02-22T19:44:33.380421+0000","last_peered":"2024-02-22T19:44:33.380421+0000","last_clean":"2024-02-22T19:44:33.380421+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:44:33.380421+0000","last_undegraded":"2024-02-22T19:44:33.380421+0000","last_fullsized":"2024-02-22T19:44:33.380421+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.048068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.048068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":100,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.370938+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:44:32.370938+0000","last_peered":"2024-02-22T19:44:32.370938+0000","last_clean":"2024-02-22T19:44:32.370938+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:44:32.370938+0000","last_undegraded":"2024-02-22T19:44:32.370938+0000","last_fullsized":"2024-02-22T19:44:32.370938+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"387'2710","reported_seq":2740,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.184564+0000","last_change":"2024-02-22T19:43:57.346250+0000","last_active":"2024-02-22T19:44:34.184564+0000","last_peered":"2024-02-22T19:44:34.184564+0000","last_clean":"2024-02-22T19:44:34.184564+0000","last_became_active":"2024-02-22T19:43:57.345992+0000","last_became_peered":"2024-02-22T19:43:57.345992+0000","last_unstale":"2024-02-22T19:44:34.184564+0000","last_undegraded":"2024-02-22T19:44:34.184564+0000","last_fullsized":"2024-02-22T19:44:34.184564+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2710,"ondisk_log_size":2710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":177602560,"num_objects":2710,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2710,"num_write_kb":173440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":373,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381192+0000","last_change":"2024-02-22T19:41:52.112553+0000","last_active":"2024-02-22T19:44:33.381192+0000","last_peered":"2024-02-22T19:44:33.381192+0000","last_clean":"2024-02-22T19:44:33.381192+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:44:33.381192+0000","last_undegraded":"2024-02-22T19:44:33.381192+0000","last_fullsized":"2024-02-22T19:44:33.381192+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:32.668407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:32.668407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"382'23","reported_seq":471,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.386511+0000","last_change":"2024-02-22T19:44:28.374931+0000","last_active":"2024-02-22T19:44:33.386511+0000","last_peered":"2024-02-22T19:44:33.386511+0000","last_clean":"2024-02-22T19:44:33.386511+0000","last_became_active":"2024-02-22T19:44:28.374620+0000","last_became_peered":"2024-02-22T19:44:28.374620+0000","last_unstale":"2024-02-22T19:44:33.386511+0000","last_undegraded":"2024-02-22T19:44:33.386511+0000","last_fullsized":"2024-02-22T19:44:33.386511+0000","mapping_epoch":381,"log_start":"190'10","ondisk_log_start":"190'10","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'17","last_scrub_stamp":"2024-02-22T19:41:28.401560+0000","last_deep_scrub":"137'9","last_deep_scrub_stamp":"2024-02-22T19:34:00.780574+0000","last_clean_scrub_stamp":"2024-02-22T19:41:28.401560+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":167,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.385935+0000","last_change":"2024-02-22T19:44:06.172481+0000","last_active":"2024-02-22T19:44:33.385935+0000","last_peered":"2024-02-22T19:44:33.385935+0000","last_clean":"2024-02-22T19:44:33.385935+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:44:33.385935+0000","last_undegraded":"2024-02-22T19:44:33.385935+0000","last_fullsized":"2024-02-22T19:44:33.385935+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:06.172363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:06.172363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":174,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363458+0000","last_change":"2024-02-22T19:44:11.716360+0000","last_active":"2024-02-22T19:44:32.363458+0000","last_peered":"2024-02-22T19:44:32.363458+0000","last_clean":"2024-02-22T19:44:32.363458+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:44:32.363458+0000","last_undegraded":"2024-02-22T19:44:32.363458+0000","last_fullsized":"2024-02-22T19:44:32.363458+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:44:11.716223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.716223+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3","version":"388'2857","reported_seq":2887,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.585810+0000","last_change":"2024-02-22T19:43:57.348166+0000","last_active":"2024-02-22T19:44:34.585810+0000","last_peered":"2024-02-22T19:44:34.585810+0000","last_clean":"2024-02-22T19:44:34.585810+0000","last_became_active":"2024-02-22T19:43:57.347832+0000","last_became_peered":"2024-02-22T19:43:57.347832+0000","last_unstale":"2024-02-22T19:44:34.585810+0000","last_undegraded":"2024-02-22T19:44:34.585810+0000","last_fullsized":"2024-02-22T19:44:34.585810+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2857,"ondisk_log_size":2857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":187236352,"num_objects":2857,"num_object_clones":0,"num_object_copies":8571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2857,"num_write_kb":182848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":184,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.369461+0000","last_change":"2024-02-22T19:39:54.478919+0000","last_active":"2024-02-22T19:44:32.369461+0000","last_peered":"2024-02-22T19:44:32.369461+0000","last_clean":"2024-02-22T19:44:32.369461+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:44:32.369461+0000","last_undegraded":"2024-02-22T19:44:32.369461+0000","last_fullsized":"2024-02-22T19:44:32.369461+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:39:54.478847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:39:54.478847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":382,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392676+0000","last_change":"2024-02-22T19:44:05.228957+0000","last_active":"2024-02-22T19:44:33.392676+0000","last_peered":"2024-02-22T19:44:33.392676+0000","last_clean":"2024-02-22T19:44:33.392676+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:44:33.392676+0000","last_undegraded":"2024-02-22T19:44:33.392676+0000","last_fullsized":"2024-02-22T19:44:33.392676+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:05.228870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:05.228870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":365,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367762+0000","last_change":"2024-02-22T19:44:25.421235+0000","last_active":"2024-02-22T19:44:32.367762+0000","last_peered":"2024-02-22T19:44:32.367762+0000","last_clean":"2024-02-22T19:44:32.367762+0000","last_became_active":"2024-02-22T19:44:25.420992+0000","last_became_peered":"2024-02-22T19:44:25.420992+0000","last_unstale":"2024-02-22T19:44:32.367762+0000","last_undegraded":"2024-02-22T19:44:32.367762+0000","last_fullsized":"2024-02-22T19:44:32.367762+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:06.753748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:06.753748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":298,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363975+0000","last_change":"2024-02-22T19:44:12.719560+0000","last_active":"2024-02-22T19:44:32.363975+0000","last_peered":"2024-02-22T19:44:32.363975+0000","last_clean":"2024-02-22T19:44:32.363975+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:44:32.363975+0000","last_undegraded":"2024-02-22T19:44:32.363975+0000","last_fullsized":"2024-02-22T19:44:32.363975+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.719494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.719494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":268,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.392632+0000","last_change":"2024-02-22T19:44:09.132549+0000","last_active":"2024-02-22T19:44:33.392632+0000","last_peered":"2024-02-22T19:44:33.392632+0000","last_clean":"2024-02-22T19:44:33.392632+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:44:33.392632+0000","last_undegraded":"2024-02-22T19:44:33.392632+0000","last_fullsized":"2024-02-22T19:44:33.392632+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:44:09.132441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.132441+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"387'2787","reported_seq":2817,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:34.158842+0000","last_change":"2024-02-22T19:43:57.351793+0000","last_active":"2024-02-22T19:44:34.158842+0000","last_peered":"2024-02-22T19:44:34.158842+0000","last_clean":"2024-02-22T19:44:34.158842+0000","last_became_active":"2024-02-22T19:43:57.351420+0000","last_became_peered":"2024-02-22T19:43:57.351420+0000","last_unstale":"2024-02-22T19:44:34.158842+0000","last_undegraded":"2024-02-22T19:44:34.158842+0000","last_fullsized":"2024-02-22T19:44:34.158842+0000","mapping_epoch":365,"log_start":"0'0","ondisk_log_start":"0'0","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":2787,"ondisk_log_size":2787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":182648832,"num_objects":2787,"num_object_clones":0,"num_object_copies":8361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2787,"num_write_kb":178368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":374,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.367058+0000","last_change":"2024-02-22T19:41:16.724733+0000","last_active":"2024-02-22T19:44:32.367058+0000","last_peered":"2024-02-22T19:44:32.367058+0000","last_clean":"2024-02-22T19:44:32.367058+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:44:32.367058+0000","last_undegraded":"2024-02-22T19:44:32.367058+0000","last_fullsized":"2024-02-22T19:44:32.367058+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.724625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.724625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":200,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.363159+0000","last_change":"2024-02-22T19:41:24.278823+0000","last_active":"2024-02-22T19:44:32.363159+0000","last_peered":"2024-02-22T19:44:32.363159+0000","last_clean":"2024-02-22T19:44:32.363159+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:44:32.363159+0000","last_undegraded":"2024-02-22T19:44:32.363159+0000","last_fullsized":"2024-02-22T19:44:32.363159+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:41:24.278750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.278750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"382'17","reported_seq":645,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-02-22T19:44:32.362936+0000","last_change":"2024-02-22T19:44:28.380051+0000","last_active":"2024-02-22T19:44:32.362936+0000","last_peered":"2024-02-22T19:44:32.362936+0000","last_clean":"2024-02-22T19:44:32.362936+0000","last_became_active":"2024-02-22T19:44:28.379682+0000","last_became_peered":"2024-02-22T19:44:28.379682+0000","last_unstale":"2024-02-22T19:44:32.362936+0000","last_undegraded":"2024-02-22T19:44:32.362936+0000","last_fullsized":"2024-02-22T19:44:32.362936+0000","mapping_epoch":381,"log_start":"100'9","ondisk_log_start":"100'9","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"221'14","last_scrub_stamp":"2024-02-22T19:41:32.453127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.453127+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":418,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381219+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:44:33.381219+0000","last_peered":"2024-02-22T19:44:33.381219+0000","last_clean":"2024-02-22T19:44:33.381219+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:44:33.381219+0000","last_undegraded":"2024-02-22T19:44:33.381219+0000","last_fullsized":"2024-02-22T19:44:33.381219+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":232,"reported_epoch":387,"state":"active+clean","last_fresh":"2024-02-22T19:44:33.381401+0000","last_change":"2024-02-22T19:41:52.575055+0000","last_active":"2024-02-22T19:44:33.381401+0000","last_peered":"2024-02-22T19:44:33.381401+0000","last_clean":"2024-02-22T19:44:33.381401+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:44:33.381401+0000","last_undegraded":"2024-02-22T19:44:33.381401+0000","last_fullsized":"2024-02-22T19:44:33.381401+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:39:53.646669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:39:53.646669+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":2958950400,"num_objects":45150,"num_object_clones":0,"num_object_copies":135450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45150,"num_write_kb":2889600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8880259072,"data_stored":8880259072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45150,"ondisk_log_size":45150,"up":48,"acting":48,"num_store_stats":5},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":916,"num_read_kb":911,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":20,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":60,"acting":60,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266878,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2194764,"kb_used_data":1609156,"kb_used_omap":0,"kb_used_meta":585600,"kb_avail":91550388,"statfs":{"total":95995035648,"available":93747597312,"internally_reserved":0,"allocated":1647775744,"data_stored":1646449708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":599654400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.415,"15min":1.524},"min":{"1min":0.252,"5min":0.245,"15min":0.19900000000000001},"max":{"1min":5.516,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.075,"15min":1.4430000000000001},"min":{"1min":0.28599999999999998,"5min":0.25600000000000001,"15min":0.20699999999999999},"max":{"1min":4.0339999999999998,"5min":7.827,"15min":42.155999999999999},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":1.8600000000000001,"15min":1.764},"min":{"1min":0.32300000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.9079999999999999,"5min":36.509,"15min":54.951000000000001},"last":0.37},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.2230000000000001,"15min":1.9399999999999999},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.214},"max":{"1min":3.984,"5min":6.5449999999999999,"15min":57.012},"last":4.9809999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.2470000000000001,"15min":1.4650000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":3.4740000000000002,"5min":6.7489999999999997,"15min":41.640999999999998},"last":2.4180000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.635,"15min":1.7769999999999999},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.23599999999999999},"max":{"1min":4.8959999999999999,"5min":8.4879999999999995,"15min":41.722000000000001},"last":2.036}]},{"osd":3,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.958,"15min":1.893},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.29499999999999998},"max":{"1min":5.2720000000000002,"5min":36.491999999999997,"15min":41.457000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.014,"5min":2.161,"15min":1.9399999999999999},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.9340000000000002,"5min":36.732999999999997,"15min":41.311999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.88300000000000001,"15min":0.71999999999999997},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":3.9039999999999999,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.315},{"interface":"front","average":{"1min":0.624,"5min":1.147,"15min":0.76600000000000001},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":2.4279999999999999,"5min":5.1790000000000003,"15min":7.6219999999999999},"last":0.439}]},{"osd":5,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":1.347,"15min":0.86399999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":1.1930000000000001,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.97699999999999998,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.29399999999999998,"15min":0.28000000000000003},"max":{"1min":4.008,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:44:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.85899999999999999,"15min":0.82699999999999996},"min":{"1min":0.42899999999999999,"5min":0.313,"15min":0.27900000000000003},"max":{"1min":3.956,"5min":6.5,"15min":12.199999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.423,"15min":0.93999999999999995},"min":{"1min":0.39100000000000001,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":6.6719999999999997,"5min":7.1230000000000002,"15min":12.067},"last":0.56899999999999995}]}]},{"osd":6,"up_from":0,"seq":193273528515,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2007128,"kb_used_data":1417424,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":91738024,"statfs":{"total":95995035648,"available":93939736576,"internally_reserved":0,"allocated":1451442176,"data_stored":1450103828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:44:05 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":2.331,"15min":1.921},"min":{"1min":0.245,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":6.6760000000000002,"5min":51.091000000000001,"15min":51.091000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.081,"5min":2.2269999999999999,"15min":1.9930000000000001},"min":{"1min":0.215,"5min":0.20999999999999999,"15min":0.17499999999999999},"max":{"1min":4.9370000000000003,"5min":51.289999999999999,"15min":51.289999999999999},"last":4.4299999999999997}]},{"osd":2,"last update":"Thu Feb 22 19:44:18 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":2.2170000000000001,"15min":2.0190000000000001},"min":{"1min":0.307,"5min":0.191,"15min":0.191},"max":{"1min":3.4590000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":3.4260000000000002},{"interface":"front","average":{"1min":1.76,"5min":2.2850000000000001,"15min":2.1619999999999999},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":6.0679999999999996,"5min":51.369999999999997,"15min":51.369999999999997},"last":4.4009999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:43:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":2.4119999999999999,"15min":2.145},"min":{"1min":0.28999999999999998,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":5.8040000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":3.8519999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":2.5419999999999998,"15min":2.206},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":6.6449999999999996,"5min":51.741,"15min":51.741},"last":0.30299999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":2.407,"15min":1.728},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.24399999999999999},"max":{"1min":2.762,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":1.673,"15min":1.387},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.245},"max":{"1min":2.1280000000000001,"5min":51.276000000000003,"15min":51.276000000000003},"last":3.9409999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:43:32 2024","interfaces":[{"interface":"back","average":{"1min":5.1959999999999997,"5min":2.7109999999999999,"15min":1.8009999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":51.183999999999997,"5min":51.183999999999997,"15min":51.183999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":4.1459999999999999,"5min":2.2629999999999999,"15min":1.6100000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":51.212000000000003,"5min":51.212000000000003,"15min":51.212000000000003},"last":4.3739999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:43:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.8069999999999999,"15min":1.4770000000000001},"min":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.28599999999999998},"max":{"1min":5.1879999999999997,"5min":51.079999999999998,"15min":51.079999999999998},"last":3.9740000000000002},{"interface":"front","average":{"1min":1.954,"5min":2.544,"15min":1.78},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.34999999999999998},"max":{"1min":8.4269999999999996,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.502}]},{"osd":7,"last update":"Thu Feb 22 19:44:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":2.3500000000000001,"15min":1.8220000000000001},"min":{"1min":0.38200000000000001,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":3.1190000000000002,"5min":51.244999999999997,"15min":51.244999999999997},"last":4.3380000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.8700000000000001,"15min":1.514},"min":{"1min":0.41399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.149,"5min":51.317999999999998,"15min":51.317999999999998},"last":3.9049999999999998}]}]},{"osd":0,"up_from":0,"seq":34359738593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411542,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2947780,"kb_used_data":2353980,"kb_used_omap":0,"kb_used_meta":593792,"kb_avail":90797372,"statfs":{"total":95995035648,"available":92976508928,"internally_reserved":0,"allocated":2410475520,"data_stored":2409155943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":608043008},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.4770000000000001,"15min":1.7529999999999999},"min":{"1min":0.217,"5min":0.19900000000000001,"15min":0.18099999999999999},"max":{"1min":3.5089999999999999,"5min":16.137,"15min":22.338999999999999},"last":2.04},{"interface":"front","average":{"1min":1.196,"5min":1.8169999999999999,"15min":2.1629999999999998},"min":{"1min":0.28599999999999998,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":9.3580000000000005,"5min":27.361999999999998,"15min":30.427},"last":1.9350000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.548,"15min":1.732},"min":{"1min":0.27400000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.516,"5min":12.349,"15min":12.349},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.71,"5min":1.6459999999999999,"15min":1.6679999999999999},"min":{"1min":0.249,"5min":0.215,"15min":0.193},"max":{"1min":6.6600000000000001,"5min":12.18,"15min":13.782},"last":3.1339999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":2.0379999999999998,"15min":2.0379999999999998},"min":{"1min":0.27000000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":8.8599999999999994,"5min":16.076000000000001,"15min":30.071000000000002},"last":2.4649999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.4940000000000002,"15min":2.3239999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.184},"max":{"1min":9.3149999999999995,"5min":28.263999999999999,"15min":32.119},"last":2.0219999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.357,"15min":1.3899999999999999},"min":{"1min":0.41299999999999998,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":5.3390000000000004,"5min":12.43,"15min":12.43},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.4890000000000001,"15min":1.27},"min":{"1min":0.38600000000000001,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":2.117,"5min":10.34,"15min":10.34},"last":2.2050000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:44:29 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.8879999999999999,"15min":1.5149999999999999},"min":{"1min":0.47099999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.431,"5min":40.633000000000003,"15min":40.633000000000003},"last":2.125},{"interface":"front","average":{"1min":1.258,"5min":2.0950000000000002,"15min":1.73},"min":{"1min":0.373,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":5.4329999999999998,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.373}]},{"osd":6,"last update":"Thu Feb 22 19:44:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.655,"15min":1.5309999999999999},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.311},"max":{"1min":5.3730000000000002,"5min":21.640000000000001,"15min":21.640000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.099,"5min":1.6339999999999999,"15min":1.335},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.29899999999999999},"max":{"1min":2.6440000000000001,"5min":10.369999999999999,"15min":10.369999999999999},"last":2.2250000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.615,"15min":1.5600000000000001},"min":{"1min":0.41299999999999998,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.4020000000000001,"5min":12.484,"15min":12.484},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.627,"15min":1.474},"min":{"1min":0.44600000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":4.0529999999999999,"5min":10.539,"15min":10.539},"last":2.1739999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149904,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2679832,"kb_used_data":1775888,"kb_used_omap":0,"kb_used_meta":903936,"kb_avail":91065320,"statfs":{"total":95995035648,"available":93250887680,"internally_reserved":0,"allocated":1818509312,"data_stored":1817170857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":925630464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.9410000000000001,"15min":1.901},"min":{"1min":0.26800000000000002,"5min":0.191,"15min":0.191},"max":{"1min":6.3220000000000001,"5min":9.9789999999999992,"15min":23.48},"last":0.82499999999999996},{"interface":"front","average":{"1min":2.2000000000000002,"5min":2.2370000000000001,"15min":2.2280000000000002},"min":{"1min":0.24199999999999999,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":10.297000000000001,"5min":18.172000000000001,"15min":23.411999999999999},"last":3.0600000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":2.4169999999999998,"5min":2.8530000000000002,"15min":2.5569999999999999},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":11.385999999999999,"5min":23.077999999999999,"15min":23.077999999999999},"last":2.9340000000000002},{"interface":"front","average":{"1min":1.51,"5min":2.2639999999999998,"15min":2.044},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":6.8040000000000003,"5min":22.187000000000001,"15min":22.187000000000001},"last":2.2010000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:43:54 2024","interfaces":[{"interface":"back","average":{"1min":2.2050000000000001,"5min":2.2650000000000001,"15min":2.1080000000000001},"min":{"1min":0.28199999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.577,"5min":20.105,"15min":20.105},"last":1.105},{"interface":"front","average":{"1min":1.9770000000000001,"5min":2.4660000000000002,"15min":2.0779999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.217},"max":{"1min":6.8390000000000004,"5min":35.939999999999998,"15min":35.939999999999998},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.9450000000000001,"15min":1.804},"min":{"1min":0.38100000000000001,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":5.3760000000000003,"5min":12.416,"15min":13.680999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.508,"15min":1.621},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.27400000000000002},"max":{"1min":7.6479999999999997,"5min":12.273,"15min":13.548},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:43:35 2024","interfaces":[{"interface":"back","average":{"1min":2.1099999999999999,"5min":1.9570000000000001,"15min":1.5800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1159999999999997,"5min":11.161,"15min":12.879},"last":3.0840000000000001},{"interface":"front","average":{"1min":2.7080000000000002,"5min":2.2029999999999998,"15min":1.7829999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":10.041,"5min":12.372999999999999,"15min":13.707000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:43:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.879,"15min":1.573},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":6.3109999999999999,"5min":11.263,"15min":12.898999999999999},"last":2.9580000000000002},{"interface":"front","average":{"1min":2.0840000000000001,"5min":2.1320000000000001,"15min":1.867},"min":{"1min":0.41799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.2859999999999996,"5min":12.396000000000001,"15min":13.803000000000001},"last":0.746}]},{"osd":7,"last update":"Thu Feb 22 19:44:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7050000000000001,"5min":1.651,"15min":1.6699999999999999},"min":{"1min":0.437,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":8.0879999999999992,"5min":12.327,"15min":13.483000000000001},"last":0.874},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.9910000000000001,"15min":1.6759999999999999},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.1749999999999998,"5min":11.539,"15min":12.846},"last":3.048}]}]},{"osd":4,"up_from":0,"seq":137438953678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724744,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1999580,"kb_used_data":1413972,"kb_used_omap":0,"kb_used_meta":585600,"kb_avail":91745572,"statfs":{"total":95995035648,"available":93947465728,"internally_reserved":0,"allocated":1447907328,"data_stored":1446564209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":599654400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":8.4299999999999997,"5min":17.684000000000001,"15min":17.684000000000001},"last":1.8500000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.7629999999999999,"15min":1.5129999999999999},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":4.7060000000000004,"5min":17.402999999999999,"15min":17.402999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.4059999999999999,"15min":1.375},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":3.0070000000000001,"5min":11.202,"15min":29.390999999999998},"last":0.25},{"interface":"front","average":{"1min":2.052,"5min":1.8089999999999999,"15min":1.5169999999999999},"min":{"1min":0.309,"5min":0.248,"15min":0.218},"max":{"1min":6.5129999999999999,"5min":8.7490000000000006,"15min":29.367000000000001},"last":1.23}]},{"osd":2,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":2.044,"15min":1.534},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.7550000000000008,"5min":11.367000000000001,"15min":14.613},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.595,"5min":1.8520000000000001,"15min":1.534},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":6.3979999999999997,"5min":16.099,"15min":16.099},"last":3.8809999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":4.0640000000000001,"5min":2.0329999999999999,"15min":1.778},"min":{"1min":0.41499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":16.370999999999999,"5min":16.370999999999999,"15min":16.370999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.6599999999999999,"15min":1.609},"min":{"1min":0.41499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.9199999999999999,"5min":8.6069999999999993,"15min":10.16},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:43:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.1379999999999999,"15min":0.73999999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.219},"max":{"1min":7.5419999999999998,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":2.234,"5min":1.496,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.23499999999999999},"max":{"1min":5.577,"5min":10.407,"15min":10.407},"last":0.32300000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:43:58 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.873,"15min":0.66900000000000004},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":2.7509999999999999,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.337,"15min":0.85599999999999998},"min":{"1min":0.36799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":4.2160000000000002,"5min":10.436,"15min":10.436},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":1.1839999999999999,"15min":0.79000000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28299999999999997},"max":{"1min":2.7730000000000001,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.873,"5min":1.3999999999999999,"15min":0.90200000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27100000000000002},"max":{"1min":5.6120000000000001,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.54300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1472790528,"data_stored":1472790528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2416508928,"data_stored":2416508928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1859584000,"data_stored":1859584000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1477705728,"data_stored":1477705728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1653669888,"data_stored":1653669888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:44:38.725 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set default.rgw.control pgp_num 12 2024-02-22T19:44:38.754 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 30 16 50192 50176 104.47 73.125 0.00493936 0.00956691 2024-02-22T19:44:38.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:38.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:37.366559+0000 mgr.y (mgr.14535) 599 : audit [DBG] from='client.25435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:38.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:37.578132+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e391: 8 total, 8 up, 5 in 2024-02-22T19:44:38.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:37.655969+0000 osd.2 (osd.2) 211 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:38.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:37.656028+0000 osd.2 (osd.2) 212 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:38.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:37.693207+0000 mgr.y (mgr.14535) 600 : cluster [DBG] pgmap v786: 146 pgs: 146 active+clean; 2.8 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:38.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:37.699622+0000 mon.a (mon.0) 1533 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-22T19:44:38.866 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:37.366559+0000 mgr.y (mgr.14535) 599 : audit [DBG] from='client.25435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:38.866 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:37.578132+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e391: 8 total, 8 up, 5 in 2024-02-22T19:44:38.866 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:37.655969+0000 osd.2 (osd.2) 211 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:38.867 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:37.656028+0000 osd.2 (osd.2) 212 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:38.867 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:37.693207+0000 mgr.y (mgr.14535) 600 : cluster [DBG] pgmap v786: 146 pgs: 146 active+clean; 2.8 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:38.867 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:37.699622+0000 mon.a (mon.0) 1533 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-22T19:44:38.878 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:39.009 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:39.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:39.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:37.366559+0000 mgr.y (mgr.14535) 599 : audit [DBG] from='client.25435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:44:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:37.578132+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e391: 8 total, 8 up, 5 in 2024-02-22T19:44:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:37.655969+0000 osd.2 (osd.2) 211 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:37.656028+0000 osd.2 (osd.2) 212 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:37.693207+0000 mgr.y (mgr.14535) 600 : cluster [DBG] pgmap v786: 146 pgs: 146 active+clean; 2.8 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:37.699622+0000 mon.a (mon.0) 1533 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-22T19:44:39.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:39.176 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:39.179 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:39.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:39.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:39.312 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:39.338 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:39.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:39.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:39.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:39.505 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:39.557 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:39.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:39.567 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:39] "GET /metrics HTTP/1.1" 200 176386 "" "Prometheus/2.33.4" 2024-02-22T19:44:39.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:39.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:39.698 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:39.724 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:39.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:39.755 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 31 16 51190 51174 103.107 62.375 0.0088137 0.00969366 2024-02-22T19:44:39.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:39.860 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:38.585061+0000 osd.1 (osd.1) 224 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:39.860 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:38.585126+0000 osd.1 (osd.1) 225 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:39.860 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:38.612135+0000 mon.a (mon.0) 1534 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-22T19:44:39.860 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:38.612249+0000 mon.a (mon.0) 1535 : cluster [DBG] osdmap e392: 8 total, 8 up, 5 in 2024-02-22T19:44:39.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:38.585061+0000 osd.1 (osd.1) 224 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:39.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:38.585126+0000 osd.1 (osd.1) 225 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:39.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:38.612135+0000 mon.a (mon.0) 1534 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-22T19:44:39.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:38.612249+0000 mon.a (mon.0) 1535 : cluster [DBG] osdmap e392: 8 total, 8 up, 5 in 2024-02-22T19:44:39.861 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:39.885 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:39.887 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:38.585061+0000 osd.1 (osd.1) 224 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:39.887 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:38.585126+0000 osd.1 (osd.1) 225 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:39.887 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:38.612135+0000 mon.a (mon.0) 1534 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-22T19:44:39.887 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:38.612249+0000 mon.a (mon.0) 1535 : cluster [DBG] osdmap e392: 8 total, 8 up, 5 in 2024-02-22T19:44:39.888 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:39.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:39.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:40.021 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:40.048 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:40.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:40.150 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:40.163 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:40.167 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:44:40.756 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 32 16 52368 52352 102.184 73.625 0.0167186 0.00977956 2024-02-22T19:44:40.960 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:40.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:40.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:40.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:39.618395+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e393: 8 total, 8 up, 5 in 2024-02-22T19:44:40.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:39.694930+0000 mgr.y (mgr.14535) 601 : cluster [DBG] pgmap v789: 146 pgs: 2 peering, 144 active+clean; 3.0 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s 2024-02-22T19:44:40.965 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:39.724458+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? 172.21.15.53:0/3754958750' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:44:40.965 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:39.618395+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e393: 8 total, 8 up, 5 in 2024-02-22T19:44:40.966 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:39.694930+0000 mgr.y (mgr.14535) 601 : cluster [DBG] pgmap v789: 146 pgs: 2 peering, 144 active+clean; 3.0 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s 2024-02-22T19:44:40.966 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:39.724458+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? 172.21.15.53:0/3754958750' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:44:41.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:41.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:39.618395+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e393: 8 total, 8 up, 5 in 2024-02-22T19:44:41.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:39.694930+0000 mgr.y (mgr.14535) 601 : cluster [DBG] pgmap v789: 146 pgs: 2 peering, 144 active+clean; 3.0 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s 2024-02-22T19:44:41.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:39.724458+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? 172.21.15.53:0/3754958750' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:44:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:41.756 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 33 16 53512 53496 101.253 71.5 0.015052 0.00987029 2024-02-22T19:44:41.981 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:44:41.981 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:44:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:40.632054+0000 mon.a (mon.0) 1538 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:40.639327+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? 172.21.15.53:0/3754958750' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:44:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:40.639369+0000 mon.a (mon.0) 1540 : cluster [DBG] osdmap e394: 8 total, 8 up, 5 in 2024-02-22T19:44:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:40.864077+0000 osd.3 (osd.3) 251 : cluster [DBG] 4.c scrub starts 2024-02-22T19:44:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:40.870673+0000 osd.3 (osd.3) 252 : cluster [DBG] 4.c scrub ok 2024-02-22T19:44:42.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:40.632054+0000 mon.a (mon.0) 1538 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:42.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:40.639327+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? 172.21.15.53:0/3754958750' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:44:42.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:40.639369+0000 mon.a (mon.0) 1540 : cluster [DBG] osdmap e394: 8 total, 8 up, 5 in 2024-02-22T19:44:42.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:40.864077+0000 osd.3 (osd.3) 251 : cluster [DBG] 4.c scrub starts 2024-02-22T19:44:42.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:40.870673+0000 osd.3 (osd.3) 252 : cluster [DBG] 4.c scrub ok 2024-02-22T19:44:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:40.632054+0000 mon.a (mon.0) 1538 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:44:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:40.639327+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? 172.21.15.53:0/3754958750' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:44:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:40.639369+0000 mon.a (mon.0) 1540 : cluster [DBG] osdmap e394: 8 total, 8 up, 5 in 2024-02-22T19:44:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:40.864077+0000 osd.3 (osd.3) 251 : cluster [DBG] 4.c scrub starts 2024-02-22T19:44:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:40.870673+0000 osd.3 (osd.3) 252 : cluster [DBG] 4.c scrub ok 2024-02-22T19:44:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:42.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:42.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:42.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:44:42.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:42.757 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 34 16 54724 54708 100.501 75.75 0.0169031 0.0099444 2024-02-22T19:44:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:42.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:41.257952+0000 osd.7 (osd.7) 236 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:44:42.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:41.260942+0000 osd.7 (osd.7) 237 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:41.394748+0000 osd.2 (osd.2) 213 : cluster [DBG] 5.e scrub starts 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:41.398530+0000 osd.2 (osd.2) 214 : cluster [DBG] 5.e scrub ok 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:41.675977+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e395: 8 total, 8 up, 5 in 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:41.695976+0000 mgr.y (mgr.14535) 602 : cluster [DBG] pgmap v792: 145 pgs: 1 peering, 144 active+clean; 3.1 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:41.698783+0000 mon.a (mon.0) 1542 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:42.361660+0000 mon.a (mon.0) 1543 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:42.361755+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e396: 8 total, 8 up, 5 in 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:41.257952+0000 osd.7 (osd.7) 236 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:41.260942+0000 osd.7 (osd.7) 237 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:41.394748+0000 osd.2 (osd.2) 213 : cluster [DBG] 5.e scrub starts 2024-02-22T19:44:42.896 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:41.398530+0000 osd.2 (osd.2) 214 : cluster [DBG] 5.e scrub ok 2024-02-22T19:44:42.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:41.675977+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e395: 8 total, 8 up, 5 in 2024-02-22T19:44:42.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:41.695976+0000 mgr.y (mgr.14535) 602 : cluster [DBG] pgmap v792: 145 pgs: 1 peering, 144 active+clean; 3.1 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:42.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:41.698783+0000 mon.a (mon.0) 1542 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-22T19:44:42.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:42.361660+0000 mon.a (mon.0) 1543 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-22T19:44:42.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:42.361755+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e396: 8 total, 8 up, 5 in 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:41.257952+0000 osd.7 (osd.7) 236 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:41.260942+0000 osd.7 (osd.7) 237 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:41.394748+0000 osd.2 (osd.2) 213 : cluster [DBG] 5.e scrub starts 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:41.398530+0000 osd.2 (osd.2) 214 : cluster [DBG] 5.e scrub ok 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:41.675977+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e395: 8 total, 8 up, 5 in 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:41.695976+0000 mgr.y (mgr.14535) 602 : cluster [DBG] pgmap v792: 145 pgs: 1 peering, 144 active+clean; 3.1 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:41.698783+0000 mon.a (mon.0) 1542 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:42.361660+0000 mon.a (mon.0) 1543 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-22T19:44:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:42.361755+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e396: 8 total, 8 up, 5 in 2024-02-22T19:44:43.760 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 35 16 55764 55748 99.4827 65 0.0079883 0.0100458 2024-02-22T19:44:43.883 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:44:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:44.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:44.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:44.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:44.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:44.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:44.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:44.440 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:44.541 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:44.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:44.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:42.818504+0000 osd.3 (osd.3) 253 : cluster [DBG] 6.c scrub starts 2024-02-22T19:44:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:42.822459+0000 osd.3 (osd.3) 254 : cluster [DBG] 6.c scrub ok 2024-02-22T19:44:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:43.364957+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e397: 8 total, 8 up, 5 in 2024-02-22T19:44:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:43.699350+0000 mgr.y (mgr.14535) 603 : cluster [DBG] pgmap v795: 145 pgs: 1 peering, 144 active+clean; 3.1 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 22 MiB/s wr, 352 op/s 2024-02-22T19:44:44.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:44.697 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:44.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:44.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:44.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:42.818504+0000 osd.3 (osd.3) 253 : cluster [DBG] 6.c scrub starts 2024-02-22T19:44:44.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:42.822459+0000 osd.3 (osd.3) 254 : cluster [DBG] 6.c scrub ok 2024-02-22T19:44:44.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:43.364957+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e397: 8 total, 8 up, 5 in 2024-02-22T19:44:44.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:43.699350+0000 mgr.y (mgr.14535) 603 : cluster [DBG] pgmap v795: 145 pgs: 1 peering, 144 active+clean; 3.1 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 22 MiB/s wr, 352 op/s 2024-02-22T19:44:44.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:42.818504+0000 osd.3 (osd.3) 253 : cluster [DBG] 6.c scrub starts 2024-02-22T19:44:44.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:42.822459+0000 osd.3 (osd.3) 254 : cluster [DBG] 6.c scrub ok 2024-02-22T19:44:44.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:43.364957+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e397: 8 total, 8 up, 5 in 2024-02-22T19:44:44.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:43.699350+0000 mgr.y (mgr.14535) 603 : cluster [DBG] pgmap v795: 145 pgs: 1 peering, 144 active+clean; 3.1 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 22 MiB/s wr, 352 op/s 2024-02-22T19:44:44.761 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 36 16 56896 56880 98.6776 70.75 0.016215 0.0101288 2024-02-22T19:44:44.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:44.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:44.891 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:44.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:45.052 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:44:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:44:45.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:44:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:43.771272+0000 osd.3 (osd.3) 255 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:44:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:43.774771+0000 osd.3 (osd.3) 256 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:44:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:44.263039+0000 osd.7 (osd.7) 238 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:44:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:44.268586+0000 osd.7 (osd.7) 239 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:44:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:44.379804+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e398: 8 total, 8 up, 5 in 2024-02-22T19:44:45.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:45.195296+0000 mgr.y (mgr.14535) 604 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:45.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:43.771272+0000 osd.3 (osd.3) 255 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:43.774771+0000 osd.3 (osd.3) 256 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:44.263039+0000 osd.7 (osd.7) 238 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:44.268586+0000 osd.7 (osd.7) 239 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:44.379804+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e398: 8 total, 8 up, 5 in 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:45.195296+0000 mgr.y (mgr.14535) 604 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:43.771272+0000 osd.3 (osd.3) 255 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:43.774771+0000 osd.3 (osd.3) 256 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:44.263039+0000 osd.7 (osd.7) 238 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:44.268586+0000 osd.7 (osd.7) 239 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:44.379804+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e398: 8 total, 8 up, 5 in 2024-02-22T19:44:45.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:45.195296+0000 mgr.y (mgr.14535) 604 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:45.761 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 37 16 57954 57938 97.798 66.125 0.00570167 0.0102198 2024-02-22T19:44:45.906 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:46.026 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.027 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.028 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.028 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.028 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.028 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.028 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:44:46.036 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:46.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:46.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:46.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:46.206 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:46.256 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:46.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:46.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:46.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:46.418 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:46.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:44.718065+0000 osd.5 (osd.5) 133 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:44:46.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:44.719540+0000 osd.5 (osd.5) 134 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:44:46.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.308519+0000 osd.7 (osd.7) 240 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:44:46.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.370826+0000 osd.7 (osd.7) 241 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:44:46.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.382996+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e399: 8 total, 8 up, 5 in 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.397942+0000 osd.2 (osd.2) 215 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.498563+0000 osd.2 (osd.2) 216 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:45.646503+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? 172.21.15.53:0/1680372010' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.699730+0000 mgr.y (mgr.14535) 605 : cluster [DBG] pgmap v798: 144 pgs: 1 peering, 143 active+clean; 3.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:45.700991+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:44.718065+0000 osd.5 (osd.5) 133 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:44.719540+0000 osd.5 (osd.5) 134 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.308519+0000 osd.7 (osd.7) 240 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.370826+0000 osd.7 (osd.7) 241 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.382996+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e399: 8 total, 8 up, 5 in 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.397942+0000 osd.2 (osd.2) 215 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.498563+0000 osd.2 (osd.2) 216 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:44:46.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:45.646503+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? 172.21.15.53:0/1680372010' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:46.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.699730+0000 mgr.y (mgr.14535) 605 : cluster [DBG] pgmap v798: 144 pgs: 1 peering, 143 active+clean; 3.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-02-22T19:44:46.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:45.700991+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-22T19:44:46.765 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 38 16 58979 58963 96.9072 64.0625 0.0148775 0.0103135 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:44.718065+0000 osd.5 (osd.5) 133 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:44.719540+0000 osd.5 (osd.5) 134 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.308519+0000 osd.7 (osd.7) 240 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.370826+0000 osd.7 (osd.7) 241 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.382996+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e399: 8 total, 8 up, 5 in 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.397942+0000 osd.2 (osd.2) 215 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:44:46.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.498563+0000 osd.2 (osd.2) 216 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:44:46.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:45.646503+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? 172.21.15.53:0/1680372010' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:46.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.699730+0000 mgr.y (mgr.14535) 605 : cluster [DBG] pgmap v798: 144 pgs: 1 peering, 143 active+clean; 3.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-02-22T19:44:46.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:45.700991+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-22T19:44:47.296 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:47.301 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:47.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:47.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:47.433 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:47.462 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:47.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:47.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:47.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:47.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:47.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.705887+0000 osd.5 (osd.5) 135 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:44:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:45.715944+0000 osd.5 (osd.5) 136 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:44:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:46.397172+0000 mon.a (mon.0) 1549 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-22T19:44:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:46.397231+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e400: 8 total, 8 up, 5 in 2024-02-22T19:44:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:46.478335+0000 mon.a (mon.0) 1551 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:44:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:46.981132+0000 mon.a (mon.0) 1552 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.705887+0000 osd.5 (osd.5) 135 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:45.715944+0000 osd.5 (osd.5) 136 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:46.397172+0000 mon.a (mon.0) 1549 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:46.397231+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e400: 8 total, 8 up, 5 in 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:46.478335+0000 mon.a (mon.0) 1551 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:46.981132+0000 mon.a (mon.0) 1552 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:47.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.705887+0000 osd.5 (osd.5) 135 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:44:47.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:45.715944+0000 osd.5 (osd.5) 136 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:44:47.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:46.397172+0000 mon.a (mon.0) 1549 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-22T19:44:47.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:46.397231+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e400: 8 total, 8 up, 5 in 2024-02-22T19:44:47.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:46.478335+0000 mon.a (mon.0) 1551 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:44:47.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:46.981132+0000 mon.a (mon.0) 1552 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:47.758 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:47.763 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 39 16 59971 59955 96.0126 62 0.00827367 0.0104103 2024-02-22T19:44:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:47.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:47.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:47.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:47.924 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:47.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:47.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:48.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:48.127 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:48.170 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:48.200 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:48.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:48.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:48.374 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:48.504 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 5 pgp_num to 12 2024-02-22T19:44:48.676 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:46.730750+0000 osd.3 (osd.3) 257 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:46.735337+0000 osd.3 (osd.3) 258 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:47.443965+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e401: 8 total, 8 up, 5 in 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:47.570233+0000 mon.a (mon.0) 1554 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:47.571251+0000 mon.a (mon.0) 1555 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:47.590258+0000 mon.a (mon.0) 1556 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:47.628801+0000 osd.1 (osd.1) 226 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:47.633237+0000 osd.1 (osd.1) 227 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:47.700209+0000 mgr.y (mgr.14535) 606 : cluster [DBG] pgmap v801: 144 pgs: 1 peering, 143 active+clean; 3.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-02-22T19:44:48.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:48.330847+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.53:0/1568693182' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-22T19:44:48.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:48.332384+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-22T19:44:48.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:46.730750+0000 osd.3 (osd.3) 257 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:44:48.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:46.735337+0000 osd.3 (osd.3) 258 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:44:48.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:47.443965+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e401: 8 total, 8 up, 5 in 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:47.570233+0000 mon.a (mon.0) 1554 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:47.571251+0000 mon.a (mon.0) 1555 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:47.590258+0000 mon.a (mon.0) 1556 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:47.628801+0000 osd.1 (osd.1) 226 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:47.633237+0000 osd.1 (osd.1) 227 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:47.700209+0000 mgr.y (mgr.14535) 606 : cluster [DBG] pgmap v801: 144 pgs: 1 peering, 143 active+clean; 3.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:48.330847+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.53:0/1568693182' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-22T19:44:48.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:48.332384+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-22T19:44:48.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:44:48.765199+0000 min lat: 0.00135253 max lat: 0.10233 avg lat: 0.0104976 2024-02-22T19:44:48.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:44:48.764 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 40 16 60997 60981 95.2136 64.125 0.0343528 0.0104976 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:46.730750+0000 osd.3 (osd.3) 257 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:46.735337+0000 osd.3 (osd.3) 258 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:47.443965+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e401: 8 total, 8 up, 5 in 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:47.570233+0000 mon.a (mon.0) 1554 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:47.571251+0000 mon.a (mon.0) 1555 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:47.590258+0000 mon.a (mon.0) 1556 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:47.628801+0000 osd.1 (osd.1) 226 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:47.633237+0000 osd.1 (osd.1) 227 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:44:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:47.700209+0000 mgr.y (mgr.14535) 606 : cluster [DBG] pgmap v801: 144 pgs: 1 peering, 143 active+clean; 3.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-02-22T19:44:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:48.330847+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.53:0/1568693182' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-22T19:44:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:48.332384+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-22T19:44:48.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:44:49.140 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:49.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:49.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:49.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:49.306 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:49.307 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:49.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:44:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:44:49.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:49.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:44:49.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:47.730157+0000 osd.3 (osd.3) 259 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:44:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:47.734935+0000 osd.3 (osd.3) 260 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:44:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:48.465118+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]': finished 2024-02-22T19:44:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:48.465197+0000 mon.a (mon.0) 1559 : cluster [DBG] osdmap e402: 8 total, 8 up, 5 in 2024-02-22T19:44:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:48.600740+0000 osd.1 (osd.1) 228 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:44:49.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:48.653688+0000 osd.1 (osd.1) 229 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:44:49.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:47.730157+0000 osd.3 (osd.3) 259 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:44:49.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:47.734935+0000 osd.3 (osd.3) 260 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:48.465118+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]': finished 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:48.465197+0000 mon.a (mon.0) 1559 : cluster [DBG] osdmap e402: 8 total, 8 up, 5 in 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:48.600740+0000 osd.1 (osd.1) 228 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:48.653688+0000 osd.1 (osd.1) 229 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:47.730157+0000 osd.3 (osd.3) 259 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:47.734935+0000 osd.3 (osd.3) 260 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:48.465118+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "12"}]': finished 2024-02-22T19:44:49.625 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:48.465197+0000 mon.a (mon.0) 1559 : cluster [DBG] osdmap e402: 8 total, 8 up, 5 in 2024-02-22T19:44:49.626 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:48.600740+0000 osd.1 (osd.1) 228 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:44:49.626 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:48.653688+0000 osd.1 (osd.1) 229 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:44:49.626 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:44:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:49] "GET /metrics HTTP/1.1" 200 176386 "" "Prometheus/2.33.4" 2024-02-22T19:44:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:49.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 41 16 62088 62072 94.5482 68.1875 0.0112438 0.0105723 2024-02-22T19:44:50.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:48.766624+0000 osd.5 (osd.5) 137 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:44:50.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:48.769595+0000 osd.5 (osd.5) 138 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:44:50.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.363460+0000 osd.7 (osd.7) 242 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.367896+0000 osd.7 (osd.7) 243 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.469381+0000 mon.a (mon.0) 1560 : cluster [DBG] osdmap e403: 8 total, 8 up, 5 in 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.701716+0000 mgr.y (mgr.14535) 607 : cluster [DBG] pgmap v804: 143 pgs: 143 active+clean; 3.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:49.704035+0000 mon.a (mon.0) 1561 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.875127+0000 osd.0 (osd.0) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.875184+0000 osd.0 (osd.0) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:48.766624+0000 osd.5 (osd.5) 137 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:48.769595+0000 osd.5 (osd.5) 138 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.363460+0000 osd.7 (osd.7) 242 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:44:50.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.367896+0000 osd.7 (osd.7) 243 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:44:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.469381+0000 mon.a (mon.0) 1560 : cluster [DBG] osdmap e403: 8 total, 8 up, 5 in 2024-02-22T19:44:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.701716+0000 mgr.y (mgr.14535) 607 : cluster [DBG] pgmap v804: 143 pgs: 143 active+clean; 3.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-02-22T19:44:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:49.704035+0000 mon.a (mon.0) 1561 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-22T19:44:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.875127+0000 osd.0 (osd.0) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.875184+0000 osd.0 (osd.0) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:50.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 42 16 63198 63182 93.9487 69.375 0.0656986 0.0106379 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:48.766624+0000 osd.5 (osd.5) 137 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:48.769595+0000 osd.5 (osd.5) 138 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.363460+0000 osd.7 (osd.7) 242 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.367896+0000 osd.7 (osd.7) 243 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.469381+0000 mon.a (mon.0) 1560 : cluster [DBG] osdmap e403: 8 total, 8 up, 5 in 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.701716+0000 mgr.y (mgr.14535) 607 : cluster [DBG] pgmap v804: 143 pgs: 143 active+clean; 3.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:49.704035+0000 mon.a (mon.0) 1561 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.875127+0000 osd.0 (osd.0) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:44:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.875184+0000 osd.0 (osd.0) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:44:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.771562+0000 osd.3 (osd.3) 261 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:44:51.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:49.777171+0000 osd.3 (osd.3) 262 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:44:51.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:50.481509+0000 mon.a (mon.0) 1562 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-22T19:44:51.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:50.481574+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e404: 8 total, 8 up, 5 in 2024-02-22T19:44:51.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.771562+0000 osd.3 (osd.3) 261 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:44:51.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:49.777171+0000 osd.3 (osd.3) 262 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:44:51.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:50.481509+0000 mon.a (mon.0) 1562 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-22T19:44:51.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:50.481574+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e404: 8 total, 8 up, 5 in 2024-02-22T19:44:51.767 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 43 16 64327 64311 93.4051 70.5625 0.0201562 0.0107001 2024-02-22T19:44:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.771562+0000 osd.3 (osd.3) 261 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:44:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:49.777171+0000 osd.3 (osd.3) 262 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:44:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:50.481509+0000 mon.a (mon.0) 1562 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-22T19:44:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:50.481574+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e404: 8 total, 8 up, 5 in 2024-02-22T19:44:52.770 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 44 16 65353 65337 92.7346 64.125 0.0084455 0.0107792 2024-02-22T19:44:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:51.509906+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e405: 8 total, 8 up, 5 in 2024-02-22T19:44:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:51.702215+0000 mgr.y (mgr.14535) 608 : cluster [DBG] pgmap v807: 143 pgs: 143 active+clean; 3.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-02-22T19:44:52.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:52.367751+0000 mon.a (mon.0) 1565 : cluster [DBG] osdmap e406: 8 total, 8 up, 5 in 2024-02-22T19:44:53.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:51.509906+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e405: 8 total, 8 up, 5 in 2024-02-22T19:44:53.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:51.702215+0000 mgr.y (mgr.14535) 608 : cluster [DBG] pgmap v807: 143 pgs: 143 active+clean; 3.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-02-22T19:44:53.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:52.367751+0000 mon.a (mon.0) 1565 : cluster [DBG] osdmap e406: 8 total, 8 up, 5 in 2024-02-22T19:44:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:51.509906+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e405: 8 total, 8 up, 5 in 2024-02-22T19:44:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:51.702215+0000 mgr.y (mgr.14535) 608 : cluster [DBG] pgmap v807: 143 pgs: 143 active+clean; 3.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-02-22T19:44:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:52.367751+0000 mon.a (mon.0) 1565 : cluster [DBG] osdmap e406: 8 total, 8 up, 5 in 2024-02-22T19:44:53.323 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:53.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:53.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:53.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:44:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:44:53.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:53.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:44:53.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:53.642 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:51.767868+0000 osd.3 (osd.3) 263 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:44:53.642 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:51.773515+0000 osd.3 (osd.3) 264 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:44:53.642 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:52.359460+0000 osd.7 (osd.7) 244 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:52.364232+0000 osd.7 (osd.7) 245 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:53.388468+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e407: 8 total, 8 up, 5 in 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:51.767868+0000 osd.3 (osd.3) 263 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:51.773515+0000 osd.3 (osd.3) 264 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:52.359460+0000 osd.7 (osd.7) 244 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:52.364232+0000 osd.7 (osd.7) 245 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:44:53.643 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:53.388468+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e407: 8 total, 8 up, 5 in 2024-02-22T19:44:53.650 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:53.773 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 45 16 66455 66439 92.1979 68.875 0.0147125 0.0108412 2024-02-22T19:44:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:53.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:53.810 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:51.767868+0000 osd.3 (osd.3) 263 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:44:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:51.773515+0000 osd.3 (osd.3) 264 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:44:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:52.359460+0000 osd.7 (osd.7) 244 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:44:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:52.364232+0000 osd.7 (osd.7) 245 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:44:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:53.388468+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e407: 8 total, 8 up, 5 in 2024-02-22T19:44:53.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:54.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:54.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:54.127 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:44:54.128 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:44:54.128 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:44:54.128 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:44:54.128 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:44:54.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:54.773 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 46 16 67545 67529 91.6748 68.125 0.0104673 0.010903 2024-02-22T19:44:54.813 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:52.793265+0000 osd.3 (osd.3) 265 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:44:54.813 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:52.796302+0000 osd.3 (osd.3) 266 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:44:54.813 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:53.628630+0000 osd.1 (osd.1) 230 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:44:54.813 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:53.631175+0000 osd.1 (osd.1) 231 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:44:54.813 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:53.702581+0000 mgr.y (mgr.14535) 609 : cluster [DBG] pgmap v810: 142 pgs: 142 active+clean; 3.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 20 MiB/s wr, 324 op/s 2024-02-22T19:44:54.813 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:53.703571+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-22T19:44:54.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:52.793265+0000 osd.3 (osd.3) 265 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:44:54.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:52.796302+0000 osd.3 (osd.3) 266 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:44:54.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:53.628630+0000 osd.1 (osd.1) 230 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:44:54.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:53.631175+0000 osd.1 (osd.1) 231 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:44:54.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:53.702581+0000 mgr.y (mgr.14535) 609 : cluster [DBG] pgmap v810: 142 pgs: 142 active+clean; 3.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 20 MiB/s wr, 324 op/s 2024-02-22T19:44:54.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:53.703571+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-22T19:44:54.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:44:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:52.793265+0000 osd.3 (osd.3) 265 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:44:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:52.796302+0000 osd.3 (osd.3) 266 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:44:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:53.628630+0000 osd.1 (osd.1) 230 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:44:54.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:53.631175+0000 osd.1 (osd.1) 231 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:44:54.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:53.702581+0000 mgr.y (mgr.14535) 609 : cluster [DBG] pgmap v810: 142 pgs: 142 active+clean; 3.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 20 MiB/s wr, 324 op/s 2024-02-22T19:44:54.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:53.703571+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-22T19:44:55.005 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:55.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:55.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:55.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:55.152 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:44:55.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:55.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:44:55.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:44:55.352 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:55.478 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:55.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:44:55.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:44:55.673 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:44:55.673 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:44:55.673 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-02-22T19:44:55.673 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:44:55.673 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:44:55.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-02-22T19:44:55.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:53.749253+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:44:55.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:53.757586+0000 osd.3 (osd.3) 268 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:44:55.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:54.538432+0000 mon.a (mon.0) 1568 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-22T19:44:55.718 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:54.538498+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e408: 8 total, 8 up, 5 in 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:54.801539+0000 osd.5 (osd.5) 139 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:54.804463+0000 osd.5 (osd.5) 140 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:55.206702+0000 mgr.y (mgr.14535) 610 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:53.749253+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:53.757586+0000 osd.3 (osd.3) 268 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:54.538432+0000 mon.a (mon.0) 1568 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:54.538498+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e408: 8 total, 8 up, 5 in 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:54.801539+0000 osd.5 (osd.5) 139 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:54.804463+0000 osd.5 (osd.5) 140 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:44:55.719 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:55.206702+0000 mgr.y (mgr.14535) 610 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:55.735 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:44:55.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:55.745 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:44:55.773 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 47 16 68596 68580 91.1222 65.6875 0.0275414 0.0109674 2024-02-22T19:44:55.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:53.749253+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:53.757586+0000 osd.3 (osd.3) 268 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:54.538432+0000 mon.a (mon.0) 1568 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:54.538498+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e408: 8 total, 8 up, 5 in 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:54.801539+0000 osd.5 (osd.5) 139 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:54.804463+0000 osd.5 (osd.5) 140 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:44:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:55.206702+0000 mgr.y (mgr.14535) 610 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:44:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:55.958 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:44:55.967 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:55.967 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:44:56.012 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:44:56.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:44:56.147 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:56.198 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:44:56.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:56.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:44:56.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:56.386 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:56.569 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:55.565804+0000 mon.a (mon.0) 1570 : cluster [DBG] osdmap e409: 8 total, 8 up, 5 in 2024-02-22T19:44:56.570 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:55.650132+0000 osd.1 (osd.1) 232 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:44:56.570 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:55.655468+0000 osd.1 (osd.1) 233 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:44:56.570 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:55.703042+0000 mgr.y (mgr.14535) 611 : cluster [DBG] pgmap v813: 142 pgs: 142 active+clean; 4.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:44:56.651 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:56.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:56.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:56.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:55.565804+0000 mon.a (mon.0) 1570 : cluster [DBG] osdmap e409: 8 total, 8 up, 5 in 2024-02-22T19:44:56.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:55.650132+0000 osd.1 (osd.1) 232 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:44:56.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:55.655468+0000 osd.1 (osd.1) 233 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:44:56.656 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:55.703042+0000 mgr.y (mgr.14535) 611 : cluster [DBG] pgmap v813: 142 pgs: 142 active+clean; 4.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:44:56.657 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:55.565804+0000 mon.a (mon.0) 1570 : cluster [DBG] osdmap e409: 8 total, 8 up, 5 in 2024-02-22T19:44:56.657 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:55.650132+0000 osd.1 (osd.1) 232 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:44:56.657 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:55.655468+0000 osd.1 (osd.1) 233 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:44:56.657 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:55.703042+0000 mgr.y (mgr.14535) 611 : cluster [DBG] pgmap v813: 142 pgs: 142 active+clean; 4.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:44:56.756 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:44:56.773 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 48 16 69527 69511 90.4364 58.1875 0.00664558 0.011052 2024-02-22T19:44:56.820 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:57.202 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:44:57.216 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:57.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:44:57.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:44:57.357 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:44:57.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:44:57.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:44:57.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:44:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:44:57.621 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:44:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:56.579501+0000 mon.a (mon.0) 1571 : cluster [DBG] osdmap e410: 8 total, 8 up, 5 in 2024-02-22T19:44:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:56.630137+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.53:0/2896037373' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:56.630778+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:56.650876+0000 osd.1 (osd.1) 234 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:44:57.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:56.653839+0000 osd.1 (osd.1) 235 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:44:57.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:56.579501+0000 mon.a (mon.0) 1571 : cluster [DBG] osdmap e410: 8 total, 8 up, 5 in 2024-02-22T19:44:57.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:56.630137+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.53:0/2896037373' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:57.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:56.630778+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:57.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:56.650876+0000 osd.1 (osd.1) 234 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:44:57.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:56.653839+0000 osd.1 (osd.1) 235 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:44:57.774 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 49 16 70531 70515 89.8717 62.75 0.0215073 0.0111218 2024-02-22T19:44:57.816 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:44:57.838 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:44:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:56.579501+0000 mon.a (mon.0) 1571 : cluster [DBG] osdmap e410: 8 total, 8 up, 5 in 2024-02-22T19:44:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:56.630137+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.53:0/2896037373' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:56.630778+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:44:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:56.650876+0000 osd.1 (osd.1) 234 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:44:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:56.653839+0000 osd.1 (osd.1) 235 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:44:57.886 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:44:57.928 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:44:57.955 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:44:57.969 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:44:57.991 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:44:58.033 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:44:58.073 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:44:58.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:56.762588+0000 osd.3 (osd.3) 269 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:44:58.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:56.776713+0000 osd.3 (osd.3) 270 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:44:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.318465+0000 osd.7 (osd.7) 246 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:44:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.360207+0000 osd.7 (osd.7) 247 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:44:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.576393+0000 mon.a (mon.0) 1573 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:44:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.576431+0000 mon.a (mon.0) 1574 : cluster [INF] Cluster is now healthy 2024-02-22T19:44:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:57.586376+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:58.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.586437+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e411: 8 total, 8 up, 5 in 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.703477+0000 mgr.y (mgr.14535) 612 : cluster [DBG] pgmap v816: 141 pgs: 141 active+clean; 4.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:57.709541+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:57.798938+0000 mon.a (mon.0) 1578 : audit [DBG] from='client.? 172.21.15.53:0/2817645943' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:56.762588+0000 osd.3 (osd.3) 269 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:56.776713+0000 osd.3 (osd.3) 270 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.318465+0000 osd.7 (osd.7) 246 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.360207+0000 osd.7 (osd.7) 247 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.576393+0000 mon.a (mon.0) 1573 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.576431+0000 mon.a (mon.0) 1574 : cluster [INF] Cluster is now healthy 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:57.586376+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:58.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.586437+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e411: 8 total, 8 up, 5 in 2024-02-22T19:44:58.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.703477+0000 mgr.y (mgr.14535) 612 : cluster [DBG] pgmap v816: 141 pgs: 141 active+clean; 4.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:44:58.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:57.709541+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-22T19:44:58.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:57.798938+0000 mon.a (mon.0) 1578 : audit [DBG] from='client.? 172.21.15.53:0/2817645943' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:58.774 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 50 16 71589 71573 89.3969 66.125 0.0214201 0.0111805 2024-02-22T19:44:59.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:56.762588+0000 osd.3 (osd.3) 269 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:56.776713+0000 osd.3 (osd.3) 270 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.318465+0000 osd.7 (osd.7) 246 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.360207+0000 osd.7 (osd.7) 247 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.576393+0000 mon.a (mon.0) 1573 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.576431+0000 mon.a (mon.0) 1574 : cluster [INF] Cluster is now healthy 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:57.586376+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.586437+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e411: 8 total, 8 up, 5 in 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.703477+0000 mgr.y (mgr.14535) 612 : cluster [DBG] pgmap v816: 141 pgs: 141 active+clean; 4.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:57.709541+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-22T19:44:59.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:57.798938+0000 mon.a (mon.0) 1578 : audit [DBG] from='client.? 172.21.15.53:0/2817645943' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:44:59.427 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:44:59] "GET /metrics HTTP/1.1" 200 176407 "" "Prometheus/2.33.4" 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.830189+0000 osd.3 (osd.3) 271 : cluster [DBG] 6.b scrub starts 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:57.962847+0000 osd.3 (osd.3) 272 : cluster [DBG] 6.b scrub ok 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:44:58.607140+0000 mon.a (mon.0) 1579 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:58.607199+0000 mon.a (mon.0) 1580 : cluster [DBG] osdmap e412: 8 total, 8 up, 5 in 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:58.795558+0000 osd.5 (osd.5) 141 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:58.799392+0000 osd.5 (osd.5) 142 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.830189+0000 osd.3 (osd.3) 271 : cluster [DBG] 6.b scrub starts 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:57.962847+0000 osd.3 (osd.3) 272 : cluster [DBG] 6.b scrub ok 2024-02-22T19:44:59.715 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:44:58.607140+0000 mon.a (mon.0) 1579 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-22T19:44:59.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:58.607199+0000 mon.a (mon.0) 1580 : cluster [DBG] osdmap e412: 8 total, 8 up, 5 in 2024-02-22T19:44:59.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:58.795558+0000 osd.5 (osd.5) 141 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:44:59.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:44:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:58.799392+0000 osd.5 (osd.5) 142 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:44:59.775 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 51 16 72673 72657 88.9712 67.75 0.0204692 0.011235 2024-02-22T19:45:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.830189+0000 osd.3 (osd.3) 271 : cluster [DBG] 6.b scrub starts 2024-02-22T19:45:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:57.962847+0000 osd.3 (osd.3) 272 : cluster [DBG] 6.b scrub ok 2024-02-22T19:45:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:44:58.607140+0000 mon.a (mon.0) 1579 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-22T19:45:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:58.607199+0000 mon.a (mon.0) 1580 : cluster [DBG] osdmap e412: 8 total, 8 up, 5 in 2024-02-22T19:45:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:58.795558+0000 osd.5 (osd.5) 141 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:45:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:44:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:58.799392+0000 osd.5 (osd.5) 142 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:45:00.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:45:00.782 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 52 16 73768 73752 88.5646 68.4375 0.0146674 0.0112869 2024-02-22T19:45:00.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:59.328442+0000 osd.7 (osd.7) 248 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:45:00.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:59.332953+0000 osd.7 (osd.7) 249 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:45:00.913 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:59.682967+0000 mon.a (mon.0) 1581 : cluster [DBG] osdmap e413: 8 total, 8 up, 5 in 2024-02-22T19:45:00.913 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:44:59.705308+0000 mgr.y (mgr.14535) 613 : cluster [DBG] pgmap v819: 141 pgs: 2 peering, 139 active+clean; 4.3 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-02-22T19:45:00.913 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:00.917 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:00.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:59.328442+0000 osd.7 (osd.7) 248 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:59.332953+0000 osd.7 (osd.7) 249 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:59.682967+0000 mon.a (mon.0) 1581 : cluster [DBG] osdmap e413: 8 total, 8 up, 5 in 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:44:59.705308+0000 mgr.y (mgr.14535) 613 : cluster [DBG] pgmap v819: 141 pgs: 2 peering, 139 active+clean; 4.3 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:59.328442+0000 osd.7 (osd.7) 248 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:59.332953+0000 osd.7 (osd.7) 249 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:59.682967+0000 mon.a (mon.0) 1581 : cluster [DBG] osdmap e413: 8 total, 8 up, 5 in 2024-02-22T19:45:00.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:44:59.705308+0000 mgr.y (mgr.14535) 613 : cluster [DBG] pgmap v819: 141 pgs: 2 peering, 139 active+clean; 4.3 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-02-22T19:45:01.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:01.083 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:01.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:45:01.228 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:45:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "" 2024-02-22T19:45:01.228 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:45:01.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:01.232 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:01.232 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:01.285 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:45:01.285 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:45:01.304 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:01.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:01.367 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:01.500 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:01.500 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:01.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:01.668 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:01.783 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 53 16 74709 74693 88.002 58.8125 0.0139034 0.0113583 2024-02-22T19:45:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:01.823 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:01.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:00.403487+0000 osd.2 (osd.2) 217 : cluster [DBG] 5.b scrub starts 2024-02-22T19:45:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:00.406444+0000 osd.2 (osd.2) 218 : cluster [DBG] 5.b scrub ok 2024-02-22T19:45:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:00.697927+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e414: 8 total, 8 up, 5 in 2024-02-22T19:45:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:00.725644+0000 osd.5 (osd.5) 143 : cluster [DBG] 1.a scrub starts 2024-02-22T19:45:01.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:00.729902+0000 osd.5 (osd.5) 144 : cluster [DBG] 1.a scrub ok 2024-02-22T19:45:01.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:01.929 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:00.403487+0000 osd.2 (osd.2) 217 : cluster [DBG] 5.b scrub starts 2024-02-22T19:45:01.929 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:00.406444+0000 osd.2 (osd.2) 218 : cluster [DBG] 5.b scrub ok 2024-02-22T19:45:01.929 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:00.697927+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e414: 8 total, 8 up, 5 in 2024-02-22T19:45:01.929 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:00.725644+0000 osd.5 (osd.5) 143 : cluster [DBG] 1.a scrub starts 2024-02-22T19:45:01.929 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:00.729902+0000 osd.5 (osd.5) 144 : cluster [DBG] 1.a scrub ok 2024-02-22T19:45:01.930 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:00.403487+0000 osd.2 (osd.2) 217 : cluster [DBG] 5.b scrub starts 2024-02-22T19:45:01.930 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:00.406444+0000 osd.2 (osd.2) 218 : cluster [DBG] 5.b scrub ok 2024-02-22T19:45:01.930 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:00.697927+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e414: 8 total, 8 up, 5 in 2024-02-22T19:45:01.930 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:00.725644+0000 osd.5 (osd.5) 143 : cluster [DBG] 1.a scrub starts 2024-02-22T19:45:01.930 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:00.729902+0000 osd.5 (osd.5) 144 : cluster [DBG] 1.a scrub ok 2024-02-22T19:45:02.041 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:02.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:45:02.503 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:02.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:02.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:02.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:45:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:45:02.669 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:02.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:45:02.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:02.783 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 54 16 75771 75755 87.6017 66.375 0.00708184 0.0114111 2024-02-22T19:45:02.804 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:02.831 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:02.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:02.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:02.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:02.966 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:01.450572+0000 osd.2 (osd.2) 219 : cluster [DBG] 3.e scrub starts 2024-02-22T19:45:02.966 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:01.461149+0000 osd.2 (osd.2) 220 : cluster [DBG] 3.e scrub ok 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:01.705624+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e415: 8 total, 8 up, 5 in 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:01.705744+0000 mgr.y (mgr.14535) 614 : cluster [DBG] pgmap v822: 140 pgs: 1 peering, 139 active+clean; 4.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.29k op/s 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:01.712420+0000 mon.a (mon.0) 1584 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:01.824699+0000 osd.3 (osd.3) 273 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:01.840219+0000 osd.3 (osd.3) 274 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:02.376635+0000 mon.a (mon.0) 1585 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-22T19:45:02.967 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:02.376694+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e416: 8 total, 8 up, 5 in 2024-02-22T19:45:02.971 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:02.995 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:03.000 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:03.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:01.450572+0000 osd.2 (osd.2) 219 : cluster [DBG] 3.e scrub starts 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:01.461149+0000 osd.2 (osd.2) 220 : cluster [DBG] 3.e scrub ok 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:01.705624+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e415: 8 total, 8 up, 5 in 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:01.705744+0000 mgr.y (mgr.14535) 614 : cluster [DBG] pgmap v822: 140 pgs: 1 peering, 139 active+clean; 4.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.29k op/s 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:01.712420+0000 mon.a (mon.0) 1584 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:01.824699+0000 osd.3 (osd.3) 273 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:01.840219+0000 osd.3 (osd.3) 274 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:02.376635+0000 mon.a (mon.0) 1585 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:02.376694+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e416: 8 total, 8 up, 5 in 2024-02-22T19:45:03.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:01.450572+0000 osd.2 (osd.2) 219 : cluster [DBG] 3.e scrub starts 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:01.461149+0000 osd.2 (osd.2) 220 : cluster [DBG] 3.e scrub ok 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:01.705624+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e415: 8 total, 8 up, 5 in 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:01.705744+0000 mgr.y (mgr.14535) 614 : cluster [DBG] pgmap v822: 140 pgs: 1 peering, 139 active+clean; 4.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.29k op/s 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:01.712420+0000 mon.a (mon.0) 1584 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:01.824699+0000 osd.3 (osd.3) 273 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:01.840219+0000 osd.3 (osd.3) 274 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:02.376635+0000 mon.a (mon.0) 1585 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-22T19:45:03.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:02.376694+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e416: 8 total, 8 up, 5 in 2024-02-22T19:45:03.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:03.499 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:03.769 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:03.774 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:03.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:03.783 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 55 16 76909 76893 87.3022 71.125 0.00832067 0.0114493 2024-02-22T19:45:03.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:03.924 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:02.821155+0000 osd.3 (osd.3) 275 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:45:04.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:02.829060+0000 osd.3 (osd.3) 276 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:45:04.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.383500+0000 mon.a (mon.0) 1587 : cluster [DBG] osdmap e417: 8 total, 8 up, 5 in 2024-02-22T19:45:04.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.421801+0000 osd.2 (osd.2) 221 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:45:04.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.431657+0000 osd.2 (osd.2) 222 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:45:04.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.706227+0000 mgr.y (mgr.14535) 615 : cluster [DBG] pgmap v825: 140 pgs: 1 peering, 139 active+clean; 4.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 19 MiB/s wr, 304 op/s 2024-02-22T19:45:04.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:02.821155+0000 osd.3 (osd.3) 275 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:45:04.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:02.829060+0000 osd.3 (osd.3) 276 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:45:04.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.383500+0000 mon.a (mon.0) 1587 : cluster [DBG] osdmap e417: 8 total, 8 up, 5 in 2024-02-22T19:45:04.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.421801+0000 osd.2 (osd.2) 221 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:45:04.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.431657+0000 osd.2 (osd.2) 222 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:45:04.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.706227+0000 mgr.y (mgr.14535) 615 : cluster [DBG] pgmap v825: 140 pgs: 1 peering, 139 active+clean; 4.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 19 MiB/s wr, 304 op/s 2024-02-22T19:45:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:02.821155+0000 osd.3 (osd.3) 275 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:45:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:02.829060+0000 osd.3 (osd.3) 276 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:45:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.383500+0000 mon.a (mon.0) 1587 : cluster [DBG] osdmap e417: 8 total, 8 up, 5 in 2024-02-22T19:45:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.421801+0000 osd.2 (osd.2) 221 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:45:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.431657+0000 osd.2 (osd.2) 222 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:45:04.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.706227+0000 mgr.y (mgr.14535) 615 : cluster [DBG] pgmap v825: 140 pgs: 1 peering, 139 active+clean; 4.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 19 MiB/s wr, 304 op/s 2024-02-22T19:45:04.784 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 56 16 77896 77880 86.8441 61.6875 0.00834804 0.0115108 2024-02-22T19:45:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:45:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:45:05.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:45:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.677789+0000 osd.1 (osd.1) 236 : cluster [DBG] 3.1a deep-scrub starts 2024-02-22T19:45:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.679095+0000 osd.1 (osd.1) 237 : cluster [DBG] 3.1a deep-scrub ok 2024-02-22T19:45:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.780163+0000 osd.5 (osd.5) 145 : cluster [DBG] 6.19 deep-scrub starts 2024-02-22T19:45:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:03.818047+0000 osd.5 (osd.5) 146 : cluster [DBG] 6.19 deep-scrub ok 2024-02-22T19:45:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:04.381235+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:45:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:04.389347+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:45:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:04.397548+0000 mon.a (mon.0) 1588 : cluster [DBG] osdmap e418: 8 total, 8 up, 5 in 2024-02-22T19:45:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:04.407067+0000 osd.7 (osd.7) 250 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:45:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:04.416665+0000 osd.7 (osd.7) 251 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:45:05.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:05.217260+0000 mgr.y (mgr.14535) 616 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:05.785 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 57 16 79029 79013 86.5613 70.8125 0.024395 0.0115481 2024-02-22T19:45:06.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.677789+0000 osd.1 (osd.1) 236 : cluster [DBG] 3.1a deep-scrub starts 2024-02-22T19:45:06.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.679095+0000 osd.1 (osd.1) 237 : cluster [DBG] 3.1a deep-scrub ok 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.780163+0000 osd.5 (osd.5) 145 : cluster [DBG] 6.19 deep-scrub starts 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:03.818047+0000 osd.5 (osd.5) 146 : cluster [DBG] 6.19 deep-scrub ok 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:04.381235+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:04.389347+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:04.397548+0000 mon.a (mon.0) 1588 : cluster [DBG] osdmap e418: 8 total, 8 up, 5 in 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:04.407067+0000 osd.7 (osd.7) 250 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:04.416665+0000 osd.7 (osd.7) 251 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:45:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:05.217260+0000 mgr.y (mgr.14535) 616 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:06.012 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.677789+0000 osd.1 (osd.1) 236 : cluster [DBG] 3.1a deep-scrub starts 2024-02-22T19:45:06.012 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.679095+0000 osd.1 (osd.1) 237 : cluster [DBG] 3.1a deep-scrub ok 2024-02-22T19:45:06.012 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.780163+0000 osd.5 (osd.5) 145 : cluster [DBG] 6.19 deep-scrub starts 2024-02-22T19:45:06.012 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:03.818047+0000 osd.5 (osd.5) 146 : cluster [DBG] 6.19 deep-scrub ok 2024-02-22T19:45:06.013 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:04.381235+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:45:06.013 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:04.389347+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:45:06.013 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:04.397548+0000 mon.a (mon.0) 1588 : cluster [DBG] osdmap e418: 8 total, 8 up, 5 in 2024-02-22T19:45:06.013 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:04.407067+0000 osd.7 (osd.7) 250 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:45:06.013 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:04.416665+0000 osd.7 (osd.7) 251 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:45:06.013 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:05.217260+0000 mgr.y (mgr.14535) 616 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:06.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:05.397200+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e419: 8 total, 8 up, 5 in 2024-02-22T19:45:06.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:05.706587+0000 mgr.y (mgr.14535) 617 : cluster [DBG] pgmap v828: 139 pgs: 1 clean+premerge+peered, 138 active+clean; 4.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:06.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:05.707854+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-22T19:45:06.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:05.397200+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e419: 8 total, 8 up, 5 in 2024-02-22T19:45:06.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:05.706587+0000 mgr.y (mgr.14535) 617 : cluster [DBG] pgmap v828: 139 pgs: 1 clean+premerge+peered, 138 active+clean; 4.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:06.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:05.707854+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-22T19:45:06.785 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 58 16 80072 80056 86.193 65.1875 0.014115 0.0115965 2024-02-22T19:45:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:05.397200+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e419: 8 total, 8 up, 5 in 2024-02-22T19:45:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:05.706587+0000 mgr.y (mgr.14535) 617 : cluster [DBG] pgmap v828: 139 pgs: 1 clean+premerge+peered, 138 active+clean; 4.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:06.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:05.707854+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-22T19:45:06.925 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:45:06.926 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:45:06.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:45:06.927 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:07.000 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:07.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:07.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:07.750 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:06.409288+0000 mon.a (mon.0) 1591 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-22T19:45:07.770 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:06.409328+0000 mon.a (mon.0) 1592 : cluster [DBG] osdmap e420: 8 total, 8 up, 5 in 2024-02-22T19:45:07.770 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:06.409288+0000 mon.a (mon.0) 1591 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-22T19:45:07.771 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:06.409328+0000 mon.a (mon.0) 1592 : cluster [DBG] osdmap e420: 8 total, 8 up, 5 in 2024-02-22T19:45:07.771 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:07.827 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 59 16 81050 81034 85.7119 61.125 0.0109879 0.0116545 2024-02-22T19:45:07.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:06.409288+0000 mon.a (mon.0) 1591 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-22T19:45:07.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:06.409328+0000 mon.a (mon.0) 1592 : cluster [DBG] osdmap e420: 8 total, 8 up, 5 in 2024-02-22T19:45:08.131 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:08.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:08.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:08.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:08.827 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:45:08.828436+0000 min lat: 0.00135253 max lat: 0.108314 avg lat: 0.0117057 2024-02-22T19:45:08.828 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:45:08.828 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 60 16 82127 82111 85.4009 67.3125 0.0146444 0.0117057 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:06.775133+0000 osd.5 (osd.5) 147 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:06.780597+0000 osd.5 (osd.5) 148 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:07.346819+0000 osd.2 (osd.2) 225 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:07.351503+0000 osd.2 (osd.2) 226 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:07.518131+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e421: 8 total, 8 up, 5 in 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:07.708894+0000 mgr.y (mgr.14535) 618 : cluster [DBG] pgmap v831: 139 pgs: 1 clean+premerge+peered, 138 active+clean; 4.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:08.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:08.377477+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:09.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:06.775133+0000 osd.5 (osd.5) 147 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:45:09.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:06.780597+0000 osd.5 (osd.5) 148 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:45:09.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:07.346819+0000 osd.2 (osd.2) 225 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:45:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:07.351503+0000 osd.2 (osd.2) 226 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:45:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:07.518131+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e421: 8 total, 8 up, 5 in 2024-02-22T19:45:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:07.708894+0000 mgr.y (mgr.14535) 618 : cluster [DBG] pgmap v831: 139 pgs: 1 clean+premerge+peered, 138 active+clean; 4.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:08.377477+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:09.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:06.775133+0000 osd.5 (osd.5) 147 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:45:09.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:06.780597+0000 osd.5 (osd.5) 148 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:45:09.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:07.346819+0000 osd.2 (osd.2) 225 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:45:09.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:07.351503+0000 osd.2 (osd.2) 226 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:45:09.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:07.518131+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e421: 8 total, 8 up, 5 in 2024-02-22T19:45:09.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:07.708894+0000 mgr.y (mgr.14535) 618 : cluster [DBG] pgmap v831: 139 pgs: 1 clean+premerge+peered, 138 active+clean; 4.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:09.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:08.377477+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:09.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:08.311999+0000 osd.2 (osd.2) 227 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:08.329362+0000 osd.2 (osd.2) 228 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:08.526787+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e422: 8 total, 8 up, 5 in 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:08.761285+0000 osd.1 (osd.1) 238 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:08.765914+0000 osd.1 (osd.1) 239 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:08.311999+0000 osd.2 (osd.2) 227 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:08.329362+0000 osd.2 (osd.2) 228 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:45:09.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:08.526787+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e422: 8 total, 8 up, 5 in 2024-02-22T19:45:09.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:08.761285+0000 osd.1 (osd.1) 238 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:45:09.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:08.765914+0000 osd.1 (osd.1) 239 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:45:09.759 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:45:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:09] "GET /metrics HTTP/1.1" 200 176417 "" "Prometheus/2.33.4" 2024-02-22T19:45:09.827 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 61 16 83199 83183 85.0996 67 0.0154937 0.0117464 2024-02-22T19:45:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:08.311999+0000 osd.2 (osd.2) 227 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:45:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:08.329362+0000 osd.2 (osd.2) 228 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:45:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:08.526787+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e422: 8 total, 8 up, 5 in 2024-02-22T19:45:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:08.761285+0000 osd.1 (osd.1) 238 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:45:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:08.765914+0000 osd.1 (osd.1) 239 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:45:10.828 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 62 16 84286 84270 84.823 67.9375 0.00980618 0.0117852 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.365073+0000 osd.7 (osd.7) 252 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.368113+0000 osd.7 (osd.7) 253 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.532790+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e423: 8 total, 8 up, 5 in 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.711251+0000 mgr.y (mgr.14535) 619 : cluster [DBG] pgmap v834: 138 pgs: 1 peering, 137 active+clean; 5.0 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 979 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.714596+0000 osd.1 (osd.1) 240 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.724550+0000 osd.1 (osd.1) 241 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.801618+0000 osd.5 (osd.5) 149 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:45:10.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.809802+0000 osd.5 (osd.5) 150 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.365073+0000 osd.7 (osd.7) 252 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.368113+0000 osd.7 (osd.7) 253 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.532790+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e423: 8 total, 8 up, 5 in 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.711251+0000 mgr.y (mgr.14535) 619 : cluster [DBG] pgmap v834: 138 pgs: 1 peering, 137 active+clean; 5.0 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 979 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.714596+0000 osd.1 (osd.1) 240 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.724550+0000 osd.1 (osd.1) 241 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:45:11.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.801618+0000 osd.5 (osd.5) 149 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.809802+0000 osd.5 (osd.5) 150 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.365073+0000 osd.7 (osd.7) 252 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.368113+0000 osd.7 (osd.7) 253 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.532790+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e423: 8 total, 8 up, 5 in 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.711251+0000 mgr.y (mgr.14535) 619 : cluster [DBG] pgmap v834: 138 pgs: 1 peering, 137 active+clean; 5.0 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 979 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.714596+0000 osd.1 (osd.1) 240 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.724550+0000 osd.1 (osd.1) 241 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.801618+0000 osd.5 (osd.5) 149 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:45:11.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.809802+0000 osd.5 (osd.5) 150 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:45:11.828 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 63 16 85329 85313 84.5117 65.1875 0.0158391 0.0118279 2024-02-22T19:45:11.849 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.680019+0000 osd.3 (osd.3) 277 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:45:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:09.692653+0000 osd.3 (osd.3) 278 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:45:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:11.230426+0000 osd.2 (osd.2) 229 : cluster [DBG] 10.3 scrub starts 2024-02-22T19:45:11.932 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:11.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.680019+0000 osd.3 (osd.3) 277 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:45:11.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:09.692653+0000 osd.3 (osd.3) 278 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:45:11.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:11.230426+0000 osd.2 (osd.2) 229 : cluster [DBG] 10.3 scrub starts 2024-02-22T19:45:11.937 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.680019+0000 osd.3 (osd.3) 277 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:45:11.937 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:09.692653+0000 osd.3 (osd.3) 278 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:45:11.937 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:11.230426+0000 osd.2 (osd.2) 229 : cluster [DBG] 10.3 scrub starts 2024-02-22T19:45:12.008 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:12.078 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:12.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:12.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:12.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:12.605 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:12.828 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 64 16 86303 86287 84.1428 60.875 0.0146828 0.0118801 2024-02-22T19:45:12.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:11.713049+0000 mgr.y (mgr.14535) 620 : cluster [DBG] pgmap v835: 138 pgs: 1 peering, 137 active+clean; 5.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 965 B/s rd, 62 MiB/s wr, 987 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:12.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:12.377910+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e424: 8 total, 8 up, 5 in 2024-02-22T19:45:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:11.713049+0000 mgr.y (mgr.14535) 620 : cluster [DBG] pgmap v835: 138 pgs: 1 peering, 137 active+clean; 5.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 965 B/s rd, 62 MiB/s wr, 987 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:13.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:12.377910+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e424: 8 total, 8 up, 5 in 2024-02-22T19:45:13.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:11.713049+0000 mgr.y (mgr.14535) 620 : cluster [DBG] pgmap v835: 138 pgs: 1 peering, 137 active+clean; 5.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 965 B/s rd, 62 MiB/s wr, 987 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:13.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:12.377910+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e424: 8 total, 8 up, 5 in 2024-02-22T19:45:13.828 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 65 16 87328 87312 83.8338 64.0625 0.0111954 0.0119241 2024-02-22T19:45:14.200 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:14.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:14.203 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:14.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:14.355 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:14.830 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 66 15 88371 88356 83.5514 65.25 0.00879254 0.0119635 2024-02-22T19:45:15.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:13.713432+0000 mgr.y (mgr.14535) 621 : cluster [DBG] pgmap v837: 138 pgs: 1 peering, 137 active+clean; 5.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 54 MiB/s wr, 872 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:15.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:14.048797+0000 osd.2 (osd.2) 230 : cluster [DBG] 10.3 scrub ok 2024-02-22T19:45:15.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:13.713432+0000 mgr.y (mgr.14535) 621 : cluster [DBG] pgmap v837: 138 pgs: 1 peering, 137 active+clean; 5.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 54 MiB/s wr, 872 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:15.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:14.048797+0000 osd.2 (osd.2) 230 : cluster [DBG] 10.3 scrub ok 2024-02-22T19:45:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:13.713432+0000 mgr.y (mgr.14535) 621 : cluster [DBG] pgmap v837: 138 pgs: 1 peering, 137 active+clean; 5.1 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 54 MiB/s wr, 872 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:14.048797+0000 osd.2 (osd.2) 230 : cluster [DBG] 10.3 scrub ok 2024-02-22T19:45:15.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:45:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:45:15.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:45:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:45:15.829 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 67 16 89478 89462 83.3363 69.125 0.00714862 0.0119942 2024-02-22T19:45:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:14.340280+0000 osd.2 (osd.2) 231 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:45:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:14.353671+0000 osd.2 (osd.2) 232 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:45:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:14.671949+0000 osd.3 (osd.3) 279 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:45:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:14.832475+0000 osd.3 (osd.3) 280 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:45:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:15.228864+0000 mgr.y (mgr.14535) 622 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:16.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:14.340280+0000 osd.2 (osd.2) 231 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:45:16.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:14.353671+0000 osd.2 (osd.2) 232 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:45:16.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:14.671949+0000 osd.3 (osd.3) 279 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:45:16.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:14.832475+0000 osd.3 (osd.3) 280 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:45:16.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:15.228864+0000 mgr.y (mgr.14535) 622 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:16.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:14.340280+0000 osd.2 (osd.2) 231 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:45:16.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:14.353671+0000 osd.2 (osd.2) 232 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:45:16.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:14.671949+0000 osd.3 (osd.3) 279 : cluster [DBG] 6.1f scrub starts 2024-02-22T19:45:16.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:14.832475+0000 osd.3 (osd.3) 280 : cluster [DBG] 6.1f scrub ok 2024-02-22T19:45:16.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:15.228864+0000 mgr.y (mgr.14535) 622 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:16.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:16.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:16.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:16.418 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:16.832 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 68 16 90568 90552 83.1094 68.125 0.0101731 0.0120279 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.715025+0000 mgr.y (mgr.14535) 623 : cluster [DBG] pgmap v838: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.4 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1.4 KiB/s rd, 91 MiB/s wr, 1.46k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.786323+0000 osd.3 (osd.3) 281 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.786396+0000 osd.3 (osd.3) 282 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.803275+0000 osd.1 (osd.1) 242 : cluster [DBG] 6.e scrub starts 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.816827+0000 osd.1 (osd.1) 243 : cluster [DBG] 6.e scrub ok 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.919371+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:15.919429+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:16.782659+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e425: 8 total, 8 up, 5 in 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.715025+0000 mgr.y (mgr.14535) 623 : cluster [DBG] pgmap v838: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.4 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1.4 KiB/s rd, 91 MiB/s wr, 1.46k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.786323+0000 osd.3 (osd.3) 281 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.786396+0000 osd.3 (osd.3) 282 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.803275+0000 osd.1 (osd.1) 242 : cluster [DBG] 6.e scrub starts 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.816827+0000 osd.1 (osd.1) 243 : cluster [DBG] 6.e scrub ok 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.919371+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:15.919429+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:16.782659+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e425: 8 total, 8 up, 5 in 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.715025+0000 mgr.y (mgr.14535) 623 : cluster [DBG] pgmap v838: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.4 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1.4 KiB/s rd, 91 MiB/s wr, 1.46k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.786323+0000 osd.3 (osd.3) 281 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.786396+0000 osd.3 (osd.3) 282 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.803275+0000 osd.1 (osd.1) 242 : cluster [DBG] 6.e scrub starts 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.816827+0000 osd.1 (osd.1) 243 : cluster [DBG] 6.e scrub ok 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.919371+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:15.919429+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:17.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:16.782659+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e425: 8 total, 8 up, 5 in 2024-02-22T19:45:17.417 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:17.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:17.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:17.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:17.580 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:17.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:17.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:17.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:17.741 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:17.834 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 69 16 91470 91454 82.7203 56.375 0.0329149 0.0120833 2024-02-22T19:45:17.854 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:17.860 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:17.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:17.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:18.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:16.349166+0000 osd.2 (osd.2) 233 : cluster [DBG] 10.e scrub starts 2024-02-22T19:45:18.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:17.377217+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e426: 8 total, 8 up, 5 in 2024-02-22T19:45:18.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:16.349166+0000 osd.2 (osd.2) 233 : cluster [DBG] 10.e scrub starts 2024-02-22T19:45:18.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:17.377217+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e426: 8 total, 8 up, 5 in 2024-02-22T19:45:18.043 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:16.349166+0000 osd.2 (osd.2) 233 : cluster [DBG] 10.e scrub starts 2024-02-22T19:45:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:17.377217+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e426: 8 total, 8 up, 5 in 2024-02-22T19:45:18.346 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:18.362 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:18.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:18.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:18.494 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:18.522 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:18.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:18.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:18.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:18.660 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:18.687 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:18.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:18.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:18.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:17.716379+0000 mgr.y (mgr.14535) 624 : cluster [DBG] pgmap v841: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.4 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 55 MiB/s wr, 873 op/s 2024-02-22T19:45:18.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:18.269117+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.53:0/739719652' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:45:18.838 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 70 16 92467 92451 82.4278 62.3125 0.0172893 0.0121264 2024-02-22T19:45:18.856 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:18.886 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:45:18.900 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:18.904 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:18.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:19.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:19.064 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:19.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:19.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:19.086 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:17.716379+0000 mgr.y (mgr.14535) 624 : cluster [DBG] pgmap v841: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.4 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 55 MiB/s wr, 873 op/s 2024-02-22T19:45:19.086 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:18.269117+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.53:0/739719652' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:45:19.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:17.716379+0000 mgr.y (mgr.14535) 624 : cluster [DBG] pgmap v841: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.4 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 852 B/s rd, 55 MiB/s wr, 873 op/s 2024-02-22T19:45:19.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:18.269117+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.53:0/739719652' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:45:19.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:19.226 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:19.395 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:19.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:19.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:19.401 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:19] "GET /metrics HTTP/1.1" 200 176417 "" "Prometheus/2.33.4" 2024-02-22T19:45:19.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:19.548 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:19.719 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:19.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:19.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:19.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:19.835 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 71 16 93534 93518 82.2063 66.6875 0.0138785 0.0121601 2024-02-22T19:45:19.885 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:19.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:18.816079+0000 mon.a (mon.0) 1601 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:19.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:18.847831+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? 172.21.15.53:0/739719652' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:45:19.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:18.847894+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e427: 8 total, 8 up, 5 in 2024-02-22T19:45:19.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:18.816079+0000 mon.a (mon.0) 1601 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:19.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:18.847831+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? 172.21.15.53:0/739719652' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:45:19.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:18.847894+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e427: 8 total, 8 up, 5 in 2024-02-22T19:45:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:18.816079+0000 mon.a (mon.0) 1601 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:18.847831+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? 172.21.15.53:0/739719652' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:45:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:18.847894+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e427: 8 total, 8 up, 5 in 2024-02-22T19:45:20.838 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 72 16 94653 94637 82.034 69.9375 0.00717659 0.012186 2024-02-22T19:45:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:19.717135+0000 mgr.y (mgr.14535) 625 : cluster [DBG] pgmap v843: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.6 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-02-22T19:45:21.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:19.717135+0000 mgr.y (mgr.14535) 625 : cluster [DBG] pgmap v843: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.6 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-02-22T19:45:21.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:19.717135+0000 mgr.y (mgr.14535) 625 : cluster [DBG] pgmap v843: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.6 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-02-22T19:45:21.838 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 73 16 95838 95822 81.9238 74.0625 0.0209999 0.0122019 2024-02-22T19:45:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:22.274 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:22.274 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:22.352 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:45:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:45:21+0000 lvl=info msg="Database locked, sleeping then retrying" logger=sqlstore error="database is locked" retry=0 2024-02-22T19:45:22.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:22.411 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:22.438 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:22.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:22.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:22.591 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:22.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:22.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:22.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:22.839 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 74 15 96951 96936 81.7577 69.625 0.0117816 0.0122266 2024-02-22T19:45:22.874 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:23.190 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:21.717687+0000 mgr.y (mgr.14535) 626 : cluster [DBG] pgmap v844: 138 pgs: 138 active+clean; 5.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 53 MiB/s wr, 849 op/s 2024-02-22T19:45:23.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:22.390553+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e428: 8 total, 8 up, 5 in 2024-02-22T19:45:23.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:21.717687+0000 mgr.y (mgr.14535) 626 : cluster [DBG] pgmap v844: 138 pgs: 138 active+clean; 5.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 53 MiB/s wr, 849 op/s 2024-02-22T19:45:23.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:22.390553+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e428: 8 total, 8 up, 5 in 2024-02-22T19:45:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:21.717687+0000 mgr.y (mgr.14535) 626 : cluster [DBG] pgmap v844: 138 pgs: 138 active+clean; 5.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 53 MiB/s wr, 849 op/s 2024-02-22T19:45:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:22.390553+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e428: 8 total, 8 up, 5 in 2024-02-22T19:45:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:23.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:23.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:23.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:23.524 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:23.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:23.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:45:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:45:23.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:23.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:45:23.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:23.816 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:23.839 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:23.839 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 75 16 97954 97938 81.5021 62.625 0.0168883 0.0122646 2024-02-22T19:45:23.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:23.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:23.944 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:24.001 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:24.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:24.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:24.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:45:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:24.166 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:45:24.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:24.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:45:24.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:24.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:45:24.302 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:24.325 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:24.329 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:24.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:24.401 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:23.392885+0000 mon.a (mon.0) 1605 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:24.401 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:23.516829+0000 mon.a (mon.0) 1606 : audit [DBG] from='client.? 172.21.15.53:0/4035608613' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:24.401 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:23.719743+0000 mgr.y (mgr.14535) 627 : cluster [DBG] pgmap v846: 138 pgs: 138 active+clean; 5.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 807 B/s rd, 50 MiB/s wr, 804 op/s 2024-02-22T19:45:24.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:24.484 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:24.489 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:24.489 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:24.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:24.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:23.392885+0000 mon.a (mon.0) 1605 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:24.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:23.516829+0000 mon.a (mon.0) 1606 : audit [DBG] from='client.? 172.21.15.53:0/4035608613' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:24.640 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:23.719743+0000 mgr.y (mgr.14535) 627 : cluster [DBG] pgmap v846: 138 pgs: 138 active+clean; 5.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 807 B/s rd, 50 MiB/s wr, 804 op/s 2024-02-22T19:45:24.641 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:23.392885+0000 mon.a (mon.0) 1605 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:24.641 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:23.516829+0000 mon.a (mon.0) 1606 : audit [DBG] from='client.? 172.21.15.53:0/4035608613' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:24.641 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:23.719743+0000 mgr.y (mgr.14535) 627 : cluster [DBG] pgmap v846: 138 pgs: 138 active+clean; 5.7 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 807 B/s rd, 50 MiB/s wr, 804 op/s 2024-02-22T19:45:24.643 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:24.839 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 76 16 99025 99009 81.3106 66.9375 0.0179419 0.0122937 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:45:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:45:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:45:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:45:25.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:45:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:45:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:25.240208+0000 mgr.y (mgr.14535) 628 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:25.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:25.240208+0000 mgr.y (mgr.14535) 628 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:25.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:25.240208+0000 mgr.y (mgr.14535) 628 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:25.839 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:25.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:25.842 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 77 16 100053 100037 81.0865 64.25 0.0213373 0.012328 2024-02-22T19:45:25.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15576.0:98031 10.b 10:d4456454:::benchmark_data_smithi053_180506_object98030:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e428)", 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:45:23.933386+0000", 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0080715209999999999, 2024-02-22T19:45:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0080983170000000007, 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15576", 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98031 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933386+0000", 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:45:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933386+0000", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933384+0000", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2059999999999999e-05 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933406+0000", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2415000000000001e-05 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933418+0000", 2024-02-22T19:45:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3480000000000003e-06 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933424+0000", 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010395699999999999 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933528+0000", 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015813399999999999 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933686+0000", 2024-02-22T19:45:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017058199999999999 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.933857+0000", 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020356200000000001 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.935892+0000", 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019726169999999999 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937865+0000", 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044787990000000003 2024-02-22T19:45:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15576.0:98034 10.1 e428/365 10:8b26ed58:::benchmark_data_smithi053_180506_object98033:head v 428'6065, mlcod=428'6063)", 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:45:23.937309+0000", 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0041480309999999999, 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042352229999999998, 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937309+0000", 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937309+0000", 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937309+0000", 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6130000000000004e-05 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937395+0000", 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2488e-05 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937407+0000", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937414+0000", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0383000000000002e-05 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937464+0000", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8240000000000002e-05 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:45:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937483+0000", 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017328299999999999 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15576.0:98036 10.4 e428/365 10:2c782699:::benchmark_data_smithi053_180506_object98035:head v 428'6017, mlcod=428'6015)", 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:45:23.937637+0000", 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0038206099999999999, 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039462280000000004, 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937637+0000", 2024-02-22T19:45:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937637+0000", 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937636+0000", 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4036000000000002e-05 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937660+0000", 2024-02-22T19:45:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6010000000000001e-06 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937670+0000", 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937675+0000", 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7206999999999998e-05 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937713+0000", 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7756000000000001e-05 2024-02-22T19:45:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:23.937730+0000", 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3583000000000002e-05 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 3 2024-02-22T19:45:26.028 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:45:26.232 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:26.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:26.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:26.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:26.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:26.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:26.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:25.720570+0000 mgr.y (mgr.14535) 629 : cluster [DBG] pgmap v847: 138 pgs: 138 active+clean; 6.0 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-02-22T19:45:26.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:25.720570+0000 mgr.y (mgr.14535) 629 : cluster [DBG] pgmap v847: 138 pgs: 138 active+clean; 6.0 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-02-22T19:45:26.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:26.581 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:26.842 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 78 16 101137 101121 80.9155 67.75 0.0124152 0.0123543 2024-02-22T19:45:26.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:25.720570+0000 mgr.y (mgr.14535) 629 : cluster [DBG] pgmap v847: 138 pgs: 138 active+clean; 6.0 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-02-22T19:45:27.457 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:27.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:27.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:27.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:45:27.568 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:27.843 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 79 16 102178 102162 80.714 65.0625 0.0257602 0.0123848 2024-02-22T19:45:27.872 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:27.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:27.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:27.978 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:28.010 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:28.032 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:28.034 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:28.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:28.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:28.212 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:28.447 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:45:28.476 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:45:28.503 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:27.721107+0000 mgr.y (mgr.14535) 630 : cluster [DBG] pgmap v848: 138 pgs: 138 active+clean; 6.0 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.1 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:45:28.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:27.721107+0000 mgr.y (mgr.14535) 630 : cluster [DBG] pgmap v848: 138 pgs: 138 active+clean; 6.0 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.1 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:45:28.512 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:28.843 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:45:28.844195+0000 min lat: 0.00135253 max lat: 0.172036 avg lat: 0.0124243 2024-02-22T19:45:28.843 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:45:28.843 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 80 16 103140 103124 80.4569 60.125 0.020439 0.0124243 2024-02-22T19:45:28.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:27.721107+0000 mgr.y (mgr.14535) 630 : cluster [DBG] pgmap v848: 138 pgs: 138 active+clean; 6.0 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.1 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-02-22T19:45:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:29.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:29.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:29.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:29.176 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:29.568 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:28.418472+0000 mgr.y (mgr.14535) 631 : audit [DBG] from='client.25462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:45:29.570 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:45:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:29] "GET /metrics HTTP/1.1" 200 176422 "" "Prometheus/2.33.4" 2024-02-22T19:45:29.570 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:28.418472+0000 mgr.y (mgr.14535) 631 : audit [DBG] from='client.25462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:45:29.571 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:29.571 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":848,"stamp":"2024-02-22T19:45:27.720861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6469195428,"num_objects":98904,"num_object_clones":0,"num_object_copies":296712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98904,"num_whiteouts":0,"num_read":2086,"num_read_kb":1838,"num_write":99426,"num_write_kb":6317609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":452,"num_bytes_recovered":15766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59267,"ondisk_log_size":59267,"up":414,"acting":414,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":414,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":22472824,"kb_used_data":18904272,"kb_used_omap":0,"kb_used_meta":3568512,"kb_avail":446252936,"statfs":{"total":479975178240,"available":456963006464,"internally_reserved":0,"allocated":19357974528,"data_stored":19350233513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3654156288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":695795712,"num_objects":10617,"num_object_clones":0,"num_object_copies":31851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10617,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10617,"num_write_kb":679488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.875497"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":385,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858562+0000","last_change":"2024-02-22T19:40:44.421798+0000","last_active":"2024-02-22T19:45:18.858562+0000","last_peered":"2024-02-22T19:45:18.858562+0000","last_clean":"2024-02-22T19:45:18.858562+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:45:18.858562+0000","last_undegraded":"2024-02-22T19:45:18.858562+0000","last_fullsized":"2024-02-22T19:45:18.858562+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:44.421713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:44.421713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":420,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858991+0000","last_change":"2024-02-22T19:43:58.780165+0000","last_active":"2024-02-22T19:45:18.858991+0000","last_peered":"2024-02-22T19:45:18.858991+0000","last_clean":"2024-02-22T19:45:18.858991+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:45:18.858991+0000","last_undegraded":"2024-02-22T19:45:18.858991+0000","last_fullsized":"2024-02-22T19:45:18.858991+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.780100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.780100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":199,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854319+0000","last_change":"2024-02-22T19:41:52.415022+0000","last_active":"2024-02-22T19:45:18.854319+0000","last_peered":"2024-02-22T19:45:18.854319+0000","last_clean":"2024-02-22T19:45:18.854319+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:45:18.854319+0000","last_undegraded":"2024-02-22T19:45:18.854319+0000","last_fullsized":"2024-02-22T19:45:18.854319+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:41:22.833850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.833850+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":400,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864616+0000","last_change":"2024-02-22T19:41:52.116243+0000","last_active":"2024-02-22T19:45:18.864616+0000","last_peered":"2024-02-22T19:45:18.864616+0000","last_clean":"2024-02-22T19:45:18.864616+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:45:18.864616+0000","last_undegraded":"2024-02-22T19:45:18.864616+0000","last_fullsized":"2024-02-22T19:45:18.864616+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:23.826943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:41:23.826943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":406,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864506+0000","last_change":"2024-02-22T19:44:56.776815+0000","last_active":"2024-02-22T19:45:18.864506+0000","last_peered":"2024-02-22T19:45:18.864506+0000","last_clean":"2024-02-22T19:45:18.864506+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:45:18.864506+0000","last_undegraded":"2024-02-22T19:45:18.864506+0000","last_fullsized":"2024-02-22T19:45:18.864506+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:56.776721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.776721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":513,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.863026+0000","last_change":"2024-02-22T19:44:07.984802+0000","last_active":"2024-02-22T19:45:18.863026+0000","last_peered":"2024-02-22T19:45:18.863026+0000","last_clean":"2024-02-22T19:45:18.863026+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:45:18.863026+0000","last_undegraded":"2024-02-22T19:45:18.863026+0000","last_fullsized":"2024-02-22T19:45:18.863026+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:44:07.984710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.984710+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":229,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.849730+0000","last_change":"2024-02-22T19:41:16.258371+0000","last_active":"2024-02-22T19:45:18.849730+0000","last_peered":"2024-02-22T19:45:18.849730+0000","last_clean":"2024-02-22T19:45:18.849730+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:45:18.849730+0000","last_undegraded":"2024-02-22T19:45:18.849730+0000","last_fullsized":"2024-02-22T19:45:18.849730+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.258272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.258272+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":243,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858700+0000","last_change":"2024-02-22T19:44:07.795043+0000","last_active":"2024-02-22T19:45:18.858700+0000","last_peered":"2024-02-22T19:45:18.858700+0000","last_clean":"2024-02-22T19:45:18.858700+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:45:18.858700+0000","last_undegraded":"2024-02-22T19:45:18.858700+0000","last_fullsized":"2024-02-22T19:45:18.858700+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:07.794943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.794943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":310,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.869877+0000","last_change":"2024-02-22T19:45:02.829140+0000","last_active":"2024-02-22T19:45:18.869877+0000","last_peered":"2024-02-22T19:45:18.869877+0000","last_clean":"2024-02-22T19:45:18.869877+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:45:18.869877+0000","last_undegraded":"2024-02-22T19:45:18.869877+0000","last_fullsized":"2024-02-22T19:45:18.869877+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:45:02.829064+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:45:02.829064+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":230,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857867+0000","last_change":"2024-02-22T19:45:03.818156+0000","last_active":"2024-02-22T19:45:18.857867+0000","last_peered":"2024-02-22T19:45:18.857867+0000","last_clean":"2024-02-22T19:45:18.857867+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:45:18.857867+0000","last_undegraded":"2024-02-22T19:45:18.857867+0000","last_fullsized":"2024-02-22T19:45:18.857867+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.818051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":260,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858414+0000","last_change":"2024-02-22T19:44:44.719650+0000","last_active":"2024-02-22T19:45:18.858414+0000","last_peered":"2024-02-22T19:45:18.858414+0000","last_clean":"2024-02-22T19:45:18.858414+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:45:18.858414+0000","last_undegraded":"2024-02-22T19:45:18.858414+0000","last_fullsized":"2024-02-22T19:45:18.858414+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:44:44.719546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:44.719546+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":233,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864266+0000","last_change":"2024-02-22T19:44:04.949624+0000","last_active":"2024-02-22T19:45:18.864266+0000","last_peered":"2024-02-22T19:45:18.864266+0000","last_clean":"2024-02-22T19:45:18.864266+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:45:18.864266+0000","last_undegraded":"2024-02-22T19:45:18.864266+0000","last_fullsized":"2024-02-22T19:45:18.864266+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:44:04.949558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.949558+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":303,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.859915+0000","last_change":"2024-02-22T19:41:52.111734+0000","last_active":"2024-02-22T19:45:18.859915+0000","last_peered":"2024-02-22T19:45:18.859915+0000","last_clean":"2024-02-22T19:45:18.859915+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:45:18.859915+0000","last_undegraded":"2024-02-22T19:45:18.859915+0000","last_fullsized":"2024-02-22T19:45:18.859915+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:27.820937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:27.820937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":206,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.856694+0000","last_change":"2024-02-22T19:45:16.384365+0000","last_active":"2024-02-22T19:45:18.856694+0000","last_peered":"2024-02-22T19:45:18.856694+0000","last_clean":"2024-02-22T19:45:18.856694+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:45:18.856694+0000","last_undegraded":"2024-02-22T19:45:18.856694+0000","last_fullsized":"2024-02-22T19:45:18.856694+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.650315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.650315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":460,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866946+0000","last_change":"2024-02-22T19:44:07.172636+0000","last_active":"2024-02-22T19:45:18.866946+0000","last_peered":"2024-02-22T19:45:18.866946+0000","last_clean":"2024-02-22T19:45:18.866946+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:45:18.866946+0000","last_undegraded":"2024-02-22T19:45:18.866946+0000","last_fullsized":"2024-02-22T19:45:18.866946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:44:07.172562+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.172562+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":423,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864430+0000","last_change":"2024-02-22T19:45:14.832581+0000","last_active":"2024-02-22T19:45:18.864430+0000","last_peered":"2024-02-22T19:45:18.864430+0000","last_clean":"2024-02-22T19:45:18.864430+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:45:18.864430+0000","last_undegraded":"2024-02-22T19:45:18.864430+0000","last_fullsized":"2024-02-22T19:45:18.864430+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.832482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.832482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":220,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853759+0000","last_change":"2024-02-22T19:45:03.679204+0000","last_active":"2024-02-22T19:45:18.853759+0000","last_peered":"2024-02-22T19:45:18.853759+0000","last_clean":"2024-02-22T19:45:18.853759+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:45:18.853759+0000","last_undegraded":"2024-02-22T19:45:18.853759+0000","last_fullsized":"2024-02-22T19:45:18.853759+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.679103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":474,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.862768+0000","last_change":"2024-02-22T19:44:43.774884+0000","last_active":"2024-02-22T19:45:18.862768+0000","last_peered":"2024-02-22T19:45:18.862768+0000","last_clean":"2024-02-22T19:45:18.862768+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:45:18.862768+0000","last_undegraded":"2024-02-22T19:45:18.862768+0000","last_fullsized":"2024-02-22T19:45:18.862768+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:44:43.774778+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:44:43.774778+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":439,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854480+0000","last_change":"2024-02-22T19:44:04.315337+0000","last_active":"2024-02-22T19:45:18.854480+0000","last_peered":"2024-02-22T19:45:18.854480+0000","last_clean":"2024-02-22T19:45:18.854480+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:45:18.854480+0000","last_undegraded":"2024-02-22T19:45:18.854480+0000","last_fullsized":"2024-02-22T19:45:18.854480+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:44:04.315265+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.315265+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":428,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854392+0000","last_change":"2024-02-22T19:45:09.724649+0000","last_active":"2024-02-22T19:45:18.854392+0000","last_peered":"2024-02-22T19:45:18.854392+0000","last_clean":"2024-02-22T19:45:18.854392+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:45:18.854392+0000","last_undegraded":"2024-02-22T19:45:18.854392+0000","last_fullsized":"2024-02-22T19:45:18.854392+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:09.724557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.724557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":240,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854539+0000","last_change":"2024-02-22T19:44:18.299947+0000","last_active":"2024-02-22T19:45:18.854539+0000","last_peered":"2024-02-22T19:45:18.854539+0000","last_clean":"2024-02-22T19:45:18.854539+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:45:18.854539+0000","last_undegraded":"2024-02-22T19:45:18.854539+0000","last_fullsized":"2024-02-22T19:45:18.854539+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:44:18.299831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:18.299831+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":428,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853815+0000","last_change":"2024-02-22T19:44:48.653802+0000","last_active":"2024-02-22T19:45:18.853815+0000","last_peered":"2024-02-22T19:45:18.853815+0000","last_clean":"2024-02-22T19:45:18.853815+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:45:18.853815+0000","last_undegraded":"2024-02-22T19:45:18.853815+0000","last_fullsized":"2024-02-22T19:45:18.853815+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.653696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.653696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":426,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.863498+0000","last_change":"2024-02-22T19:44:10.017728+0000","last_active":"2024-02-22T19:45:18.863498+0000","last_peered":"2024-02-22T19:45:18.863498+0000","last_clean":"2024-02-22T19:45:18.863498+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:45:18.863498+0000","last_undegraded":"2024-02-22T19:45:18.863498+0000","last_fullsized":"2024-02-22T19:45:18.863498+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:10.017666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:44:10.017666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":468,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.859058+0000","last_change":"2024-02-22T19:45:09.809874+0000","last_active":"2024-02-22T19:45:18.859058+0000","last_peered":"2024-02-22T19:45:18.859058+0000","last_clean":"2024-02-22T19:45:18.859058+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:45:18.859058+0000","last_undegraded":"2024-02-22T19:45:18.859058+0000","last_fullsized":"2024-02-22T19:45:18.859058+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:09.809806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.809806+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":408,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854667+0000","last_change":"2024-02-22T19:43:59.334492+0000","last_active":"2024-02-22T19:45:18.854667+0000","last_peered":"2024-02-22T19:45:18.854667+0000","last_clean":"2024-02-22T19:45:18.854667+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:45:18.854667+0000","last_undegraded":"2024-02-22T19:45:18.854667+0000","last_fullsized":"2024-02-22T19:45:18.854667+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.334420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.334420+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":420,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858907+0000","last_change":"2024-02-22T19:44:13.694618+0000","last_active":"2024-02-22T19:45:18.858907+0000","last_peered":"2024-02-22T19:45:18.858907+0000","last_clean":"2024-02-22T19:45:18.858907+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:45:18.858907+0000","last_undegraded":"2024-02-22T19:45:18.858907+0000","last_fullsized":"2024-02-22T19:45:18.858907+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.694552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.694552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":422,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.863586+0000","last_change":"2024-02-22T19:41:52.308580+0000","last_active":"2024-02-22T19:45:18.863586+0000","last_peered":"2024-02-22T19:45:18.863586+0000","last_clean":"2024-02-22T19:45:18.863586+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:45:18.863586+0000","last_undegraded":"2024-02-22T19:45:18.863586+0000","last_fullsized":"2024-02-22T19:45:18.863586+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:41:29.630421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.630421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":429,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864779+0000","last_change":"2024-02-22T19:43:59.063011+0000","last_active":"2024-02-22T19:45:18.864779+0000","last_peered":"2024-02-22T19:45:18.864779+0000","last_clean":"2024-02-22T19:45:18.864779+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:45:18.864779+0000","last_undegraded":"2024-02-22T19:45:18.864779+0000","last_fullsized":"2024-02-22T19:45:18.864779+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:59.062912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:59.062912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":225,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.859893+0000","last_change":"2024-02-22T19:44:12.674961+0000","last_active":"2024-02-22T19:45:18.859893+0000","last_peered":"2024-02-22T19:45:18.859893+0000","last_clean":"2024-02-22T19:45:18.859893+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:45:18.859893+0000","last_undegraded":"2024-02-22T19:45:18.859893+0000","last_fullsized":"2024-02-22T19:45:18.859893+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.674856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.674856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":447,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.862847+0000","last_change":"2024-02-22T19:44:11.041545+0000","last_active":"2024-02-22T19:45:18.862847+0000","last_peered":"2024-02-22T19:45:18.862847+0000","last_clean":"2024-02-22T19:45:18.862847+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:45:18.862847+0000","last_undegraded":"2024-02-22T19:45:18.862847+0000","last_fullsized":"2024-02-22T19:45:18.862847+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:44:11.041458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.041458+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":411,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858091+0000","last_change":"2024-02-22T19:44:14.661384+0000","last_active":"2024-02-22T19:45:18.858091+0000","last_peered":"2024-02-22T19:45:18.858091+0000","last_clean":"2024-02-22T19:45:18.858091+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:45:18.858091+0000","last_undegraded":"2024-02-22T19:45:18.858091+0000","last_fullsized":"2024-02-22T19:45:18.858091+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:14.661322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:14.661322+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":443,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854391+0000","last_change":"2024-02-22T19:41:52.111171+0000","last_active":"2024-02-22T19:45:18.854391+0000","last_peered":"2024-02-22T19:45:18.854391+0000","last_clean":"2024-02-22T19:45:18.854391+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:45:18.854391+0000","last_undegraded":"2024-02-22T19:45:18.854391+0000","last_fullsized":"2024-02-22T19:45:18.854391+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.611187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.611187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":447,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854036+0000","last_change":"2024-02-22T19:41:52.122300+0000","last_active":"2024-02-22T19:45:18.854036+0000","last_peered":"2024-02-22T19:45:18.854036+0000","last_clean":"2024-02-22T19:45:18.854036+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:45:18.854036+0000","last_undegraded":"2024-02-22T19:45:18.854036+0000","last_fullsized":"2024-02-22T19:45:18.854036+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:41:18.275065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.275065+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":441,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.862453+0000","last_change":"2024-02-22T19:44:52.796416+0000","last_active":"2024-02-22T19:45:18.862453+0000","last_peered":"2024-02-22T19:45:18.862453+0000","last_clean":"2024-02-22T19:45:18.862453+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:45:18.862453+0000","last_undegraded":"2024-02-22T19:45:18.862453+0000","last_fullsized":"2024-02-22T19:45:18.862453+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:52.796310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.796310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":213,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854414+0000","last_change":"2024-02-22T19:45:04.416774+0000","last_active":"2024-02-22T19:45:18.854414+0000","last_peered":"2024-02-22T19:45:18.854414+0000","last_clean":"2024-02-22T19:45:18.854414+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:45:18.854414+0000","last_undegraded":"2024-02-22T19:45:18.854414+0000","last_fullsized":"2024-02-22T19:45:18.854414+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:04.416671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:04.416671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":210,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.855583+0000","last_change":"2024-02-22T19:41:52.124754+0000","last_active":"2024-02-22T19:45:18.855583+0000","last_peered":"2024-02-22T19:45:18.855583+0000","last_clean":"2024-02-22T19:45:18.855583+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:45:18.855583+0000","last_undegraded":"2024-02-22T19:45:18.855583+0000","last_fullsized":"2024-02-22T19:45:18.855583+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:17.864427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.864427+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":228,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854435+0000","last_change":"2024-02-22T19:44:41.261006+0000","last_active":"2024-02-22T19:45:18.854435+0000","last_peered":"2024-02-22T19:45:18.854435+0000","last_clean":"2024-02-22T19:45:18.854435+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:45:18.854435+0000","last_undegraded":"2024-02-22T19:45:18.854435+0000","last_fullsized":"2024-02-22T19:45:18.854435+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:41.260945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:41.260945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":421,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.856538+0000","last_change":"2024-02-22T19:45:09.368245+0000","last_active":"2024-02-22T19:45:18.856538+0000","last_peered":"2024-02-22T19:45:18.856538+0000","last_clean":"2024-02-22T19:45:18.856538+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:45:18.856538+0000","last_undegraded":"2024-02-22T19:45:18.856538+0000","last_fullsized":"2024-02-22T19:45:18.856538+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:09.368133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.368133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":430,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.868509+0000","last_change":"2024-02-22T19:41:24.739687+0000","last_active":"2024-02-22T19:45:18.868509+0000","last_peered":"2024-02-22T19:45:18.868509+0000","last_clean":"2024-02-22T19:45:18.868509+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:45:18.868509+0000","last_undegraded":"2024-02-22T19:45:18.868509+0000","last_fullsized":"2024-02-22T19:45:18.868509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:41:24.739564+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.739564+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":413,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866680+0000","last_change":"2024-02-22T19:44:45.498665+0000","last_active":"2024-02-22T19:45:18.866680+0000","last_peered":"2024-02-22T19:45:18.866680+0000","last_clean":"2024-02-22T19:45:18.866680+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:45:18.866680+0000","last_undegraded":"2024-02-22T19:45:18.866680+0000","last_fullsized":"2024-02-22T19:45:18.866680+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:45.498567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.498567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":214,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854574+0000","last_change":"2024-02-22T19:41:34.587724+0000","last_active":"2024-02-22T19:45:18.854574+0000","last_peered":"2024-02-22T19:45:18.854574+0000","last_clean":"2024-02-22T19:45:18.854574+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:45:18.854574+0000","last_undegraded":"2024-02-22T19:45:18.854574+0000","last_fullsized":"2024-02-22T19:45:18.854574+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:34.587620+0000","last_clean_scrub_stamp":"2024-02-22T19:41:34.587620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":244,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858807+0000","last_change":"2024-02-22T19:41:31.388212+0000","last_active":"2024-02-22T19:45:18.858807+0000","last_peered":"2024-02-22T19:45:18.858807+0000","last_clean":"2024-02-22T19:45:18.858807+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:45:18.858807+0000","last_undegraded":"2024-02-22T19:45:18.858807+0000","last_fullsized":"2024-02-22T19:45:18.858807+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.388143+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":436,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.867093+0000","last_change":"2024-02-22T19:45:07.351611+0000","last_active":"2024-02-22T19:45:18.867093+0000","last_peered":"2024-02-22T19:45:18.867093+0000","last_clean":"2024-02-22T19:45:18.867093+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:45:18.867093+0000","last_undegraded":"2024-02-22T19:45:18.867093+0000","last_fullsized":"2024-02-22T19:45:18.867093+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:07.351512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:45:07.351512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":439,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853941+0000","last_change":"2024-02-22T19:41:52.112047+0000","last_active":"2024-02-22T19:45:18.853941+0000","last_peered":"2024-02-22T19:45:18.853941+0000","last_clean":"2024-02-22T19:45:18.853941+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:45:18.853941+0000","last_undegraded":"2024-02-22T19:45:18.853941+0000","last_fullsized":"2024-02-22T19:45:18.853941+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:31.641131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:41:31.641131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":221,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864643+0000","last_change":"2024-02-22T19:44:47.735006+0000","last_active":"2024-02-22T19:45:18.864643+0000","last_peered":"2024-02-22T19:45:18.864643+0000","last_clean":"2024-02-22T19:45:18.864643+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:45:18.864643+0000","last_undegraded":"2024-02-22T19:45:18.864643+0000","last_fullsized":"2024-02-22T19:45:18.864643+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:44:47.734940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.734940+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":441,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854307+0000","last_change":"2024-02-22T19:43:57.440300+0000","last_active":"2024-02-22T19:45:18.854307+0000","last_peered":"2024-02-22T19:45:18.854307+0000","last_clean":"2024-02-22T19:45:18.854307+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:45:18.854307+0000","last_undegraded":"2024-02-22T19:45:18.854307+0000","last_fullsized":"2024-02-22T19:45:18.854307+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.440217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.440217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":397,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857900+0000","last_change":"2024-02-22T19:44:58.799472+0000","last_active":"2024-02-22T19:45:18.857900+0000","last_peered":"2024-02-22T19:45:18.857900+0000","last_clean":"2024-02-22T19:45:18.857900+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:45:18.857900+0000","last_undegraded":"2024-02-22T19:45:18.857900+0000","last_fullsized":"2024-02-22T19:45:18.857900+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:58.799397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:44:58.799397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":435,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864476+0000","last_change":"2024-02-22T19:45:01.840336+0000","last_active":"2024-02-22T19:45:18.864476+0000","last_peered":"2024-02-22T19:45:18.864476+0000","last_clean":"2024-02-22T19:45:18.864476+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:45:18.864476+0000","last_undegraded":"2024-02-22T19:45:18.864476+0000","last_fullsized":"2024-02-22T19:45:18.864476+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:45:01.840227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.840227+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"428'6218","reported_seq":6289,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.422939+0000","last_change":"2024-02-22T19:43:57.347864+0000","last_active":"2024-02-22T19:45:24.422939+0000","last_peered":"2024-02-22T19:45:24.422939+0000","last_clean":"2024-02-22T19:45:24.422939+0000","last_became_active":"2024-02-22T19:43:57.347537+0000","last_became_peered":"2024-02-22T19:43:57.347537+0000","last_unstale":"2024-02-22T19:45:24.422939+0000","last_undegraded":"2024-02-22T19:45:24.422939+0000","last_fullsized":"2024-02-22T19:45:24.422939+0000","mapping_epoch":365,"log_start":"385'2700","ondisk_log_start":"385'2700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3518,"ondisk_log_size":3518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":407502848,"num_objects":6218,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6218,"num_write_kb":397952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":261,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858477+0000","last_change":"2024-02-22T19:41:52.107796+0000","last_active":"2024-02-22T19:45:18.858477+0000","last_peered":"2024-02-22T19:45:18.858477+0000","last_clean":"2024-02-22T19:45:18.858477+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:45:18.858477+0000","last_undegraded":"2024-02-22T19:45:18.858477+0000","last_fullsized":"2024-02-22T19:45:18.858477+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:49.502343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:40:49.502343+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":493,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854892+0000","last_change":"2024-02-22T19:44:08.271084+0000","last_active":"2024-02-22T19:45:18.854892+0000","last_peered":"2024-02-22T19:45:18.854892+0000","last_clean":"2024-02-22T19:45:18.854892+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:45:18.854892+0000","last_undegraded":"2024-02-22T19:45:18.854892+0000","last_fullsized":"2024-02-22T19:45:18.854892+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:44:08.271004+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:44:08.271004+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":284,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858148+0000","last_change":"2024-02-22T19:41:52.721927+0000","last_active":"2024-02-22T19:45:18.858148+0000","last_peered":"2024-02-22T19:45:18.858148+0000","last_clean":"2024-02-22T19:45:18.858148+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:45:18.858148+0000","last_undegraded":"2024-02-22T19:45:18.858148+0000","last_fullsized":"2024-02-22T19:45:18.858148+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:46.420351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:46.420351+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":675,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:22.406918+0000","last_change":"2024-02-22T19:41:54.317912+0000","last_active":"2024-02-22T19:45:22.406918+0000","last_peered":"2024-02-22T19:45:22.406918+0000","last_clean":"2024-02-22T19:45:22.406918+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:45:22.406918+0000","last_undegraded":"2024-02-22T19:45:22.406918+0000","last_fullsized":"2024-02-22T19:45:22.406918+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'13","last_scrub_stamp":"2024-02-22T19:41:17.712487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:17.712487+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"428'6247","reported_seq":6318,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.775411+0000","last_change":"2024-02-22T19:43:57.350818+0000","last_active":"2024-02-22T19:45:24.775411+0000","last_peered":"2024-02-22T19:45:24.775411+0000","last_clean":"2024-02-22T19:45:24.775411+0000","last_became_active":"2024-02-22T19:43:57.350467+0000","last_became_peered":"2024-02-22T19:43:57.350467+0000","last_unstale":"2024-02-22T19:45:24.775411+0000","last_undegraded":"2024-02-22T19:45:24.775411+0000","last_fullsized":"2024-02-22T19:45:24.775411+0000","mapping_epoch":365,"log_start":"388'2900","ondisk_log_start":"388'2900","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3347,"ondisk_log_size":3347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":409403392,"num_objects":6247,"num_object_clones":0,"num_object_copies":18741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":399808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":391,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864697+0000","last_change":"2024-02-22T19:44:57.962949+0000","last_active":"2024-02-22T19:45:18.864697+0000","last_peered":"2024-02-22T19:45:18.864697+0000","last_clean":"2024-02-22T19:45:18.864697+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:45:18.864697+0000","last_undegraded":"2024-02-22T19:45:18.864697+0000","last_fullsized":"2024-02-22T19:45:18.864697+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.962855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.962855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":135,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.865820+0000","last_change":"2024-02-22T19:45:01.461246+0000","last_active":"2024-02-22T19:45:18.865820+0000","last_peered":"2024-02-22T19:45:18.865820+0000","last_clean":"2024-02-22T19:45:18.865820+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:45:18.865820+0000","last_undegraded":"2024-02-22T19:45:18.865820+0000","last_fullsized":"2024-02-22T19:45:18.865820+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:01.461156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.461156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":279,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854177+0000","last_change":"2024-02-22T19:45:08.766029+0000","last_active":"2024-02-22T19:45:18.854177+0000","last_peered":"2024-02-22T19:45:18.854177+0000","last_clean":"2024-02-22T19:45:18.854177+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:45:18.854177+0000","last_undegraded":"2024-02-22T19:45:18.854177+0000","last_fullsized":"2024-02-22T19:45:18.854177+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:45:08.765921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:08.765921+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":451,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866573+0000","last_change":"2024-02-22T19:45:03.431763+0000","last_active":"2024-02-22T19:45:18.866573+0000","last_peered":"2024-02-22T19:45:18.866573+0000","last_clean":"2024-02-22T19:45:18.866573+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:45:18.866573+0000","last_undegraded":"2024-02-22T19:45:18.866573+0000","last_fullsized":"2024-02-22T19:45:18.866573+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.431664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.431664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"428'6071","reported_seq":6141,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.816438+0000","last_change":"2024-02-22T19:43:57.349932+0000","last_active":"2024-02-22T19:45:24.816438+0000","last_peered":"2024-02-22T19:45:24.816438+0000","last_clean":"2024-02-22T19:45:24.816438+0000","last_became_active":"2024-02-22T19:43:57.349636+0000","last_became_peered":"2024-02-22T19:43:57.349636+0000","last_unstale":"2024-02-22T19:45:24.816438+0000","last_undegraded":"2024-02-22T19:45:24.816438+0000","last_fullsized":"2024-02-22T19:45:24.816438+0000","mapping_epoch":365,"log_start":"387'2700","ondisk_log_start":"387'2700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3371,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":397869056,"num_objects":6071,"num_object_clones":0,"num_object_copies":18213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":388544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":392,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853780+0000","last_change":"2024-02-22T19:41:25.602463+0000","last_active":"2024-02-22T19:45:18.853780+0000","last_peered":"2024-02-22T19:45:18.853780+0000","last_clean":"2024-02-22T19:45:18.853780+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:45:18.853780+0000","last_undegraded":"2024-02-22T19:45:18.853780+0000","last_fullsized":"2024-02-22T19:45:18.853780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.602356+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":477,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.867044+0000","last_change":"2024-02-22T19:45:00.406556+0000","last_active":"2024-02-22T19:45:18.867044+0000","last_peered":"2024-02-22T19:45:18.867044+0000","last_clean":"2024-02-22T19:45:18.867044+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:45:18.867044+0000","last_undegraded":"2024-02-22T19:45:18.867044+0000","last_fullsized":"2024-02-22T19:45:18.867044+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.406451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.406451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":430,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854600+0000","last_change":"2024-02-22T19:41:52.109740+0000","last_active":"2024-02-22T19:45:18.854600+0000","last_peered":"2024-02-22T19:45:18.854600+0000","last_clean":"2024-02-22T19:45:18.854600+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:45:18.854600+0000","last_undegraded":"2024-02-22T19:45:18.854600+0000","last_fullsized":"2024-02-22T19:45:18.854600+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:26.608998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:26.608998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":426,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853929+0000","last_change":"2024-02-22T19:44:19.749503+0000","last_active":"2024-02-22T19:45:18.853929+0000","last_peered":"2024-02-22T19:45:18.853929+0000","last_clean":"2024-02-22T19:45:18.853929+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:45:18.853929+0000","last_undegraded":"2024-02-22T19:45:18.853929+0000","last_fullsized":"2024-02-22T19:45:18.853929+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:19.749430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.749430+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"428'6114","reported_seq":6185,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.612403+0000","last_change":"2024-02-22T19:43:57.347961+0000","last_active":"2024-02-22T19:45:24.612403+0000","last_peered":"2024-02-22T19:45:24.612403+0000","last_clean":"2024-02-22T19:45:24.612403+0000","last_became_active":"2024-02-22T19:43:57.347657+0000","last_became_peered":"2024-02-22T19:43:57.347657+0000","last_unstale":"2024-02-22T19:45:24.612403+0000","last_undegraded":"2024-02-22T19:45:24.612403+0000","last_fullsized":"2024-02-22T19:45:24.612403+0000","mapping_epoch":365,"log_start":"382'2300","ondisk_log_start":"382'2300","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3814,"ondisk_log_size":3814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":400687104,"num_objects":6114,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6114,"num_write_kb":391296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":219,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854092+0000","last_change":"2024-02-22T19:44:57.360296+0000","last_active":"2024-02-22T19:45:18.854092+0000","last_peered":"2024-02-22T19:45:18.854092+0000","last_clean":"2024-02-22T19:45:18.854092+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:45:18.854092+0000","last_undegraded":"2024-02-22T19:45:18.854092+0000","last_fullsized":"2024-02-22T19:45:18.854092+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.360215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.360215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":456,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866790+0000","last_change":"2024-02-22T19:44:17.122626+0000","last_active":"2024-02-22T19:45:18.866790+0000","last_peered":"2024-02-22T19:45:18.866790+0000","last_clean":"2024-02-22T19:45:18.866790+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:45:18.866790+0000","last_undegraded":"2024-02-22T19:45:18.866790+0000","last_fullsized":"2024-02-22T19:45:18.866790+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.122559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.122559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":234,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858380+0000","last_change":"2024-02-22T19:41:52.097366+0000","last_active":"2024-02-22T19:45:18.858380+0000","last_peered":"2024-02-22T19:45:18.858380+0000","last_clean":"2024-02-22T19:45:18.858380+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:45:18.858380+0000","last_undegraded":"2024-02-22T19:45:18.858380+0000","last_fullsized":"2024-02-22T19:45:18.858380+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.486550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":231,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.867251+0000","last_change":"2024-02-22T19:44:11.175908+0000","last_active":"2024-02-22T19:45:18.867251+0000","last_peered":"2024-02-22T19:45:18.867251+0000","last_clean":"2024-02-22T19:45:18.867251+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:45:18.867251+0000","last_undegraded":"2024-02-22T19:45:18.867251+0000","last_fullsized":"2024-02-22T19:45:18.867251+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:44:11.175784+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.125098+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.175784+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"428'6180","reported_seq":6251,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.402483+0000","last_change":"2024-02-22T19:43:57.351947+0000","last_active":"2024-02-22T19:45:24.402483+0000","last_peered":"2024-02-22T19:45:24.402483+0000","last_clean":"2024-02-22T19:45:24.402483+0000","last_became_active":"2024-02-22T19:43:57.351647+0000","last_became_peered":"2024-02-22T19:43:57.351647+0000","last_unstale":"2024-02-22T19:45:24.402483+0000","last_undegraded":"2024-02-22T19:45:24.402483+0000","last_fullsized":"2024-02-22T19:45:24.402483+0000","mapping_epoch":365,"log_start":"384'2600","ondisk_log_start":"384'2600","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3580,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":405012480,"num_objects":6180,"num_object_clones":0,"num_object_copies":18540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6180,"num_write_kb":395520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":261,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857860+0000","last_change":"2024-02-22T19:40:54.553601+0000","last_active":"2024-02-22T19:45:18.857860+0000","last_peered":"2024-02-22T19:45:18.857860+0000","last_clean":"2024-02-22T19:45:18.857860+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:45:18.857860+0000","last_undegraded":"2024-02-22T19:45:18.857860+0000","last_fullsized":"2024-02-22T19:45:18.857860+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:40:54.553511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:40:54.553511+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":409,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864726+0000","last_change":"2024-02-22T19:41:52.104097+0000","last_active":"2024-02-22T19:45:18.864726+0000","last_peered":"2024-02-22T19:45:18.864726+0000","last_clean":"2024-02-22T19:45:18.864726+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:45:18.864726+0000","last_undegraded":"2024-02-22T19:45:18.864726+0000","last_fullsized":"2024-02-22T19:45:18.864726+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:32.641411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:32.641411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":234,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858390+0000","last_change":"2024-02-22T19:43:57.722289+0000","last_active":"2024-02-22T19:45:18.858390+0000","last_peered":"2024-02-22T19:45:18.858390+0000","last_clean":"2024-02-22T19:45:18.858390+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:45:18.858390+0000","last_undegraded":"2024-02-22T19:45:18.858390+0000","last_fullsized":"2024-02-22T19:45:18.858390+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.722226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.722226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":349,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.856230+0000","last_change":"2024-02-22T19:44:16.734407+0000","last_active":"2024-02-22T19:45:18.856230+0000","last_peered":"2024-02-22T19:45:18.856230+0000","last_clean":"2024-02-22T19:45:18.856230+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:45:18.856230+0000","last_undegraded":"2024-02-22T19:45:18.856230+0000","last_fullsized":"2024-02-22T19:45:18.856230+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:16.734314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:16.734314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":673,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:19.383366+0000","last_change":"2024-02-22T19:41:29.209622+0000","last_active":"2024-02-22T19:45:19.383366+0000","last_peered":"2024-02-22T19:45:19.383366+0000","last_clean":"2024-02-22T19:45:19.383366+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:45:19.383366+0000","last_undegraded":"2024-02-22T19:45:19.383366+0000","last_fullsized":"2024-02-22T19:45:19.383366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:41:29.209507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:29.209507+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"419'20","reported_seq":289,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:19.379732+0000","last_change":"2024-02-22T19:45:05.413797+0000","last_active":"2024-02-22T19:45:19.379732+0000","last_peered":"2024-02-22T19:45:19.379732+0000","last_clean":"2024-02-22T19:45:19.379732+0000","last_became_active":"2024-02-22T19:45:05.413407+0000","last_became_peered":"2024-02-22T19:45:05.413407+0000","last_unstale":"2024-02-22T19:45:19.379732+0000","last_undegraded":"2024-02-22T19:45:19.379732+0000","last_fullsized":"2024-02-22T19:45:19.379732+0000","mapping_epoch":322,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.782332+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"428'6202","reported_seq":6273,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.411767+0000","last_change":"2024-02-22T19:43:57.347092+0000","last_active":"2024-02-22T19:45:24.411767+0000","last_peered":"2024-02-22T19:45:24.411767+0000","last_clean":"2024-02-22T19:45:24.411767+0000","last_became_active":"2024-02-22T19:43:57.346743+0000","last_became_peered":"2024-02-22T19:43:57.346743+0000","last_unstale":"2024-02-22T19:45:24.411767+0000","last_undegraded":"2024-02-22T19:45:24.411767+0000","last_fullsized":"2024-02-22T19:45:24.411767+0000","mapping_epoch":365,"log_start":"386'2700","ondisk_log_start":"386'2700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3502,"ondisk_log_size":3502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406454272,"num_objects":6202,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6202,"num_write_kb":396928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":456,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854768+0000","last_change":"2024-02-22T19:44:44.268687+0000","last_active":"2024-02-22T19:45:18.854768+0000","last_peered":"2024-02-22T19:45:18.854768+0000","last_clean":"2024-02-22T19:45:18.854768+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:45:18.854768+0000","last_undegraded":"2024-02-22T19:45:18.854768+0000","last_fullsized":"2024-02-22T19:45:18.854768+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:44.268593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:44.268593+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":406,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854723+0000","last_change":"2024-02-22T19:44:12.295154+0000","last_active":"2024-02-22T19:45:18.854723+0000","last_peered":"2024-02-22T19:45:18.854723+0000","last_clean":"2024-02-22T19:45:18.854723+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:45:18.854723+0000","last_undegraded":"2024-02-22T19:45:18.854723+0000","last_fullsized":"2024-02-22T19:45:18.854723+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.295048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.295048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":414,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854193+0000","last_change":"2024-02-22T19:41:52.097076+0000","last_active":"2024-02-22T19:45:18.854193+0000","last_peered":"2024-02-22T19:45:18.854193+0000","last_clean":"2024-02-22T19:45:18.854193+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:45:18.854193+0000","last_undegraded":"2024-02-22T19:45:18.854193+0000","last_fullsized":"2024-02-22T19:45:18.854193+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:22.216558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:22.216558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":223,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866311+0000","last_change":"2024-02-22T19:45:08.329467+0000","last_active":"2024-02-22T19:45:18.866311+0000","last_peered":"2024-02-22T19:45:18.866311+0000","last_clean":"2024-02-22T19:45:18.866311+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:45:18.866311+0000","last_undegraded":"2024-02-22T19:45:18.866311+0000","last_fullsized":"2024-02-22T19:45:18.866311+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:08.329372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:45:08.329372+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":507,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866772+0000","last_change":"2024-02-22T19:44:20.206193+0000","last_active":"2024-02-22T19:45:18.866772+0000","last_peered":"2024-02-22T19:45:18.866772+0000","last_clean":"2024-02-22T19:45:18.866772+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:45:18.866772+0000","last_undegraded":"2024-02-22T19:45:18.866772+0000","last_fullsized":"2024-02-22T19:45:18.866772+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:44:20.206076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:20.206076+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"415'30","reported_seq":474,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:22.403671+0000","last_change":"2024-02-22T19:45:04.389455+0000","last_active":"2024-02-22T19:45:22.403671+0000","last_peered":"2024-02-22T19:45:22.403671+0000","last_clean":"2024-02-22T19:45:22.403671+0000","last_became_active":"2024-02-22T19:45:01.732679+0000","last_became_peered":"2024-02-22T19:45:01.732679+0000","last_unstale":"2024-02-22T19:45:22.403671+0000","last_undegraded":"2024-02-22T19:45:22.403671+0000","last_fullsized":"2024-02-22T19:45:22.403671+0000","mapping_epoch":217,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":415,"parent":"0.0","parent_split_bits":0,"last_scrub":"415'30","last_scrub_stamp":"2024-02-22T19:45:04.389357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:45:04.389357+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"428'6329","reported_seq":6400,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.200003+0000","last_change":"2024-02-22T19:43:57.344606+0000","last_active":"2024-02-22T19:45:24.200003+0000","last_peered":"2024-02-22T19:45:24.200003+0000","last_clean":"2024-02-22T19:45:24.200003+0000","last_became_active":"2024-02-22T19:43:57.344303+0000","last_became_peered":"2024-02-22T19:43:57.344303+0000","last_unstale":"2024-02-22T19:45:24.200003+0000","last_undegraded":"2024-02-22T19:45:24.200003+0000","last_fullsized":"2024-02-22T19:45:24.200003+0000","mapping_epoch":365,"log_start":"383'2500","ondisk_log_start":"383'2500","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3829,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":414777344,"num_objects":6329,"num_object_clones":0,"num_object_copies":18987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6329,"num_write_kb":405056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":239,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.859127+0000","last_change":"2024-02-22T19:44:17.656581+0000","last_active":"2024-02-22T19:45:18.859127+0000","last_peered":"2024-02-22T19:45:18.859127+0000","last_clean":"2024-02-22T19:45:18.859127+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:45:18.859127+0000","last_undegraded":"2024-02-22T19:45:18.859127+0000","last_fullsized":"2024-02-22T19:45:18.859127+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.656518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":456,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853907+0000","last_change":"2024-02-22T19:44:49.368004+0000","last_active":"2024-02-22T19:45:18.853907+0000","last_peered":"2024-02-22T19:45:18.853907+0000","last_clean":"2024-02-22T19:45:18.853907+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:45:18.853907+0000","last_undegraded":"2024-02-22T19:45:18.853907+0000","last_fullsized":"2024-02-22T19:45:18.853907+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:49.367906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:49.367906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":424,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.865245+0000","last_change":"2024-02-22T19:44:09.027439+0000","last_active":"2024-02-22T19:45:18.865245+0000","last_peered":"2024-02-22T19:45:18.865245+0000","last_clean":"2024-02-22T19:45:18.865245+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:45:18.865245+0000","last_undegraded":"2024-02-22T19:45:18.865245+0000","last_fullsized":"2024-02-22T19:45:18.865245+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.027320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":459,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854502+0000","last_change":"2024-02-22T19:41:52.726772+0000","last_active":"2024-02-22T19:45:18.854502+0000","last_peered":"2024-02-22T19:45:18.854502+0000","last_clean":"2024-02-22T19:45:18.854502+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:45:18.854502+0000","last_undegraded":"2024-02-22T19:45:18.854502+0000","last_fullsized":"2024-02-22T19:45:18.854502+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:41:16.790226+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.790226+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"423'13","reported_seq":465,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:22.407413+0000","last_change":"2024-02-22T19:45:09.548942+0000","last_active":"2024-02-22T19:45:22.407413+0000","last_peered":"2024-02-22T19:45:22.407413+0000","last_clean":"2024-02-22T19:45:22.407413+0000","last_became_active":"2024-02-22T19:45:09.547565+0000","last_became_peered":"2024-02-22T19:45:09.547565+0000","last_unstale":"2024-02-22T19:45:22.407413+0000","last_undegraded":"2024-02-22T19:45:22.407413+0000","last_fullsized":"2024-02-22T19:45:22.407413+0000","mapping_epoch":217,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'11","last_scrub_stamp":"2024-02-22T19:44:52.364238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.364238+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"428'6118","reported_seq":6189,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.615788+0000","last_change":"2024-02-22T19:43:57.346858+0000","last_active":"2024-02-22T19:45:24.615788+0000","last_peered":"2024-02-22T19:45:24.615788+0000","last_clean":"2024-02-22T19:45:24.615788+0000","last_became_active":"2024-02-22T19:43:57.346554+0000","last_became_peered":"2024-02-22T19:43:57.346554+0000","last_unstale":"2024-02-22T19:45:24.615788+0000","last_undegraded":"2024-02-22T19:45:24.615788+0000","last_fullsized":"2024-02-22T19:45:24.615788+0000","mapping_epoch":365,"log_start":"382'2300","ondisk_log_start":"382'2300","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3818,"ondisk_log_size":3818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":400949248,"num_objects":6118,"num_object_clones":0,"num_object_copies":18354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6118,"num_write_kb":391552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":310,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.855380+0000","last_change":"2024-02-22T19:44:55.655563+0000","last_active":"2024-02-22T19:45:18.855380+0000","last_peered":"2024-02-22T19:45:18.855380+0000","last_clean":"2024-02-22T19:45:18.855380+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:45:18.855380+0000","last_undegraded":"2024-02-22T19:45:18.855380+0000","last_fullsized":"2024-02-22T19:45:18.855380+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:55.655474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:55.655474+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":417,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858024+0000","last_change":"2024-02-22T19:41:35.211931+0000","last_active":"2024-02-22T19:45:18.858024+0000","last_peered":"2024-02-22T19:45:18.858024+0000","last_clean":"2024-02-22T19:45:18.858024+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:45:18.858024+0000","last_undegraded":"2024-02-22T19:45:18.858024+0000","last_fullsized":"2024-02-22T19:45:18.858024+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:35.211863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:35.211863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":207,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.862603+0000","last_change":"2024-02-22T19:44:15.990043+0000","last_active":"2024-02-22T19:45:18.862603+0000","last_peered":"2024-02-22T19:45:18.862603+0000","last_clean":"2024-02-22T19:45:18.862603+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:45:18.862603+0000","last_undegraded":"2024-02-22T19:45:18.862603+0000","last_fullsized":"2024-02-22T19:45:18.862603+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.989937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.989937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1426,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:23.017208+0000","last_change":"2024-02-22T19:44:19.273347+0000","last_active":"2024-02-22T19:45:23.017208+0000","last_peered":"2024-02-22T19:45:23.017208+0000","last_clean":"2024-02-22T19:45:23.017208+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:45:23.017208+0000","last_undegraded":"2024-02-22T19:45:23.017208+0000","last_fullsized":"2024-02-22T19:45:23.017208+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:44:19.273191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.273191+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":966,"num_read_kb":961,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":239,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857754+0000","last_change":"2024-02-22T19:44:56.653960+0000","last_active":"2024-02-22T19:45:18.857754+0000","last_peered":"2024-02-22T19:45:18.857754+0000","last_clean":"2024-02-22T19:45:18.857754+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:45:18.857754+0000","last_undegraded":"2024-02-22T19:45:18.857754+0000","last_fullsized":"2024-02-22T19:45:18.857754+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:44:56.653848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.653848+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"412'25","reported_seq":309,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:22.400461+0000","last_change":"2024-02-22T19:45:06.780716+0000","last_active":"2024-02-22T19:45:22.400461+0000","last_peered":"2024-02-22T19:45:22.400461+0000","last_clean":"2024-02-22T19:45:22.400461+0000","last_became_active":"2024-02-22T19:44:57.599424+0000","last_became_peered":"2024-02-22T19:44:57.599424+0000","last_unstale":"2024-02-22T19:45:22.400461+0000","last_undegraded":"2024-02-22T19:45:22.400461+0000","last_fullsized":"2024-02-22T19:45:22.400461+0000","mapping_epoch":267,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":411,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'25","last_scrub_stamp":"2024-02-22T19:45:06.780605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:45:06.780605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"428'6079","reported_seq":6150,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.802053+0000","last_change":"2024-02-22T19:43:57.347743+0000","last_active":"2024-02-22T19:45:24.802053+0000","last_peered":"2024-02-22T19:45:24.802053+0000","last_clean":"2024-02-22T19:45:24.802053+0000","last_became_active":"2024-02-22T19:43:57.347455+0000","last_became_peered":"2024-02-22T19:43:57.347455+0000","last_unstale":"2024-02-22T19:45:24.802053+0000","last_undegraded":"2024-02-22T19:45:24.802053+0000","last_fullsized":"2024-02-22T19:45:24.802053+0000","mapping_epoch":365,"log_start":"387'2700","ondisk_log_start":"387'2700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3379,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":398393344,"num_objects":6079,"num_object_clones":0,"num_object_copies":18237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6079,"num_write_kb":389056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":281,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864430+0000","last_change":"2024-02-22T19:41:52.104000+0000","last_active":"2024-02-22T19:45:18.864430+0000","last_peered":"2024-02-22T19:45:18.864430+0000","last_clean":"2024-02-22T19:45:18.864430+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:45:18.864430+0000","last_undegraded":"2024-02-22T19:45:18.864430+0000","last_fullsized":"2024-02-22T19:45:18.864430+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:25.816369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:41:25.816369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":408,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854265+0000","last_change":"2024-02-22T19:44:53.631267+0000","last_active":"2024-02-22T19:45:18.854265+0000","last_peered":"2024-02-22T19:45:18.854265+0000","last_clean":"2024-02-22T19:45:18.854265+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:45:18.854265+0000","last_undegraded":"2024-02-22T19:45:18.854265+0000","last_fullsized":"2024-02-22T19:45:18.854265+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:53.631188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.631188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":423,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858289+0000","last_change":"2024-02-22T19:44:13.731072+0000","last_active":"2024-02-22T19:45:18.858289+0000","last_peered":"2024-02-22T19:45:18.858289+0000","last_clean":"2024-02-22T19:45:18.858289+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:45:18.858289+0000","last_undegraded":"2024-02-22T19:45:18.858289+0000","last_fullsized":"2024-02-22T19:45:18.858289+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.730975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.730975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":244,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858827+0000","last_change":"2024-02-22T19:44:15.640361+0000","last_active":"2024-02-22T19:45:18.858827+0000","last_peered":"2024-02-22T19:45:18.858827+0000","last_clean":"2024-02-22T19:45:18.858827+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:45:18.858827+0000","last_undegraded":"2024-02-22T19:45:18.858827+0000","last_fullsized":"2024-02-22T19:45:18.858827+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:44:15.640271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.640271+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":411,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858136+0000","last_change":"2024-02-22T19:44:54.804568+0000","last_active":"2024-02-22T19:45:18.858136+0000","last_peered":"2024-02-22T19:45:18.858136+0000","last_clean":"2024-02-22T19:45:18.858136+0000","last_became_active":"2024-02-22T19:44:45.398192+0000","last_became_peered":"2024-02-22T19:44:45.398192+0000","last_unstale":"2024-02-22T19:45:18.858136+0000","last_undegraded":"2024-02-22T19:45:18.858136+0000","last_fullsized":"2024-02-22T19:45:18.858136+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":399,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:54.804469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:54.804469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"428'6201","reported_seq":6272,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.424053+0000","last_change":"2024-02-22T19:43:57.351727+0000","last_active":"2024-02-22T19:45:24.424053+0000","last_peered":"2024-02-22T19:45:24.424053+0000","last_clean":"2024-02-22T19:45:24.424053+0000","last_became_active":"2024-02-22T19:43:57.351385+0000","last_became_peered":"2024-02-22T19:43:57.351385+0000","last_unstale":"2024-02-22T19:45:24.424053+0000","last_undegraded":"2024-02-22T19:45:24.424053+0000","last_fullsized":"2024-02-22T19:45:24.424053+0000","mapping_epoch":365,"log_start":"382'2400","ondisk_log_start":"382'2400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3801,"ondisk_log_size":3801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":406388736,"num_objects":6201,"num_object_clones":0,"num_object_copies":18603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":396864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":138,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864025+0000","last_change":"2024-02-22T19:44:46.735448+0000","last_active":"2024-02-22T19:45:18.864025+0000","last_peered":"2024-02-22T19:45:18.864025+0000","last_clean":"2024-02-22T19:45:18.864025+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:45:18.864025+0000","last_undegraded":"2024-02-22T19:45:18.864025+0000","last_fullsized":"2024-02-22T19:45:18.864025+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:46.735345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:44:46.735345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":219,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.863418+0000","last_change":"2024-02-22T19:44:49.777279+0000","last_active":"2024-02-22T19:45:18.863418+0000","last_peered":"2024-02-22T19:45:18.863418+0000","last_clean":"2024-02-22T19:45:18.863418+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:45:18.863418+0000","last_undegraded":"2024-02-22T19:45:18.863418+0000","last_fullsized":"2024-02-22T19:45:18.863418+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:49.777178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:44:49.777178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":225,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858854+0000","last_change":"2024-02-22T19:41:30.423493+0000","last_active":"2024-02-22T19:45:18.858854+0000","last_peered":"2024-02-22T19:45:18.858854+0000","last_clean":"2024-02-22T19:45:18.858854+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:45:18.858854+0000","last_undegraded":"2024-02-22T19:45:18.858854+0000","last_fullsized":"2024-02-22T19:45:18.858854+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.423423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":538,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853984+0000","last_change":"2024-02-22T19:41:36.170173+0000","last_active":"2024-02-22T19:45:18.853984+0000","last_peered":"2024-02-22T19:45:18.853984+0000","last_clean":"2024-02-22T19:45:18.853984+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:45:18.853984+0000","last_undegraded":"2024-02-22T19:45:18.853984+0000","last_fullsized":"2024-02-22T19:45:18.853984+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:41:36.170049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:41:36.170049+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"395'16","reported_seq":490,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:22.400142+0000","last_change":"2024-02-22T19:44:41.683660+0000","last_active":"2024-02-22T19:45:22.400142+0000","last_peered":"2024-02-22T19:45:22.400142+0000","last_clean":"2024-02-22T19:45:22.400142+0000","last_became_active":"2024-02-22T19:44:41.682349+0000","last_became_peered":"2024-02-22T19:44:41.682349+0000","last_unstale":"2024-02-22T19:45:22.400142+0000","last_undegraded":"2024-02-22T19:45:22.400142+0000","last_fullsized":"2024-02-22T19:45:22.400142+0000","mapping_epoch":322,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'13","last_scrub_stamp":"2024-02-22T19:41:24.850179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:41:24.850179+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"428'6127","reported_seq":6198,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.818230+0000","last_change":"2024-02-22T19:43:57.348552+0000","last_active":"2024-02-22T19:45:24.818230+0000","last_peered":"2024-02-22T19:45:24.818230+0000","last_clean":"2024-02-22T19:45:24.818230+0000","last_became_active":"2024-02-22T19:43:57.348228+0000","last_became_peered":"2024-02-22T19:43:57.348228+0000","last_unstale":"2024-02-22T19:45:24.818230+0000","last_undegraded":"2024-02-22T19:45:24.818230+0000","last_fullsized":"2024-02-22T19:45:24.818230+0000","mapping_epoch":365,"log_start":"386'2800","ondisk_log_start":"386'2800","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3327,"ondisk_log_size":3327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":401539072,"num_objects":6127,"num_object_clones":0,"num_object_copies":18381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6127,"num_write_kb":392128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":310,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.868292+0000","last_change":"2024-02-22T19:43:58.000275+0000","last_active":"2024-02-22T19:45:18.868292+0000","last_peered":"2024-02-22T19:45:18.868292+0000","last_clean":"2024-02-22T19:45:18.868292+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:45:18.868292+0000","last_undegraded":"2024-02-22T19:45:18.868292+0000","last_fullsized":"2024-02-22T19:45:18.868292+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.000163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.000163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":409,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.856822+0000","last_change":"2024-02-22T19:44:45.370939+0000","last_active":"2024-02-22T19:45:18.856822+0000","last_peered":"2024-02-22T19:45:18.856822+0000","last_clean":"2024-02-22T19:45:18.856822+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:45:18.856822+0000","last_undegraded":"2024-02-22T19:45:18.856822+0000","last_fullsized":"2024-02-22T19:45:18.856822+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:45.370834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.370834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":228,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857539+0000","last_change":"2024-02-22T19:44:47.633310+0000","last_active":"2024-02-22T19:45:18.857539+0000","last_peered":"2024-02-22T19:45:18.857539+0000","last_clean":"2024-02-22T19:45:18.857539+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:45:18.857539+0000","last_undegraded":"2024-02-22T19:45:18.857539+0000","last_fullsized":"2024-02-22T19:45:18.857539+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:47.633242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.633242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":545,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857993+0000","last_change":"2024-02-22T19:44:45.716073+0000","last_active":"2024-02-22T19:45:18.857993+0000","last_peered":"2024-02-22T19:45:18.857993+0000","last_clean":"2024-02-22T19:45:18.857993+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:45:18.857993+0000","last_undegraded":"2024-02-22T19:45:18.857993+0000","last_fullsized":"2024-02-22T19:45:18.857993+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:45.715955+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.715955+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"407'13","reported_seq":658,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:22.399762+0000","last_change":"2024-02-22T19:44:53.757698+0000","last_active":"2024-02-22T19:45:22.399762+0000","last_peered":"2024-02-22T19:45:22.399762+0000","last_clean":"2024-02-22T19:45:22.399762+0000","last_became_active":"2024-02-22T19:44:53.415371+0000","last_became_peered":"2024-02-22T19:44:53.415371+0000","last_unstale":"2024-02-22T19:45:22.399762+0000","last_undegraded":"2024-02-22T19:45:22.399762+0000","last_fullsized":"2024-02-22T19:45:22.399762+0000","mapping_epoch":267,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":407,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:44:53.757594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.757594+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"428'6236","reported_seq":6309,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.644297+0000","last_change":"2024-02-22T19:45:18.873179+0000","last_active":"2024-02-22T19:45:24.644297+0000","last_peered":"2024-02-22T19:45:24.644297+0000","last_clean":"2024-02-22T19:45:24.644297+0000","last_became_active":"2024-02-22T19:43:57.348032+0000","last_became_peered":"2024-02-22T19:43:57.348032+0000","last_unstale":"2024-02-22T19:45:24.644297+0000","last_undegraded":"2024-02-22T19:45:24.644297+0000","last_fullsized":"2024-02-22T19:45:24.644297+0000","mapping_epoch":365,"log_start":"382'2400","ondisk_log_start":"382'2400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3836,"ondisk_log_size":3836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":408682496,"num_objects":6236,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6236,"num_write_kb":399104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":217,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.873510+0000","last_change":"2024-02-22T19:45:14.353778+0000","last_active":"2024-02-22T19:45:18.873510+0000","last_peered":"2024-02-22T19:45:18.873510+0000","last_clean":"2024-02-22T19:45:18.873510+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:45:18.873510+0000","last_undegraded":"2024-02-22T19:45:18.873510+0000","last_fullsized":"2024-02-22T19:45:18.873510+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.353679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.353679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":206,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.869972+0000","last_change":"2024-02-22T19:41:52.103793+0000","last_active":"2024-02-22T19:45:18.869972+0000","last_peered":"2024-02-22T19:45:18.869972+0000","last_clean":"2024-02-22T19:45:18.869972+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:45:18.869972+0000","last_undegraded":"2024-02-22T19:45:18.869972+0000","last_fullsized":"2024-02-22T19:45:18.869972+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:16.678317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:16.678317+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":424,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864216+0000","last_change":"2024-02-22T19:41:21.735045+0000","last_active":"2024-02-22T19:45:18.864216+0000","last_peered":"2024-02-22T19:45:18.864216+0000","last_clean":"2024-02-22T19:45:18.864216+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:45:18.864216+0000","last_undegraded":"2024-02-22T19:45:18.864216+0000","last_fullsized":"2024-02-22T19:45:18.864216+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:21.734935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:41:21.734935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":533,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.863864+0000","last_change":"2024-02-22T19:45:09.692752+0000","last_active":"2024-02-22T19:45:18.863864+0000","last_peered":"2024-02-22T19:45:18.863864+0000","last_clean":"2024-02-22T19:45:18.863864+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:45:18.863864+0000","last_undegraded":"2024-02-22T19:45:18.863864+0000","last_fullsized":"2024-02-22T19:45:18.863864+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:45:09.692661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.692661+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":238,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.862756+0000","last_change":"2024-02-22T19:44:51.773621+0000","last_active":"2024-02-22T19:45:18.862756+0000","last_peered":"2024-02-22T19:45:18.862756+0000","last_clean":"2024-02-22T19:45:18.862756+0000","last_became_active":"2024-02-22T19:44:49.491617+0000","last_became_peered":"2024-02-22T19:44:49.491617+0000","last_unstale":"2024-02-22T19:45:18.862756+0000","last_undegraded":"2024-02-22T19:45:18.862756+0000","last_fullsized":"2024-02-22T19:45:18.862756+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":403,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:51.773527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:51.773527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"428'6101","reported_seq":6171,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.402806+0000","last_change":"2024-02-22T19:43:57.347525+0000","last_active":"2024-02-22T19:45:24.402806+0000","last_peered":"2024-02-22T19:45:24.402806+0000","last_clean":"2024-02-22T19:45:24.402806+0000","last_became_active":"2024-02-22T19:43:57.347224+0000","last_became_peered":"2024-02-22T19:43:57.347224+0000","last_unstale":"2024-02-22T19:45:24.402806+0000","last_undegraded":"2024-02-22T19:45:24.402806+0000","last_fullsized":"2024-02-22T19:45:24.402806+0000","mapping_epoch":365,"log_start":"381'2300","ondisk_log_start":"381'2300","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3801,"ondisk_log_size":3801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":399835136,"num_objects":6101,"num_object_clones":0,"num_object_copies":18303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6101,"num_write_kb":390464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":242,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858644+0000","last_change":"2024-02-22T19:45:00.729978+0000","last_active":"2024-02-22T19:45:18.858644+0000","last_peered":"2024-02-22T19:45:18.858644+0000","last_clean":"2024-02-22T19:45:18.858644+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:45:18.858644+0000","last_undegraded":"2024-02-22T19:45:18.858644+0000","last_fullsized":"2024-02-22T19:45:18.858644+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.729907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.729907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":252,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858680+0000","last_change":"2024-02-22T19:41:18.527770+0000","last_active":"2024-02-22T19:45:18.858680+0000","last_peered":"2024-02-22T19:45:18.858680+0000","last_clean":"2024-02-22T19:45:18.858680+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:45:18.858680+0000","last_undegraded":"2024-02-22T19:45:18.858680+0000","last_fullsized":"2024-02-22T19:45:18.858680+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:41:18.527703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:41:18.527703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":402,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.863750+0000","last_change":"2024-02-22T19:44:12.048204+0000","last_active":"2024-02-22T19:45:18.863750+0000","last_peered":"2024-02-22T19:45:18.863750+0000","last_clean":"2024-02-22T19:45:18.863750+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:45:18.863750+0000","last_undegraded":"2024-02-22T19:45:18.863750+0000","last_fullsized":"2024-02-22T19:45:18.863750+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.048068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.048068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":141,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.858959+0000","last_change":"2024-02-22T19:41:26.853471+0000","last_active":"2024-02-22T19:45:18.858959+0000","last_peered":"2024-02-22T19:45:18.858959+0000","last_clean":"2024-02-22T19:45:18.858959+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:45:18.858959+0000","last_undegraded":"2024-02-22T19:45:18.858959+0000","last_fullsized":"2024-02-22T19:45:18.858959+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:40:53.383914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:40:53.383914+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"428'6087","reported_seq":6158,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.144366+0000","last_change":"2024-02-22T19:43:57.346250+0000","last_active":"2024-02-22T19:45:24.144366+0000","last_peered":"2024-02-22T19:45:24.144366+0000","last_clean":"2024-02-22T19:45:24.144366+0000","last_became_active":"2024-02-22T19:43:57.345992+0000","last_became_peered":"2024-02-22T19:43:57.345992+0000","last_unstale":"2024-02-22T19:45:24.144366+0000","last_undegraded":"2024-02-22T19:45:24.144366+0000","last_fullsized":"2024-02-22T19:45:24.144366+0000","mapping_epoch":365,"log_start":"381'2200","ondisk_log_start":"381'2200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3887,"ondisk_log_size":3887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":398917632,"num_objects":6087,"num_object_clones":0,"num_object_copies":18261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6087,"num_write_kb":389568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":418,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.868428+0000","last_change":"2024-02-22T19:44:42.822533+0000","last_active":"2024-02-22T19:45:18.868428+0000","last_peered":"2024-02-22T19:45:18.868428+0000","last_clean":"2024-02-22T19:45:18.868428+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:45:18.868428+0000","last_undegraded":"2024-02-22T19:45:18.868428+0000","last_fullsized":"2024-02-22T19:45:18.868428+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:42.822463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:42.822463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":207,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866503+0000","last_change":"2024-02-22T19:44:06.172481+0000","last_active":"2024-02-22T19:45:18.866503+0000","last_peered":"2024-02-22T19:45:18.866503+0000","last_clean":"2024-02-22T19:45:18.866503+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:45:18.866503+0000","last_undegraded":"2024-02-22T19:45:18.866503+0000","last_fullsized":"2024-02-22T19:45:18.866503+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:06.172363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:06.172363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":215,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857670+0000","last_change":"2024-02-22T19:44:11.716360+0000","last_active":"2024-02-22T19:45:18.857670+0000","last_peered":"2024-02-22T19:45:18.857670+0000","last_clean":"2024-02-22T19:45:18.857670+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:45:18.857670+0000","last_undegraded":"2024-02-22T19:45:18.857670+0000","last_fullsized":"2024-02-22T19:45:18.857670+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:44:11.716223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.716223+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3","version":"428'6222","reported_seq":6298,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.635573+0000","last_change":"2024-02-22T19:45:14.048917+0000","last_active":"2024-02-22T19:45:24.635573+0000","last_peered":"2024-02-22T19:45:24.635573+0000","last_clean":"2024-02-22T19:45:24.635573+0000","last_became_active":"2024-02-22T19:43:57.347832+0000","last_became_peered":"2024-02-22T19:43:57.347832+0000","last_unstale":"2024-02-22T19:45:24.635573+0000","last_undegraded":"2024-02-22T19:45:24.635573+0000","last_fullsized":"2024-02-22T19:45:24.635573+0000","mapping_epoch":365,"log_start":"382'2400","ondisk_log_start":"382'2400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"424'5534","last_scrub_stamp":"2024-02-22T19:45:14.048806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.048806+0000","log_size":3822,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":407764992,"num_objects":6222,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":398208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":230,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857991+0000","last_change":"2024-02-22T19:44:48.769697+0000","last_active":"2024-02-22T19:45:18.857991+0000","last_peered":"2024-02-22T19:45:18.857991+0000","last_clean":"2024-02-22T19:45:18.857991+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:45:18.857991+0000","last_undegraded":"2024-02-22T19:45:18.857991+0000","last_fullsized":"2024-02-22T19:45:18.857991+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.769602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.769602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":422,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.866599+0000","last_change":"2024-02-22T19:44:05.228957+0000","last_active":"2024-02-22T19:45:18.866599+0000","last_peered":"2024-02-22T19:45:18.866599+0000","last_clean":"2024-02-22T19:45:18.866599+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:45:18.866599+0000","last_undegraded":"2024-02-22T19:45:18.866599+0000","last_fullsized":"2024-02-22T19:45:18.866599+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:05.228870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:05.228870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":339,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.853807+0000","last_change":"2024-02-22T19:44:12.719560+0000","last_active":"2024-02-22T19:45:18.853807+0000","last_peered":"2024-02-22T19:45:18.853807+0000","last_clean":"2024-02-22T19:45:18.853807+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:45:18.853807+0000","last_undegraded":"2024-02-22T19:45:18.853807+0000","last_fullsized":"2024-02-22T19:45:18.853807+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.719494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.719494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":308,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.873575+0000","last_change":"2024-02-22T19:44:09.132549+0000","last_active":"2024-02-22T19:45:18.873575+0000","last_peered":"2024-02-22T19:45:18.873575+0000","last_clean":"2024-02-22T19:45:18.873575+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:45:18.873575+0000","last_undegraded":"2024-02-22T19:45:18.873575+0000","last_fullsized":"2024-02-22T19:45:18.873575+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:44:09.132441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.132441+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"428'6180","reported_seq":6251,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-22T19:45:24.139485+0000","last_change":"2024-02-22T19:43:57.351793+0000","last_active":"2024-02-22T19:45:24.139485+0000","last_peered":"2024-02-22T19:45:24.139485+0000","last_clean":"2024-02-22T19:45:24.139485+0000","last_became_active":"2024-02-22T19:43:57.351420+0000","last_became_peered":"2024-02-22T19:43:57.351420+0000","last_unstale":"2024-02-22T19:45:24.139485+0000","last_undegraded":"2024-02-22T19:45:24.139485+0000","last_fullsized":"2024-02-22T19:45:24.139485+0000","mapping_epoch":365,"log_start":"382'2300","ondisk_log_start":"382'2300","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3880,"ondisk_log_size":3880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":405012480,"num_objects":6180,"num_object_clones":0,"num_object_copies":18540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6180,"num_write_kb":395520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":420,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.854124+0000","last_change":"2024-02-22T19:44:59.333051+0000","last_active":"2024-02-22T19:45:18.854124+0000","last_peered":"2024-02-22T19:45:18.854124+0000","last_clean":"2024-02-22T19:45:18.854124+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:45:18.854124+0000","last_undegraded":"2024-02-22T19:45:18.854124+0000","last_fullsized":"2024-02-22T19:45:18.854124+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:59.332960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:59.332960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":248,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.857444+0000","last_change":"2024-02-22T19:45:15.816947+0000","last_active":"2024-02-22T19:45:18.857444+0000","last_peered":"2024-02-22T19:45:18.857444+0000","last_clean":"2024-02-22T19:45:18.857444+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:45:18.857444+0000","last_undegraded":"2024-02-22T19:45:18.857444+0000","last_fullsized":"2024-02-22T19:45:18.857444+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:45:15.816835+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:45:15.816835+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":458,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.864526+0000","last_change":"2024-02-22T19:41:30.613764+0000","last_active":"2024-02-22T19:45:18.864526+0000","last_peered":"2024-02-22T19:45:18.864526+0000","last_clean":"2024-02-22T19:45:18.864526+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:45:18.864526+0000","last_undegraded":"2024-02-22T19:45:18.864526+0000","last_fullsized":"2024-02-22T19:45:18.864526+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:41:30.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:41:30.613654+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":277,"reported_epoch":427,"state":"active+clean","last_fresh":"2024-02-22T19:45:18.868649+0000","last_change":"2024-02-22T19:44:40.870790+0000","last_active":"2024-02-22T19:45:18.868649+0000","last_peered":"2024-02-22T19:45:18.868649+0000","last_clean":"2024-02-22T19:45:18.868649+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:45:18.868649+0000","last_undegraded":"2024-02-22T19:45:18.868649+0000","last_fullsized":"2024-02-22T19:45:18.868649+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:44:40.870681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:40.870681+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":6469189632,"num_objects":98712,"num_object_clones":0,"num_object_copies":296136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98712,"num_write_kb":6317568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19405406208,"data_stored":19405406208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58512,"ondisk_log_size":58512,"up":48,"acting":48,"num_store_stats":5},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":966,"num_read_kb":961,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266888,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4136696,"kb_used_data":3546992,"kb_used_omap":0,"kb_used_meta":589696,"kb_avail":89608456,"statfs":{"total":95995035648,"available":91759058944,"internally_reserved":0,"allocated":3632119808,"data_stored":3630578676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":603848704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.4690000000000001,"15min":1.5569999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.19900000000000001},"max":{"1min":2.8660000000000001,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.302,"5min":1.0880000000000001,"15min":1.4950000000000001},"min":{"1min":0.25900000000000001,"5min":0.25600000000000001,"15min":0.20699999999999999},"max":{"1min":5.1349999999999998,"5min":7.827,"15min":42.155999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.9670000000000001,"15min":1.8089999999999999},"min":{"1min":0.224,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":4.085,"5min":36.509,"15min":54.951000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":2.1269999999999998,"5min":1.5169999999999999,"15min":2.0449999999999999},"min":{"1min":0.25800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":7.1449999999999996,"5min":7.1449999999999996,"15min":57.012},"last":0.312}]},{"osd":2,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.9059999999999999,"5min":1.4470000000000001,"15min":1.5549999999999999},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":9.6310000000000002,"5min":9.6310000000000002,"15min":41.640999999999998},"last":4.3949999999999996},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.744,"15min":1.8180000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.5300000000000002,"5min":8.4879999999999995,"15min":41.722000000000001},"last":1.224}]},{"osd":3,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":2.133,"15min":1.972},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":7.5199999999999996,"5min":36.491999999999997,"15min":41.457000000000001},"last":1.319},{"interface":"front","average":{"1min":1.4770000000000001,"5min":2.2919999999999998,"15min":1.996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.5149999999999997,"5min":36.732999999999997,"15min":41.311999999999998},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.875,"15min":0.73099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23999999999999999},"max":{"1min":1.6859999999999999,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":1.075,"15min":0.76200000000000001},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":1.123,"5min":5.1790000000000003,"15min":7.6219999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":1.2929999999999999,"15min":0.85999999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27000000000000002},"max":{"1min":1.1579999999999999,"5min":6.9039999999999999,"15min":6.9039999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.022,"15min":0.81000000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":4.2110000000000003,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":1.607}]},{"osd":6,"last update":"Thu Feb 22 19:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":1.718,"5min":6.5,"15min":12.199999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.44,"5min":1.373,"15min":0.93300000000000005},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.57199999999999995,"5min":7.1230000000000002,"15min":12.067},"last":0.41999999999999998}]}]},{"osd":6,"up_from":0,"seq":193273528525,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4039684,"kb_used_data":3135996,"kb_used_omap":0,"kb_used_meta":903680,"kb_avail":89705468,"statfs":{"total":95995035648,"available":91858399232,"internally_reserved":0,"allocated":3211259904,"data_stored":3209706460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":925368320},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:45:05 2024","interfaces":[{"interface":"back","average":{"1min":1.77,"5min":2.3119999999999998,"15min":2.0150000000000001},"min":{"1min":0.20599999999999999,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":7.0679999999999996,"5min":51.091000000000001,"15min":51.091000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.623,"5min":2.2730000000000001,"15min":2.0760000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.17499999999999999},"max":{"1min":7.8339999999999996,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:45:22 2024","interfaces":[{"interface":"back","average":{"1min":3.8940000000000001,"5min":2.7629999999999999,"15min":2.2549999999999999},"min":{"1min":0.217,"5min":0.191,"15min":0.191},"max":{"1min":12.154,"5min":51.027000000000001,"15min":51.027000000000001},"last":8.141},{"interface":"front","average":{"1min":3.1179999999999999,"5min":2.7719999999999998,"15min":2.3460000000000001},"min":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":12.961,"5min":51.369999999999997,"15min":51.369999999999997},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.929,"5min":2.3399999999999999,"15min":2.2450000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.214},"max":{"1min":12.4,"5min":51.755000000000003,"15min":51.755000000000003},"last":3.7389999999999999},{"interface":"front","average":{"1min":2.4529999999999998,"5min":2.472,"15min":2.3359999999999999},"min":{"1min":0.25600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":11.878,"5min":51.741,"15min":51.741},"last":8.173}]},{"osd":4,"last update":"Thu Feb 22 19:45:14 2024","interfaces":[{"interface":"back","average":{"1min":2.0230000000000001,"5min":2.4940000000000002,"15min":1.827},"min":{"1min":0.39700000000000002,"5min":0.307,"15min":0.24399999999999999},"max":{"1min":11.706,"5min":51.158999999999999,"15min":51.158999999999999},"last":8.0800000000000001},{"interface":"front","average":{"1min":1.8,"5min":1.863,"15min":1.4730000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":11.763999999999999,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.34200000000000003}]},{"osd":5,"last update":"Thu Feb 22 19:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":2.399,"15min":1.8340000000000001},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":4.9989999999999997,"5min":51.183999999999997,"15min":51.183999999999997},"last":8.0310000000000006},{"interface":"front","average":{"1min":1.119,"5min":2.1699999999999999,"15min":1.6459999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.3739999999999997,"5min":51.212000000000003,"15min":51.212000000000003},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:45:01 2024","interfaces":[{"interface":"back","average":{"1min":1.8959999999999999,"5min":2.016,"15min":1.5669999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.28599999999999998},"max":{"1min":11.798999999999999,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":2.2949999999999999,"5min":2.6669999999999998,"15min":1.8939999999999999},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.34999999999999998},"max":{"1min":11.65,"5min":51.119999999999997,"15min":51.119999999999997},"last":8.1110000000000007}]},{"osd":7,"last update":"Thu Feb 22 19:45:22 2024","interfaces":[{"interface":"back","average":{"1min":1.8240000000000001,"5min":2.577,"15min":1.909},"min":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":11.826000000000001,"5min":51.244999999999997,"15min":51.244999999999997},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.8460000000000001,"5min":2.0680000000000001,"15min":1.601},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":11.603,"5min":51.317999999999998,"15min":51.317999999999998},"last":0.379}]}]},{"osd":0,"up_from":0,"seq":34359738603,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411552,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5714352,"kb_used_data":5128744,"kb_used_omap":0,"kb_used_meta":585600,"kb_avail":88030800,"statfs":{"total":95995035648,"available":90143539200,"internally_reserved":0,"allocated":5251833856,"data_stored":5250299183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":599654400},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:45:20 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.216,"15min":1.831},"min":{"1min":0.245,"5min":0.215,"15min":0.18099999999999999},"max":{"1min":5.7350000000000003,"5min":6.46,"15min":22.338999999999999},"last":5.0279999999999996},{"interface":"front","average":{"1min":1.903,"5min":1.7629999999999999,"15min":2.266},"min":{"1min":0.216,"5min":0.216,"15min":0.17599999999999999},"max":{"1min":5.2809999999999997,"5min":27.361999999999998,"15min":30.427},"last":0.311}]},{"osd":1,"last update":"Thu Feb 22 19:45:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1269999999999998,"5min":1.6000000000000001,"15min":1.851},"min":{"1min":0.26200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":8.5670000000000002,"5min":12.101000000000001,"15min":12.349},"last":5.2110000000000003},{"interface":"front","average":{"1min":2.1419999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.193},"max":{"1min":10.212999999999999,"5min":12.18,"15min":13.782},"last":4.992}]},{"osd":3,"last update":"Thu Feb 22 19:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":1.8009999999999999,"15min":2.1669999999999998},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":5.8239999999999998,"5min":12.151999999999999,"15min":30.071000000000002},"last":5.7009999999999996},{"interface":"front","average":{"1min":1.6539999999999999,"5min":2.0659999999999998,"15min":2.4060000000000001},"min":{"1min":0.31,"5min":0.26800000000000002,"15min":0.184},"max":{"1min":4.423,"5min":28.263999999999999,"15min":32.119},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:45:09 2024","interfaces":[{"interface":"back","average":{"1min":1.7509999999999999,"5min":1.389,"15min":1.47},"min":{"1min":0.34200000000000003,"5min":0.29799999999999999,"15min":0.26400000000000001},"max":{"1min":8.2289999999999992,"5min":8.2289999999999992,"15min":12.43},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.3240000000000001,"15min":1.345},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.253},"max":{"1min":5.7999999999999998,"5min":5.7999999999999998,"15min":10.34},"last":5.117}]},{"osd":5,"last update":"Thu Feb 22 19:44:29 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.8879999999999999,"15min":1.5149999999999999},"min":{"1min":0.47099999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.431,"5min":40.633000000000003,"15min":40.633000000000003},"last":5.0679999999999996},{"interface":"front","average":{"1min":1.258,"5min":2.0950000000000002,"15min":1.73},"min":{"1min":0.373,"5min":0.30599999999999999,"15min":0.26600000000000001},"max":{"1min":5.4329999999999998,"5min":40.997999999999998,"15min":40.997999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:45:09 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.5429999999999999,"15min":1.607},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.311},"max":{"1min":8.2530000000000001,"5min":8.2530000000000001,"15min":21.640000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.552,"15min":1.4099999999999999},"min":{"1min":0.371,"5min":0.34100000000000003,"15min":0.29899999999999999},"max":{"1min":5.8869999999999996,"5min":7.968,"15min":10.369999999999999},"last":5.1449999999999996}]},{"osd":7,"last update":"Thu Feb 22 19:45:20 2024","interfaces":[{"interface":"back","average":{"1min":2.4369999999999998,"5min":1.7609999999999999,"15min":1.6839999999999999},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":14.199,"5min":14.199,"15min":14.199},"last":0.54800000000000004},{"interface":"front","average":{"1min":2.2549999999999999,"5min":1.7529999999999999,"15min":1.585},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":18.021999999999998,"5min":18.021999999999998,"15min":18.021999999999998},"last":5.0919999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149914,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4845060,"kb_used_data":3949308,"kb_used_omap":0,"kb_used_meta":895744,"kb_avail":88900092,"statfs":{"total":95995035648,"available":91033694208,"internally_reserved":0,"allocated":4044091392,"data_stored":4042537841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":917241856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.5169999999999999,"15min":1.9350000000000001},"min":{"1min":0.20999999999999999,"5min":0.191,"15min":0.191},"max":{"1min":4.3899999999999997,"5min":9.9789999999999992,"15min":23.48},"last":1.327},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.9630000000000001,"15min":2.3029999999999999},"min":{"1min":0.25,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":3.9039999999999999,"5min":18.172000000000001,"15min":23.411999999999999},"last":1.873}]},{"osd":1,"last update":"Thu Feb 22 19:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":2.1059999999999999,"15min":2.5950000000000002},"min":{"1min":0.28299999999999997,"5min":0.222,"15min":0.20999999999999999},"max":{"1min":4.4630000000000001,"5min":18.213999999999999,"15min":23.077999999999999},"last":8.7260000000000009},{"interface":"front","average":{"1min":1.956,"5min":1.8,"15min":2.1459999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":6.0990000000000002,"5min":12.234,"15min":22.187000000000001},"last":3.9340000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:44:55 2024","interfaces":[{"interface":"back","average":{"1min":2.0169999999999999,"5min":1.895,"15min":2.2109999999999999},"min":{"1min":0.35899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.6260000000000003,"5min":20.105,"15min":20.105},"last":3.851},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.702,"15min":2.1339999999999999},"min":{"1min":0.28599999999999998,"5min":0.23100000000000001,"15min":0.217},"max":{"1min":5.5800000000000001,"5min":10.066000000000001,"15min":35.939999999999998},"last":1.7509999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:45:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.5589999999999999,"15min":1.823},"min":{"1min":0.432,"5min":0.27100000000000002,"15min":0.249},"max":{"1min":3.2909999999999999,"5min":9.968,"15min":13.680999999999999},"last":2.1230000000000002},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.389,"15min":1.6830000000000001},"min":{"1min":0.38300000000000001,"5min":0.32300000000000001,"15min":0.27400000000000002},"max":{"1min":4.7199999999999998,"5min":7.6479999999999997,"15min":13.548},"last":3.8849999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.627,"15min":1.601},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.0840000000000001,"5min":11.161,"15min":12.879},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.7729999999999999,"15min":1.8280000000000001},"min":{"1min":0.34499999999999997,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":6.1040000000000001,"5min":12.372999999999999,"15min":13.707000000000001},"last":2.0910000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.6659999999999999,"15min":1.5940000000000001},"min":{"1min":0.34499999999999997,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":2.9580000000000002,"5min":11.263,"15min":12.898999999999999},"last":1.661},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.7410000000000001,"15min":1.8959999999999999},"min":{"1min":0.37,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.331,"5min":10.02,"15min":13.803000000000001},"last":2.1509999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.569,"15min":1.74},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":5.3369999999999997,"5min":8.0879999999999992,"15min":13.483000000000001},"last":3.911},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.9350000000000001,"15min":1.7310000000000001},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":9.3930000000000007,"5min":11.539,"15min":12.846},"last":1.7230000000000001}]}]},{"osd":4,"up_from":0,"seq":137438953688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724754,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3737032,"kb_used_data":3143232,"kb_used_omap":0,"kb_used_meta":593792,"kb_avail":90008120,"statfs":{"total":95995035648,"available":92168314880,"internally_reserved":0,"allocated":3218669568,"data_stored":3217111353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":608043008},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:44:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.226,"15min":1.3799999999999999},"min":{"1min":0.254,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":5.7880000000000003,"5min":8.4299999999999997,"15min":17.684000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.516,"15min":1.5409999999999999},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.23999999999999999},"max":{"1min":5.1159999999999997,"5min":11.9,"15min":17.402999999999999},"last":0.67200000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":1.151,"15min":1.3999999999999999},"min":{"1min":0.25,"5min":0.19,"15min":0.19},"max":{"1min":3.1909999999999998,"5min":6.0750000000000002,"15min":29.390999999999998},"last":2.6779999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.6559999999999999,"15min":1.548},"min":{"1min":0.434,"5min":0.248,"15min":0.218},"max":{"1min":2.5430000000000001,"5min":6.5129999999999999,"15min":29.367000000000001},"last":3.3220000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:44:45 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.6259999999999999,"15min":1.579},"min":{"1min":0.28599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.548,"5min":9.7550000000000008,"15min":14.613},"last":3.004},{"interface":"front","average":{"1min":1.224,"5min":1.4370000000000001,"15min":1.579},"min":{"1min":0.377,"5min":0.251,"15min":0.251},"max":{"1min":4.1580000000000004,"5min":6.3979999999999997,"15min":16.099},"last":2.5870000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.8620000000000001,"15min":1.8029999999999999},"min":{"1min":0.376,"5min":0.254,"15min":0.254},"max":{"1min":2.734,"5min":16.370999999999999,"15min":16.370999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.5209999999999999,"15min":1.6599999999999999},"min":{"1min":0.34799999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.5529999999999999,"5min":7.2729999999999997,"15min":10.16},"last":1.1180000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":1.1399999999999999,"15min":0.75600000000000001},"min":{"1min":0.32100000000000001,"5min":0.28199999999999997,"15min":0.223},"max":{"1min":2.5840000000000001,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":1.419,"15min":0.84899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.25800000000000001},"max":{"1min":1.107,"5min":10.407,"15min":10.407},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.89400000000000002,"15min":0.67900000000000005},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.247},"max":{"1min":2.2730000000000001,"5min":5.1150000000000002,"15min":5.6040000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.3049999999999999,"15min":0.86899999999999999},"min":{"1min":0.34399999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.2549999999999999,"5min":10.436,"15min":10.436},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":1.1839999999999999,"15min":0.79000000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28299999999999997},"max":{"1min":2.7730000000000001,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.379},{"interface":"front","average":{"1min":0.873,"5min":1.3999999999999999,"15min":0.90200000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27100000000000002},"max":{"1min":5.6120000000000001,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.37}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3223257088,"data_stored":3223257088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5261492224,"data_stored":5261492224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4049010688,"data_stored":4049010688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3241345024,"data_stored":3241345024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3630301184,"data_stored":3630301184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:45:29.577 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:29.578 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:29.678 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:29.713 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:28.418472+0000 mgr.y (mgr.14535) 631 : audit [DBG] from='client.25462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:45:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:29.738 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:29.741 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:29.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:29.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:29.843 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 81 16 104241 104225 80.3133 68.8125 0.0103023 0.0124465 2024-02-22T19:45:29.906 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:30.034 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:30.037 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:30.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:30.045 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:30.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:30.177 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:30.200 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:30.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:30.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:30.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:30.357 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:30.432 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:29.721672+0000 mgr.y (mgr.14535) 632 : cluster [DBG] pgmap v849: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-02-22T19:45:30.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:45:30.626 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:30.654 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:30.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:30.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:29.721672+0000 mgr.y (mgr.14535) 632 : cluster [DBG] pgmap v849: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-02-22T19:45:30.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:30.784 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:29.721672+0000 mgr.y (mgr.14535) 632 : cluster [DBG] pgmap v849: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-02-22T19:45:30.787 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:30.811 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:30.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:30.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:30.844 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 82 16 105288 105272 80.1308 65.4375 0.0182453 0.0124754 2024-02-22T19:45:30.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:30.987 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:31.523 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:31.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:31.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:31.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:31.661 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:31.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:31.694 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:31.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:31.827 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:31.844 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 83 16 106367 106351 79.978 67.4375 0.0151464 0.0124994 2024-02-22T19:45:31.850 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:31.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:31.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:31.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:32.014 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:32.056 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:32.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:32.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:32.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:32.845 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 84 16 107416 107400 79.8065 65.5625 0.0167026 0.0125266 2024-02-22T19:45:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:31.722202+0000 mgr.y (mgr.14535) 633 : cluster [DBG] pgmap v850: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-02-22T19:45:33.157 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:31.722202+0000 mgr.y (mgr.14535) 633 : cluster [DBG] pgmap v850: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-02-22T19:45:33.159 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:31.722202+0000 mgr.y (mgr.14535) 633 : cluster [DBG] pgmap v850: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-02-22T19:45:33.217 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:33.226 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:33.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:33.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:33.380 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:33.646 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:33.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:33.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:33.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:33.858 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 85 16 108502 108486 79.6662 67.875 0.0258907 0.0125476 2024-02-22T19:45:33.905 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:33.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:33.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:34.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:34.076 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:34.096 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:45:34.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:45:34.096 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:45:34.096 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:45:34.096 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:45:34.577 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:45:34.578 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:45:34.578 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-02-22T19:45:34.578 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-02-22T19:45:34.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 1 1 2024-02-22T19:45:34.853 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 86 16 109401 109385 79.3935 56.1875 0.0521482 0.01259 2024-02-22T19:45:34.915 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:45:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:33.637751+0000 osd.6 (osd.6) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:33.637802+0000 osd.6 (osd.6) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:33.722617+0000 mgr.y (mgr.14535) 634 : cluster [DBG] pgmap v851: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 903 B/s rd, 59 MiB/s wr, 945 op/s 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:33.800122+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e429: 8 total, 8 up, 5 in 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:34.297110+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.53:0/3100648840' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:33.637751+0000 osd.6 (osd.6) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:33.637802+0000 osd.6 (osd.6) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:33.722617+0000 mgr.y (mgr.14535) 634 : cluster [DBG] pgmap v851: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 903 B/s rd, 59 MiB/s wr, 945 op/s 2024-02-22T19:45:34.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:33.800122+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e429: 8 total, 8 up, 5 in 2024-02-22T19:45:34.960 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:34.297110+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.53:0/3100648840' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:45:34.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:34.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:35.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:35.091 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:33.637751+0000 osd.6 (osd.6) 158 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:35.091 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:33.637802+0000 osd.6 (osd.6) 159 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:35.091 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:33.722617+0000 mgr.y (mgr.14535) 634 : cluster [DBG] pgmap v851: 138 pgs: 138 active+clean; 6.3 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 903 B/s rd, 59 MiB/s wr, 945 op/s 2024-02-22T19:45:35.092 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:33.800122+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e429: 8 total, 8 up, 5 in 2024-02-22T19:45:35.092 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:34.297110+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.53:0/3100648840' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:45:35.092 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:45:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:45:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:35.114 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:35.118 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:35.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:35.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:35.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:45:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:45:35.845 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 87 16 110367 110351 79.1751 60.375 0.0119031 0.0126258 2024-02-22T19:45:35.852 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:35.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:35.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:35.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:36.012 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:36.050 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:36.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:36.061 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:34.798517+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:36.061 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:34.812480+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.53:0/3100648840' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:45:36.061 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:34.812529+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e430: 8 total, 8 up, 5 in 2024-02-22T19:45:36.061 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:35.250336+0000 mgr.y (mgr.14535) 635 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:36.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:36.063 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:34.798517+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:36.063 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:34.812480+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.53:0/3100648840' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:45:36.063 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:34.812529+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e430: 8 total, 8 up, 5 in 2024-02-22T19:45:36.063 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:35.250336+0000 mgr.y (mgr.14535) 635 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:34.798517+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:34.812480+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.53:0/3100648840' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:45:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:34.812529+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e430: 8 total, 8 up, 5 in 2024-02-22T19:45:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:35.250336+0000 mgr.y (mgr.14535) 635 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:36.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:36.197 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:36.224 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:36.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:36.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:36.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:36.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:36.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:36.553 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:36.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:36.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:36.714 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:36.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:36.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:36.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:36.852 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 88 16 111483 111467 79.0621 69.75 0.0149154 0.0126437 2024-02-22T19:45:36.853 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:36.878 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:36.881 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:36.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:36.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:45:36.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:37.078 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:35.723180+0000 mgr.y (mgr.14535) 636 : cluster [DBG] pgmap v854: 138 pgs: 138 active+clean; 6.7 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-02-22T19:45:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:36.243382+0000 osd.7 (osd.7) 254 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:36.243459+0000 osd.7 (osd.7) 255 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:35.723180+0000 mgr.y (mgr.14535) 636 : cluster [DBG] pgmap v854: 138 pgs: 138 active+clean; 6.7 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-02-22T19:45:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:36.243382+0000 osd.7 (osd.7) 254 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:36.243459+0000 osd.7 (osd.7) 255 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:37.255 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:37.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:35.723180+0000 mgr.y (mgr.14535) 636 : cluster [DBG] pgmap v854: 138 pgs: 138 active+clean; 6.7 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-02-22T19:45:37.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:36.243382+0000 osd.7 (osd.7) 254 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:37.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:36.243459+0000 osd.7 (osd.7) 255 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:37.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:37.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:37.270 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:37.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:37.377 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:37.442 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:37.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:37.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:37.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:37.853 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 89 16 112442 112426 78.8462 59.9375 0.0197258 0.0126783 2024-02-22T19:45:37.888 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:36.830355+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e431: 8 total, 8 up, 5 in 2024-02-22T19:45:37.888 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:37.425374+0000 mon.a (mon.0) 1613 : cluster [DBG] osdmap e432: 8 total, 8 up, 5 in 2024-02-22T19:45:37.888 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:36.830355+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e431: 8 total, 8 up, 5 in 2024-02-22T19:45:37.888 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:37.425374+0000 mon.a (mon.0) 1613 : cluster [DBG] osdmap e432: 8 total, 8 up, 5 in 2024-02-22T19:45:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:36.830355+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e431: 8 total, 8 up, 5 in 2024-02-22T19:45:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:37.425374+0000 mon.a (mon.0) 1613 : cluster [DBG] osdmap e432: 8 total, 8 up, 5 in 2024-02-22T19:45:38.290 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:38.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:38.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:38.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:38.854 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total time run: 90.0065 2024-02-22T19:45:39.204 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total writes made: 113376 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Write size: 65536 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object size: 65536 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Bandwidth (MB/sec): 78.7277 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Bandwidth: 25.4661 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max bandwidth (MB/sec): 183.438 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min bandwidth (MB/sec): 56.1875 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average IOPS: 1259 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev IOPS: 407.457 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max IOPS: 2935 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min IOPS: 899 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average Latency(s): 0.0126986 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Latency(s): 0.00732961 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max latency(s): 0.172036 2024-02-22T19:45:39.205 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min latency(s): 0.00135253 2024-02-22T19:45:39.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:37.725692+0000 mgr.y (mgr.14535) 637 : cluster [DBG] pgmap v857: 138 pgs: 138 active+clean; 6.7 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-02-22T19:45:39.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:38.606027+0000 osd.4 (osd.4) 157 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:39.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:38.606072+0000 osd.4 (osd.4) 158 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:39.206 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Cleaning up (deleting benchmark objects) 2024-02-22T19:45:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:37.725692+0000 mgr.y (mgr.14535) 637 : cluster [DBG] pgmap v857: 138 pgs: 138 active+clean; 6.7 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-02-22T19:45:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:38.606027+0000 osd.4 (osd.4) 157 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:38.606072+0000 osd.4 (osd.4) 158 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:37.725692+0000 mgr.y (mgr.14535) 637 : cluster [DBG] pgmap v857: 138 pgs: 138 active+clean; 6.7 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-02-22T19:45:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:38.606027+0000 osd.4 (osd.4) 157 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:39.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:38.606072+0000 osd.4 (osd.4) 158 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:39.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:45:39.756 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:45:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:39] "GET /metrics HTTP/1.1" 200 176435 "" "Prometheus/2.33.4" 2024-02-22T19:45:40.038 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:40.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:40.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:40.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:39.447295+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e433: 8 total, 8 up, 5 in 2024-02-22T19:45:40.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:39.726368+0000 mgr.y (mgr.14535) 638 : cluster [DBG] pgmap v859: 138 pgs: 138 active+clean; 6.8 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1.0 KiB/s rd, 57 MiB/s wr, 1.01k op/s 2024-02-22T19:45:40.515 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:39.447295+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e433: 8 total, 8 up, 5 in 2024-02-22T19:45:40.515 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:39.726368+0000 mgr.y (mgr.14535) 638 : cluster [DBG] pgmap v859: 138 pgs: 138 active+clean; 6.8 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1.0 KiB/s rd, 57 MiB/s wr, 1.01k op/s 2024-02-22T19:45:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:39.447295+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e433: 8 total, 8 up, 5 in 2024-02-22T19:45:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:39.726368+0000 mgr.y (mgr.14535) 638 : cluster [DBG] pgmap v859: 138 pgs: 138 active+clean; 6.8 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1.0 KiB/s rd, 57 MiB/s wr, 1.01k op/s 2024-02-22T19:45:41.181 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:41.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:41.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:41.199 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:41.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:41.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:45:41.448 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:45:41.461 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:41.343346+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? 172.21.15.53:0/2699396059' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:41.489 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:45:41.543 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:45:41.555 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:45:41.586 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:45:41.617 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:45:41.660 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:45:41.673 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:45:41.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:41.343346+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? 172.21.15.53:0/2699396059' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:41.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:41.343346+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? 172.21.15.53:0/2699396059' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:42.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:42.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:42.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:42.611 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:42.615 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:42.616 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:41.726907+0000 mgr.y (mgr.14535) 639 : cluster [DBG] pgmap v860: 138 pgs: 138 active+clean; 6.9 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1023 B/s rd, 44 MiB/s wr, 820 op/s 2024-02-22T19:45:42.616 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:42.424114+0000 mon.a (mon.0) 1615 : cluster [DBG] osdmap e434: 8 total, 8 up, 5 in 2024-02-22T19:45:42.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:41.726907+0000 mgr.y (mgr.14535) 639 : cluster [DBG] pgmap v860: 138 pgs: 138 active+clean; 6.9 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1023 B/s rd, 44 MiB/s wr, 820 op/s 2024-02-22T19:45:42.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:42.424114+0000 mon.a (mon.0) 1615 : cluster [DBG] osdmap e434: 8 total, 8 up, 5 in 2024-02-22T19:45:42.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:42.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:42.748 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:41.726907+0000 mgr.y (mgr.14535) 639 : cluster [DBG] pgmap v860: 138 pgs: 138 active+clean; 6.9 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1023 B/s rd, 44 MiB/s wr, 820 op/s 2024-02-22T19:45:42.748 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:42.424114+0000 mon.a (mon.0) 1615 : cluster [DBG] osdmap e434: 8 total, 8 up, 5 in 2024-02-22T19:45:42.751 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:42.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:42.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:42.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:45:42.919 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:42.941 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:45:42.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:42.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:45:43.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:43.105 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:43.477 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:42.556142+0000 osd.2 (osd.2) 234 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:43.477 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:42.556193+0000 osd.2 (osd.2) 235 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:42.556142+0000 osd.2 (osd.2) 234 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:42.556193+0000 osd.2 (osd.2) 235 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:42.556142+0000 osd.2 (osd.2) 234 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:42.556193+0000 osd.2 (osd.2) 235 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:43.899 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:43.926 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:43.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:43.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:44.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:44.095 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:44.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:44.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:44.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:44.289 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:44.323 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:44.327 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:44.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:44.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:45:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:44.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:45:44.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:45:44.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:44.672 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:43.486611+0000 mon.a (mon.0) 1616 : cluster [DBG] osdmap e435: 8 total, 8 up, 5 in 2024-02-22T19:45:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:43.612753+0000 osd.1 (osd.1) 244 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:43.612804+0000 osd.1 (osd.1) 245 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:43.727338+0000 mgr.y (mgr.14535) 640 : cluster [DBG] pgmap v863: 138 pgs: 138 active+clean; 6.9 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1023 B/s rd, 44 MiB/s wr, 821 op/s 2024-02-22T19:45:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:43.486611+0000 mon.a (mon.0) 1616 : cluster [DBG] osdmap e435: 8 total, 8 up, 5 in 2024-02-22T19:45:44.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:43.612753+0000 osd.1 (osd.1) 244 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:44.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:43.612804+0000 osd.1 (osd.1) 245 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:44.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:43.727338+0000 mgr.y (mgr.14535) 640 : cluster [DBG] pgmap v863: 138 pgs: 138 active+clean; 6.9 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1023 B/s rd, 44 MiB/s wr, 821 op/s 2024-02-22T19:45:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:45:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:45:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:43.486611+0000 mon.a (mon.0) 1616 : cluster [DBG] osdmap e435: 8 total, 8 up, 5 in 2024-02-22T19:45:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:43.612753+0000 osd.1 (osd.1) 244 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:45:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:43.612804+0000 osd.1 (osd.1) 245 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:45:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:43.727338+0000 mgr.y (mgr.14535) 640 : cluster [DBG] pgmap v863: 138 pgs: 138 active+clean; 6.9 GiB data, 24 GiB used, 423 GiB / 447 GiB avail; 1023 B/s rd, 44 MiB/s wr, 821 op/s 2024-02-22T19:45:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-02-22T19:45:45.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:45:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:45:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:44.494982+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e436: 8 total, 8 up, 5 in 2024-02-22T19:45:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:45.058485+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.53:0/1572688583' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:45:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:45.058865+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:45:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:45.263615+0000 mgr.y (mgr.14535) 641 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:44.494982+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e436: 8 total, 8 up, 5 in 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:45.058485+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.53:0/1572688583' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:45.058865+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:45.263615+0000 mgr.y (mgr.14535) 641 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:44.494982+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e436: 8 total, 8 up, 5 in 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:45.058485+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.53:0/1572688583' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:45.058865+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:45:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:45.263615+0000 mgr.y (mgr.14535) 641 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:45.863 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:45.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:45.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:45.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:45:45.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:46.039 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:45.498544+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:45:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:45.498609+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e437: 8 total, 8 up, 5 in 2024-02-22T19:45:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:45.728098+0000 mgr.y (mgr.14535) 642 : cluster [DBG] pgmap v866: 138 pgs: 138 active+clean; 6.5 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:45:47.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:45.498544+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:45:47.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:45.498609+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e437: 8 total, 8 up, 5 in 2024-02-22T19:45:47.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:45.728098+0000 mgr.y (mgr.14535) 642 : cluster [DBG] pgmap v866: 138 pgs: 138 active+clean; 6.5 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:45:47.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:45.498544+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:45:47.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:45.498609+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e437: 8 total, 8 up, 5 in 2024-02-22T19:45:47.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:45.728098+0000 mgr.y (mgr.14535) 642 : cluster [DBG] pgmap v866: 138 pgs: 138 active+clean; 6.5 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:45:47.129 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:47.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:47.132 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:47.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:47.286 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:47.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:47.291 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:47.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:45:47.424 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:47.448 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:45:47.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:47.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:45:47.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:47.611 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:47.833 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:47.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:47.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:47.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:48.015 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:48.390 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:48.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:48.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:48.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:48.532 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:48.555 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:48.558 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:48.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:48.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:48.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:47.421387+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e438: 8 total, 8 up, 5 in 2024-02-22T19:45:48.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:47.598957+0000 mon.a (mon.0) 1622 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:45:48.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:47.728550+0000 mgr.y (mgr.14535) 643 : cluster [DBG] pgmap v868: 138 pgs: 138 active+clean; 6.5 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-02-22T19:45:48.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:47.914386+0000 mon.a (mon.0) 1623 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:48.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:48.298494+0000 mon.a (mon.0) 1624 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:45:48.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:48.299750+0000 mon.a (mon.0) 1625 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:45:48.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:48.310268+0000 mon.a (mon.0) 1626 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:47.421387+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e438: 8 total, 8 up, 5 in 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:47.598957+0000 mon.a (mon.0) 1622 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:47.728550+0000 mgr.y (mgr.14535) 643 : cluster [DBG] pgmap v868: 138 pgs: 138 active+clean; 6.5 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:47.914386+0000 mon.a (mon.0) 1623 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:48.298494+0000 mon.a (mon.0) 1624 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:48.299750+0000 mon.a (mon.0) 1625 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:45:48.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:48.310268+0000 mon.a (mon.0) 1626 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:48.715 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:47.421387+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e438: 8 total, 8 up, 5 in 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:47.598957+0000 mon.a (mon.0) 1622 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:47.728550+0000 mgr.y (mgr.14535) 643 : cluster [DBG] pgmap v868: 138 pgs: 138 active+clean; 6.5 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:47.914386+0000 mon.a (mon.0) 1623 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:48.298494+0000 mon.a (mon.0) 1624 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:48.299750+0000 mon.a (mon.0) 1625 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:45:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:48.310268+0000 mon.a (mon.0) 1626 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:45:48.878 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:48.884 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:48.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:48.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:45:49.020 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:49.043 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:45:49.046 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:49.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:45:49.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:49.363 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:45:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:49] "GET /metrics HTTP/1.1" 200 176435 "" "Prometheus/2.33.4" 2024-02-22T19:45:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:49.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:49.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:49.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:49.523 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:49.546 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:45:49.547 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:45:49.999 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:50.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:50.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:50.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:50.169 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:50.538 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:50.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:50.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:50.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:45:50.651 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:50.781 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:50.850 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:50.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:50.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:50.892 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:50.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:51.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:49.729267+0000 mgr.y (mgr.14535) 644 : cluster [DBG] pgmap v869: 138 pgs: 138 active+clean; 6.3 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.61k op/s 2024-02-22T19:45:51.033 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:49.729267+0000 mgr.y (mgr.14535) 644 : cluster [DBG] pgmap v869: 138 pgs: 138 active+clean; 6.3 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.61k op/s 2024-02-22T19:45:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:49.729267+0000 mgr.y (mgr.14535) 644 : cluster [DBG] pgmap v869: 138 pgs: 138 active+clean; 6.3 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.61k op/s 2024-02-22T19:45:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:51.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:51.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:51.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:51.818 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:45:51.846 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:51.850 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:51.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:51.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:52.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:52.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:52.070 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:45:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:51.223899+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.53:0/320897203' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:45:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:51.226446+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:45:52.115 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:52.162 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:51.223899+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.53:0/320897203' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:45:52.162 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:51.226446+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:45:52.167 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:51.223899+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.53:0/320897203' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:45:52.167 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:51.226446+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:45:52.167 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:52.207 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:52.219 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:52.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:52.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:52.375 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:52.638 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:52.645 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:52.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:52.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:52.803 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:52.871 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:52.875 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:51.730532+0000 mgr.y (mgr.14535) 645 : cluster [DBG] pgmap v870: 138 pgs: 138 active+clean; 6.2 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 707 B/s rd, 0 B/s wr, 1.14k op/s 2024-02-22T19:45:52.875 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:51.801888+0000 mon.a (mon.0) 1628 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:52.875 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:51.809042+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:45:52.875 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:51.809135+0000 mon.a (mon.0) 1630 : cluster [DBG] osdmap e439: 8 total, 8 up, 5 in 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:52.248634+0000 osd.7 (osd.7) 256 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:52.251956+0000 osd.7 (osd.7) 257 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:52.427318+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e440: 8 total, 8 up, 5 in 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:51.730532+0000 mgr.y (mgr.14535) 645 : cluster [DBG] pgmap v870: 138 pgs: 138 active+clean; 6.2 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 707 B/s rd, 0 B/s wr, 1.14k op/s 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:51.801888+0000 mon.a (mon.0) 1628 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:51.809042+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:51.809135+0000 mon.a (mon.0) 1630 : cluster [DBG] osdmap e439: 8 total, 8 up, 5 in 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:52.248634+0000 osd.7 (osd.7) 256 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:52.251956+0000 osd.7 (osd.7) 257 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:45:52.877 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:52.427318+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e440: 8 total, 8 up, 5 in 2024-02-22T19:45:52.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:52.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:52.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:45:52.984 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:45:52.984 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:45:52.984 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-02-22T19:45:52.984 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:45:52.984 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:45:52.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-02-22T19:45:53.047 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:51.730532+0000 mgr.y (mgr.14535) 645 : cluster [DBG] pgmap v870: 138 pgs: 138 active+clean; 6.2 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 707 B/s rd, 0 B/s wr, 1.14k op/s 2024-02-22T19:45:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:51.801888+0000 mon.a (mon.0) 1628 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:45:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:51.809042+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:45:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:51.809135+0000 mon.a (mon.0) 1630 : cluster [DBG] osdmap e439: 8 total, 8 up, 5 in 2024-02-22T19:45:53.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:52.248634+0000 osd.7 (osd.7) 256 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:45:53.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:52.251956+0000 osd.7 (osd.7) 257 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:45:53.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:52.427318+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e440: 8 total, 8 up, 5 in 2024-02-22T19:45:53.111 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:45:53.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:53.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:45:53.215 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:53.276 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:53.335 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:45:53.339 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:53.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:53.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:53.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:53.505 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:52.692576+0000 osd.5 (osd.5) 153 : cluster [DBG] 6.a scrub starts 2024-02-22T19:45:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:52.702468+0000 osd.5 (osd.5) 154 : cluster [DBG] 6.a scrub ok 2024-02-22T19:45:54.224 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:52.692576+0000 osd.5 (osd.5) 153 : cluster [DBG] 6.a scrub starts 2024-02-22T19:45:54.224 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:52.702468+0000 osd.5 (osd.5) 154 : cluster [DBG] 6.a scrub ok 2024-02-22T19:45:54.224 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:52.692576+0000 osd.5 (osd.5) 153 : cluster [DBG] 6.a scrub starts 2024-02-22T19:45:54.224 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:52.702468+0000 osd.5 (osd.5) 154 : cluster [DBG] 6.a scrub ok 2024-02-22T19:45:54.480 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:54.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:54.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:54.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:54.737 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:54.741 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:45:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:45:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:54.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:54.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:45:54.911 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:55.093 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:45:55.096 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:45:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:53.731153+0000 mgr.y (mgr.14535) 646 : cluster [DBG] pgmap v873: 138 pgs: 138 active+clean; 6.2 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 811 B/s rd, 0 B/s wr, 877 op/s 2024-02-22T19:45:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:53.868675+0000 osd.1 (osd.1) 246 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:45:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:53.877950+0000 osd.1 (osd.1) 247 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:45:55.142 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:45:55.167 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:45:55.169 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:53.731153+0000 mgr.y (mgr.14535) 646 : cluster [DBG] pgmap v873: 138 pgs: 138 active+clean; 6.2 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 811 B/s rd, 0 B/s wr, 877 op/s 2024-02-22T19:45:55.169 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:53.868675+0000 osd.1 (osd.1) 246 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:45:55.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:53.877950+0000 osd.1 (osd.1) 247 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:45:55.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:53.731153+0000 mgr.y (mgr.14535) 646 : cluster [DBG] pgmap v873: 138 pgs: 138 active+clean; 6.2 GiB data, 23 GiB used, 424 GiB / 447 GiB avail; 811 B/s rd, 0 B/s wr, 877 op/s 2024-02-22T19:45:55.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:53.868675+0000 osd.1 (osd.1) 246 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:45:55.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:53.877950+0000 osd.1 (osd.1) 247 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:45:55.187 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15576.0:131415 10.d e440/365 10:b6444dff:::benchmark_data_smithi053_180506_object18036:head v 440'8045, mlcod=440'8041)", 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:45:55.139857+0000", 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.001761032, 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017904380000000001, 2024-02-22T19:45:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139857+0000", 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139857+0000", 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:45:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139856+0000", 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5220000000000007e-06 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139865+0000", 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139873+0000", 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3910000000000003e-06 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:45:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139880+0000", 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7181000000000002e-05 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139917+0000", 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0775000000000003e-05 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139948+0000", 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0667000000000005e-05 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15576.0:131416 10.f e440/365 10:ff9557f8:::benchmark_data_smithi053_180506_object18037:head v 440'8226, mlcod=440'8224)", 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:45:55.139892+0000", 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0017263199999999999, 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001829638, 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139892+0000", 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139892+0000", 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139891+0000", 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989999999999997e-06 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139896+0000", 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8710000000000002e-06 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139902+0000", 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999999e-06 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139906+0000", 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0895000000000001e-05 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139947+0000", 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9018e-05 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.139966+0000", 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4103000000000005e-05 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.15576.0:131417 10.7 e440/365 10:e522fdf6:::benchmark_data_smithi053_180506_object18038:head v 440'8252, mlcod=440'8248)", 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:45:55.140597+0000", 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.001020962, 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012004979999999999, 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:45:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.140597+0000", 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.140597+0000", 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.140596+0000", 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3999999999999997e-06 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.140603+0000", 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9540000000000004e-06 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.140611+0000", 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8999999999999997e-06 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.140616+0000", 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091784600000000005 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.141533+0000", 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1789000000000001e-05 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:45:55.141555+0000", 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095816800000000002 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 3 2024-02-22T19:45:55.222 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:45:55.227 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:45:55.267 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:45:55.289 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:45:55.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:45:55.379 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:45:55.473 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:45:55.480 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:55.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:45:55.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:55.601 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:45:55.688 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:56.064 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:45:56.067 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:56.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:45:56.168 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:45:56.201 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:55.017044+0000 osd.3 (osd.3) 283 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:45:56.201 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:55.024745+0000 osd.3 (osd.3) 284 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:45:56.201 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:55.036669+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? 172.21.15.53:0/2408895564' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:56.201 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:55.197544+0000 osd.7 (osd.7) 258 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:45:56.201 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:55.202512+0000 osd.7 (osd.7) 259 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:45:56.201 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:45:55.274507+0000 mgr.y (mgr.14535) 647 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:55.017044+0000 osd.3 (osd.3) 283 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:45:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:55.024745+0000 osd.3 (osd.3) 284 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:45:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:55.036669+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? 172.21.15.53:0/2408895564' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:55.197544+0000 osd.7 (osd.7) 258 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:45:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:55.202512+0000 osd.7 (osd.7) 259 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:45:56.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:45:55.274507+0000 mgr.y (mgr.14535) 647 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:56.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:55.017044+0000 osd.3 (osd.3) 283 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:45:56.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:55.024745+0000 osd.3 (osd.3) 284 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:45:56.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:55.036669+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? 172.21.15.53:0/2408895564' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:45:56.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:55.197544+0000 osd.7 (osd.7) 258 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:45:56.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:55.202512+0000 osd.7 (osd.7) 259 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:45:56.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:45:55.274507+0000 mgr.y (mgr.14535) 647 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:45:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:45:57.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:57.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:45:57.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:45:57.202 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:55.731821+0000 mgr.y (mgr.14535) 648 : cluster [DBG] pgmap v874: 138 pgs: 138 active+clean; 5.9 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-02-22T19:45:57.204 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:45:57.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:57.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:45:57.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:55.731821+0000 mgr.y (mgr.14535) 648 : cluster [DBG] pgmap v874: 138 pgs: 138 active+clean; 5.9 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-02-22T19:45:57.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:55.731821+0000 mgr.y (mgr.14535) 648 : cluster [DBG] pgmap v874: 138 pgs: 138 active+clean; 5.9 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-02-22T19:45:57.342 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:45:57.407 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:57.507 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:45:57.507 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "" 2024-02-22T19:45:57.507 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:45:57.630 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:45:57.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:45:57.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:45:57.741 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:45:57.800 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:45:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:56.592042+0000 osd.5 (osd.5) 155 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:45:58.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:56.598060+0000 osd.5 (osd.5) 156 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:45:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:56.592042+0000 osd.5 (osd.5) 155 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:45:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:56.598060+0000 osd.5 (osd.5) 156 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:45:58.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:56.592042+0000 osd.5 (osd.5) 155 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:45:58.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:56.598060+0000 osd.5 (osd.5) 156 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:45:59.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:57.732650+0000 mgr.y (mgr.14535) 649 : cluster [DBG] pgmap v875: 138 pgs: 138 active+clean; 5.9 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 882 op/s 2024-02-22T19:45:59.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:57.914524+0000 osd.1 (osd.1) 248 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:45:59.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:57.920782+0000 osd.1 (osd.1) 249 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:45:59.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:57.732650+0000 mgr.y (mgr.14535) 649 : cluster [DBG] pgmap v875: 138 pgs: 138 active+clean; 5.9 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 882 op/s 2024-02-22T19:45:59.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:57.914524+0000 osd.1 (osd.1) 248 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:45:59.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:57.920782+0000 osd.1 (osd.1) 249 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:45:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:57.732650+0000 mgr.y (mgr.14535) 649 : cluster [DBG] pgmap v875: 138 pgs: 138 active+clean; 5.9 GiB data, 22 GiB used, 425 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 882 op/s 2024-02-22T19:45:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:57.914524+0000 osd.1 (osd.1) 248 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:45:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:57.920782+0000 osd.1 (osd.1) 249 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:45:59.643 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:45:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:45:59] "GET /metrics HTTP/1.1" 200 176472 "" "Prometheus/2.33.4" 2024-02-22T19:45:59.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:45:59.903 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:58.992076+0000 osd.3 (osd.3) 285 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:45:59.904 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:45:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:58.995847+0000 osd.3 (osd.3) 286 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:46:00.157 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:00.188 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:00.194 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:00.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:00.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:58.992076+0000 osd.3 (osd.3) 285 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:46:00.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:45:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:58.995847+0000 osd.3 (osd.3) 286 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:46:00.296 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:00.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:58.992076+0000 osd.3 (osd.3) 285 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:46:00.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:45:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:58.995847+0000 osd.3 (osd.3) 286 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:46:00.331 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:00.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:00.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:00.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:00.566 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:00.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:00.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:00.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:00.764 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:00.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:00.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:00.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:00.987 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:01.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:01.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:01.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:01.151 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:01.175 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:01.177 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:59.733376+0000 mgr.y (mgr.14535) 650 : cluster [DBG] pgmap v876: 138 pgs: 138 active+clean; 5.6 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:46:01.177 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:45:59.736548+0000 osd.2 (osd.2) 236 : cluster [DBG] 10.e scrub starts 2024-02-22T19:46:01.178 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:01.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:01.217 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:01.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:59.733376+0000 mgr.y (mgr.14535) 650 : cluster [DBG] pgmap v876: 138 pgs: 138 active+clean; 5.6 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:46:01.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:45:59.736548+0000 osd.2 (osd.2) 236 : cluster [DBG] 10.e scrub starts 2024-02-22T19:46:01.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:59.733376+0000 mgr.y (mgr.14535) 650 : cluster [DBG] pgmap v876: 138 pgs: 138 active+clean; 5.6 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:46:01.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:45:59.736548+0000 osd.2 (osd.2) 236 : cluster [DBG] 10.e scrub starts 2024-02-22T19:46:01.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:01.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:01.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:01.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:01.508 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:01.539 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:01.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:01.543 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:01.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:01.674 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:01.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:01.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:01.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:01.925 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:02.066 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:02.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:02.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:02.152 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:46:02.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:02.237 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:02.240 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:02.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:02.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:02.399 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:03.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:01.735092+0000 mgr.y (mgr.14535) 651 : cluster [DBG] pgmap v877: 138 pgs: 138 active+clean; 5.5 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:46:03.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:01.735092+0000 mgr.y (mgr.14535) 651 : cluster [DBG] pgmap v877: 138 pgs: 138 active+clean; 5.5 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:46:03.280 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5] out_osds: [4, 0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:46:03.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 1 1 2024-02-22T19:46:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:01.735092+0000 mgr.y (mgr.14535) 651 : cluster [DBG] pgmap v877: 138 pgs: 138 active+clean; 5.5 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:46:03.679 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:03.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:03.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:03.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:03.842 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:03.846 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:03.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:03.941 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:02.481063+0000 osd.2 (osd.2) 237 : cluster [DBG] 10.e scrub ok 2024-02-22T19:46:03.941 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:03.478622+0000 osd.5 (osd.5) 157 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:46:03.942 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:03.485060+0000 osd.5 (osd.5) 158 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:46:03.942 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:02.481063+0000 osd.2 (osd.2) 237 : cluster [DBG] 10.e scrub ok 2024-02-22T19:46:03.942 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:03.478622+0000 osd.5 (osd.5) 157 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:46:03.942 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:03.485060+0000 osd.5 (osd.5) 158 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:46:03.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:02.481063+0000 osd.2 (osd.2) 237 : cluster [DBG] 10.e scrub ok 2024-02-22T19:46:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:03.478622+0000 osd.5 (osd.5) 157 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:46:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:03.485060+0000 osd.5 (osd.5) 158 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:46:04.936 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:46:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:46:05.163 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:05.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:05.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:03.735489+0000 mgr.y (mgr.14535) 652 : cluster [DBG] pgmap v878: 138 pgs: 138 active+clean; 5.5 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 905 B/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:46:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:03.765821+0000 osd.2 (osd.2) 238 : cluster [DBG] 10.5 scrub starts 2024-02-22T19:46:05.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:03.735489+0000 mgr.y (mgr.14535) 652 : cluster [DBG] pgmap v878: 138 pgs: 138 active+clean; 5.5 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 905 B/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:46:05.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:03.765821+0000 osd.2 (osd.2) 238 : cluster [DBG] 10.5 scrub starts 2024-02-22T19:46:05.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:46:05.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:03.735489+0000 mgr.y (mgr.14535) 652 : cluster [DBG] pgmap v878: 138 pgs: 138 active+clean; 5.5 GiB data, 21 GiB used, 426 GiB / 447 GiB avail; 905 B/s rd, 0 B/s wr, 1.01k op/s 2024-02-22T19:46:05.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:03.765821+0000 osd.2 (osd.2) 238 : cluster [DBG] 10.5 scrub starts 2024-02-22T19:46:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:46:05.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:05.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:46:05.356 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:05.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:05.467 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:05.488 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:05.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:05.493 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:05.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:05.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:46:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:46:05.711 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:06.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:05.284234+0000 mgr.y (mgr.14535) 653 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:06.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:05.284234+0000 mgr.y (mgr.14535) 653 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:05.284234+0000 mgr.y (mgr.14535) 653 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:07.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:05.737109+0000 mgr.y (mgr.14535) 654 : cluster [DBG] pgmap v879: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.2 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:46:07.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:05.737109+0000 mgr.y (mgr.14535) 654 : cluster [DBG] pgmap v879: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.2 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:46:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:05.737109+0000 mgr.y (mgr.14535) 654 : cluster [DBG] pgmap v879: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.2 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:46:08.084 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:46:08.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:06.484956+0000 osd.2 (osd.2) 239 : cluster [DBG] 10.5 scrub ok 2024-02-22T19:46:08.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:06.798752+0000 osd.2 (osd.2) 240 : cluster [DBG] 10.8 scrub starts 2024-02-22T19:46:08.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:07.048137+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.53:0/4278684133' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:08.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:07.051251+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:07.142772+0000 osd.0 (osd.0) 173 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:07.142827+0000 osd.0 (osd.0) 174 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:06.484956+0000 osd.2 (osd.2) 239 : cluster [DBG] 10.5 scrub ok 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:06.798752+0000 osd.2 (osd.2) 240 : cluster [DBG] 10.8 scrub starts 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:07.048137+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.53:0/4278684133' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:07.051251+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:07.142772+0000 osd.0 (osd.0) 173 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:08.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:07.142827+0000 osd.0 (osd.0) 174 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:06.484956+0000 osd.2 (osd.2) 239 : cluster [DBG] 10.5 scrub ok 2024-02-22T19:46:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:06.798752+0000 osd.2 (osd.2) 240 : cluster [DBG] 10.8 scrub starts 2024-02-22T19:46:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:07.048137+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.53:0/4278684133' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:07.051251+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:07.142772+0000 osd.0 (osd.0) 173 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:07.142827+0000 osd.0 (osd.0) 174 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:08.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:08.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:09.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:09.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:07.737502+0000 mgr.y (mgr.14535) 655 : cluster [DBG] pgmap v880: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.2 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 916 op/s 2024-02-22T19:46:09.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:07.945669+0000 mon.a (mon.0) 1633 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:46:09.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:07.945706+0000 mon.a (mon.0) 1634 : cluster [INF] Cluster is now healthy 2024-02-22T19:46:09.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:07.957639+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:46:09.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:07.957703+0000 mon.a (mon.0) 1636 : cluster [DBG] osdmap e441: 8 total, 8 up, 5 in 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:08.384524+0000 mon.a (mon.0) 1637 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:07.737502+0000 mgr.y (mgr.14535) 655 : cluster [DBG] pgmap v880: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.2 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 916 op/s 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:07.945669+0000 mon.a (mon.0) 1633 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:07.945706+0000 mon.a (mon.0) 1634 : cluster [INF] Cluster is now healthy 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:07.957639+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:07.957703+0000 mon.a (mon.0) 1636 : cluster [DBG] osdmap e441: 8 total, 8 up, 5 in 2024-02-22T19:46:09.150 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:08.384524+0000 mon.a (mon.0) 1637 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:09.311 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:09.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:09.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:07.737502+0000 mgr.y (mgr.14535) 655 : cluster [DBG] pgmap v880: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 5.2 GiB data, 20 GiB used, 427 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 916 op/s 2024-02-22T19:46:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:07.945669+0000 mon.a (mon.0) 1633 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:46:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:07.945706+0000 mon.a (mon.0) 1634 : cluster [INF] Cluster is now healthy 2024-02-22T19:46:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:07.957639+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:46:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:07.957703+0000 mon.a (mon.0) 1636 : cluster [DBG] osdmap e441: 8 total, 8 up, 5 in 2024-02-22T19:46:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:08.384524+0000 mon.a (mon.0) 1637 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:09.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:09.477 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:09] "GET /metrics HTTP/1.1" 200 176477 "" "Prometheus/2.33.4" 2024-02-22T19:46:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:09.827 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:46:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:46:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:46:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:46:09.828 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:46:09.988 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:08.745811+0000 osd.2 (osd.2) 241 : cluster [DBG] 10.8 scrub ok 2024-02-22T19:46:09.988 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.011496+0000 osd.3 (osd.3) 287 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:46:09.988 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.018312+0000 osd.3 (osd.3) 288 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:46:10.166 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:10.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:10.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:08.745811+0000 osd.2 (osd.2) 241 : cluster [DBG] 10.8 scrub ok 2024-02-22T19:46:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.011496+0000 osd.3 (osd.3) 287 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:46:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.018312+0000 osd.3 (osd.3) 288 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:46:10.263 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:10.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:10.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:08.745811+0000 osd.2 (osd.2) 241 : cluster [DBG] 10.8 scrub ok 2024-02-22T19:46:10.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.011496+0000 osd.3 (osd.3) 287 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:46:10.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.018312+0000 osd.3 (osd.3) 288 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:46:10.305 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:10.332 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:10.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:10.367 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:46:10.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:46:10.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:10.442 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:46:10.488 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:46:10.490 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:10.520 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:46:10.556 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:46:10.602 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:46:10.635 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:46:10.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:46:11.150 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.738095+0000 mgr.y (mgr.14535) 656 : cluster [DBG] pgmap v882: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.843221+0000 osd.2 (osd.2) 242 : cluster [DBG] 4.d scrub starts 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.851633+0000 osd.2 (osd.2) 243 : cluster [DBG] 4.d scrub ok 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.891816+0000 osd.1 (osd.1) 250 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:09.901342+0000 osd.1 (osd.1) 251 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:10.057282+0000 osd.3 (osd.3) 289 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:10.060776+0000 osd.3 (osd.3) 290 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:10.323373+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? 172.21.15.53:0/568493634' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.738095+0000 mgr.y (mgr.14535) 656 : cluster [DBG] pgmap v882: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-02-22T19:46:11.151 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.843221+0000 osd.2 (osd.2) 242 : cluster [DBG] 4.d scrub starts 2024-02-22T19:46:11.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.851633+0000 osd.2 (osd.2) 243 : cluster [DBG] 4.d scrub ok 2024-02-22T19:46:11.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.891816+0000 osd.1 (osd.1) 250 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:46:11.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:09.901342+0000 osd.1 (osd.1) 251 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:46:11.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:10.057282+0000 osd.3 (osd.3) 289 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:46:11.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:10.060776+0000 osd.3 (osd.3) 290 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:46:11.152 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:10.323373+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? 172.21.15.53:0/568493634' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.738095+0000 mgr.y (mgr.14535) 656 : cluster [DBG] pgmap v882: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.843221+0000 osd.2 (osd.2) 242 : cluster [DBG] 4.d scrub starts 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.851633+0000 osd.2 (osd.2) 243 : cluster [DBG] 4.d scrub ok 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.891816+0000 osd.1 (osd.1) 250 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:09.901342+0000 osd.1 (osd.1) 251 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:10.057282+0000 osd.3 (osd.3) 289 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:10.060776+0000 osd.3 (osd.3) 290 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:46:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:10.323373+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? 172.21.15.53:0/568493634' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:11.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:46:12.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.268698+0000 osd.7 (osd.7) 260 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:46:12.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.272322+0000 osd.7 (osd.7) 261 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:46:12.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.424691+0000 osd.5 (osd.5) 159 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:46:12.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.428893+0000 osd.5 (osd.5) 160 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:46:12.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.268698+0000 osd.7 (osd.7) 260 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:46:12.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.272322+0000 osd.7 (osd.7) 261 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:46:12.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.424691+0000 osd.5 (osd.5) 159 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:46:12.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.428893+0000 osd.5 (osd.5) 160 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:46:12.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.268698+0000 osd.7 (osd.7) 260 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:46:12.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.272322+0000 osd.7 (osd.7) 261 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:46:12.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.424691+0000 osd.5 (osd.5) 159 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:46:12.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.428893+0000 osd.5 (osd.5) 160 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:46:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:12.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:12.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:12.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:12.961 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:12.982 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:12.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:12.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:13.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.738632+0000 mgr.y (mgr.14535) 657 : cluster [DBG] pgmap v883: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.814403+0000 osd.2 (osd.2) 244 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:11.826704+0000 osd.2 (osd.2) 245 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:12.041780+0000 osd.3 (osd.3) 291 : cluster [DBG] 3.b scrub starts 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:12.047687+0000 osd.3 (osd.3) 292 : cluster [DBG] 3.b scrub ok 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:12.312488+0000 osd.7 (osd.7) 262 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:46:13.128 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:12.320298+0000 osd.7 (osd.7) 263 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:46:13.129 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:12.441292+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e442: 8 total, 8 up, 5 in 2024-02-22T19:46:13.130 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.738632+0000 mgr.y (mgr.14535) 657 : cluster [DBG] pgmap v883: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.814403+0000 osd.2 (osd.2) 244 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:11.826704+0000 osd.2 (osd.2) 245 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:12.041780+0000 osd.3 (osd.3) 291 : cluster [DBG] 3.b scrub starts 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:12.047687+0000 osd.3 (osd.3) 292 : cluster [DBG] 3.b scrub ok 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:12.312488+0000 osd.7 (osd.7) 262 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:12.320298+0000 osd.7 (osd.7) 263 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:46:13.131 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:12.441292+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e442: 8 total, 8 up, 5 in 2024-02-22T19:46:13.131 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.738632+0000 mgr.y (mgr.14535) 657 : cluster [DBG] pgmap v883: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.814403+0000 osd.2 (osd.2) 244 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:11.826704+0000 osd.2 (osd.2) 245 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:12.041780+0000 osd.3 (osd.3) 291 : cluster [DBG] 3.b scrub starts 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:12.047687+0000 osd.3 (osd.3) 292 : cluster [DBG] 3.b scrub ok 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:12.312488+0000 osd.7 (osd.7) 262 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:12.320298+0000 osd.7 (osd.7) 263 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:46:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:12.441292+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e442: 8 total, 8 up, 5 in 2024-02-22T19:46:13.701 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:13.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:13.706 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:13.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:14.086 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:14.093 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:14.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:14.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:14.255 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:13.019587+0000 osd.3 (osd.3) 293 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:46:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:13.030859+0000 osd.3 (osd.3) 294 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:46:14.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:13.019587+0000 osd.3 (osd.3) 293 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:46:14.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:13.030859+0000 osd.3 (osd.3) 294 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:46:14.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:13.019587+0000 osd.3 (osd.3) 293 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:46:14.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:13.030859+0000 osd.3 (osd.3) 294 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:46:14.693 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:14.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:14.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:14.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:14.904 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:15.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:13.740260+0000 mgr.y (mgr.14535) 658 : cluster [DBG] pgmap v885: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 650 op/s 2024-02-22T19:46:15.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:13.904286+0000 osd.1 (osd.1) 252 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:46:15.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:13.905874+0000 osd.1 (osd.1) 253 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:46:15.047 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:14.038891+0000 osd.3 (osd.3) 295 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:46:15.048 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:14.159006+0000 osd.3 (osd.3) 296 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:46:15.048 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:14.314151+0000 osd.7 (osd.7) 264 : cluster [DBG] 3.d scrub starts 2024-02-22T19:46:15.048 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:14.317974+0000 osd.7 (osd.7) 265 : cluster [DBG] 3.d scrub ok 2024-02-22T19:46:15.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:46:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:46:15.324 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:13.740260+0000 mgr.y (mgr.14535) 658 : cluster [DBG] pgmap v885: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 650 op/s 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:13.904286+0000 osd.1 (osd.1) 252 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:13.905874+0000 osd.1 (osd.1) 253 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:14.038891+0000 osd.3 (osd.3) 295 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:14.159006+0000 osd.3 (osd.3) 296 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:14.314151+0000 osd.7 (osd.7) 264 : cluster [DBG] 3.d scrub starts 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:14.317974+0000 osd.7 (osd.7) 265 : cluster [DBG] 3.d scrub ok 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:13.740260+0000 mgr.y (mgr.14535) 658 : cluster [DBG] pgmap v885: 138 pgs: 138 active+clean; 4.9 GiB data, 19 GiB used, 428 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 650 op/s 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:13.904286+0000 osd.1 (osd.1) 252 : cluster [DBG] 6.18 scrub starts 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:13.905874+0000 osd.1 (osd.1) 253 : cluster [DBG] 6.18 scrub ok 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:14.038891+0000 osd.3 (osd.3) 295 : cluster [DBG] 6.1b scrub starts 2024-02-22T19:46:15.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:14.159006+0000 osd.3 (osd.3) 296 : cluster [DBG] 6.1b scrub ok 2024-02-22T19:46:15.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:14.314151+0000 osd.7 (osd.7) 264 : cluster [DBG] 3.d scrub starts 2024-02-22T19:46:15.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:14.317974+0000 osd.7 (osd.7) 265 : cluster [DBG] 3.d scrub ok 2024-02-22T19:46:15.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:46:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:46:15.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:46:16.066 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:14.865512+0000 osd.1 (osd.1) 254 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:46:16.066 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:14.875942+0000 osd.1 (osd.1) 255 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:46:16.066 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:15.295532+0000 mgr.y (mgr.14535) 659 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:16.067 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:15.305872+0000 osd.7 (osd.7) 266 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:46:16.067 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:15.312646+0000 osd.7 (osd.7) 267 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:46:16.067 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:15.451238+0000 osd.5 (osd.5) 161 : cluster [DBG] 4.f scrub starts 2024-02-22T19:46:16.067 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:15.453043+0000 osd.5 (osd.5) 162 : cluster [DBG] 4.f scrub ok 2024-02-22T19:46:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:14.865512+0000 osd.1 (osd.1) 254 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:46:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:14.875942+0000 osd.1 (osd.1) 255 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:46:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:15.295532+0000 mgr.y (mgr.14535) 659 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:15.305872+0000 osd.7 (osd.7) 266 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:46:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:15.312646+0000 osd.7 (osd.7) 267 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:46:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:15.451238+0000 osd.5 (osd.5) 161 : cluster [DBG] 4.f scrub starts 2024-02-22T19:46:16.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:15.453043+0000 osd.5 (osd.5) 162 : cluster [DBG] 4.f scrub ok 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:14.865512+0000 osd.1 (osd.1) 254 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:14.875942+0000 osd.1 (osd.1) 255 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:15.295532+0000 mgr.y (mgr.14535) 659 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:15.305872+0000 osd.7 (osd.7) 266 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:15.312646+0000 osd.7 (osd.7) 267 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:15.451238+0000 osd.5 (osd.5) 161 : cluster [DBG] 4.f scrub starts 2024-02-22T19:46:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:15.453043+0000 osd.5 (osd.5) 162 : cluster [DBG] 4.f scrub ok 2024-02-22T19:46:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:17.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:17.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:17.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:17.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:17.247 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:17.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:15.742170+0000 mgr.y (mgr.14535) 660 : cluster [DBG] pgmap v886: 138 pgs: 138 active+clean; 4.5 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-02-22T19:46:17.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:16.110761+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.53:0/1704789366' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:46:17.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:16.114473+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:46:17.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:15.742170+0000 mgr.y (mgr.14535) 660 : cluster [DBG] pgmap v886: 138 pgs: 138 active+clean; 4.5 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-02-22T19:46:17.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:16.110761+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.53:0/1704789366' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:46:17.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:16.114473+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:46:17.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:15.742170+0000 mgr.y (mgr.14535) 660 : cluster [DBG] pgmap v886: 138 pgs: 138 active+clean; 4.5 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-02-22T19:46:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:16.110761+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.53:0/1704789366' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:46:17.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:16.114473+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:46:17.413 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:17.434 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-02-22T19:46:18.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:16.795953+0000 osd.2 (osd.2) 246 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:16.806785+0000 osd.2 (osd.2) 247 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:17.073562+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.073633+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e443: 8 total, 8 up, 5 in 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.322398+0000 osd.7 (osd.7) 268 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.328234+0000 osd.7 (osd.7) 269 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.442860+0000 mon.a (mon.0) 1642 : cluster [DBG] osdmap e444: 8 total, 8 up, 5 in 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.479652+0000 osd.5 (osd.5) 163 : cluster [DBG] 6.d scrub starts 2024-02-22T19:46:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.513832+0000 osd.5 (osd.5) 164 : cluster [DBG] 6.d scrub ok 2024-02-22T19:46:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:16.795953+0000 osd.2 (osd.2) 246 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:46:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:16.806785+0000 osd.2 (osd.2) 247 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:17.073562+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.073633+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e443: 8 total, 8 up, 5 in 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.322398+0000 osd.7 (osd.7) 268 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.328234+0000 osd.7 (osd.7) 269 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.442860+0000 mon.a (mon.0) 1642 : cluster [DBG] osdmap e444: 8 total, 8 up, 5 in 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.479652+0000 osd.5 (osd.5) 163 : cluster [DBG] 6.d scrub starts 2024-02-22T19:46:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.513832+0000 osd.5 (osd.5) 164 : cluster [DBG] 6.d scrub ok 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:16.795953+0000 osd.2 (osd.2) 246 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:16.806785+0000 osd.2 (osd.2) 247 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:17.073562+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.073633+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e443: 8 total, 8 up, 5 in 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.322398+0000 osd.7 (osd.7) 268 : cluster [DBG] 3.12 deep-scrub starts 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.328234+0000 osd.7 (osd.7) 269 : cluster [DBG] 3.12 deep-scrub ok 2024-02-22T19:46:18.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.442860+0000 mon.a (mon.0) 1642 : cluster [DBG] osdmap e444: 8 total, 8 up, 5 in 2024-02-22T19:46:18.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.479652+0000 osd.5 (osd.5) 163 : cluster [DBG] 6.d scrub starts 2024-02-22T19:46:18.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.513832+0000 osd.5 (osd.5) 164 : cluster [DBG] 6.d scrub ok 2024-02-22T19:46:19.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 2 1 2024-02-22T19:46:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.742924+0000 mgr.y (mgr.14535) 661 : cluster [DBG] pgmap v889: 138 pgs: 138 active+clean; 4.5 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 948 op/s 2024-02-22T19:46:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.791209+0000 osd.1 (osd.1) 256 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:46:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.792729+0000 osd.1 (osd.1) 257 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:46:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.806698+0000 osd.2 (osd.2) 248 : cluster [DBG] 6.f scrub starts 2024-02-22T19:46:19.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:17.949721+0000 osd.2 (osd.2) 249 : cluster [DBG] 6.f scrub ok 2024-02-22T19:46:19.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:18.485240+0000 osd.5 (osd.5) 165 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:46:19.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:18.486260+0000 osd.5 (osd.5) 166 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:46:19.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.742924+0000 mgr.y (mgr.14535) 661 : cluster [DBG] pgmap v889: 138 pgs: 138 active+clean; 4.5 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 948 op/s 2024-02-22T19:46:19.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.791209+0000 osd.1 (osd.1) 256 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:46:19.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.792729+0000 osd.1 (osd.1) 257 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:46:19.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.806698+0000 osd.2 (osd.2) 248 : cluster [DBG] 6.f scrub starts 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:17.949721+0000 osd.2 (osd.2) 249 : cluster [DBG] 6.f scrub ok 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:18.485240+0000 osd.5 (osd.5) 165 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:18.486260+0000 osd.5 (osd.5) 166 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:19] "GET /metrics HTTP/1.1" 200 176477 "" "Prometheus/2.33.4" 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.742924+0000 mgr.y (mgr.14535) 661 : cluster [DBG] pgmap v889: 138 pgs: 138 active+clean; 4.5 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 948 op/s 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.791209+0000 osd.1 (osd.1) 256 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.792729+0000 osd.1 (osd.1) 257 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.806698+0000 osd.2 (osd.2) 248 : cluster [DBG] 6.f scrub starts 2024-02-22T19:46:19.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:17.949721+0000 osd.2 (osd.2) 249 : cluster [DBG] 6.f scrub ok 2024-02-22T19:46:19.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:18.485240+0000 osd.5 (osd.5) 165 : cluster [DBG] 6.1e scrub starts 2024-02-22T19:46:19.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:18.486260+0000 osd.5 (osd.5) 166 : cluster [DBG] 6.1e scrub ok 2024-02-22T19:46:20.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:18.826326+0000 osd.1 (osd.1) 258 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:46:20.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:18.828876+0000 osd.1 (osd.1) 259 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:46:20.487 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:18.826326+0000 osd.1 (osd.1) 258 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:46:20.487 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:18.828876+0000 osd.1 (osd.1) 259 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:46:20.532 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:20.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:20.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:18.826326+0000 osd.1 (osd.1) 258 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:46:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:18.828876+0000 osd.1 (osd.1) 259 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:46:20.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:20.675 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:20.702 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:20.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:20.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:20.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:20.865 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:20.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:20.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:21.021 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:21.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:21.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:21.120 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:19.537072+0000 osd.5 (osd.5) 167 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:46:21.121 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:19.537985+0000 osd.5 (osd.5) 168 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:46:21.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:21.161 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:21.185 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:21.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:21.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:21.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:21.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:19.537072+0000 osd.5 (osd.5) 167 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:46:21.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:19.537985+0000 osd.5 (osd.5) 168 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:46:21.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:19.743770+0000 mgr.y (mgr.14535) 662 : cluster [DBG] pgmap v890: 138 pgs: 138 active+clean; 4.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-02-22T19:46:21.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:20.069764+0000 osd.3 (osd.3) 297 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:46:21.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:20.075863+0000 osd.3 (osd.3) 298 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:46:21.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:20.351763+0000 osd.7 (osd.7) 270 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:46:21.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:20.359127+0000 osd.7 (osd.7) 271 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:46:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:21.349 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:21.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:21.450 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:19.743770+0000 mgr.y (mgr.14535) 662 : cluster [DBG] pgmap v890: 138 pgs: 138 active+clean; 4.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-02-22T19:46:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:20.069764+0000 osd.3 (osd.3) 297 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:46:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:20.075863+0000 osd.3 (osd.3) 298 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:46:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:20.351763+0000 osd.7 (osd.7) 270 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:46:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:20.359127+0000 osd.7 (osd.7) 271 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:19.537072+0000 osd.5 (osd.5) 167 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:19.537985+0000 osd.5 (osd.5) 168 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:19.743770+0000 mgr.y (mgr.14535) 662 : cluster [DBG] pgmap v890: 138 pgs: 138 active+clean; 4.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:20.069764+0000 osd.3 (osd.3) 297 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:20.075863+0000 osd.3 (osd.3) 298 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:20.351763+0000 osd.7 (osd.7) 270 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:46:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:20.359127+0000 osd.7 (osd.7) 271 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:46:21.528 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:21.584 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:21.594 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:21.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:21.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:21.766 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:22.036 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:22.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:22.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:22.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:22.215 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:22.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:20.783024+0000 osd.2 (osd.2) 250 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:46:22.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:20.790728+0000 osd.2 (osd.2) 251 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:46:22.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:21.086894+0000 osd.3 (osd.3) 299 : cluster [DBG] 10.4 scrub starts 2024-02-22T19:46:22.334 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:20.783024+0000 osd.2 (osd.2) 250 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:46:22.334 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:20.790728+0000 osd.2 (osd.2) 251 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:46:22.334 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:21.086894+0000 osd.3 (osd.3) 299 : cluster [DBG] 10.4 scrub starts 2024-02-22T19:46:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:20.783024+0000 osd.2 (osd.2) 250 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:46:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:20.790728+0000 osd.2 (osd.2) 251 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:46:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:21.086894+0000 osd.3 (osd.3) 299 : cluster [DBG] 10.4 scrub starts 2024-02-22T19:46:23.053 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:46:23.053 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:46:23.389 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:23.395 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:23.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:21.744424+0000 mgr.y (mgr.14535) 663 : cluster [DBG] pgmap v891: 138 pgs: 138 active+clean; 4.2 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:46:23.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:22.239536+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.53:0/3778617626' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:46:23.395 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:22.241446+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:46:23.395 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:21.744424+0000 mgr.y (mgr.14535) 663 : cluster [DBG] pgmap v891: 138 pgs: 138 active+clean; 4.2 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:46:23.396 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:22.239536+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.53:0/3778617626' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:46:23.396 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:22.241446+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:46:23.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:23.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:23.569 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:21.744424+0000 mgr.y (mgr.14535) 663 : cluster [DBG] pgmap v891: 138 pgs: 138 active+clean; 4.2 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-02-22T19:46:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:22.239536+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.53:0/3778617626' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:46:23.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:22.241446+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:46:23.628 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:46:24.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:23.126653+0000 mon.a (mon.0) 1644 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:23.132648+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:23.132691+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e445: 8 total, 8 up, 5 in 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:23.387533+0000 mon.a (mon.0) 1647 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:23.126653+0000 mon.a (mon.0) 1644 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:23.132648+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:23.132691+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e445: 8 total, 8 up, 5 in 2024-02-22T19:46:24.441 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:23.387533+0000 mon.a (mon.0) 1647 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:23.126653+0000 mon.a (mon.0) 1644 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:23.132648+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:46:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:23.132691+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e445: 8 total, 8 up, 5 in 2024-02-22T19:46:24.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:23.387533+0000 mon.a (mon.0) 1647 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:24.875 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:24.884 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:24.885 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:24.902 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:24.907 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:24.967 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:24.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:25.051 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:46:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:46:25.195 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:25.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:25.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:25.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:25.370 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:25.373 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:25.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:25.466 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:23.747376+0000 mgr.y (mgr.14535) 664 : cluster [DBG] pgmap v893: 138 pgs: 138 active+clean; 4.2 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 767 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:46:25.468 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:23.747376+0000 mgr.y (mgr.14535) 664 : cluster [DBG] pgmap v893: 138 pgs: 138 active+clean; 4.2 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 767 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:46:25.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:25.506 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:25.533 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:25.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:25.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:46:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:46:25.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:23.747376+0000 mgr.y (mgr.14535) 664 : cluster [DBG] pgmap v893: 138 pgs: 138 active+clean; 4.2 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 767 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:46:25.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:25.683 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:25.898 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:25.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:25.907 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:26.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:26.057 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:26.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:26.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:26.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:26.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:25.308742+0000 mgr.y (mgr.14535) 665 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:26.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:25.308742+0000 mgr.y (mgr.14535) 665 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:26.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:46:26.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:25.308742+0000 mgr.y (mgr.14535) 665 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:27.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:25.748037+0000 mgr.y (mgr.14535) 666 : cluster [DBG] pgmap v894: 138 pgs: 138 active+clean; 3.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-02-22T19:46:27.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:25.748037+0000 mgr.y (mgr.14535) 666 : cluster [DBG] pgmap v894: 138 pgs: 138 active+clean; 3.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-02-22T19:46:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:25.748037+0000 mgr.y (mgr.14535) 666 : cluster [DBG] pgmap v894: 138 pgs: 138 active+clean; 3.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-02-22T19:46:28.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:27.443527+0000 mon.a (mon.0) 1648 : cluster [DBG] osdmap e446: 8 total, 8 up, 5 in 2024-02-22T19:46:28.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:27.748327+0000 mgr.y (mgr.14535) 667 : cluster [DBG] pgmap v896: 138 pgs: 138 active+clean; 3.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 850 op/s 2024-02-22T19:46:28.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:27.443527+0000 mon.a (mon.0) 1648 : cluster [DBG] osdmap e446: 8 total, 8 up, 5 in 2024-02-22T19:46:28.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:27.748327+0000 mgr.y (mgr.14535) 667 : cluster [DBG] pgmap v896: 138 pgs: 138 active+clean; 3.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 850 op/s 2024-02-22T19:46:28.803 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:28.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:28.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:27.443527+0000 mon.a (mon.0) 1648 : cluster [DBG] osdmap e446: 8 total, 8 up, 5 in 2024-02-22T19:46:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:27.748327+0000 mgr.y (mgr.14535) 667 : cluster [DBG] pgmap v896: 138 pgs: 138 active+clean; 3.9 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 850 op/s 2024-02-22T19:46:28.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:28.973 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:28.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:29.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:29.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:29.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:29.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:29.507 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:46:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:29] "GET /metrics HTTP/1.1" 200 176481 "" "Prometheus/2.33.4" 2024-02-22T19:46:29.582 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:29.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:29.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:29.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:29.724 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:29.751 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:29.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:29.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:29.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:29.889 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:29.914 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:29.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:29.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:30.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:30.056 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:30.078 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:30.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:30.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:30.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:30.213 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:30.240 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:30.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:30.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:30.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:30.401 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:30.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:30.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:30.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:30.570 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:28.816615+0000 osd.5 (osd.5) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:30.571 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:28.816655+0000 osd.5 (osd.5) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:30.571 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:29.466888+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e447: 8 total, 8 up, 5 in 2024-02-22T19:46:30.571 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:29.748969+0000 mgr.y (mgr.14535) 668 : cluster [DBG] pgmap v898: 138 pgs: 138 active+clean; 3.6 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-02-22T19:46:30.571 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:28.816615+0000 osd.5 (osd.5) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:30.571 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:28.816655+0000 osd.5 (osd.5) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:30.571 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:29.466888+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e447: 8 total, 8 up, 5 in 2024-02-22T19:46:30.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:29.748969+0000 mgr.y (mgr.14535) 668 : cluster [DBG] pgmap v898: 138 pgs: 138 active+clean; 3.6 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-02-22T19:46:30.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:30.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:30.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:30.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:28.816615+0000 osd.5 (osd.5) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:28.816655+0000 osd.5 (osd.5) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:29.466888+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e447: 8 total, 8 up, 5 in 2024-02-22T19:46:30.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:29.748969+0000 mgr.y (mgr.14535) 668 : cluster [DBG] pgmap v898: 138 pgs: 138 active+clean; 3.6 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-02-22T19:46:31.524 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-02-22T19:46:31.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:31.091865+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.53:0/1922073414' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:46:31.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:31.095379+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:46:31.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:31.091865+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.53:0/1922073414' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:46:31.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:31.095379+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:46:31.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:31.091865+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.53:0/1922073414' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:46:31.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:31.095379+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:46:32.088 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:46:32.088 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:46:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:31.497223+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:46:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:31.497266+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e448: 8 total, 8 up, 5 in 2024-02-22T19:46:32.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:31.749368+0000 mgr.y (mgr.14535) 669 : cluster [DBG] pgmap v900: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 919 op/s 2024-02-22T19:46:32.937 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:31.497223+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:46:32.938 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:31.497266+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e448: 8 total, 8 up, 5 in 2024-02-22T19:46:32.938 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:31.749368+0000 mgr.y (mgr.14535) 669 : cluster [DBG] pgmap v900: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 919 op/s 2024-02-22T19:46:32.938 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:31.497223+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:46:32.938 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:31.497266+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e448: 8 total, 8 up, 5 in 2024-02-22T19:46:32.938 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:31.749368+0000 mgr.y (mgr.14535) 669 : cluster [DBG] pgmap v900: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 919 op/s 2024-02-22T19:46:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:32.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:32.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:33.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:33.080 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:33.104 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:33.107 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:33.108 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:33.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:33.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:33.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:33.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:33.400 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:33.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:33.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:33.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:33.563 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:35.054 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:46:35.054 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:46:35.054 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:46:35.054 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:46:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:33.749772+0000 mgr.y (mgr.14535) 670 : cluster [DBG] pgmap v901: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 812 B/s rd, 0 B/s wr, 875 op/s 2024-02-22T19:46:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:34.061600+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? 172.21.15.53:0/3519256408' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:35.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:46:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:46:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:33.749772+0000 mgr.y (mgr.14535) 670 : cluster [DBG] pgmap v901: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 812 B/s rd, 0 B/s wr, 875 op/s 2024-02-22T19:46:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:34.061600+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? 172.21.15.53:0/3519256408' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:33.749772+0000 mgr.y (mgr.14535) 670 : cluster [DBG] pgmap v901: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 812 B/s rd, 0 B/s wr, 875 op/s 2024-02-22T19:46:35.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:34.061600+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? 172.21.15.53:0/3519256408' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:35.285 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:35.292 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:35.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:35.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:35.446 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:35.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:46:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:46:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:35.321757+0000 mgr.y (mgr.14535) 671 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:36.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:35.321757+0000 mgr.y (mgr.14535) 671 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:35.321757+0000 mgr.y (mgr.14535) 671 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:36.738 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:36.746 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:36.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:36.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:46:36.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 3 1 2024-02-22T19:46:36.847 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:36.949 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:35.756802+0000 mgr.y (mgr.14535) 672 : cluster [DBG] pgmap v902: 138 pgs: 138 active+clean; 3.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s 2024-02-22T19:46:37.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:35.756802+0000 mgr.y (mgr.14535) 672 : cluster [DBG] pgmap v902: 138 pgs: 138 active+clean; 3.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s 2024-02-22T19:46:37.047 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:37.052 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:37.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:35.756802+0000 mgr.y (mgr.14535) 672 : cluster [DBG] pgmap v902: 138 pgs: 138 active+clean; 3.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s 2024-02-22T19:46:37.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:37.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:37.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:37.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:37.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:37.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:37.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:37.697 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:37.701 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:37.701 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:37.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:37.836 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:37.860 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:37.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:37.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:37.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:38.036 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:38.158 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:38.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:38.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:38.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:38.269 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:38.355 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:38.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:38.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:38.596 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:38.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:37.757179+0000 mgr.y (mgr.14535) 673 : cluster [DBG] pgmap v903: 138 pgs: 138 active+clean; 3.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-02-22T19:46:38.607 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:37.757179+0000 mgr.y (mgr.14535) 673 : cluster [DBG] pgmap v903: 138 pgs: 138 active+clean; 3.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-02-22T19:46:38.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:38.730 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:37.757179+0000 mgr.y (mgr.14535) 673 : cluster [DBG] pgmap v903: 138 pgs: 138 active+clean; 3.2 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-02-22T19:46:39.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:46:39.551 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:46:39.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:38.636866+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.53:0/2839319113' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:39.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:38.637610+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:39.583 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:38.636866+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.53:0/2839319113' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:39.584 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:38.637610+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:39.584 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:46:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:39] "GET /metrics HTTP/1.1" 200 176480 "" "Prometheus/2.33.4" 2024-02-22T19:46:39.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:38.636866+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.53:0/2839319113' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:39.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:38.637610+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:46:40.413 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:40.422 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:40.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:40.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:40.600 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:40.713 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:39.502995+0000 mon.a (mon.0) 1654 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:39.521302+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:39.521362+0000 mon.a (mon.0) 1656 : cluster [DBG] osdmap e449: 8 total, 8 up, 5 in 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:39.757638+0000 mgr.y (mgr.14535) 674 : cluster [DBG] pgmap v905: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:39.502995+0000 mon.a (mon.0) 1654 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:39.521302+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:39.521362+0000 mon.a (mon.0) 1656 : cluster [DBG] osdmap e449: 8 total, 8 up, 5 in 2024-02-22T19:46:40.714 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:39.757638+0000 mgr.y (mgr.14535) 674 : cluster [DBG] pgmap v905: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:46:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:39.502995+0000 mon.a (mon.0) 1654 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:39.521302+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:46:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:39.521362+0000 mon.a (mon.0) 1656 : cluster [DBG] osdmap e449: 8 total, 8 up, 5 in 2024-02-22T19:46:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:39.757638+0000 mgr.y (mgr.14535) 674 : cluster [DBG] pgmap v905: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-02-22T19:46:42.495 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:42.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:42.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:42.599 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:42.676 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:42.768 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:42.785 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:42.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:42.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:42.938 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:41.759245+0000 mgr.y (mgr.14535) 675 : cluster [DBG] pgmap v906: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-02-22T19:46:43.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:42.454126+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e450: 8 total, 8 up, 5 in 2024-02-22T19:46:43.116 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:41.759245+0000 mgr.y (mgr.14535) 675 : cluster [DBG] pgmap v906: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-02-22T19:46:43.116 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:42.454126+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e450: 8 total, 8 up, 5 in 2024-02-22T19:46:43.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:41.759245+0000 mgr.y (mgr.14535) 675 : cluster [DBG] pgmap v906: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-02-22T19:46:43.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:42.454126+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e450: 8 total, 8 up, 5 in 2024-02-22T19:46:43.413 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:43.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:43.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:43.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:43.553 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:43.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:43.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:43.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:43.713 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:43.738 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:43.740 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:43.741 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:43.840 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:43.605431+0000 osd.7 (osd.7) 272 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:43.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:43.934 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:43.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:43.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:43.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:43.605431+0000 osd.7 (osd.7) 272 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:43.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:43.605471+0000 osd.7 (osd.7) 273 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:43.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:43.605431+0000 osd.7 (osd.7) 272 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:43.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:43.605471+0000 osd.7 (osd.7) 273 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:44.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:44.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:43.605471+0000 osd.7 (osd.7) 273 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:44.117 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:44.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:44.233 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:44.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:44.335 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:46:44.335 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:46:44.335 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:46:44.336 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:46:44.336 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:46:44.431 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:44.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:46:44.614 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:44.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:44.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:44.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:44.790 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:44.903 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:43.760060+0000 mgr.y (mgr.14535) 676 : cluster [DBG] pgmap v908: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 695 op/s 2024-02-22T19:46:44.903 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:43.849374+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e451: 8 total, 8 up, 5 in 2024-02-22T19:46:44.903 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:44.700007+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? 172.21.15.53:0/967070798' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:46:44.904 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:43.760060+0000 mgr.y (mgr.14535) 676 : cluster [DBG] pgmap v908: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 695 op/s 2024-02-22T19:46:44.904 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:43.849374+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e451: 8 total, 8 up, 5 in 2024-02-22T19:46:44.904 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:44.700007+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? 172.21.15.53:0/967070798' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:46:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.3 to 1 (10000) 2024-02-22T19:46:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:43.760060+0000 mgr.y (mgr.14535) 676 : cluster [DBG] pgmap v908: 138 pgs: 138 active+clean; 2.9 GiB data, 13 GiB used, 434 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 695 op/s 2024-02-22T19:46:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:43.849374+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e451: 8 total, 8 up, 5 in 2024-02-22T19:46:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:44.700007+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? 172.21.15.53:0/967070798' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:46:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:46:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:46:45.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:46:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:46:45.787 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:45.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:45.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:45.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:45.948 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:45.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:45.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:46.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:46:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:46.125 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:46:46.129 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:44.853384+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.53:0/967070798' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:46:46.129 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:44.853448+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e452: 8 total, 8 up, 5 in 2024-02-22T19:46:46.129 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:45.333178+0000 mgr.y (mgr.14535) 677 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:46.130 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:46.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:46:46.139 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:44.853384+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.53:0/967070798' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:46:46.140 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:44.853448+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e452: 8 total, 8 up, 5 in 2024-02-22T19:46:46.140 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:45.333178+0000 mgr.y (mgr.14535) 677 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:46.140 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:44.853384+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.53:0/967070798' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:46:46.140 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:44.853448+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e452: 8 total, 8 up, 5 in 2024-02-22T19:46:46.140 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:45.333178+0000 mgr.y (mgr.14535) 677 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:46.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:46.265 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:46.291 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:46.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:46.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:46.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:47.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:45.762056+0000 mgr.y (mgr.14535) 678 : cluster [DBG] pgmap v911: 138 pgs: 138 active+clean; 2.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s 2024-02-22T19:46:47.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:45.762056+0000 mgr.y (mgr.14535) 678 : cluster [DBG] pgmap v911: 138 pgs: 138 active+clean; 2.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s 2024-02-22T19:46:47.234 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:47.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:47.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:47.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 7 1 2024-02-22T19:46:47.297 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:47.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:46:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:45.762056+0000 mgr.y (mgr.14535) 678 : cluster [DBG] pgmap v911: 138 pgs: 138 active+clean; 2.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s 2024-02-22T19:46:47.373 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:46:47.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:47.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:46:47.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:47.561 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:47.887 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:47.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:45.955003+0000 osd.3 (osd.3) 300 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:47.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:45.955056+0000 osd.3 (osd.3) 301 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:47.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:46.714915+0000 osd.6 (osd.6) 160 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:47.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:46.714975+0000 osd.6 (osd.6) 161 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:47.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:46.892631+0000 mon.a (mon.0) 1662 : cluster [DBG] osdmap e453: 8 total, 8 up, 5 in 2024-02-22T19:46:47.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:47.461783+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e454: 8 total, 8 up, 5 in 2024-02-22T19:46:47.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:47.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:47.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:46:47.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:48.124 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:48.192 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:48.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:48.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:45.955003+0000 osd.3 (osd.3) 300 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:45.955056+0000 osd.3 (osd.3) 301 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:46.714915+0000 osd.6 (osd.6) 160 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:46.714975+0000 osd.6 (osd.6) 161 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:46.892631+0000 mon.a (mon.0) 1662 : cluster [DBG] osdmap e453: 8 total, 8 up, 5 in 2024-02-22T19:46:48.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:47.461783+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e454: 8 total, 8 up, 5 in 2024-02-22T19:46:48.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:48.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:45.955003+0000 osd.3 (osd.3) 300 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:48.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:45.955056+0000 osd.3 (osd.3) 301 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:48.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:46.714915+0000 osd.6 (osd.6) 160 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:46:48.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:46.714975+0000 osd.6 (osd.6) 161 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:46:48.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:46.892631+0000 mon.a (mon.0) 1662 : cluster [DBG] osdmap e453: 8 total, 8 up, 5 in 2024-02-22T19:46:48.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:47.461783+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e454: 8 total, 8 up, 5 in 2024-02-22T19:46:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:48.355 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:48.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:48.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:48.460 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:48.570 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:47.764428+0000 mgr.y (mgr.14535) 679 : cluster [DBG] pgmap v914: 138 pgs: 138 active+clean; 2.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-02-22T19:46:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:48.320283+0000 mon.a (mon.0) 1664 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:46:49.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:47.764428+0000 mgr.y (mgr.14535) 679 : cluster [DBG] pgmap v914: 138 pgs: 138 active+clean; 2.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-02-22T19:46:49.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:48.320283+0000 mon.a (mon.0) 1664 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:46:49.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:47.764428+0000 mgr.y (mgr.14535) 679 : cluster [DBG] pgmap v914: 138 pgs: 138 active+clean; 2.5 GiB data, 12 GiB used, 435 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-02-22T19:46:49.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:48.320283+0000 mon.a (mon.0) 1664 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:46:49.431 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:49.444 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:49.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:49.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:49.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:49.627 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:49.629 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:49] "GET /metrics HTTP/1.1" 200 176480 "" "Prometheus/2.33.4" 2024-02-22T19:46:49.901 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:49.683450+0000 mon.a (mon.0) 1665 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:46:49.902 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:49.684285+0000 mon.a (mon.0) 1666 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:46:49.902 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:49.697885+0000 mon.a (mon.0) 1667 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:49.903 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:49.683450+0000 mon.a (mon.0) 1665 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:46:49.903 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:49.684285+0000 mon.a (mon.0) 1666 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:46:49.903 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:49.697885+0000 mon.a (mon.0) 1667 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:50.301 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:50.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:50.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:49.683450+0000 mon.a (mon.0) 1665 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:46:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:49.684285+0000 mon.a (mon.0) 1666 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:46:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:49.697885+0000 mon.a (mon.0) 1667 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:46:50.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:46:50.442 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:46:50.467 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:50.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:46:50.569 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:51.225 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:49.765115+0000 mgr.y (mgr.14535) 680 : cluster [DBG] pgmap v915: 138 pgs: 138 active+clean; 2.3 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-02-22T19:46:51.226 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:49.765115+0000 mgr.y (mgr.14535) 680 : cluster [DBG] pgmap v915: 138 pgs: 138 active+clean; 2.3 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-02-22T19:46:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:49.765115+0000 mgr.y (mgr.14535) 680 : cluster [DBG] pgmap v915: 138 pgs: 138 active+clean; 2.3 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-02-22T19:46:52.473 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:46:52.494 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:46:52.512 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:46:52.602 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:46:52.645 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:46:52.681 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:46:52.722 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:46:52.750 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:46:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:46:53.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:51.767066+0000 mgr.y (mgr.14535) 681 : cluster [DBG] pgmap v916: 138 pgs: 138 active+clean; 2.2 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 740 B/s rd, 0 B/s wr, 963 op/s 2024-02-22T19:46:53.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:52.428079+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.53:0/1589383522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:53.188 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:51.767066+0000 mgr.y (mgr.14535) 681 : cluster [DBG] pgmap v916: 138 pgs: 138 active+clean; 2.2 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 740 B/s rd, 0 B/s wr, 963 op/s 2024-02-22T19:46:53.188 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:52.428079+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.53:0/1589383522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:53.226 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:53.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:53.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:53.332 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:46:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:51.767066+0000 mgr.y (mgr.14535) 681 : cluster [DBG] pgmap v916: 138 pgs: 138 active+clean; 2.2 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 740 B/s rd, 0 B/s wr, 963 op/s 2024-02-22T19:46:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:52.428079+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.53:0/1589383522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:46:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:53.474 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:46:53.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:53.485 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:46:53.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:53.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:53.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:53.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:46:53.803 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:54.577 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:46:54.587 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:54.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:46:54.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:54.746 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:54.749 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:54.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:54.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:46:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:54.908 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:46:54.911 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:54.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:46:54.912 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:46:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:46:54.940 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.7 to 1 (10000) 2024-02-22T19:46:55.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:55.068 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:55.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:53.767554+0000 mgr.y (mgr.14535) 682 : cluster [DBG] pgmap v917: 138 pgs: 138 active+clean; 2.2 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 671 op/s 2024-02-22T19:46:55.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:54.488221+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.53:0/3354651631' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:46:55.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:54.488826+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:46:55.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:53.767554+0000 mgr.y (mgr.14535) 682 : cluster [DBG] pgmap v917: 138 pgs: 138 active+clean; 2.2 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 671 op/s 2024-02-22T19:46:55.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:54.488221+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.53:0/3354651631' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:46:55.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:54.488826+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:46:55.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:53.767554+0000 mgr.y (mgr.14535) 682 : cluster [DBG] pgmap v917: 138 pgs: 138 active+clean; 2.2 GiB data, 11 GiB used, 436 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 671 op/s 2024-02-22T19:46:55.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:54.488221+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.53:0/3354651631' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:46:55.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:54.488826+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:46:55.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:46:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:46:55.778 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:46:56.212 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:54.931098+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:46:56.212 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:54.931167+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e455: 8 total, 8 up, 5 in 2024-02-22T19:46:56.212 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:55.343198+0000 mgr.y (mgr.14535) 683 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:56.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:54.931098+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:46:56.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:54.931167+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e455: 8 total, 8 up, 5 in 2024-02-22T19:46:56.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:55.343198+0000 mgr.y (mgr.14535) 683 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:56.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 5 1 2024-02-22T19:46:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:54.931098+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:46:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:54.931167+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e455: 8 total, 8 up, 5 in 2024-02-22T19:46:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:55.343198+0000 mgr.y (mgr.14535) 683 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:46:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:46:56.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:56.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:46:56.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:56.513 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:56.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:56.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:56.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:56.719 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:56.818 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:56.826 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:56.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:56.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:46:56.960 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:56.983 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:55.769291+0000 mgr.y (mgr.14535) 684 : cluster [DBG] pgmap v919: 138 pgs: 138 active+clean; 1.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-02-22T19:46:56.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:56.885777+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.53:0/361520690' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:46:56.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:56.886232+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:46:56.988 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:46:56.990 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:56.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:46:57.004 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:46:57.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:46:57.124 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:46:57.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:57.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:46:57.257 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:46:57.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:55.769291+0000 mgr.y (mgr.14535) 684 : cluster [DBG] pgmap v919: 138 pgs: 138 active+clean; 1.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-02-22T19:46:57.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:56.885777+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.53:0/361520690' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:46:57.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:56.886232+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:46:57.314 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:46:57.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:55.769291+0000 mgr.y (mgr.14535) 684 : cluster [DBG] pgmap v919: 138 pgs: 138 active+clean; 1.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-02-22T19:46:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:56.885777+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.53:0/361520690' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:46:57.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:56.886232+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:46:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:57.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:46:57.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:46:57.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:46:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:57.509 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:46:57.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:57.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:46:57.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:46:57.647 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:57.671 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:46:57.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:46:57.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:46:57.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:46:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:46:58.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:56.955590+0000 mon.a (mon.0) 1672 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:46:56.971007+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:56.971065+0000 mon.a (mon.0) 1674 : cluster [DBG] osdmap e456: 8 total, 8 up, 5 in 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:57.299920+0000 osd.7 (osd.7) 274 : cluster [DBG] 10.c scrub starts 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:56.955590+0000 mon.a (mon.0) 1672 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:46:56.971007+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:56.971065+0000 mon.a (mon.0) 1674 : cluster [DBG] osdmap e456: 8 total, 8 up, 5 in 2024-02-22T19:46:58.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:57.299920+0000 osd.7 (osd.7) 274 : cluster [DBG] 10.c scrub starts 2024-02-22T19:46:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:56.955590+0000 mon.a (mon.0) 1672 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:46:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:46:56.971007+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:46:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:56.971065+0000 mon.a (mon.0) 1674 : cluster [DBG] osdmap e456: 8 total, 8 up, 5 in 2024-02-22T19:46:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:57.299920+0000 osd.7 (osd.7) 274 : cluster [DBG] 10.c scrub starts 2024-02-22T19:46:59.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:57.770981+0000 mgr.y (mgr.14535) 685 : cluster [DBG] pgmap v921: 138 pgs: 138 active+clean; 1.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 859 op/s 2024-02-22T19:46:59.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:57.952942+0000 osd.7 (osd.7) 275 : cluster [DBG] 10.c scrub ok 2024-02-22T19:46:59.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:58.508450+0000 osd.5 (osd.5) 171 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:46:59.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:58.513468+0000 osd.5 (osd.5) 172 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:46:59.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:57.770981+0000 mgr.y (mgr.14535) 685 : cluster [DBG] pgmap v921: 138 pgs: 138 active+clean; 1.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 859 op/s 2024-02-22T19:46:59.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:57.952942+0000 osd.7 (osd.7) 275 : cluster [DBG] 10.c scrub ok 2024-02-22T19:46:59.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:58.508450+0000 osd.5 (osd.5) 171 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:46:59.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:46:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:58.513468+0000 osd.5 (osd.5) 172 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:46:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:57.770981+0000 mgr.y (mgr.14535) 685 : cluster [DBG] pgmap v921: 138 pgs: 138 active+clean; 1.9 GiB data, 10 GiB used, 437 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 859 op/s 2024-02-22T19:46:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:57.952942+0000 osd.7 (osd.7) 275 : cluster [DBG] 10.c scrub ok 2024-02-22T19:46:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:58.508450+0000 osd.5 (osd.5) 171 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:46:59.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:46:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:58.513468+0000 osd.5 (osd.5) 172 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:46:59.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:46:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:46:59] "GET /metrics HTTP/1.1" 200 176498 "" "Prometheus/2.33.4" 2024-02-22T19:47:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:58.643814+0000 osd.1 (osd.1) 260 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:47:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:58.646205+0000 osd.1 (osd.1) 261 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:47:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:58.870775+0000 osd.2 (osd.2) 252 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:47:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:58.874343+0000 osd.2 (osd.2) 253 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:47:00.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:59.147320+0000 osd.3 (osd.3) 302 : cluster [DBG] 10.4 scrub starts 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:58.643814+0000 osd.1 (osd.1) 260 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:58.646205+0000 osd.1 (osd.1) 261 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:58.870775+0000 osd.2 (osd.2) 252 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:58.874343+0000 osd.2 (osd.2) 253 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:59.147320+0000 osd.3 (osd.3) 302 : cluster [DBG] 10.4 scrub starts 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:58.643814+0000 osd.1 (osd.1) 260 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:47:00.486 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:58.646205+0000 osd.1 (osd.1) 261 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:47:00.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:58.870775+0000 osd.2 (osd.2) 252 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:47:00.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:58.874343+0000 osd.2 (osd.2) 253 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:47:00.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:59.147320+0000 osd.3 (osd.3) 302 : cluster [DBG] 10.4 scrub starts 2024-02-22T19:47:00.901 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:00.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:00.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:00.950 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:00.978 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:01.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:01.069 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:01.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:01.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:01.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:01.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:46:59.772237+0000 mgr.y (mgr.14535) 686 : cluster [DBG] pgmap v922: 138 pgs: 138 active+clean; 1.7 GiB data, 9.5 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:01.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:00.151536+0000 osd.3 (osd.3) 303 : cluster [DBG] 10.4 scrub ok 2024-02-22T19:47:01.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:46:59.772237+0000 mgr.y (mgr.14535) 686 : cluster [DBG] pgmap v922: 138 pgs: 138 active+clean; 1.7 GiB data, 9.5 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:01.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:00.151536+0000 osd.3 (osd.3) 303 : cluster [DBG] 10.4 scrub ok 2024-02-22T19:47:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:46:59.772237+0000 mgr.y (mgr.14535) 686 : cluster [DBG] pgmap v922: 138 pgs: 138 active+clean; 1.7 GiB data, 9.5 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:00.151536+0000 osd.3 (osd.3) 303 : cluster [DBG] 10.4 scrub ok 2024-02-22T19:47:02.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:00.855284+0000 osd.2 (osd.2) 254 : cluster [DBG] 5.a scrub starts 2024-02-22T19:47:02.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:00.864199+0000 osd.2 (osd.2) 255 : cluster [DBG] 5.a scrub ok 2024-02-22T19:47:02.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:01.212591+0000 osd.7 (osd.7) 276 : cluster [DBG] 10.1 scrub starts 2024-02-22T19:47:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:00.855284+0000 osd.2 (osd.2) 254 : cluster [DBG] 5.a scrub starts 2024-02-22T19:47:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:00.864199+0000 osd.2 (osd.2) 255 : cluster [DBG] 5.a scrub ok 2024-02-22T19:47:02.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:01.212591+0000 osd.7 (osd.7) 276 : cluster [DBG] 10.1 scrub starts 2024-02-22T19:47:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:00.855284+0000 osd.2 (osd.2) 254 : cluster [DBG] 5.a scrub starts 2024-02-22T19:47:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:00.864199+0000 osd.2 (osd.2) 255 : cluster [DBG] 5.a scrub ok 2024-02-22T19:47:02.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:01.212591+0000 osd.7 (osd.7) 276 : cluster [DBG] 10.1 scrub starts 2024-02-22T19:47:02.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:47:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:01.723977+0000 osd.7 (osd.7) 277 : cluster [DBG] 10.1 scrub ok 2024-02-22T19:47:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:01.772775+0000 mgr.y (mgr.14535) 687 : cluster [DBG] pgmap v923: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 1.5 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-02-22T19:47:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:02.072061+0000 osd.3 (osd.3) 304 : cluster [DBG] 10.f scrub starts 2024-02-22T19:47:03.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:02.888392+0000 osd.3 (osd.3) 305 : cluster [DBG] 10.f scrub ok 2024-02-22T19:47:03.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:01.723977+0000 osd.7 (osd.7) 277 : cluster [DBG] 10.1 scrub ok 2024-02-22T19:47:03.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:01.772775+0000 mgr.y (mgr.14535) 687 : cluster [DBG] pgmap v923: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 1.5 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-02-22T19:47:03.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:02.072061+0000 osd.3 (osd.3) 304 : cluster [DBG] 10.f scrub starts 2024-02-22T19:47:03.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:02.888392+0000 osd.3 (osd.3) 305 : cluster [DBG] 10.f scrub ok 2024-02-22T19:47:03.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:01.723977+0000 osd.7 (osd.7) 277 : cluster [DBG] 10.1 scrub ok 2024-02-22T19:47:03.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:01.772775+0000 mgr.y (mgr.14535) 687 : cluster [DBG] pgmap v923: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 1.5 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-02-22T19:47:03.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:02.072061+0000 osd.3 (osd.3) 304 : cluster [DBG] 10.f scrub starts 2024-02-22T19:47:03.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:02.888392+0000 osd.3 (osd.3) 305 : cluster [DBG] 10.f scrub ok 2024-02-22T19:47:04.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.260771+0000 osd.7 (osd.7) 278 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:47:04.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.314514+0000 osd.7 (osd.7) 279 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:47:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.260771+0000 osd.7 (osd.7) 278 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:47:04.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.314514+0000 osd.7 (osd.7) 279 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:47:04.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.260771+0000 osd.7 (osd.7) 278 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:47:04.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.314514+0000 osd.7 (osd.7) 279 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:47:05.058 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:47:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.611616+0000 osd.5 (osd.5) 173 : cluster [DBG] 3.c scrub starts 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.617522+0000 osd.5 (osd.5) 174 : cluster [DBG] 3.c scrub ok 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.665357+0000 osd.1 (osd.1) 262 : cluster [DBG] 10.0 scrub starts 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.690941+0000 osd.2 (osd.2) 256 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.690992+0000 osd.2 (osd.2) 257 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.774378+0000 mgr.y (mgr.14535) 688 : cluster [DBG] pgmap v924: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 1.5 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 578 B/s rd, 0 B/s wr, 614 op/s 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.902816+0000 osd.1 (osd.1) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:03.902854+0000 osd.1 (osd.1) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:04.056287+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e457: 8 total, 8 up, 5 in 2024-02-22T19:47:05.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:04.138950+0000 osd.1 (osd.1) 265 : cluster [DBG] 10.0 scrub ok 2024-02-22T19:47:05.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.611616+0000 osd.5 (osd.5) 173 : cluster [DBG] 3.c scrub starts 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.617522+0000 osd.5 (osd.5) 174 : cluster [DBG] 3.c scrub ok 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.665357+0000 osd.1 (osd.1) 262 : cluster [DBG] 10.0 scrub starts 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.690941+0000 osd.2 (osd.2) 256 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.690992+0000 osd.2 (osd.2) 257 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.774378+0000 mgr.y (mgr.14535) 688 : cluster [DBG] pgmap v924: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 1.5 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 578 B/s rd, 0 B/s wr, 614 op/s 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.902816+0000 osd.1 (osd.1) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:03.902854+0000 osd.1 (osd.1) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:04.056287+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e457: 8 total, 8 up, 5 in 2024-02-22T19:47:05.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:04.138950+0000 osd.1 (osd.1) 265 : cluster [DBG] 10.0 scrub ok 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.611616+0000 osd.5 (osd.5) 173 : cluster [DBG] 3.c scrub starts 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.617522+0000 osd.5 (osd.5) 174 : cluster [DBG] 3.c scrub ok 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.665357+0000 osd.1 (osd.1) 262 : cluster [DBG] 10.0 scrub starts 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.690941+0000 osd.2 (osd.2) 256 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.690992+0000 osd.2 (osd.2) 257 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.774378+0000 mgr.y (mgr.14535) 688 : cluster [DBG] pgmap v924: 138 pgs: 1 active+clean+scrubbing, 137 active+clean; 1.5 GiB data, 9.0 GiB used, 438 GiB / 447 GiB avail; 578 B/s rd, 0 B/s wr, 614 op/s 2024-02-22T19:47:05.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.902816+0000 osd.1 (osd.1) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:05.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:03.902854+0000 osd.1 (osd.1) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:05.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:04.056287+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e457: 8 total, 8 up, 5 in 2024-02-22T19:47:05.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:04.138950+0000 osd.1 (osd.1) 265 : cluster [DBG] 10.0 scrub ok 2024-02-22T19:47:05.546 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:05.547 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:05.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:05.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:05.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:47:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:47:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:05.703 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:05.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:05.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:05.729 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:47:05.729 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:47:05.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:05.869 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:06.026 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:06.035 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:06.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:06.081 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:05.290495+0000 osd.7 (osd.7) 280 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:47:06.081 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:05.294609+0000 osd.7 (osd.7) 281 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:47:06.081 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:05.353260+0000 mgr.y (mgr.14535) 689 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:06.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:06.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:05.290495+0000 osd.7 (osd.7) 280 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:47:06.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:05.294609+0000 osd.7 (osd.7) 281 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:47:06.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:05.353260+0000 mgr.y (mgr.14535) 689 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:05.290495+0000 osd.7 (osd.7) 280 : cluster [DBG] 6.1c scrub starts 2024-02-22T19:47:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:05.294609+0000 osd.7 (osd.7) 281 : cluster [DBG] 6.1c scrub ok 2024-02-22T19:47:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:05.353260+0000 mgr.y (mgr.14535) 689 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:05.776826+0000 mgr.y (mgr.14535) 690 : cluster [DBG] pgmap v926: 138 pgs: 138 active+clean; 1.2 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:47:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:05.979433+0000 osd.2 (osd.2) 258 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:47:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:05.985172+0000 osd.2 (osd.2) 259 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:47:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:06.258643+0000 osd.7 (osd.7) 282 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:47:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:06.262149+0000 osd.7 (osd.7) 283 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:47:07.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:05.776826+0000 mgr.y (mgr.14535) 690 : cluster [DBG] pgmap v926: 138 pgs: 138 active+clean; 1.2 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:47:07.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:05.979433+0000 osd.2 (osd.2) 258 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:05.985172+0000 osd.2 (osd.2) 259 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:06.258643+0000 osd.7 (osd.7) 282 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:06.262149+0000 osd.7 (osd.7) 283 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:05.776826+0000 mgr.y (mgr.14535) 690 : cluster [DBG] pgmap v926: 138 pgs: 138 active+clean; 1.2 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:05.979433+0000 osd.2 (osd.2) 258 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:05.985172+0000 osd.2 (osd.2) 259 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:06.258643+0000 osd.7 (osd.7) 282 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:47:07.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:06.262149+0000 osd.7 (osd.7) 283 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:47:08.194 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:08.197 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:08.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:08.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:06.788557+0000 osd.4 (osd.4) 159 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:06.788631+0000 osd.4 (osd.4) 160 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:06.961815+0000 osd.2 (osd.2) 260 : cluster [DBG] 5.b scrub starts 2024-02-22T19:47:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:06.968578+0000 osd.2 (osd.2) 261 : cluster [DBG] 5.b scrub ok 2024-02-22T19:47:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:07.087053+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e458: 8 total, 8 up, 5 in 2024-02-22T19:47:08.354 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:08.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:08.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:08.382 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:06.788557+0000 osd.4 (osd.4) 159 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:08.382 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:06.788631+0000 osd.4 (osd.4) 160 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:08.382 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:06.961815+0000 osd.2 (osd.2) 260 : cluster [DBG] 5.b scrub starts 2024-02-22T19:47:08.382 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:06.968578+0000 osd.2 (osd.2) 261 : cluster [DBG] 5.b scrub ok 2024-02-22T19:47:08.383 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:07.087053+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e458: 8 total, 8 up, 5 in 2024-02-22T19:47:08.383 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:06.788557+0000 osd.4 (osd.4) 159 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:08.383 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:06.788631+0000 osd.4 (osd.4) 160 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:08.383 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:06.961815+0000 osd.2 (osd.2) 260 : cluster [DBG] 5.b scrub starts 2024-02-22T19:47:08.383 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:06.968578+0000 osd.2 (osd.2) 261 : cluster [DBG] 5.b scrub ok 2024-02-22T19:47:08.384 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:07.087053+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e458: 8 total, 8 up, 5 in 2024-02-22T19:47:08.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:47:08.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:08.528 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:08.625 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:08.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:08.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:08.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:08.812 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:08.817 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:08.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:08.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:09.146 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:09.147 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:09.150 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:09.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:09.218 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:47:09.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:07.643720+0000 osd.5 (osd.5) 175 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:07.646801+0000 osd.5 (osd.5) 176 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:07.777301+0000 mgr.y (mgr.14535) 691 : cluster [DBG] pgmap v928: 138 pgs: 138 active+clean; 1.2 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:07.969256+0000 osd.2 (osd.2) 262 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:08.073139+0000 osd.2 (osd.2) 263 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:07.643720+0000 osd.5 (osd.5) 175 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:47:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:07.646801+0000 osd.5 (osd.5) 176 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:47:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:07.777301+0000 mgr.y (mgr.14535) 691 : cluster [DBG] pgmap v928: 138 pgs: 138 active+clean; 1.2 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-02-22T19:47:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:07.969256+0000 osd.2 (osd.2) 262 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:47:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:08.073139+0000 osd.2 (osd.2) 263 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:47:09.276 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:47:09.287 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:09.323 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:47:09.365 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:09.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:07.643720+0000 osd.5 (osd.5) 175 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:47:09.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:07.646801+0000 osd.5 (osd.5) 176 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:47:09.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:07.777301+0000 mgr.y (mgr.14535) 691 : cluster [DBG] pgmap v928: 138 pgs: 138 active+clean; 1.2 GiB data, 8.0 GiB used, 439 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-02-22T19:47:09.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:07.969256+0000 osd.2 (osd.2) 262 : cluster [DBG] 6.17 scrub starts 2024-02-22T19:47:09.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:08.073139+0000 osd.2 (osd.2) 263 : cluster [DBG] 6.17 scrub ok 2024-02-22T19:47:09.369 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:47:09.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:09.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:09.418 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:47:09.455 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:47:09.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:09.494 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:47:09.522 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:09.534 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:47:09.555 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.556 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:47:09.559 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:09.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:09.559 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:47:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:09] "GET /metrics HTTP/1.1" 200 176499 "" "Prometheus/2.33.4" 2024-02-22T19:47:09.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:09.698 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:09.724 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:09.729 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:09.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:09.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:09.863 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:10.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:10.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:10.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-02-22T19:47:10.140 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:10.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:09.016325+0000 osd.2 (osd.2) 264 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:47:10.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:09.025528+0000 osd.2 (osd.2) 265 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:47:10.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:09.183228+0000 mon.a (mon.0) 1677 : audit [DBG] from='client.? 172.21.15.53:0/2308448610' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:10.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:09.459298+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.53:0/4106642224' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:47:10.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:09.459867+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:47:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:09.016325+0000 osd.2 (osd.2) 264 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:47:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:09.025528+0000 osd.2 (osd.2) 265 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:47:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:09.183228+0000 mon.a (mon.0) 1677 : audit [DBG] from='client.? 172.21.15.53:0/2308448610' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:09.459298+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.53:0/4106642224' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:47:10.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:09.459867+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:47:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:09.016325+0000 osd.2 (osd.2) 264 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:47:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:09.025528+0000 osd.2 (osd.2) 265 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:47:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:09.183228+0000 mon.a (mon.0) 1677 : audit [DBG] from='client.? 172.21.15.53:0/2308448610' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:09.459298+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.53:0/4106642224' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:47:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:09.459867+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:47:10.693 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:10.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:10.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:10.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:10.836 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:10.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:10.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:11.014 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:11.014 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:47:11.014 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:47:11.014 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:11.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:11.138 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-22T19:47:11.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:47:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:09.577824+0000 osd.5 (osd.5) 177 : cluster [DBG] 10.9 scrub starts 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:09.777903+0000 mgr.y (mgr.14535) 692 : cluster [DBG] pgmap v929: 138 pgs: 138 active+clean; 1.0 GiB data, 7.5 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:09.916387+0000 osd.5 (osd.5) 178 : cluster [DBG] 10.9 scrub ok 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:09.995874+0000 osd.2 (osd.2) 266 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:10.006171+0000 osd.2 (osd.2) 267 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:10.053561+0000 osd.3 (osd.3) 306 : cluster [DBG] 10.d scrub starts 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:10.110449+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:47:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:10.110532+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e459: 8 total, 8 up, 5 in 2024-02-22T19:47:11.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:10.417933+0000 osd.3 (osd.3) 307 : cluster [DBG] 10.d scrub ok 2024-02-22T19:47:11.495 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:09.577824+0000 osd.5 (osd.5) 177 : cluster [DBG] 10.9 scrub starts 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:09.777903+0000 mgr.y (mgr.14535) 692 : cluster [DBG] pgmap v929: 138 pgs: 138 active+clean; 1.0 GiB data, 7.5 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:09.916387+0000 osd.5 (osd.5) 178 : cluster [DBG] 10.9 scrub ok 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:09.995874+0000 osd.2 (osd.2) 266 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:10.006171+0000 osd.2 (osd.2) 267 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:10.053561+0000 osd.3 (osd.3) 306 : cluster [DBG] 10.d scrub starts 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:10.110449+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:10.110532+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e459: 8 total, 8 up, 5 in 2024-02-22T19:47:11.496 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:10.417933+0000 osd.3 (osd.3) 307 : cluster [DBG] 10.d scrub ok 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:09.577824+0000 osd.5 (osd.5) 177 : cluster [DBG] 10.9 scrub starts 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:09.777903+0000 mgr.y (mgr.14535) 692 : cluster [DBG] pgmap v929: 138 pgs: 138 active+clean; 1.0 GiB data, 7.5 GiB used, 440 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:09.916387+0000 osd.5 (osd.5) 178 : cluster [DBG] 10.9 scrub ok 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:09.995874+0000 osd.2 (osd.2) 266 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:10.006171+0000 osd.2 (osd.2) 267 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:10.053561+0000 osd.3 (osd.3) 306 : cluster [DBG] 10.d scrub starts 2024-02-22T19:47:11.497 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:10.110449+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:47:11.498 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:10.110532+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e459: 8 total, 8 up, 5 in 2024-02-22T19:47:11.498 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:10.417933+0000 osd.3 (osd.3) 307 : cluster [DBG] 10.d scrub ok 2024-02-22T19:47:11.512 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:11.516 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:11.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:11.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:11.650 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:12.005 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:12.008 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:12.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:12.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:47:12.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:12.182 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:12.187 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:12.188 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:12.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:12.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:10.689303+0000 osd.1 (osd.1) 266 : cluster [DBG] 4.e scrub starts 2024-02-22T19:47:12.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:10.692924+0000 osd.1 (osd.1) 267 : cluster [DBG] 4.e scrub ok 2024-02-22T19:47:12.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:11.080318+0000 osd.3 (osd.3) 308 : cluster [DBG] 10.7 scrub starts 2024-02-22T19:47:12.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:11.595029+0000 osd.3 (osd.3) 309 : cluster [DBG] 10.7 scrub ok 2024-02-22T19:47:12.327 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:12.356 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:12.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:12.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:12.433 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:12.460 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:12.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:10.689303+0000 osd.1 (osd.1) 266 : cluster [DBG] 4.e scrub starts 2024-02-22T19:47:12.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:10.692924+0000 osd.1 (osd.1) 267 : cluster [DBG] 4.e scrub ok 2024-02-22T19:47:12.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:11.080318+0000 osd.3 (osd.3) 308 : cluster [DBG] 10.7 scrub starts 2024-02-22T19:47:12.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:11.595029+0000 osd.3 (osd.3) 309 : cluster [DBG] 10.7 scrub ok 2024-02-22T19:47:12.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:10.689303+0000 osd.1 (osd.1) 266 : cluster [DBG] 4.e scrub starts 2024-02-22T19:47:12.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:10.692924+0000 osd.1 (osd.1) 267 : cluster [DBG] 4.e scrub ok 2024-02-22T19:47:12.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:11.080318+0000 osd.3 (osd.3) 308 : cluster [DBG] 10.7 scrub starts 2024-02-22T19:47:12.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:11.595029+0000 osd.3 (osd.3) 309 : cluster [DBG] 10.7 scrub ok 2024-02-22T19:47:12.540 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:12.911 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:12.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:12.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:13.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:13.076 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:13.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:13.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:13.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:11.778312+0000 mgr.y (mgr.14535) 693 : cluster [DBG] pgmap v931: 138 pgs: 138 active+clean; 883 MiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-02-22T19:47:13.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:12.054868+0000 osd.3 (osd.3) 310 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:47:13.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:12.135973+0000 osd.3 (osd.3) 311 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:47:13.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:12.315487+0000 osd.7 (osd.7) 284 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:47:13.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:12.323907+0000 osd.7 (osd.7) 285 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:47:13.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:12.473164+0000 mon.a (mon.0) 1681 : cluster [DBG] osdmap e460: 8 total, 8 up, 5 in 2024-02-22T19:47:13.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:11.778312+0000 mgr.y (mgr.14535) 693 : cluster [DBG] pgmap v931: 138 pgs: 138 active+clean; 883 MiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-02-22T19:47:13.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:12.054868+0000 osd.3 (osd.3) 310 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:47:13.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:12.135973+0000 osd.3 (osd.3) 311 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:47:13.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:12.315487+0000 osd.7 (osd.7) 284 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:47:13.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:12.323907+0000 osd.7 (osd.7) 285 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:47:13.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:12.473164+0000 mon.a (mon.0) 1681 : cluster [DBG] osdmap e460: 8 total, 8 up, 5 in 2024-02-22T19:47:13.468 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:11.778312+0000 mgr.y (mgr.14535) 693 : cluster [DBG] pgmap v931: 138 pgs: 138 active+clean; 883 MiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-02-22T19:47:13.468 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:12.054868+0000 osd.3 (osd.3) 310 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:47:13.468 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:12.135973+0000 osd.3 (osd.3) 311 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:47:13.468 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:12.315487+0000 osd.7 (osd.7) 284 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:47:13.468 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:12.323907+0000 osd.7 (osd.7) 285 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:47:13.468 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:12.473164+0000 mon.a (mon.0) 1681 : cluster [DBG] osdmap e460: 8 total, 8 up, 5 in 2024-02-22T19:47:13.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:47:13.538 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:13.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:13.796 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:13.801 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:13.801 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:13.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:13.964 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:14.335 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:47:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:47:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:47:14.335 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:14.341 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:14.352 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:14.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:14.453 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:14.531 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:14.560 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:14.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:14.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:14.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:14.724 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:15.010 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:47:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:47:15.030 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:13.701664+0000 osd.1 (osd.1) 268 : cluster [DBG] 10.2 scrub starts 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:13.778924+0000 mgr.y (mgr.14535) 694 : cluster [DBG] pgmap v933: 138 pgs: 138 active+clean; 883 MiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 764 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:13.940203+0000 osd.1 (osd.1) 269 : cluster [DBG] 10.2 scrub ok 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:14.057165+0000 osd.2 (osd.2) 268 : cluster [DBG] 3.e scrub starts 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:14.069461+0000 osd.2 (osd.2) 269 : cluster [DBG] 3.e scrub ok 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:15.040324+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.53:0/1498449255' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:15.042166+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:15.377 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:15.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:13.701664+0000 osd.1 (osd.1) 268 : cluster [DBG] 10.2 scrub starts 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:13.778924+0000 mgr.y (mgr.14535) 694 : cluster [DBG] pgmap v933: 138 pgs: 138 active+clean; 883 MiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 764 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:13.940203+0000 osd.1 (osd.1) 269 : cluster [DBG] 10.2 scrub ok 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:14.057165+0000 osd.2 (osd.2) 268 : cluster [DBG] 3.e scrub starts 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:14.069461+0000 osd.2 (osd.2) 269 : cluster [DBG] 3.e scrub ok 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:15.040324+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.53:0/1498449255' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:15.042166+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:13.701664+0000 osd.1 (osd.1) 268 : cluster [DBG] 10.2 scrub starts 2024-02-22T19:47:15.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:13.778924+0000 mgr.y (mgr.14535) 694 : cluster [DBG] pgmap v933: 138 pgs: 138 active+clean; 883 MiB data, 7.0 GiB used, 440 GiB / 447 GiB avail; 764 B/s rd, 0 B/s wr, 815 op/s 2024-02-22T19:47:15.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:13.940203+0000 osd.1 (osd.1) 269 : cluster [DBG] 10.2 scrub ok 2024-02-22T19:47:15.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:14.057165+0000 osd.2 (osd.2) 268 : cluster [DBG] 3.e scrub starts 2024-02-22T19:47:15.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:14.069461+0000 osd.2 (osd.2) 269 : cluster [DBG] 3.e scrub ok 2024-02-22T19:47:15.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:15.040324+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.53:0/1498449255' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:15.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:15.042166+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:15.515 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:15.522 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:15.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:15.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:15.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:47:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:47:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:15.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:15.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:15.716 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:47:15.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:15.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:15.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:15.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:16.031 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:16.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:14.703229+0000 osd.1 (osd.1) 270 : cluster [DBG] 3.a scrub starts 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:14.714500+0000 osd.1 (osd.1) 271 : cluster [DBG] 3.a scrub ok 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.161470+0000 mon.a (mon.0) 1683 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.161503+0000 mon.a (mon.0) 1684 : cluster [INF] Cluster is now healthy 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:15.174165+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.174250+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e461: 8 total, 8 up, 5 in 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:15.363673+0000 mgr.y (mgr.14535) 695 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.395129+0000 osd.7 (osd.7) 286 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.447421+0000 osd.7 (osd.7) 287 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:14.703229+0000 osd.1 (osd.1) 270 : cluster [DBG] 3.a scrub starts 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:14.714500+0000 osd.1 (osd.1) 271 : cluster [DBG] 3.a scrub ok 2024-02-22T19:47:16.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.161470+0000 mon.a (mon.0) 1683 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:47:16.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.161503+0000 mon.a (mon.0) 1684 : cluster [INF] Cluster is now healthy 2024-02-22T19:47:16.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:15.174165+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:47:16.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.174250+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e461: 8 total, 8 up, 5 in 2024-02-22T19:47:16.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:15.363673+0000 mgr.y (mgr.14535) 695 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:16.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.395129+0000 osd.7 (osd.7) 286 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:47:16.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.447421+0000 osd.7 (osd.7) 287 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:47:16.572 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:14.703229+0000 osd.1 (osd.1) 270 : cluster [DBG] 3.a scrub starts 2024-02-22T19:47:16.572 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:14.714500+0000 osd.1 (osd.1) 271 : cluster [DBG] 3.a scrub ok 2024-02-22T19:47:16.572 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.161470+0000 mon.a (mon.0) 1683 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:47:16.572 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.161503+0000 mon.a (mon.0) 1684 : cluster [INF] Cluster is now healthy 2024-02-22T19:47:16.572 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:15.174165+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:47:16.573 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.174250+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e461: 8 total, 8 up, 5 in 2024-02-22T19:47:16.573 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:15.363673+0000 mgr.y (mgr.14535) 695 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:16.573 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.395129+0000 osd.7 (osd.7) 286 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:47:16.573 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.447421+0000 osd.7 (osd.7) 287 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:47:17.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.546595+0000 osd.5 (osd.5) 179 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.553320+0000 osd.5 (osd.5) 180 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.724178+0000 osd.1 (osd.1) 272 : cluster [DBG] 10.b scrub starts 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.779336+0000 mgr.y (mgr.14535) 696 : cluster [DBG] pgmap v935: 138 pgs: 138 active+clean; 541 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 1.37k op/s 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:15.918624+0000 osd.1 (osd.1) 273 : cluster [DBG] 10.b scrub ok 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:16.009676+0000 osd.3 (osd.3) 312 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:16.015880+0000 osd.3 (osd.3) 313 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:16.368891+0000 osd.7 (osd.7) 288 : cluster [DBG] 3.f deep-scrub starts 2024-02-22T19:47:17.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:16.375362+0000 osd.7 (osd.7) 289 : cluster [DBG] 3.f deep-scrub ok 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.546595+0000 osd.5 (osd.5) 179 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.553320+0000 osd.5 (osd.5) 180 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.724178+0000 osd.1 (osd.1) 272 : cluster [DBG] 10.b scrub starts 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.779336+0000 mgr.y (mgr.14535) 696 : cluster [DBG] pgmap v935: 138 pgs: 138 active+clean; 541 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 1.37k op/s 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:15.918624+0000 osd.1 (osd.1) 273 : cluster [DBG] 10.b scrub ok 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:16.009676+0000 osd.3 (osd.3) 312 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:16.015880+0000 osd.3 (osd.3) 313 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:16.368891+0000 osd.7 (osd.7) 288 : cluster [DBG] 3.f deep-scrub starts 2024-02-22T19:47:17.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:16.375362+0000 osd.7 (osd.7) 289 : cluster [DBG] 3.f deep-scrub ok 2024-02-22T19:47:17.440 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:17.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:17.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:17.500 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:17.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:17.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.546595+0000 osd.5 (osd.5) 179 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:47:17.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.553320+0000 osd.5 (osd.5) 180 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:47:17.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.724178+0000 osd.1 (osd.1) 272 : cluster [DBG] 10.b scrub starts 2024-02-22T19:47:17.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.779336+0000 mgr.y (mgr.14535) 696 : cluster [DBG] pgmap v935: 138 pgs: 138 active+clean; 541 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 1.37k op/s 2024-02-22T19:47:17.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:15.918624+0000 osd.1 (osd.1) 273 : cluster [DBG] 10.b scrub ok 2024-02-22T19:47:17.580 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:16.009676+0000 osd.3 (osd.3) 312 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:47:17.581 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:16.015880+0000 osd.3 (osd.3) 313 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:47:17.581 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:16.368891+0000 osd.7 (osd.7) 288 : cluster [DBG] 3.f deep-scrub starts 2024-02-22T19:47:17.581 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:16.375362+0000 osd.7 (osd.7) 289 : cluster [DBG] 3.f deep-scrub ok 2024-02-22T19:47:17.582 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:18.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:18.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:18.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:18.325 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:17.066269+0000 osd.2 (osd.2) 270 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:47:18.326 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:17.081871+0000 osd.2 (osd.2) 271 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:47:18.326 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:17.478703+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e462: 8 total, 8 up, 5 in 2024-02-22T19:47:18.326 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:18.327 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:17.066269+0000 osd.2 (osd.2) 270 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:47:18.327 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:17.081871+0000 osd.2 (osd.2) 271 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:47:18.327 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:17.478703+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e462: 8 total, 8 up, 5 in 2024-02-22T19:47:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:17.066269+0000 osd.2 (osd.2) 270 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:47:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:17.081871+0000 osd.2 (osd.2) 271 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:47:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:17.478703+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e462: 8 total, 8 up, 5 in 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:156980 8.2 8:446b8e74:::benchmark_data_smithi053_130774_object45246:head [delete] snapc 0=[] ondisk+write+known_if_redirected e208)", 2024-02-22T19:47:18.429 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 570.20970787600004, 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064537973999999998, 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 156980 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:47:18.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217205+0000", 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217204+0000", 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6099999999999999e-06 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217208+0000", 2024-02-22T19:47:18.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4339999999999997e-06 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217215+0000", 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1739999999999999e-06 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.217220+0000", 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002322302 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219542+0000", 2024-02-22T19:47:18.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9874e-05 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219632+0000", 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9351000000000006e-05 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.219711+0000", 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012168019999999999 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.220928+0000", 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004561847 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.225490+0000", 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.056184824000000001 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281675+0000", 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2453999999999998e-05 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:18.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281707+0000", 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5172000000000001e-05 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.281743+0000", 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.064537973999999998 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:157456 8.2 8:47cb7be2:::benchmark_data_smithi053_130774_object45722:head [delete] snapc 0=[] ondisk+write+known_if_redirected e209)", 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 569.68747614999995, 2024-02-22T19:47:18.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065090921999999996, 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 157456 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739436+0000", 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739435+0000", 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4889999999999998e-06 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739443+0000", 2024-02-22T19:47:18.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6900000000000004e-06 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739453+0000", 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5810000000000002e-06 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739459+0000", 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5296000000000001e-05 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739505+0000", 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7620999999999994e-05 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739582+0000", 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6128e-05 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.739658+0000", 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022481889999999998 2024-02-22T19:47:18.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.741906+0000", 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00070410199999999996 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.742611+0000", 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061844286999999998 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804455+0000", 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6128999999999998e-05 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804501+0000", 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6259e-05 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:48.804527+0000", 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065090921999999996 2024-02-22T19:47:18.441 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:158760 8.2 8:4b5b1757:::benchmark_data_smithi053_130774_object47026:head [delete] snapc 0=[] ondisk+write+known_if_redirected e210)", 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 568.35831162500006, 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.070793354000000003, 2024-02-22T19:47:18.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 158760 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:47:18.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068601+0000", 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068599+0000", 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3269999999999997e-06 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068605+0000", 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0729e-05 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068616+0000", 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9429999999999999e-06 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068621+0000", 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4260000000000001e-05 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068655+0000", 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110695 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068766+0000", 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104616 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.068871+0000", 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019013000000000001 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.070772+0000", 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.065132173000000002 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.135904+0000", 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033978099999999998 2024-02-22T19:47:18.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139302+0000", 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1437000000000002e-05 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139353+0000", 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0763999999999998e-05 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:50.139394+0000", 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.070793354000000003 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165007 8.2 8:4b3d7df9:::benchmark_data_smithi053_130774_object53273:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 561.788311292, 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.689438121, 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:18.449 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165007 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638601+0000", 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638600+0000", 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8800000000000008e-06 2024-02-22T19:47:18.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638608+0000", 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0635999999999999e-05 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638618+0000", 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9420000000000003e-06 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638623+0000", 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5618999999999998e-05 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638669+0000", 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012975000000000001 2024-02-22T19:47:18.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638799+0000", 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015391499999999999 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638953+0000", 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040805320000000004 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.643033+0000", 2024-02-22T19:47:18.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012758660000000001 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.644309+0000", 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015138571999999999 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659448+0000", 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0815999999999999e-05 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659458+0000", 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50944174600000003 2024-02-22T19:47:18.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168900+0000", 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.302e-06 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168909+0000", 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1590766800000001 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327986+0000", 2024-02-22T19:47:18.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3182e-05 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328039+0000", 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.689438121 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165013 8.2 8:42c6cb88:::benchmark_data_smithi053_130774_object53279:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 561.7881476, 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6893490839999998, 2024-02-22T19:47:18.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165013 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638765+0000", 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638764+0000", 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9380000000000003e-06 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638771+0000", 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4669999999999993e-06 2024-02-22T19:47:18.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638780+0000", 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9320000000000001e-06 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.638786+0000", 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024245949999999999 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641211+0000", 2024-02-22T19:47:18.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011383399999999999 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641324+0000", 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000133826 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.641458+0000", 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017005779999999999 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.643159+0000", 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005364382 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.648523+0000", 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010966531999999999 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659490+0000", 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3050000000000004e-06 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659497+0000", 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50944507900000002 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168942+0000", 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4430000000000003e-06 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168951+0000", 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1591219230000001 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328073+0000", 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1409999999999998e-05 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328114+0000", 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6893490839999998 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1372 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 4] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 561.75462595199997, 2024-02-22T19:47:18.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.542875494, 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1372 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672287+0000", 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672285+0000", 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6209999999999998e-06 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672291+0000", 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0467e-05 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672302+0000", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0189999999999995e-06 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672311+0000", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3849999999999993e-05 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672375+0000", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3922e-05 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672389+0000", 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.496154551 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168543+0000", 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3271999999999999e-05 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168556+0000", 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0231974429999999 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.191754+0000", 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000199951 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.191954+0000", 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023208168000000001 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.215162+0000", 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.542875494 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1374 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 5] snapc 0=[] ondisk+write+known_if_redirected e218)", 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 561.25917670800004, 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0476472020000001, 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1374 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167736+0000", 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167734+0000", 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7470000000000005e-06 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167742+0000", 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5179999999999993e-06 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167751+0000", 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1430000000000001e-06 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.167758+0000", 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00082432800000000004 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168583+0000", 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3889999999999994e-06 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.168592+0000", 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.023928116 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.192520+0000", 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8975999999999997e-05 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.192599+0000", 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022783739000000001 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.215383+0000", 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0476472020000001 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1382 5.e 5:7759931f:::notify.3:head [watch ping cookie 94129597099392 gen 5] snapc 0=[] ondisk+write+known_if_redirected e219)", 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 559.16308720999996, 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95001531500000003, 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1382 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263825+0000", 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0230000000000001e-06 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263828+0000", 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3229999999999999e-06 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263832+0000", 2024-02-22T19:47:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2860000000000001e-06 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263836+0000", 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4635000000000005e-05 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263921+0000", 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.167e-05 2024-02-22T19:47:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263933+0000", 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94966063199999995 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213593+0000", 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000170743 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213764+0000", 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6550000000000004e-05 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.213841+0000", 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95001531500000003 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1387 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94129597099392 gen 6] snapc 0=[] ondisk+write+known_if_redirected e220)", 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 559.16306903199995, 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96457482999999999, 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1387 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.481 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:47:18.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263843+0000", 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5679999999999998e-06 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263846+0000", 2024-02-22T19:47:18.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1229999999999997e-06 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263852+0000", 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1789999999999999e-06 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263857+0000", 2024-02-22T19:47:18.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104499 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263961+0000", 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6989999999999997e-06 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:59.263968+0000", 2024-02-22T19:47:18.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95095029499999995 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.214918+0000", 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7822999999999998e-05 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.214976+0000", 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01344215 2024-02-22T19:47:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:00.228418+0000", 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96457482999999999 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:197539 8.2 8:421bd531:::benchmark_data_smithi053_130774_object85805:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:46.112931+0000", 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 512.31398110400005, 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1586733920000001, 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 197539 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112931+0000", 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112931+0000", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112930+0000", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1970000000000001e-06 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112936+0000", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0519999999999999e-05 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112947+0000", 2024-02-22T19:47:18.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5189999999999998e-06 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.112953+0000", 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2479999999999998e-05 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.113016+0000", 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5166e-05 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.113031+0000", 2024-02-22T19:47:18.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1526715759999999 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.265702+0000", 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0606000000000006e-05 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.265793+0000", 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011398199999999999 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5,6", 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.265907+0000", 2024-02-22T19:47:18.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0049431880000000003 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270850+0000", 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8307999999999998e-05 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270909+0000", 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7954000000000003e-05 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270946+0000", 2024-02-22T19:47:18.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026440200000000001 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271211+0000", 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000175093 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271386+0000", 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0007000000000003e-05 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271446+0000", 2024-02-22T19:47:18.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000158803 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271605+0000", 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1586733920000001 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:197560 8.2 8:4ef66ec7:::benchmark_data_smithi053_130774_object85826:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:46.162279+0000", 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 512.26463342900001, 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1127937880000001, 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 197560 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162279+0000", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162279+0000", 2024-02-22T19:47:18.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162277+0000", 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0739999999999999e-06 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162286+0000", 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5593000000000001e-05 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162302+0000", 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3080000000000004e-06 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162308+0000", 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9870000000000002e-05 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162358+0000", 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6190000000000006e-06 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.162368+0000", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.105709786 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.268078+0000", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146847 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.268224+0000", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000168157 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5,6", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.268393+0000", 2024-02-22T19:47:18.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002563877 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.270956+0000", 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5355999999999997e-05 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271002+0000", 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3988000000000001e-05 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271036+0000", 2024-02-22T19:47:18.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00093006199999999995 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.271966+0000", 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028799379999999999 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.274846+0000", 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8474999999999997e-05 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:18.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.274914+0000", 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015857499999999999 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.275073+0000", 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1127937880000001 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:209454 8.2 8.b21c7012 (undecoded) ondisk+write+known_if_redirected e260)", 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:39:07.009940+0000", 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 491.41697251300002, 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.21687268300000001, 2024-02-22T19:47:18.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 209454 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009940+0000", 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009940+0000", 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009938+0000", 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.024e-06 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009945+0000", 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3556e-05 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009959+0000", 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8280000000000003e-06 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.009967+0000", 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8288000000000003e-05 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.010015+0000", 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9039999999999992e-06 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.010023+0000", 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.216777631 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.226801+0000", 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2084e-05 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.226813+0000", 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.21687268300000001 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:18.506 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:47:18.663 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:47:19.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:47:19.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:47:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:17.713291+0000 osd.1 (osd.1) 274 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:47:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:17.718306+0000 osd.1 (osd.1) 275 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:47:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:17.779744+0000 mgr.y (mgr.14535) 697 : cluster [DBG] pgmap v937: 138 pgs: 138 active+clean; 541 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 913 op/s 2024-02-22T19:47:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:18.015265+0000 osd.3 (osd.3) 314 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:47:19.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:18.018178+0000 osd.3 (osd.3) 315 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:47:19.461 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:19.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:17.713291+0000 osd.1 (osd.1) 274 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:47:19.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:17.718306+0000 osd.1 (osd.1) 275 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:47:19.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:17.779744+0000 mgr.y (mgr.14535) 697 : cluster [DBG] pgmap v937: 138 pgs: 138 active+clean; 541 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 913 op/s 2024-02-22T19:47:19.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:18.015265+0000 osd.3 (osd.3) 314 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:47:19.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:18.018178+0000 osd.3 (osd.3) 315 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:47:19.507 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:19] "GET /metrics HTTP/1.1" 200 176499 "" "Prometheus/2.33.4" 2024-02-22T19:47:19.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:17.713291+0000 osd.1 (osd.1) 274 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:47:19.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:17.718306+0000 osd.1 (osd.1) 275 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:47:19.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:17.779744+0000 mgr.y (mgr.14535) 697 : cluster [DBG] pgmap v937: 138 pgs: 138 active+clean; 541 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 913 op/s 2024-02-22T19:47:19.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:18.015265+0000 osd.3 (osd.3) 314 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:47:19.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:18.018178+0000 osd.3 (osd.3) 315 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:47:19.622 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:19.633 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:19.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:19.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:19.775 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:19.800 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:19.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:19.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:19.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:18.623854+0000 mgr.y (mgr.14535) 698 : audit [DBG] from='client.25577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:47:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:19.435024+0000 osd.7 (osd.7) 290 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:47:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:19.438610+0000 osd.7 (osd.7) 291 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:47:20.434 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:20.447 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:20.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:18.623854+0000 mgr.y (mgr.14535) 698 : audit [DBG] from='client.25577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:47:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:19.435024+0000 osd.7 (osd.7) 290 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:47:20.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:19.438610+0000 osd.7 (osd.7) 291 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:47:20.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:18.623854+0000 mgr.y (mgr.14535) 698 : audit [DBG] from='client.25577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:47:20.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:19.435024+0000 osd.7 (osd.7) 290 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:47:20.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:19.438610+0000 osd.7 (osd.7) 291 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:47:20.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:20.741 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:20.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:20.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:20.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:20.879 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:20.990 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:20.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:21.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:21.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:19.781857+0000 mgr.y (mgr.14535) 699 : cluster [DBG] pgmap v938: 138 pgs: 138 active+clean; 368 MiB data, 5.4 GiB used, 442 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.13k op/s 2024-02-22T19:47:21.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:20.066681+0000 osd.2 (osd.2) 272 : cluster [DBG] 10.3 scrub starts 2024-02-22T19:47:21.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:20.159410+0000 osd.2 (osd.2) 273 : cluster [DBG] 10.3 scrub ok 2024-02-22T19:47:21.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:20.454110+0000 osd.5 (osd.5) 181 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:47:21.249 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:20.457613+0000 osd.5 (osd.5) 182 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:47:21.261 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:21.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:21.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:21.297 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:21.297 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:47:21.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:225548 10.a 10:5a8441c0:::benchmark_data_smithi053_180506_object112169:head [delete] snapc 0=[] ondisk+write+known_if_redirected e462)", 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:47:21.287635+0000", 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010046658, 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010068046000000001, 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 225548 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287635+0000", 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287635+0000", 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287634+0000", 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5949999999999998e-06 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287640+0000", 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7189999999999996e-06 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287650+0000", 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.651e-06 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287654+0000", 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5713999999999997e-05 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287690+0000", 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7469e-05 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287768+0000", 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5079000000000002e-05 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.287863+0000", 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016341680000000001 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289497+0000", 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002554579 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.292051+0000", 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044160830000000003 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:225553 10.a 10:53cace03:::benchmark_data_smithi053_180506_object112174:head [delete] snapc 0=[] ondisk+write+known_if_redirected e462)", 2024-02-22T19:47:21.302 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:47:21.289933+0000", 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0077487459999999999, 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078296649999999995, 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 225553 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:21.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289933+0000", 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289933+0000", 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289932+0000", 2024-02-22T19:47:21.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5770000000000004e-06 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289937+0000", 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7559999999999994e-06 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289947+0000", 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0010000000000001e-06 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289952+0000", 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5450000000000001e-05 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.289987+0000", 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2046999999999996e-05 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.290069+0000", 2024-02-22T19:47:21.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5895000000000004e-05 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.290145+0000", 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001490718 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.291636+0000", 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014343680000000001 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:21.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293070+0000", 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031370030000000002 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:225558 10.6 10:60bb25a5:::benchmark_data_smithi053_180506_object112179:head [delete] snapc 0=[] ondisk+write+known_if_redirected e462)", 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:47:21.293432+0000", 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042501199999999996, 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043827930000000003, 2024-02-22T19:47:21.308 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 225558 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293432+0000", 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293432+0000", 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293431+0000", 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8810000000000001e-06 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293436+0000", 2024-02-22T19:47:21.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7367000000000002e-05 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293454+0000", 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7510000000000008e-06 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293462+0000", 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.122e-05 2024-02-22T19:47:21.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293523+0000", 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011547100000000001 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293638+0000", 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013350699999999999 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.293772+0000", 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002130407 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:21.295902+0000", 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024702719999999999 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:21.313 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:47:21.314 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 3 2024-02-22T19:47:21.314 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:21.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:21.389 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":937,"stamp":"2024-02-22T19:47:17.779501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":566761148,"num_objects":8841,"num_object_clones":0,"num_object_copies":26523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8841,"num_whiteouts":0,"num_read":2197,"num_read_kb":1949,"num_write":218819,"num_write_kb":7256106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":452,"num_bytes_recovered":15766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59460,"ondisk_log_size":59460,"up":414,"acting":414,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":414,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":6282244,"kb_used_data":1815708,"kb_used_omap":0,"kb_used_meta":4466240,"kb_avail":462443516,"statfs":{"total":479975178240,"available":473542160384,"internally_reserved":0,"allocated":1859284992,"data_stored":1850674131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4573429760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,4,6,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-359137280,"num_objects":-5480,"num_object_clones":0,"num_object_copies":-16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5480,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5480,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001439"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":422,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114655+0000","last_change":"2024-02-22T19:45:56.598157+0000","last_active":"2024-02-22T19:47:10.114655+0000","last_peered":"2024-02-22T19:47:10.114655+0000","last_clean":"2024-02-22T19:47:10.114655+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:47:10.114655+0000","last_undegraded":"2024-02-22T19:47:10.114655+0000","last_fullsized":"2024-02-22T19:47:10.114655+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:56.598068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:56.598068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":452,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115590+0000","last_change":"2024-02-22T19:43:58.780165+0000","last_active":"2024-02-22T19:47:10.115590+0000","last_peered":"2024-02-22T19:47:10.115590+0000","last_clean":"2024-02-22T19:47:10.115590+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:47:10.115590+0000","last_undegraded":"2024-02-22T19:47:10.115590+0000","last_fullsized":"2024-02-22T19:47:10.115590+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.780100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.780100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":238,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117304+0000","last_change":"2024-02-22T19:46:09.901457+0000","last_active":"2024-02-22T19:47:10.117304+0000","last_peered":"2024-02-22T19:47:10.117304+0000","last_clean":"2024-02-22T19:47:10.117304+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:47:10.117304+0000","last_undegraded":"2024-02-22T19:47:10.117304+0000","last_fullsized":"2024-02-22T19:47:10.117304+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:46:09.901351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.901351+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":437,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129037+0000","last_change":"2024-02-22T19:46:14.159107+0000","last_active":"2024-02-22T19:47:10.129037+0000","last_peered":"2024-02-22T19:47:10.129037+0000","last_clean":"2024-02-22T19:47:10.129037+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:47:10.129037+0000","last_undegraded":"2024-02-22T19:47:10.129037+0000","last_fullsized":"2024-02-22T19:47:10.129037+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.159012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.159012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":438,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129113+0000","last_change":"2024-02-22T19:44:56.776815+0000","last_active":"2024-02-22T19:47:10.129113+0000","last_peered":"2024-02-22T19:47:10.129113+0000","last_clean":"2024-02-22T19:47:10.129113+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:47:10.129113+0000","last_undegraded":"2024-02-22T19:47:10.129113+0000","last_fullsized":"2024-02-22T19:47:10.129113+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:56.776721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.776721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":545,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.127310+0000","last_change":"2024-02-22T19:44:07.984802+0000","last_active":"2024-02-22T19:47:10.127310+0000","last_peered":"2024-02-22T19:47:10.127310+0000","last_clean":"2024-02-22T19:47:10.127310+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:47:10.127310+0000","last_undegraded":"2024-02-22T19:47:10.127310+0000","last_fullsized":"2024-02-22T19:47:10.127310+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:44:07.984710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.984710+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":266,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.116635+0000","last_change":"2024-02-22T19:46:13.905945+0000","last_active":"2024-02-22T19:47:10.116635+0000","last_peered":"2024-02-22T19:47:10.116635+0000","last_clean":"2024-02-22T19:47:10.116635+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:47:10.116635+0000","last_undegraded":"2024-02-22T19:47:10.116635+0000","last_fullsized":"2024-02-22T19:47:10.116635+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.905878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.905878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":275,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115031+0000","last_change":"2024-02-22T19:44:07.795043+0000","last_active":"2024-02-22T19:47:10.115031+0000","last_peered":"2024-02-22T19:47:10.115031+0000","last_clean":"2024-02-22T19:47:10.115031+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:47:10.115031+0000","last_undegraded":"2024-02-22T19:47:10.115031+0000","last_fullsized":"2024-02-22T19:47:10.115031+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:07.794943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.794943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":342,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128845+0000","last_change":"2024-02-22T19:45:02.829140+0000","last_active":"2024-02-22T19:47:10.128845+0000","last_peered":"2024-02-22T19:47:10.128845+0000","last_clean":"2024-02-22T19:47:10.128845+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:47:10.128845+0000","last_undegraded":"2024-02-22T19:47:10.128845+0000","last_fullsized":"2024-02-22T19:47:10.128845+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:45:02.829064+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:45:02.829064+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":262,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113659+0000","last_change":"2024-02-22T19:45:03.818156+0000","last_active":"2024-02-22T19:47:10.113659+0000","last_peered":"2024-02-22T19:47:10.113659+0000","last_clean":"2024-02-22T19:47:10.113659+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:47:10.113659+0000","last_undegraded":"2024-02-22T19:47:10.113659+0000","last_fullsized":"2024-02-22T19:47:10.113659+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.818051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":292,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114326+0000","last_change":"2024-02-22T19:44:44.719650+0000","last_active":"2024-02-22T19:47:10.114326+0000","last_peered":"2024-02-22T19:47:10.114326+0000","last_clean":"2024-02-22T19:47:10.114326+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:47:10.114326+0000","last_undegraded":"2024-02-22T19:47:10.114326+0000","last_fullsized":"2024-02-22T19:47:10.114326+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:44:44.719546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:44.719546+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":265,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128952+0000","last_change":"2024-02-22T19:44:04.949624+0000","last_active":"2024-02-22T19:47:10.128952+0000","last_peered":"2024-02-22T19:47:10.128952+0000","last_clean":"2024-02-22T19:47:10.128952+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:47:10.128952+0000","last_undegraded":"2024-02-22T19:47:10.128952+0000","last_fullsized":"2024-02-22T19:47:10.128952+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:44:04.949558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.949558+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":340,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115260+0000","last_change":"2024-02-22T19:46:18.486367+0000","last_active":"2024-02-22T19:47:10.115260+0000","last_peered":"2024-02-22T19:47:10.115260+0000","last_clean":"2024-02-22T19:47:10.115260+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:47:10.115260+0000","last_undegraded":"2024-02-22T19:47:10.115260+0000","last_fullsized":"2024-02-22T19:47:10.115260+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:18.486267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.486267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":243,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119255+0000","last_change":"2024-02-22T19:45:52.252022+0000","last_active":"2024-02-22T19:47:10.119255+0000","last_peered":"2024-02-22T19:47:10.119255+0000","last_clean":"2024-02-22T19:47:10.119255+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:47:10.119255+0000","last_undegraded":"2024-02-22T19:47:10.119255+0000","last_fullsized":"2024-02-22T19:47:10.119255+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.251960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.251960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":497,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114397+0000","last_change":"2024-02-22T19:46:16.806927+0000","last_active":"2024-02-22T19:47:10.114397+0000","last_peered":"2024-02-22T19:47:10.114397+0000","last_clean":"2024-02-22T19:47:10.114397+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:47:10.114397+0000","last_undegraded":"2024-02-22T19:47:10.114397+0000","last_fullsized":"2024-02-22T19:47:10.114397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:46:16.806791+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:46:16.806791+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":455,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129914+0000","last_change":"2024-02-22T19:45:14.832581+0000","last_active":"2024-02-22T19:47:10.129914+0000","last_peered":"2024-02-22T19:47:10.129914+0000","last_clean":"2024-02-22T19:47:10.129914+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:47:10.129914+0000","last_undegraded":"2024-02-22T19:47:10.129914+0000","last_fullsized":"2024-02-22T19:47:10.129914+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.832482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.832482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":252,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117394+0000","last_change":"2024-02-22T19:45:03.679204+0000","last_active":"2024-02-22T19:47:10.117394+0000","last_peered":"2024-02-22T19:47:10.117394+0000","last_clean":"2024-02-22T19:47:10.117394+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:47:10.117394+0000","last_undegraded":"2024-02-22T19:47:10.117394+0000","last_fullsized":"2024-02-22T19:47:10.117394+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.679103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":506,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.134341+0000","last_change":"2024-02-22T19:44:43.774884+0000","last_active":"2024-02-22T19:47:10.134341+0000","last_peered":"2024-02-22T19:47:10.134341+0000","last_clean":"2024-02-22T19:47:10.134341+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:47:10.134341+0000","last_undegraded":"2024-02-22T19:47:10.134341+0000","last_fullsized":"2024-02-22T19:47:10.134341+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:44:43.774778+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:44:43.774778+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":476,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119842+0000","last_change":"2024-02-22T19:47:05.294685+0000","last_active":"2024-02-22T19:47:10.119842+0000","last_peered":"2024-02-22T19:47:10.119842+0000","last_clean":"2024-02-22T19:47:10.119842+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:47:10.119842+0000","last_undegraded":"2024-02-22T19:47:10.119842+0000","last_fullsized":"2024-02-22T19:47:10.119842+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:47:05.294613+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.294613+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":460,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117239+0000","last_change":"2024-02-22T19:45:09.724649+0000","last_active":"2024-02-22T19:47:10.117239+0000","last_peered":"2024-02-22T19:47:10.117239+0000","last_clean":"2024-02-22T19:47:10.117239+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:47:10.117239+0000","last_undegraded":"2024-02-22T19:47:10.117239+0000","last_fullsized":"2024-02-22T19:47:10.117239+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:09.724557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.724557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":279,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.324065+0000","last_change":"2024-02-22T19:47:12.324026+0000","last_active":"2024-02-22T19:47:12.324065+0000","last_peered":"2024-02-22T19:47:12.324065+0000","last_clean":"2024-02-22T19:47:12.324065+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:47:12.324065+0000","last_undegraded":"2024-02-22T19:47:12.324065+0000","last_fullsized":"2024-02-22T19:47:12.324065+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:12.323915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.323915+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":460,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117282+0000","last_change":"2024-02-22T19:44:48.653802+0000","last_active":"2024-02-22T19:47:10.117282+0000","last_peered":"2024-02-22T19:47:10.117282+0000","last_clean":"2024-02-22T19:47:10.117282+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:47:10.117282+0000","last_undegraded":"2024-02-22T19:47:10.117282+0000","last_fullsized":"2024-02-22T19:47:10.117282+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.653696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.653696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":458,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.127997+0000","last_change":"2024-02-22T19:44:10.017728+0000","last_active":"2024-02-22T19:47:10.127997+0000","last_peered":"2024-02-22T19:47:10.127997+0000","last_clean":"2024-02-22T19:47:10.127997+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:47:10.127997+0000","last_undegraded":"2024-02-22T19:47:10.127997+0000","last_fullsized":"2024-02-22T19:47:10.127997+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:10.017666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:44:10.017666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":500,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115783+0000","last_change":"2024-02-22T19:45:09.809874+0000","last_active":"2024-02-22T19:47:10.115783+0000","last_peered":"2024-02-22T19:47:10.115783+0000","last_clean":"2024-02-22T19:47:10.115783+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:47:10.115783+0000","last_undegraded":"2024-02-22T19:47:10.115783+0000","last_fullsized":"2024-02-22T19:47:10.115783+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:09.809806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.809806+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":445,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119090+0000","last_change":"2024-02-22T19:47:06.262224+0000","last_active":"2024-02-22T19:47:10.119090+0000","last_peered":"2024-02-22T19:47:10.119090+0000","last_clean":"2024-02-22T19:47:10.119090+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:47:10.119090+0000","last_undegraded":"2024-02-22T19:47:10.119090+0000","last_fullsized":"2024-02-22T19:47:10.119090+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.262153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.262153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":452,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115424+0000","last_change":"2024-02-22T19:44:13.694618+0000","last_active":"2024-02-22T19:47:10.115424+0000","last_peered":"2024-02-22T19:47:10.115424+0000","last_clean":"2024-02-22T19:47:10.115424+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:47:10.115424+0000","last_undegraded":"2024-02-22T19:47:10.115424+0000","last_fullsized":"2024-02-22T19:47:10.115424+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.694552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.694552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":459,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128042+0000","last_change":"2024-02-22T19:46:09.018391+0000","last_active":"2024-02-22T19:47:10.128042+0000","last_peered":"2024-02-22T19:47:10.128042+0000","last_clean":"2024-02-22T19:47:10.128042+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:47:10.128042+0000","last_undegraded":"2024-02-22T19:47:10.128042+0000","last_fullsized":"2024-02-22T19:47:10.128042+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:46:09.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.018318+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":466,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.136119+0000","last_change":"2024-02-22T19:47:12.136077+0000","last_active":"2024-02-22T19:47:12.136119+0000","last_peered":"2024-02-22T19:47:12.136119+0000","last_clean":"2024-02-22T19:47:12.136119+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:47:12.136119+0000","last_undegraded":"2024-02-22T19:47:12.136119+0000","last_fullsized":"2024-02-22T19:47:12.136119+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:12.135980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.135980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":257,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115215+0000","last_change":"2024-02-22T19:44:12.674961+0000","last_active":"2024-02-22T19:47:10.115215+0000","last_peered":"2024-02-22T19:47:10.115215+0000","last_clean":"2024-02-22T19:47:10.115215+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:47:10.115215+0000","last_undegraded":"2024-02-22T19:47:10.115215+0000","last_fullsized":"2024-02-22T19:47:10.115215+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.674856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.674856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":479,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.127137+0000","last_change":"2024-02-22T19:44:11.041545+0000","last_active":"2024-02-22T19:47:10.127137+0000","last_peered":"2024-02-22T19:47:10.127137+0000","last_clean":"2024-02-22T19:47:10.127137+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:47:10.127137+0000","last_undegraded":"2024-02-22T19:47:10.127137+0000","last_fullsized":"2024-02-22T19:47:10.127137+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:44:11.041458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.041458+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":443,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113997+0000","last_change":"2024-02-22T19:44:14.661384+0000","last_active":"2024-02-22T19:47:10.113997+0000","last_peered":"2024-02-22T19:47:10.113997+0000","last_clean":"2024-02-22T19:47:10.113997+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:47:10.113997+0000","last_undegraded":"2024-02-22T19:47:10.113997+0000","last_fullsized":"2024-02-22T19:47:10.113997+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:14.661322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:14.661322+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":480,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119307+0000","last_change":"2024-02-22T19:46:20.359190+0000","last_active":"2024-02-22T19:47:10.119307+0000","last_peered":"2024-02-22T19:47:10.119307+0000","last_clean":"2024-02-22T19:47:10.119307+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:47:10.119307+0000","last_undegraded":"2024-02-22T19:47:10.119307+0000","last_fullsized":"2024-02-22T19:47:10.119307+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.359130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.359130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":484,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117935+0000","last_change":"2024-02-22T19:45:53.878055+0000","last_active":"2024-02-22T19:47:10.117935+0000","last_peered":"2024-02-22T19:47:10.117935+0000","last_clean":"2024-02-22T19:47:10.117935+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:47:10.117935+0000","last_undegraded":"2024-02-22T19:47:10.117935+0000","last_fullsized":"2024-02-22T19:47:10.117935+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:45:53.877957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:53.877957+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":473,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.126783+0000","last_change":"2024-02-22T19:44:52.796416+0000","last_active":"2024-02-22T19:47:10.126783+0000","last_peered":"2024-02-22T19:47:10.126783+0000","last_clean":"2024-02-22T19:47:10.126783+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:47:10.126783+0000","last_undegraded":"2024-02-22T19:47:10.126783+0000","last_fullsized":"2024-02-22T19:47:10.126783+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:52.796310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.796310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":245,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119746+0000","last_change":"2024-02-22T19:45:04.416774+0000","last_active":"2024-02-22T19:47:10.119746+0000","last_peered":"2024-02-22T19:47:10.119746+0000","last_clean":"2024-02-22T19:47:10.119746+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:47:10.119746+0000","last_undegraded":"2024-02-22T19:47:10.119746+0000","last_fullsized":"2024-02-22T19:47:10.119746+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:04.416671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:04.416671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":247,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118586+0000","last_change":"2024-02-22T19:45:57.920895+0000","last_active":"2024-02-22T19:47:10.118586+0000","last_peered":"2024-02-22T19:47:10.118586+0000","last_clean":"2024-02-22T19:47:10.118586+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:47:10.118586+0000","last_undegraded":"2024-02-22T19:47:10.118586+0000","last_fullsized":"2024-02-22T19:47:10.118586+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:57.920791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:57.920791+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":260,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119364+0000","last_change":"2024-02-22T19:44:41.261006+0000","last_active":"2024-02-22T19:47:10.119364+0000","last_peered":"2024-02-22T19:47:10.119364+0000","last_clean":"2024-02-22T19:47:10.119364+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:47:10.119364+0000","last_undegraded":"2024-02-22T19:47:10.119364+0000","last_fullsized":"2024-02-22T19:47:10.119364+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:41.260945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:41.260945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":453,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119176+0000","last_change":"2024-02-22T19:45:09.368245+0000","last_active":"2024-02-22T19:47:10.119176+0000","last_peered":"2024-02-22T19:47:10.119176+0000","last_clean":"2024-02-22T19:47:10.119176+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:47:10.119176+0000","last_undegraded":"2024-02-22T19:47:10.119176+0000","last_fullsized":"2024-02-22T19:47:10.119176+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:09.368133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.368133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":467,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.131481+0000","last_change":"2024-02-22T19:46:10.060876+0000","last_active":"2024-02-22T19:47:10.131481+0000","last_peered":"2024-02-22T19:47:10.131481+0000","last_clean":"2024-02-22T19:47:10.131481+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:47:10.131481+0000","last_undegraded":"2024-02-22T19:47:10.131481+0000","last_fullsized":"2024-02-22T19:47:10.131481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:46:10.060783+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:46:10.060783+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":450,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115637+0000","last_change":"2024-02-22T19:47:08.073243+0000","last_active":"2024-02-22T19:47:10.115637+0000","last_peered":"2024-02-22T19:47:10.115637+0000","last_clean":"2024-02-22T19:47:10.115637+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:47:10.115637+0000","last_undegraded":"2024-02-22T19:47:10.115637+0000","last_fullsized":"2024-02-22T19:47:10.115637+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:08.073147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:47:08.073147+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":251,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118920+0000","last_change":"2024-02-22T19:46:17.328336+0000","last_active":"2024-02-22T19:47:10.118920+0000","last_peered":"2024-02-22T19:47:10.118920+0000","last_clean":"2024-02-22T19:47:10.118920+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:47:10.118920+0000","last_undegraded":"2024-02-22T19:47:10.118920+0000","last_fullsized":"2024-02-22T19:47:10.118920+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.328243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":281,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114932+0000","last_change":"2024-02-22T19:47:07.646871+0000","last_active":"2024-02-22T19:47:10.114932+0000","last_peered":"2024-02-22T19:47:10.114932+0000","last_clean":"2024-02-22T19:47:10.114932+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:47:10.114932+0000","last_undegraded":"2024-02-22T19:47:10.114932+0000","last_fullsized":"2024-02-22T19:47:10.114932+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:07.646805+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:47:07.646805+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":473,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114581+0000","last_change":"2024-02-22T19:47:05.985304+0000","last_active":"2024-02-22T19:47:10.114581+0000","last_peered":"2024-02-22T19:47:10.114581+0000","last_clean":"2024-02-22T19:47:10.114581+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:47:10.114581+0000","last_undegraded":"2024-02-22T19:47:10.114581+0000","last_fullsized":"2024-02-22T19:47:10.114581+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:05.985178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.985178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":480,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118784+0000","last_change":"2024-02-22T19:46:11.272398+0000","last_active":"2024-02-22T19:47:10.118784+0000","last_peered":"2024-02-22T19:47:10.118784+0000","last_clean":"2024-02-22T19:47:10.118784+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:47:10.118784+0000","last_undegraded":"2024-02-22T19:47:10.118784+0000","last_fullsized":"2024-02-22T19:47:10.118784+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.272326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.272326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":253,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129273+0000","last_change":"2024-02-22T19:44:47.735006+0000","last_active":"2024-02-22T19:47:10.129273+0000","last_peered":"2024-02-22T19:47:10.129273+0000","last_clean":"2024-02-22T19:47:10.129273+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:47:10.129273+0000","last_undegraded":"2024-02-22T19:47:10.129273+0000","last_fullsized":"2024-02-22T19:47:10.129273+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:44:47.734940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.734940+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":480,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119211+0000","last_change":"2024-02-22T19:47:03.314628+0000","last_active":"2024-02-22T19:47:10.119211+0000","last_peered":"2024-02-22T19:47:10.119211+0000","last_clean":"2024-02-22T19:47:10.119211+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:47:10.119211+0000","last_undegraded":"2024-02-22T19:47:10.119211+0000","last_fullsized":"2024-02-22T19:47:10.119211+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.314522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.314522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":429,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113826+0000","last_change":"2024-02-22T19:44:58.799472+0000","last_active":"2024-02-22T19:47:10.113826+0000","last_peered":"2024-02-22T19:47:10.113826+0000","last_clean":"2024-02-22T19:47:10.113826+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:47:10.113826+0000","last_undegraded":"2024-02-22T19:47:10.113826+0000","last_fullsized":"2024-02-22T19:47:10.113826+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:58.799397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:44:58.799397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":467,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129958+0000","last_change":"2024-02-22T19:45:01.840336+0000","last_active":"2024-02-22T19:47:10.129958+0000","last_peered":"2024-02-22T19:47:10.129958+0000","last_clean":"2024-02-22T19:47:10.129958+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:47:10.129958+0000","last_undegraded":"2024-02-22T19:47:10.129958+0000","last_fullsized":"2024-02-22T19:47:10.129958+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:45:01.840227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.840227+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"460'13709","reported_seq":13810,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.477098+0000","last_change":"2024-02-22T19:43:57.347864+0000","last_active":"2024-02-22T19:47:14.477098+0000","last_peered":"2024-02-22T19:47:14.477098+0000","last_clean":"2024-02-22T19:47:14.477098+0000","last_became_active":"2024-02-22T19:43:57.347537+0000","last_became_peered":"2024-02-22T19:43:57.347537+0000","last_unstale":"2024-02-22T19:47:14.477098+0000","last_undegraded":"2024-02-22T19:47:14.477098+0000","last_fullsized":"2024-02-22T19:47:14.477098+0000","mapping_epoch":365,"log_start":"445'10200","ondisk_log_start":"445'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3509,"ondisk_log_size":3509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34406400,"num_objects":525,"num_object_clones":0,"num_object_copies":1575,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13709,"num_write_kb":455488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":298,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114569+0000","last_change":"2024-02-22T19:45:52.702554+0000","last_active":"2024-02-22T19:47:10.114569+0000","last_peered":"2024-02-22T19:47:10.114569+0000","last_clean":"2024-02-22T19:47:10.114569+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:47:10.114569+0000","last_undegraded":"2024-02-22T19:47:10.114569+0000","last_fullsized":"2024-02-22T19:47:10.114569+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.702473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":525,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119313+0000","last_change":"2024-02-22T19:44:08.271084+0000","last_active":"2024-02-22T19:47:10.119313+0000","last_peered":"2024-02-22T19:47:10.119313+0000","last_clean":"2024-02-22T19:47:10.119313+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:47:10.119313+0000","last_undegraded":"2024-02-22T19:47:10.119313+0000","last_fullsized":"2024-02-22T19:47:10.119313+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:44:08.271004+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:30:17.547567+0000","last_clean_scrub_stamp":"2024-02-22T19:44:08.271004+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":321,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113694+0000","last_change":"2024-02-22T19:46:03.485142+0000","last_active":"2024-02-22T19:47:10.113694+0000","last_peered":"2024-02-22T19:47:10.113694+0000","last_clean":"2024-02-22T19:47:10.113694+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:47:10.113694+0000","last_undegraded":"2024-02-22T19:47:10.113694+0000","last_fullsized":"2024-02-22T19:47:10.113694+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:03.485065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:03.485065+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":734,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.477495+0000","last_change":"2024-02-22T19:46:15.312716+0000","last_active":"2024-02-22T19:47:12.477495+0000","last_peered":"2024-02-22T19:47:12.477495+0000","last_clean":"2024-02-22T19:47:12.477495+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:47:12.477495+0000","last_undegraded":"2024-02-22T19:47:12.477495+0000","last_fullsized":"2024-02-22T19:47:12.477495+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'16","last_scrub_stamp":"2024-02-22T19:46:15.312651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.312651+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"460'13745","reported_seq":13853,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.923317+0000","last_change":"2024-02-22T19:47:11.595137+0000","last_active":"2024-02-22T19:47:14.923317+0000","last_peered":"2024-02-22T19:47:14.923317+0000","last_clean":"2024-02-22T19:47:14.923317+0000","last_became_active":"2024-02-22T19:43:57.350467+0000","last_became_peered":"2024-02-22T19:43:57.350467+0000","last_unstale":"2024-02-22T19:47:14.923317+0000","last_undegraded":"2024-02-22T19:47:14.923317+0000","last_fullsized":"2024-02-22T19:47:14.923317+0000","mapping_epoch":365,"log_start":"445'10400","ondisk_log_start":"445'10400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"459'13519","last_scrub_stamp":"2024-02-22T19:47:11.595037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:11.595037+0000","log_size":3345,"ondisk_log_size":3345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32833536,"num_objects":501,"num_object_clones":0,"num_object_copies":1503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":455872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":423,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129119+0000","last_change":"2024-02-22T19:44:57.962949+0000","last_active":"2024-02-22T19:47:10.129119+0000","last_peered":"2024-02-22T19:47:10.129119+0000","last_clean":"2024-02-22T19:47:10.129119+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:47:10.129119+0000","last_undegraded":"2024-02-22T19:47:10.129119+0000","last_fullsized":"2024-02-22T19:47:10.129119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.962855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.962855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":172,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.069558+0000","last_change":"2024-02-22T19:47:14.069558+0000","last_active":"2024-02-22T19:47:14.069558+0000","last_peered":"2024-02-22T19:47:14.069558+0000","last_clean":"2024-02-22T19:47:14.069558+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:47:14.069558+0000","last_undegraded":"2024-02-22T19:47:14.069558+0000","last_fullsized":"2024-02-22T19:47:14.069558+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.069465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.069465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":311,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117005+0000","last_change":"2024-02-22T19:45:08.766029+0000","last_active":"2024-02-22T19:47:10.117005+0000","last_peered":"2024-02-22T19:47:10.117005+0000","last_clean":"2024-02-22T19:47:10.117005+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:47:10.117005+0000","last_undegraded":"2024-02-22T19:47:10.117005+0000","last_fullsized":"2024-02-22T19:47:10.117005+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:45:08.765921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:08.765921+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":488,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118019+0000","last_change":"2024-02-22T19:47:09.025632+0000","last_active":"2024-02-22T19:47:10.118019+0000","last_peered":"2024-02-22T19:47:10.118019+0000","last_clean":"2024-02-22T19:47:10.118019+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:47:10.118019+0000","last_undegraded":"2024-02-22T19:47:10.118019+0000","last_fullsized":"2024-02-22T19:47:10.118019+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:09.025536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.025536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"460'13476","reported_seq":13587,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.923813+0000","last_change":"2024-02-22T19:47:00.151647+0000","last_active":"2024-02-22T19:47:14.923813+0000","last_peered":"2024-02-22T19:47:14.923813+0000","last_clean":"2024-02-22T19:47:14.923813+0000","last_became_active":"2024-02-22T19:43:57.349636+0000","last_became_peered":"2024-02-22T19:43:57.349636+0000","last_unstale":"2024-02-22T19:47:14.923813+0000","last_undegraded":"2024-02-22T19:47:14.923813+0000","last_fullsized":"2024-02-22T19:47:14.923813+0000","mapping_epoch":365,"log_start":"445'10100","ondisk_log_start":"445'10100","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12423","last_scrub_stamp":"2024-02-22T19:47:00.151543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.151543+0000","log_size":3376,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36569088,"num_objects":558,"num_object_clones":0,"num_object_copies":1674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":449088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":429,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118612+0000","last_change":"2024-02-22T19:45:55.202582+0000","last_active":"2024-02-22T19:47:10.118612+0000","last_peered":"2024-02-22T19:47:10.118612+0000","last_clean":"2024-02-22T19:47:10.118612+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:47:10.118612+0000","last_undegraded":"2024-02-22T19:47:10.118612+0000","last_fullsized":"2024-02-22T19:47:10.118612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.202515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.202515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":514,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.116125+0000","last_change":"2024-02-22T19:47:06.968685+0000","last_active":"2024-02-22T19:47:10.116125+0000","last_peered":"2024-02-22T19:47:10.116125+0000","last_clean":"2024-02-22T19:47:10.116125+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:47:10.116125+0000","last_undegraded":"2024-02-22T19:47:10.116125+0000","last_fullsized":"2024-02-22T19:47:10.116125+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.968586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.968586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":467,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119496+0000","last_change":"2024-02-22T19:46:14.318040+0000","last_active":"2024-02-22T19:47:10.119496+0000","last_peered":"2024-02-22T19:47:10.119496+0000","last_clean":"2024-02-22T19:47:10.119496+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:47:10.119496+0000","last_undegraded":"2024-02-22T19:47:10.119496+0000","last_fullsized":"2024-02-22T19:47:10.119496+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.317978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.317978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":458,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117708+0000","last_change":"2024-02-22T19:44:19.749503+0000","last_active":"2024-02-22T19:47:10.117708+0000","last_peered":"2024-02-22T19:47:10.117708+0000","last_clean":"2024-02-22T19:47:10.117708+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:47:10.117708+0000","last_undegraded":"2024-02-22T19:47:10.117708+0000","last_fullsized":"2024-02-22T19:47:10.117708+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:19.749430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.749430+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"460'13417","reported_seq":13525,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.710143+0000","last_change":"2024-02-22T19:46:06.485063+0000","last_active":"2024-02-22T19:47:14.710143+0000","last_peered":"2024-02-22T19:47:14.710143+0000","last_clean":"2024-02-22T19:47:14.710143+0000","last_became_active":"2024-02-22T19:43:57.347657+0000","last_became_peered":"2024-02-22T19:43:57.347657+0000","last_unstale":"2024-02-22T19:47:14.710143+0000","last_undegraded":"2024-02-22T19:47:14.710143+0000","last_fullsized":"2024-02-22T19:47:14.710143+0000","mapping_epoch":365,"log_start":"444'9600","ondisk_log_start":"444'9600","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'8844","last_scrub_stamp":"2024-02-22T19:46:06.484965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:46:06.484965+0000","log_size":3817,"ondisk_log_size":3817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33357824,"num_objects":509,"num_object_clones":0,"num_object_copies":1527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13417,"num_write_kb":445632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":251,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118991+0000","last_change":"2024-02-22T19:44:57.360296+0000","last_active":"2024-02-22T19:47:10.118991+0000","last_peered":"2024-02-22T19:47:10.118991+0000","last_clean":"2024-02-22T19:47:10.118991+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:47:10.118991+0000","last_undegraded":"2024-02-22T19:47:10.118991+0000","last_fullsized":"2024-02-22T19:47:10.118991+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.360215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.360215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":493,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113757+0000","last_change":"2024-02-22T19:47:00.864299+0000","last_active":"2024-02-22T19:47:10.113757+0000","last_peered":"2024-02-22T19:47:10.113757+0000","last_clean":"2024-02-22T19:47:10.113757+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:47:10.113757+0000","last_undegraded":"2024-02-22T19:47:10.113757+0000","last_fullsized":"2024-02-22T19:47:10.113757+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:00.864208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.864208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":275,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114237+0000","last_change":"2024-02-22T19:47:03.617591+0000","last_active":"2024-02-22T19:47:10.114237+0000","last_peered":"2024-02-22T19:47:10.114237+0000","last_clean":"2024-02-22T19:47:10.114237+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:47:10.114237+0000","last_undegraded":"2024-02-22T19:47:10.114237+0000","last_fullsized":"2024-02-22T19:47:10.114237+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.617526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.617526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":268,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.116320+0000","last_change":"2024-02-22T19:46:11.826808+0000","last_active":"2024-02-22T19:47:10.116320+0000","last_peered":"2024-02-22T19:47:10.116320+0000","last_clean":"2024-02-22T19:47:10.116320+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:47:10.116320+0000","last_undegraded":"2024-02-22T19:47:10.116320+0000","last_fullsized":"2024-02-22T19:47:10.116320+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.826712+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"460'13755","reported_seq":13858,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.472710+0000","last_change":"2024-02-22T19:43:57.351947+0000","last_active":"2024-02-22T19:47:14.472710+0000","last_peered":"2024-02-22T19:47:14.472710+0000","last_clean":"2024-02-22T19:47:14.472710+0000","last_became_active":"2024-02-22T19:43:57.351647+0000","last_became_peered":"2024-02-22T19:43:57.351647+0000","last_unstale":"2024-02-22T19:47:14.472710+0000","last_undegraded":"2024-02-22T19:47:14.472710+0000","last_fullsized":"2024-02-22T19:47:14.472710+0000","mapping_epoch":365,"log_start":"444'10200","ondisk_log_start":"444'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3555,"ondisk_log_size":3555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37289984,"num_objects":569,"num_object_clones":0,"num_object_copies":1707,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13755,"num_write_kb":458368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":300,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113613+0000","last_change":"2024-02-22T19:46:11.428955+0000","last_active":"2024-02-22T19:47:10.113613+0000","last_peered":"2024-02-22T19:47:10.113613+0000","last_clean":"2024-02-22T19:47:10.113613+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:47:10.113613+0000","last_undegraded":"2024-02-22T19:47:10.113613+0000","last_fullsized":"2024-02-22T19:47:10.113613+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.428898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.428898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":446,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129362+0000","last_change":"2024-02-22T19:46:13.030959+0000","last_active":"2024-02-22T19:47:10.129362+0000","last_peered":"2024-02-22T19:47:10.129362+0000","last_clean":"2024-02-22T19:47:10.129362+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:47:10.129362+0000","last_undegraded":"2024-02-22T19:47:10.129362+0000","last_fullsized":"2024-02-22T19:47:10.129362+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.030866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.030866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":266,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114475+0000","last_change":"2024-02-22T19:43:57.722289+0000","last_active":"2024-02-22T19:47:10.114475+0000","last_peered":"2024-02-22T19:47:10.114475+0000","last_clean":"2024-02-22T19:47:10.114475+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:47:10.114475+0000","last_undegraded":"2024-02-22T19:47:10.114475+0000","last_fullsized":"2024-02-22T19:47:10.114475+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.722226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.722226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":381,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118184+0000","last_change":"2024-02-22T19:44:16.734407+0000","last_active":"2024-02-22T19:47:10.118184+0000","last_peered":"2024-02-22T19:47:10.118184+0000","last_clean":"2024-02-22T19:47:10.118184+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:47:10.118184+0000","last_undegraded":"2024-02-22T19:47:10.118184+0000","last_fullsized":"2024-02-22T19:47:10.118184+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:16.734314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:16.734314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":732,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118529+0000","last_change":"2024-02-22T19:46:14.876056+0000","last_active":"2024-02-22T19:47:10.118529+0000","last_peered":"2024-02-22T19:47:10.118529+0000","last_clean":"2024-02-22T19:47:10.118529+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:47:10.118529+0000","last_undegraded":"2024-02-22T19:47:10.118529+0000","last_fullsized":"2024-02-22T19:47:10.118529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:46:14.875950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.875950+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"419'20","reported_seq":365,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117189+0000","last_change":"2024-02-22T19:45:05.413797+0000","last_active":"2024-02-22T19:47:10.117189+0000","last_peered":"2024-02-22T19:47:10.117189+0000","last_clean":"2024-02-22T19:47:10.117189+0000","last_became_active":"2024-02-22T19:45:05.413407+0000","last_became_peered":"2024-02-22T19:45:05.413407+0000","last_unstale":"2024-02-22T19:47:10.117189+0000","last_undegraded":"2024-02-22T19:47:10.117189+0000","last_fullsized":"2024-02-22T19:47:10.117189+0000","mapping_epoch":322,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.782332+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"460'13781","reported_seq":13889,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.451109+0000","last_change":"2024-02-22T19:47:09.916480+0000","last_active":"2024-02-22T19:47:14.451109+0000","last_peered":"2024-02-22T19:47:14.451109+0000","last_clean":"2024-02-22T19:47:14.451109+0000","last_became_active":"2024-02-22T19:43:57.346743+0000","last_became_peered":"2024-02-22T19:43:57.346743+0000","last_unstale":"2024-02-22T19:47:14.451109+0000","last_undegraded":"2024-02-22T19:47:14.451109+0000","last_fullsized":"2024-02-22T19:47:14.451109+0000","mapping_epoch":365,"log_start":"444'10200","ondisk_log_start":"444'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"458'13446","last_scrub_stamp":"2024-02-22T19:47:09.916392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.916392+0000","log_size":3581,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37158912,"num_objects":567,"num_object_clones":0,"num_object_copies":1701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13781,"num_write_kb":459136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":488,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119533+0000","last_change":"2024-02-22T19:44:44.268687+0000","last_active":"2024-02-22T19:47:10.119533+0000","last_peered":"2024-02-22T19:47:10.119533+0000","last_clean":"2024-02-22T19:47:10.119533+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:47:10.119533+0000","last_undegraded":"2024-02-22T19:47:10.119533+0000","last_fullsized":"2024-02-22T19:47:10.119533+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:44.268593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:44.268593+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":438,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119718+0000","last_change":"2024-02-22T19:44:12.295154+0000","last_active":"2024-02-22T19:47:10.119718+0000","last_peered":"2024-02-22T19:47:10.119718+0000","last_clean":"2024-02-22T19:47:10.119718+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:47:10.119718+0000","last_undegraded":"2024-02-22T19:47:10.119718+0000","last_fullsized":"2024-02-22T19:47:10.119718+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.295048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.295048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":451,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117048+0000","last_change":"2024-02-22T19:46:58.646300+0000","last_active":"2024-02-22T19:47:10.117048+0000","last_peered":"2024-02-22T19:47:10.117048+0000","last_clean":"2024-02-22T19:47:10.117048+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:47:10.117048+0000","last_undegraded":"2024-02-22T19:47:10.117048+0000","last_fullsized":"2024-02-22T19:47:10.117048+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:58.646210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.646210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":255,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118948+0000","last_change":"2024-02-22T19:45:08.329467+0000","last_active":"2024-02-22T19:47:10.118948+0000","last_peered":"2024-02-22T19:47:10.118948+0000","last_clean":"2024-02-22T19:47:10.118948+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:47:10.118948+0000","last_undegraded":"2024-02-22T19:47:10.118948+0000","last_fullsized":"2024-02-22T19:47:10.118948+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:08.329372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:45:08.329372+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":544,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115714+0000","last_change":"2024-02-22T19:46:58.874444+0000","last_active":"2024-02-22T19:47:10.115714+0000","last_peered":"2024-02-22T19:47:10.115714+0000","last_clean":"2024-02-22T19:47:10.115714+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:47:10.115714+0000","last_undegraded":"2024-02-22T19:47:10.115714+0000","last_fullsized":"2024-02-22T19:47:10.115714+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:46:58.874350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.874350+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"415'30","reported_seq":533,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.476325+0000","last_change":"2024-02-22T19:47:10.006289+0000","last_active":"2024-02-22T19:47:12.476325+0000","last_peered":"2024-02-22T19:47:12.476325+0000","last_clean":"2024-02-22T19:47:12.476325+0000","last_became_active":"2024-02-22T19:45:01.732679+0000","last_became_peered":"2024-02-22T19:45:01.732679+0000","last_unstale":"2024-02-22T19:47:12.476325+0000","last_undegraded":"2024-02-22T19:47:12.476325+0000","last_fullsized":"2024-02-22T19:47:12.476325+0000","mapping_epoch":217,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":415,"parent":"0.0","parent_split_bits":0,"last_scrub":"415'30","last_scrub_stamp":"2024-02-22T19:47:10.006180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.006180+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"460'13959","reported_seq":14062,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.261605+0000","last_change":"2024-02-22T19:43:57.344606+0000","last_active":"2024-02-22T19:47:14.261605+0000","last_peered":"2024-02-22T19:47:14.261605+0000","last_clean":"2024-02-22T19:47:14.261605+0000","last_became_active":"2024-02-22T19:43:57.344303+0000","last_became_peered":"2024-02-22T19:43:57.344303+0000","last_unstale":"2024-02-22T19:47:14.261605+0000","last_undegraded":"2024-02-22T19:47:14.261605+0000","last_fullsized":"2024-02-22T19:47:14.261605+0000","mapping_epoch":365,"log_start":"444'10100","ondisk_log_start":"444'10100","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3859,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36241408,"num_objects":553,"num_object_clones":0,"num_object_copies":1659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13959,"num_write_kb":464384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":271,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115862+0000","last_change":"2024-02-22T19:44:17.656581+0000","last_active":"2024-02-22T19:47:10.115862+0000","last_peered":"2024-02-22T19:47:10.115862+0000","last_clean":"2024-02-22T19:47:10.115862+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:47:10.115862+0000","last_undegraded":"2024-02-22T19:47:10.115862+0000","last_fullsized":"2024-02-22T19:47:10.115862+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.656518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":488,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118706+0000","last_change":"2024-02-22T19:44:49.368004+0000","last_active":"2024-02-22T19:47:10.118706+0000","last_peered":"2024-02-22T19:47:10.118706+0000","last_clean":"2024-02-22T19:47:10.118706+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:47:10.118706+0000","last_undegraded":"2024-02-22T19:47:10.118706+0000","last_fullsized":"2024-02-22T19:47:10.118706+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:49.367906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:49.367906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":456,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128700+0000","last_change":"2024-02-22T19:44:09.027439+0000","last_active":"2024-02-22T19:47:10.128700+0000","last_peered":"2024-02-22T19:47:10.128700+0000","last_clean":"2024-02-22T19:47:10.128700+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:47:10.128700+0000","last_undegraded":"2024-02-22T19:47:10.128700+0000","last_fullsized":"2024-02-22T19:47:10.128700+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.027320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":496,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119405+0000","last_change":"2024-02-22T19:46:12.320420+0000","last_active":"2024-02-22T19:47:10.119405+0000","last_peered":"2024-02-22T19:47:10.119405+0000","last_clean":"2024-02-22T19:47:10.119405+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:47:10.119405+0000","last_undegraded":"2024-02-22T19:47:10.119405+0000","last_fullsized":"2024-02-22T19:47:10.119405+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:46:12.320306+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.320306+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"423'13","reported_seq":519,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.478698+0000","last_change":"2024-02-22T19:45:09.548942+0000","last_active":"2024-02-22T19:47:12.478698+0000","last_peered":"2024-02-22T19:47:12.478698+0000","last_clean":"2024-02-22T19:47:12.478698+0000","last_became_active":"2024-02-22T19:45:09.547565+0000","last_became_peered":"2024-02-22T19:45:09.547565+0000","last_unstale":"2024-02-22T19:47:12.478698+0000","last_undegraded":"2024-02-22T19:47:12.478698+0000","last_fullsized":"2024-02-22T19:47:12.478698+0000","mapping_epoch":217,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'11","last_scrub_stamp":"2024-02-22T19:44:52.364238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.364238+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"460'13515","reported_seq":13623,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.747200+0000","last_change":"2024-02-22T19:46:08.745923+0000","last_active":"2024-02-22T19:47:14.747200+0000","last_peered":"2024-02-22T19:47:14.747200+0000","last_clean":"2024-02-22T19:47:14.747200+0000","last_became_active":"2024-02-22T19:43:57.346554+0000","last_became_peered":"2024-02-22T19:43:57.346554+0000","last_unstale":"2024-02-22T19:47:14.747200+0000","last_undegraded":"2024-02-22T19:47:14.747200+0000","last_fullsized":"2024-02-22T19:47:14.747200+0000","mapping_epoch":365,"log_start":"444'9700","ondisk_log_start":"444'9700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'8994","last_scrub_stamp":"2024-02-22T19:46:08.745819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:46:08.745819+0000","log_size":3815,"ondisk_log_size":3815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33226752,"num_objects":507,"num_object_clones":0,"num_object_copies":1521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13515,"num_write_kb":448704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":342,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118375+0000","last_change":"2024-02-22T19:44:55.655563+0000","last_active":"2024-02-22T19:47:10.118375+0000","last_peered":"2024-02-22T19:47:10.118375+0000","last_clean":"2024-02-22T19:47:10.118375+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:47:10.118375+0000","last_undegraded":"2024-02-22T19:47:10.118375+0000","last_fullsized":"2024-02-22T19:47:10.118375+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:55.655474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:55.655474+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":454,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118157+0000","last_change":"2024-02-22T19:46:17.792837+0000","last_active":"2024-02-22T19:47:10.118157+0000","last_peered":"2024-02-22T19:47:10.118157+0000","last_clean":"2024-02-22T19:47:10.118157+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:47:10.118157+0000","last_undegraded":"2024-02-22T19:47:10.118157+0000","last_fullsized":"2024-02-22T19:47:10.118157+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.792736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.792736+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":239,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.126906+0000","last_change":"2024-02-22T19:44:15.990043+0000","last_active":"2024-02-22T19:47:10.126906+0000","last_peered":"2024-02-22T19:47:10.126906+0000","last_clean":"2024-02-22T19:47:10.126906+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:47:10.126906+0000","last_undegraded":"2024-02-22T19:47:10.126906+0000","last_fullsized":"2024-02-22T19:47:10.126906+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.989937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.989937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1568,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:13.139169+0000","last_change":"2024-02-22T19:44:19.273347+0000","last_active":"2024-02-22T19:47:13.139169+0000","last_peered":"2024-02-22T19:47:13.139169+0000","last_clean":"2024-02-22T19:47:13.139169+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:47:13.139169+0000","last_undegraded":"2024-02-22T19:47:13.139169+0000","last_fullsized":"2024-02-22T19:47:13.139169+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:44:19.273191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.273191+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1076,"num_read_kb":1071,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":271,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117777+0000","last_change":"2024-02-22T19:44:56.653960+0000","last_active":"2024-02-22T19:47:10.117777+0000","last_peered":"2024-02-22T19:47:10.117777+0000","last_clean":"2024-02-22T19:47:10.117777+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:47:10.117777+0000","last_undegraded":"2024-02-22T19:47:10.117777+0000","last_fullsized":"2024-02-22T19:47:10.117777+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:44:56.653848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.653848+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"412'25","reported_seq":363,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.479528+0000","last_change":"2024-02-22T19:45:06.780716+0000","last_active":"2024-02-22T19:47:12.479528+0000","last_peered":"2024-02-22T19:47:12.479528+0000","last_clean":"2024-02-22T19:47:12.479528+0000","last_became_active":"2024-02-22T19:44:57.599424+0000","last_became_peered":"2024-02-22T19:44:57.599424+0000","last_unstale":"2024-02-22T19:47:12.479528+0000","last_undegraded":"2024-02-22T19:47:12.479528+0000","last_fullsized":"2024-02-22T19:47:12.479528+0000","mapping_epoch":267,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":411,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'25","last_scrub_stamp":"2024-02-22T19:45:06.780605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:45:06.780605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"460'13345","reported_seq":13456,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.905272+0000","last_change":"2024-02-22T19:47:10.418043+0000","last_active":"2024-02-22T19:47:14.905272+0000","last_peered":"2024-02-22T19:47:14.905272+0000","last_clean":"2024-02-22T19:47:14.905272+0000","last_became_active":"2024-02-22T19:43:57.347455+0000","last_became_peered":"2024-02-22T19:43:57.347455+0000","last_unstale":"2024-02-22T19:47:14.905272+0000","last_undegraded":"2024-02-22T19:47:14.905272+0000","last_fullsized":"2024-02-22T19:47:14.905272+0000","mapping_epoch":365,"log_start":"445'10000","ondisk_log_start":"445'10000","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"459'13065","last_scrub_stamp":"2024-02-22T19:47:10.417940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.417940+0000","log_size":3345,"ondisk_log_size":3345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32899096,"num_objects":503,"num_object_clones":0,"num_object_copies":1509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13345,"num_write_kb":443073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":318,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129018+0000","last_change":"2024-02-22T19:45:55.024849+0000","last_active":"2024-02-22T19:47:10.129018+0000","last_peered":"2024-02-22T19:47:10.129018+0000","last_clean":"2024-02-22T19:47:10.129018+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:47:10.129018+0000","last_undegraded":"2024-02-22T19:47:10.129018+0000","last_fullsized":"2024-02-22T19:47:10.129018+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.024753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.024753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":440,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117101+0000","last_change":"2024-02-22T19:44:53.631267+0000","last_active":"2024-02-22T19:47:10.117101+0000","last_peered":"2024-02-22T19:47:10.117101+0000","last_clean":"2024-02-22T19:47:10.117101+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:47:10.117101+0000","last_undegraded":"2024-02-22T19:47:10.117101+0000","last_fullsized":"2024-02-22T19:47:10.117101+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:53.631188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.631188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":455,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.118439+0000","last_change":"2024-02-22T19:44:13.731072+0000","last_active":"2024-02-22T19:47:10.118439+0000","last_peered":"2024-02-22T19:47:10.118439+0000","last_clean":"2024-02-22T19:47:10.118439+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:47:10.118439+0000","last_undegraded":"2024-02-22T19:47:10.118439+0000","last_fullsized":"2024-02-22T19:47:10.118439+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.730975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.730975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"264'29","reported_seq":276,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114600+0000","last_change":"2024-02-22T19:44:15.640361+0000","last_active":"2024-02-22T19:47:10.114600+0000","last_peered":"2024-02-22T19:47:10.114600+0000","last_clean":"2024-02-22T19:47:10.114600+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:47:10.114600+0000","last_undegraded":"2024-02-22T19:47:10.114600+0000","last_fullsized":"2024-02-22T19:47:10.114600+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-02-22T19:44:15.640271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.640271+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":443,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114243+0000","last_change":"2024-02-22T19:44:54.804568+0000","last_active":"2024-02-22T19:47:10.114243+0000","last_peered":"2024-02-22T19:47:10.114243+0000","last_clean":"2024-02-22T19:47:10.114243+0000","last_became_active":"2024-02-22T19:44:45.398192+0000","last_became_peered":"2024-02-22T19:44:45.398192+0000","last_unstale":"2024-02-22T19:47:10.114243+0000","last_undegraded":"2024-02-22T19:47:10.114243+0000","last_fullsized":"2024-02-22T19:47:10.114243+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":399,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:54.804469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:54.804469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"460'13721","reported_seq":13829,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.471844+0000","last_change":"2024-02-22T19:46:57.953017+0000","last_active":"2024-02-22T19:47:14.471844+0000","last_peered":"2024-02-22T19:47:14.471844+0000","last_clean":"2024-02-22T19:47:14.471844+0000","last_became_active":"2024-02-22T19:43:57.351385+0000","last_became_peered":"2024-02-22T19:43:57.351385+0000","last_unstale":"2024-02-22T19:47:14.471844+0000","last_undegraded":"2024-02-22T19:47:14.471844+0000","last_fullsized":"2024-02-22T19:47:14.471844+0000","mapping_epoch":365,"log_start":"444'9900","ondisk_log_start":"444'9900","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12580","last_scrub_stamp":"2024-02-22T19:46:57.952946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:46:57.952946+0000","log_size":3821,"ondisk_log_size":3821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37027840,"num_objects":565,"num_object_clones":0,"num_object_copies":1695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13721,"num_write_kb":457152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":170,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129675+0000","last_change":"2024-02-22T19:44:46.735448+0000","last_active":"2024-02-22T19:47:10.129675+0000","last_peered":"2024-02-22T19:47:10.129675+0000","last_clean":"2024-02-22T19:47:10.129675+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:47:10.129675+0000","last_undegraded":"2024-02-22T19:47:10.129675+0000","last_fullsized":"2024-02-22T19:47:10.129675+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:46.735345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:44:46.735345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":251,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.127953+0000","last_change":"2024-02-22T19:44:49.777279+0000","last_active":"2024-02-22T19:47:10.127953+0000","last_peered":"2024-02-22T19:47:10.127953+0000","last_clean":"2024-02-22T19:47:10.127953+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:47:10.127953+0000","last_undegraded":"2024-02-22T19:47:10.127953+0000","last_fullsized":"2024-02-22T19:47:10.127953+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:49.777178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:44:49.777178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":262,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114827+0000","last_change":"2024-02-22T19:46:19.538051+0000","last_active":"2024-02-22T19:47:10.114827+0000","last_peered":"2024-02-22T19:47:10.114827+0000","last_clean":"2024-02-22T19:47:10.114827+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:47:10.114827+0000","last_undegraded":"2024-02-22T19:47:10.114827+0000","last_fullsized":"2024-02-22T19:47:10.114827+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:19.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:46:19.537988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":575,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117816+0000","last_change":"2024-02-22T19:46:18.828987+0000","last_active":"2024-02-22T19:47:10.117816+0000","last_peered":"2024-02-22T19:47:10.117816+0000","last_clean":"2024-02-22T19:47:10.117816+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:47:10.117816+0000","last_undegraded":"2024-02-22T19:47:10.117816+0000","last_fullsized":"2024-02-22T19:47:10.117816+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:46:18.828884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.828884+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"395'16","reported_seq":551,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.478768+0000","last_change":"2024-02-22T19:46:58.513539+0000","last_active":"2024-02-22T19:47:12.478768+0000","last_peered":"2024-02-22T19:47:12.478768+0000","last_clean":"2024-02-22T19:47:12.478768+0000","last_became_active":"2024-02-22T19:44:41.682349+0000","last_became_peered":"2024-02-22T19:44:41.682349+0000","last_unstale":"2024-02-22T19:47:12.478768+0000","last_undegraded":"2024-02-22T19:47:12.478768+0000","last_fullsized":"2024-02-22T19:47:12.478768+0000","mapping_epoch":322,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'16","last_scrub_stamp":"2024-02-22T19:46:58.513473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.513473+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"460'13574","reported_seq":13682,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.891797+0000","last_change":"2024-02-22T19:47:02.888460+0000","last_active":"2024-02-22T19:47:14.891797+0000","last_peered":"2024-02-22T19:47:14.891797+0000","last_clean":"2024-02-22T19:47:14.891797+0000","last_became_active":"2024-02-22T19:43:57.348228+0000","last_became_peered":"2024-02-22T19:43:57.348228+0000","last_unstale":"2024-02-22T19:47:14.891797+0000","last_undegraded":"2024-02-22T19:47:14.891797+0000","last_fullsized":"2024-02-22T19:47:14.891797+0000","mapping_epoch":365,"log_start":"445'10200","ondisk_log_start":"445'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12733","last_scrub_stamp":"2024-02-22T19:47:02.888397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:02.888397+0000","log_size":3374,"ondisk_log_size":3374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34996224,"num_objects":534,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13574,"num_write_kb":451456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":342,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.131277+0000","last_change":"2024-02-22T19:43:58.000275+0000","last_active":"2024-02-22T19:47:10.131277+0000","last_peered":"2024-02-22T19:47:10.131277+0000","last_clean":"2024-02-22T19:47:10.131277+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:47:10.131277+0000","last_undegraded":"2024-02-22T19:47:10.131277+0000","last_fullsized":"2024-02-22T19:47:10.131277+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:58.000163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:43:58.000163+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":441,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119407+0000","last_change":"2024-02-22T19:44:45.370939+0000","last_active":"2024-02-22T19:47:10.119407+0000","last_peered":"2024-02-22T19:47:10.119407+0000","last_clean":"2024-02-22T19:47:10.119407+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:47:10.119407+0000","last_undegraded":"2024-02-22T19:47:10.119407+0000","last_fullsized":"2024-02-22T19:47:10.119407+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:45.370834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.370834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":260,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117641+0000","last_change":"2024-02-22T19:44:47.633310+0000","last_active":"2024-02-22T19:47:10.117641+0000","last_peered":"2024-02-22T19:47:10.117641+0000","last_clean":"2024-02-22T19:47:10.117641+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:47:10.117641+0000","last_undegraded":"2024-02-22T19:47:10.117641+0000","last_fullsized":"2024-02-22T19:47:10.117641+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:47.633242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.633242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":577,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113760+0000","last_change":"2024-02-22T19:44:45.716073+0000","last_active":"2024-02-22T19:47:10.113760+0000","last_peered":"2024-02-22T19:47:10.113760+0000","last_clean":"2024-02-22T19:47:10.113760+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:47:10.113760+0000","last_undegraded":"2024-02-22T19:47:10.113760+0000","last_fullsized":"2024-02-22T19:47:10.113760+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:45.715955+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.715955+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"407'13","reported_seq":712,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:12.484540+0000","last_change":"2024-02-22T19:44:53.757698+0000","last_active":"2024-02-22T19:47:12.484540+0000","last_peered":"2024-02-22T19:47:12.484540+0000","last_clean":"2024-02-22T19:47:12.484540+0000","last_became_active":"2024-02-22T19:44:53.415371+0000","last_became_peered":"2024-02-22T19:44:53.415371+0000","last_unstale":"2024-02-22T19:47:12.484540+0000","last_undegraded":"2024-02-22T19:47:12.484540+0000","last_fullsized":"2024-02-22T19:47:12.484540+0000","mapping_epoch":267,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":407,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:44:53.757594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.757594+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"460'13764","reported_seq":13875,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.662316+0000","last_change":"2024-02-22T19:46:02.481207+0000","last_active":"2024-02-22T19:47:14.662316+0000","last_peered":"2024-02-22T19:47:14.662316+0000","last_clean":"2024-02-22T19:47:14.662316+0000","last_became_active":"2024-02-22T19:43:57.348032+0000","last_became_peered":"2024-02-22T19:43:57.348032+0000","last_unstale":"2024-02-22T19:47:14.662316+0000","last_undegraded":"2024-02-22T19:47:14.662316+0000","last_fullsized":"2024-02-22T19:47:14.662316+0000","mapping_epoch":365,"log_start":"444'9900","ondisk_log_start":"444'9900","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'8759","last_scrub_stamp":"2024-02-22T19:46:02.481072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:46:02.481072+0000","log_size":3864,"ondisk_log_size":3864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34340864,"num_objects":524,"num_object_clones":0,"num_object_copies":1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13764,"num_write_kb":457216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":249,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.116064+0000","last_change":"2024-02-22T19:45:14.353778+0000","last_active":"2024-02-22T19:47:10.116064+0000","last_peered":"2024-02-22T19:47:10.116064+0000","last_clean":"2024-02-22T19:47:10.116064+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:47:10.116064+0000","last_undegraded":"2024-02-22T19:47:10.116064+0000","last_fullsized":"2024-02-22T19:47:10.116064+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.353679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.353679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":243,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128944+0000","last_change":"2024-02-22T19:46:20.075957+0000","last_active":"2024-02-22T19:47:10.128944+0000","last_peered":"2024-02-22T19:47:10.128944+0000","last_clean":"2024-02-22T19:47:10.128944+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:47:10.128944+0000","last_undegraded":"2024-02-22T19:47:10.128944+0000","last_fullsized":"2024-02-22T19:47:10.128944+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.075867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.075867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":461,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129781+0000","last_change":"2024-02-22T19:45:58.995918+0000","last_active":"2024-02-22T19:47:10.129781+0000","last_peered":"2024-02-22T19:47:10.129781+0000","last_clean":"2024-02-22T19:47:10.129781+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:47:10.129781+0000","last_undegraded":"2024-02-22T19:47:10.129781+0000","last_fullsized":"2024-02-22T19:47:10.129781+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:58.995851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:45:58.995851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":565,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128243+0000","last_change":"2024-02-22T19:45:09.692752+0000","last_active":"2024-02-22T19:47:10.128243+0000","last_peered":"2024-02-22T19:47:10.128243+0000","last_clean":"2024-02-22T19:47:10.128243+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:47:10.128243+0000","last_undegraded":"2024-02-22T19:47:10.128243+0000","last_fullsized":"2024-02-22T19:47:10.128243+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:45:09.692661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.692661+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":270,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.127032+0000","last_change":"2024-02-22T19:44:51.773621+0000","last_active":"2024-02-22T19:47:10.127032+0000","last_peered":"2024-02-22T19:47:10.127032+0000","last_clean":"2024-02-22T19:47:10.127032+0000","last_became_active":"2024-02-22T19:44:49.491617+0000","last_became_peered":"2024-02-22T19:44:49.491617+0000","last_unstale":"2024-02-22T19:47:10.127032+0000","last_undegraded":"2024-02-22T19:47:10.127032+0000","last_fullsized":"2024-02-22T19:47:10.127032+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":403,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:51.773527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:51.773527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"460'13465","reported_seq":13573,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.454905+0000","last_change":"2024-02-22T19:47:01.724088+0000","last_active":"2024-02-22T19:47:14.454905+0000","last_peered":"2024-02-22T19:47:14.454905+0000","last_clean":"2024-02-22T19:47:14.454905+0000","last_became_active":"2024-02-22T19:43:57.347224+0000","last_became_peered":"2024-02-22T19:43:57.347224+0000","last_unstale":"2024-02-22T19:47:14.454905+0000","last_undegraded":"2024-02-22T19:47:14.454905+0000","last_fullsized":"2024-02-22T19:47:14.454905+0000","mapping_epoch":365,"log_start":"442'9600","ondisk_log_start":"442'9600","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12645","last_scrub_stamp":"2024-02-22T19:47:01.723982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:01.723982+0000","log_size":3865,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37814272,"num_objects":577,"num_object_clones":0,"num_object_copies":1731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13465,"num_write_kb":449344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":274,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114713+0000","last_change":"2024-02-22T19:45:00.729978+0000","last_active":"2024-02-22T19:47:10.114713+0000","last_peered":"2024-02-22T19:47:10.114713+0000","last_clean":"2024-02-22T19:47:10.114713+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:47:10.114713+0000","last_undegraded":"2024-02-22T19:47:10.114713+0000","last_fullsized":"2024-02-22T19:47:10.114713+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.729907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.729907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":289,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.114728+0000","last_change":"2024-02-22T19:46:17.513957+0000","last_active":"2024-02-22T19:47:10.114728+0000","last_peered":"2024-02-22T19:47:10.114728+0000","last_clean":"2024-02-22T19:47:10.114728+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:47:10.114728+0000","last_undegraded":"2024-02-22T19:47:10.114728+0000","last_fullsized":"2024-02-22T19:47:10.114728+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.513841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.513841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":434,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.128156+0000","last_change":"2024-02-22T19:44:12.048204+0000","last_active":"2024-02-22T19:47:10.128156+0000","last_peered":"2024-02-22T19:47:10.128156+0000","last_clean":"2024-02-22T19:47:10.128156+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:47:10.128156+0000","last_undegraded":"2024-02-22T19:47:10.128156+0000","last_fullsized":"2024-02-22T19:47:10.128156+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.048068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.048068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":178,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115509+0000","last_change":"2024-02-22T19:46:15.453110+0000","last_active":"2024-02-22T19:47:10.115509+0000","last_peered":"2024-02-22T19:47:10.115509+0000","last_clean":"2024-02-22T19:47:10.115509+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:47:10.115509+0000","last_undegraded":"2024-02-22T19:47:10.115509+0000","last_fullsized":"2024-02-22T19:47:10.115509+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:15.453047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.453047+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"460'13439","reported_seq":13547,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.256616+0000","last_change":"2024-02-22T19:47:04.139059+0000","last_active":"2024-02-22T19:47:14.256616+0000","last_peered":"2024-02-22T19:47:14.256616+0000","last_clean":"2024-02-22T19:47:14.256616+0000","last_became_active":"2024-02-22T19:43:57.345992+0000","last_became_peered":"2024-02-22T19:43:57.345992+0000","last_unstale":"2024-02-22T19:47:14.256616+0000","last_undegraded":"2024-02-22T19:47:14.256616+0000","last_fullsized":"2024-02-22T19:47:14.256616+0000","mapping_epoch":365,"log_start":"444'9600","ondisk_log_start":"444'9600","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'12751","last_scrub_stamp":"2024-02-22T19:47:04.138958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:04.138958+0000","log_size":3839,"ondisk_log_size":3839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36765696,"num_objects":561,"num_object_clones":0,"num_object_copies":1683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13439,"num_write_kb":448000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":450,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.131405+0000","last_change":"2024-02-22T19:44:42.822533+0000","last_active":"2024-02-22T19:47:10.131405+0000","last_peered":"2024-02-22T19:47:10.131405+0000","last_clean":"2024-02-22T19:47:10.131405+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:47:10.131405+0000","last_undegraded":"2024-02-22T19:47:10.131405+0000","last_fullsized":"2024-02-22T19:47:10.131405+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:42.822463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:42.822463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":244,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119053+0000","last_change":"2024-02-22T19:46:20.790826+0000","last_active":"2024-02-22T19:47:10.119053+0000","last_peered":"2024-02-22T19:47:10.119053+0000","last_clean":"2024-02-22T19:47:10.119053+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:47:10.119053+0000","last_undegraded":"2024-02-22T19:47:10.119053+0000","last_fullsized":"2024-02-22T19:47:10.119053+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.790735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.790735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":254,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.693021+0000","last_change":"2024-02-22T19:47:10.692993+0000","last_active":"2024-02-22T19:47:10.693021+0000","last_peered":"2024-02-22T19:47:10.693021+0000","last_clean":"2024-02-22T19:47:10.693021+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:47:10.693021+0000","last_undegraded":"2024-02-22T19:47:10.693021+0000","last_fullsized":"2024-02-22T19:47:10.693021+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:10.692928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.692928+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3","version":"460'13778","reported_seq":13886,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.716896+0000","last_change":"2024-02-22T19:45:14.048917+0000","last_active":"2024-02-22T19:47:14.716896+0000","last_peered":"2024-02-22T19:47:14.716896+0000","last_clean":"2024-02-22T19:47:14.716896+0000","last_became_active":"2024-02-22T19:43:57.347832+0000","last_became_peered":"2024-02-22T19:43:57.347832+0000","last_unstale":"2024-02-22T19:47:14.716896+0000","last_undegraded":"2024-02-22T19:47:14.716896+0000","last_fullsized":"2024-02-22T19:47:14.716896+0000","mapping_epoch":365,"log_start":"444'9900","ondisk_log_start":"444'9900","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"424'5534","last_scrub_stamp":"2024-02-22T19:45:14.048806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.048806+0000","log_size":3878,"ondisk_log_size":3878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34603008,"num_objects":528,"num_object_clones":0,"num_object_copies":1584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13778,"num_write_kb":457792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":262,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.113912+0000","last_change":"2024-02-22T19:44:48.769697+0000","last_active":"2024-02-22T19:47:10.113912+0000","last_peered":"2024-02-22T19:47:10.113912+0000","last_clean":"2024-02-22T19:47:10.113912+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:47:10.113912+0000","last_undegraded":"2024-02-22T19:47:10.113912+0000","last_fullsized":"2024-02-22T19:47:10.113912+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.769602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.769602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":459,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.115563+0000","last_change":"2024-02-22T19:46:17.949817+0000","last_active":"2024-02-22T19:47:10.115563+0000","last_peered":"2024-02-22T19:47:10.115563+0000","last_clean":"2024-02-22T19:47:10.115563+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:47:10.115563+0000","last_undegraded":"2024-02-22T19:47:10.115563+0000","last_fullsized":"2024-02-22T19:47:10.115563+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.949728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.949728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":371,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117471+0000","last_change":"2024-02-22T19:44:12.719560+0000","last_active":"2024-02-22T19:47:10.117471+0000","last_peered":"2024-02-22T19:47:10.117471+0000","last_clean":"2024-02-22T19:47:10.117471+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:47:10.117471+0000","last_undegraded":"2024-02-22T19:47:10.117471+0000","last_fullsized":"2024-02-22T19:47:10.117471+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.719494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.719494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":345,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.116158+0000","last_change":"2024-02-22T19:46:09.851700+0000","last_active":"2024-02-22T19:47:10.116158+0000","last_peered":"2024-02-22T19:47:10.116158+0000","last_clean":"2024-02-22T19:47:10.116158+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:47:10.116158+0000","last_undegraded":"2024-02-22T19:47:10.116158+0000","last_fullsized":"2024-02-22T19:47:10.116158+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:46:09.851636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.851636+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"460'13662","reported_seq":13770,"reported_epoch":460,"state":"active+clean","last_fresh":"2024-02-22T19:47:14.256586+0000","last_change":"2024-02-22T19:47:13.940276+0000","last_active":"2024-02-22T19:47:14.256586+0000","last_peered":"2024-02-22T19:47:14.256586+0000","last_clean":"2024-02-22T19:47:14.256586+0000","last_became_active":"2024-02-22T19:43:57.351420+0000","last_became_peered":"2024-02-22T19:43:57.351420+0000","last_unstale":"2024-02-22T19:47:14.256586+0000","last_undegraded":"2024-02-22T19:47:14.256586+0000","last_fullsized":"2024-02-22T19:47:14.256586+0000","mapping_epoch":365,"log_start":"444'9800","ondisk_log_start":"444'9800","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"460'13632","last_scrub_stamp":"2024-02-22T19:47:13.940207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:13.940207+0000","log_size":3862,"ondisk_log_size":3862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37224448,"num_objects":568,"num_object_clones":0,"num_object_copies":1704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13662,"num_write_kb":455360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":452,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.119806+0000","last_change":"2024-02-22T19:44:59.333051+0000","last_active":"2024-02-22T19:47:10.119806+0000","last_peered":"2024-02-22T19:47:10.119806+0000","last_clean":"2024-02-22T19:47:10.119806+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:47:10.119806+0000","last_undegraded":"2024-02-22T19:47:10.119806+0000","last_fullsized":"2024-02-22T19:47:10.119806+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:59.332960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:59.332960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":280,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.117562+0000","last_change":"2024-02-22T19:45:15.816947+0000","last_active":"2024-02-22T19:47:10.117562+0000","last_peered":"2024-02-22T19:47:10.117562+0000","last_clean":"2024-02-22T19:47:10.117562+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:47:10.117562+0000","last_undegraded":"2024-02-22T19:47:10.117562+0000","last_fullsized":"2024-02-22T19:47:10.117562+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:45:15.816835+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:45:15.816835+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":495,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.129038+0000","last_change":"2024-02-22T19:46:12.047830+0000","last_active":"2024-02-22T19:47:10.129038+0000","last_peered":"2024-02-22T19:47:10.129038+0000","last_clean":"2024-02-22T19:47:10.129038+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:47:10.129038+0000","last_undegraded":"2024-02-22T19:47:10.129038+0000","last_fullsized":"2024-02-22T19:47:10.129038+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:46:12.047696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.047696+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":309,"reported_epoch":459,"state":"active+clean","last_fresh":"2024-02-22T19:47:10.131607+0000","last_change":"2024-02-22T19:44:40.870790+0000","last_active":"2024-02-22T19:47:10.131607+0000","last_peered":"2024-02-22T19:47:10.131607+0000","last_clean":"2024-02-22T19:47:10.131607+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:47:10.131607+0000","last_undegraded":"2024-02-22T19:47:10.131607+0000","last_fullsized":"2024-02-22T19:47:10.131607+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:44:40.870681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:40.870681+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":566755352,"num_objects":8649,"num_object_clones":0,"num_object_copies":25947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218105,"num_write_kb":7256065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1701195776,"data_stored":1701183560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58705,"ondisk_log_size":58705,"up":48,"acting":48,"num_store_stats":5},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1076,"num_read_kb":1071,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266910,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1211132,"kb_used_data":322420,"kb_used_omap":0,"kb_used_meta":888704,"kb_avail":92534020,"statfs":{"total":95995035648,"available":94754836480,"internally_reserved":0,"allocated":330158080,"data_stored":328441350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":910032896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":1.252,"15min":1.5760000000000001},"min":{"1min":0.23100000000000001,"5min":0.219,"15min":0.19900000000000001},"max":{"1min":3.0390000000000001,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.98699999999999999,"15min":1.524},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.20699999999999999},"max":{"1min":3.7389999999999999,"5min":5.1349999999999998,"15min":42.155999999999999},"last":0.495}]},{"osd":1,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8580000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.20000000000000001},"max":{"1min":4.1180000000000003,"5min":36.509,"15min":54.951000000000001},"last":0.312},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.4830000000000001,"15min":2.133},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":6.1559999999999997,"5min":7.1449999999999996,"15min":57.012},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":2.698,"5min":1.6819999999999999,"15min":1.7050000000000001},"min":{"1min":0.33800000000000002,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":25.053999999999998,"5min":25.053999999999998,"15min":41.640999999999998},"last":1.873},{"interface":"front","average":{"1min":3.234,"5min":1.8979999999999999,"15min":2},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":24.956,"5min":24.956,"15min":41.722000000000001},"last":1.806}]},{"osd":3,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.97,"15min":2.0230000000000001},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":6.694,"5min":36.491999999999997,"15min":41.457000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.5029999999999999,"5min":2.157,"15min":2.0619999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.5,"5min":36.732999999999997,"15min":41.311999999999998},"last":1.5129999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.70399999999999996,"15min":0.73899999999999999},"min":{"1min":0.28100000000000003,"5min":0.25900000000000001,"15min":0.23999999999999999},"max":{"1min":4.0910000000000002,"5min":4.4219999999999997,"15min":6.4720000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.94599999999999995,"15min":0.77600000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":3.1120000000000001,"5min":5.0910000000000002,"15min":7.6219999999999999},"last":0.438}]},{"osd":5,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.0029999999999999,"15min":0.86799999999999999},"min":{"1min":0.28699999999999998,"5min":0.27200000000000002,"15min":0.27000000000000002},"max":{"1min":1.8069999999999999,"5min":6.625,"15min":6.9039999999999999},"last":0.373},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.80700000000000005,"15min":0.81599999999999995},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":1.776,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.67400000000000004,"15min":0.84099999999999997},"min":{"1min":0.35199999999999998,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":1.9339999999999999,"5min":3.956,"15min":12.199999999999999},"last":2.6840000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":1.1040000000000001,"15min":0.93700000000000006},"min":{"1min":0.36099999999999999,"5min":0.317,"15min":0.28100000000000003},"max":{"1min":1.952,"5min":6.6719999999999997,"15min":12.067},"last":0.40100000000000002}]}]},{"osd":6,"up_from":0,"seq":193273528547,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575100,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1215556,"kb_used_data":315900,"kb_used_omap":0,"kb_used_meta":899584,"kb_avail":92529596,"statfs":{"total":95995035648,"available":94750306304,"internally_reserved":0,"allocated":323481600,"data_stored":321756630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":921174016},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:10 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":2.4430000000000001,"15min":2.1869999999999998},"min":{"1min":0.20999999999999999,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":4.6769999999999996,"5min":51.091000000000001,"15min":51.091000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.8380000000000001,"5min":2.2999999999999998,"15min":2.2650000000000001},"min":{"1min":0.28499999999999998,"5min":0.215,"15min":0.17499999999999999},"max":{"1min":5.8630000000000004,"5min":51.289999999999999,"15min":51.289999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":2.4369999999999998,"5min":2.9049999999999998,"15min":2.387},"min":{"1min":0.23000000000000001,"5min":0.191,"15min":0.191},"max":{"1min":7.0430000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":2.7320000000000002},{"interface":"front","average":{"1min":3.1339999999999999,"5min":3.0659999999999998,"15min":2.528},"min":{"1min":0.29899999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":12.957000000000001,"5min":51.369999999999997,"15min":51.369999999999997},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:46:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":2.4380000000000002,"15min":2.3919999999999999},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.214},"max":{"1min":6.2670000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":1.026},{"interface":"front","average":{"1min":2.9710000000000001,"5min":2.895,"15min":2.6320000000000001},"min":{"1min":0.44500000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":9.6820000000000004,"5min":51.741,"15min":51.741},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:46:15 2024","interfaces":[{"interface":"back","average":{"1min":1.9399999999999999,"5min":2.5790000000000002,"15min":1.923},"min":{"1min":0.40400000000000003,"5min":0.307,"15min":0.24399999999999999},"max":{"1min":8.0800000000000001,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.887,"15min":1.512},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":5.2690000000000001,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.746}]},{"osd":5,"last update":"Thu Feb 22 19:46:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":2.6680000000000001,"15min":2.0209999999999999},"min":{"1min":0.34399999999999997,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":6.2210000000000001,"5min":51.183999999999997,"15min":51.183999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.591,"5min":2.3999999999999999,"15min":1.7989999999999999},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":6.3129999999999997,"5min":51.212000000000003,"15min":51.212000000000003},"last":2.4780000000000002}]},{"osd":6,"last update":"Thu Feb 22 19:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.8560000000000001,"15min":1.6319999999999999},"min":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.28599999999999998},"max":{"1min":2.8130000000000002,"5min":51.079999999999998,"15min":51.079999999999998},"last":2.577},{"interface":"front","average":{"1min":1.859,"5min":2.8109999999999999,"15min":2.056},"min":{"1min":0.51100000000000001,"5min":0.44,"15min":0.34999999999999998},"max":{"1min":4.657,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.58999999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":2.5640000000000001,"15min":2.0019999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":6.327,"5min":51.244999999999997,"15min":51.244999999999997},"last":2.008},{"interface":"front","average":{"1min":1.1459999999999999,"5min":2.028,"15min":1.641},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.3040000000000003,"5min":51.317999999999998,"15min":51.317999999999998},"last":0.95499999999999996}]}]},{"osd":0,"up_from":0,"seq":34359738625,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411574,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1380340,"kb_used_data":484716,"kb_used_omap":0,"kb_used_meta":895424,"kb_avail":92364812,"statfs":{"total":95995035648,"available":94581567488,"internally_reserved":0,"allocated":496349184,"data_stored":494638913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":916914176},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:46:21 2024","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.3320000000000001,"15min":1.907},"min":{"1min":0.28899999999999998,"5min":0.215,"15min":0.18099999999999999},"max":{"1min":5.0279999999999996,"5min":5.7350000000000003,"15min":22.338999999999999},"last":6.1589999999999998},{"interface":"front","average":{"1min":3.0819999999999999,"5min":2.0779999999999998,"15min":2.4460000000000002},"min":{"1min":0.24099999999999999,"5min":0.216,"15min":0.17599999999999999},"max":{"1min":7.7450000000000001,"5min":27.361999999999998,"15min":30.427},"last":3.6240000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:46:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0419999999999998,"5min":1.8400000000000001,"15min":1.9630000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":5.2110000000000003,"5min":12.101000000000001,"15min":12.349},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.849,"5min":1.9079999999999999,"15min":1.879},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.193},"max":{"1min":4.992,"5min":12.18,"15min":13.782},"last":6.3289999999999997}]},{"osd":3,"last update":"Thu Feb 22 19:46:55 2024","interfaces":[{"interface":"back","average":{"1min":2.2320000000000002,"5min":2.0939999999999999,"15min":2.379},"min":{"1min":0.312,"5min":0.27000000000000002,"15min":0.20999999999999999},"max":{"1min":6.5049999999999999,"5min":12.151999999999999,"15min":30.071000000000002},"last":6.875},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.3540000000000001,"15min":2.6309999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":9.2829999999999995,"5min":28.263999999999999,"15min":32.119},"last":6.9850000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4850000000000001,"15min":1.579},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.26400000000000001},"max":{"1min":4.0789999999999997,"5min":8.2289999999999992,"15min":12.43},"last":0.32800000000000001},{"interface":"front","average":{"1min":1.742,"5min":1.375,"15min":1.4670000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.253},"max":{"1min":5.5039999999999996,"5min":5.7999999999999998,"15min":10.34},"last":6.2060000000000004}]},{"osd":5,"last update":"Thu Feb 22 19:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.6160000000000001,"15min":1.673},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.5190000000000001,"5min":14.045,"15min":40.633000000000003},"last":6.2329999999999997},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.9750000000000001,"15min":1.9379999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.26600000000000001},"max":{"1min":7.9450000000000003,"5min":14.513999999999999,"15min":40.997999999999998},"last":0.30199999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.532,"15min":1.696},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.311},"max":{"1min":2.4649999999999999,"5min":8.2530000000000001,"15min":21.640000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.583,"15min":1.552},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":5.5380000000000003,"5min":5.8869999999999996,"15min":10.369999999999999},"last":6.2649999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:46:21 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.8600000000000001,"15min":1.7509999999999999},"min":{"1min":0.48999999999999999,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.0750000000000002,"5min":14.199,"15min":14.199},"last":0.372},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.8089999999999999,"15min":1.647},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":5.0919999999999996,"5min":18.021999999999998,"15min":18.021999999999998},"last":6.2960000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149936,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1265032,"kb_used_data":377472,"kb_used_omap":0,"kb_used_meta":887552,"kb_avail":92480120,"statfs":{"total":95995035648,"available":94699642880,"internally_reserved":0,"allocated":386531328,"data_stored":384802179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":908853248},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:01 2024","interfaces":[{"interface":"back","average":{"1min":1.976,"5min":1.911,"15min":2.1819999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":6.6440000000000001,"5min":9.3599999999999994,"15min":23.48},"last":0.29699999999999999},{"interface":"front","average":{"1min":1.97,"5min":2.141,"15min":2.5329999999999999},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.20799999999999999},"max":{"1min":6.7350000000000003,"5min":10.297000000000001,"15min":23.411999999999999},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:47:01 2024","interfaces":[{"interface":"back","average":{"1min":8.9220000000000006,"5min":3.839,"15min":3.383},"min":{"1min":0.214,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":94.454999999999998,"5min":94.454999999999998,"15min":94.454999999999998},"last":1.837},{"interface":"front","average":{"1min":9.5449999999999999,"5min":3.5539999999999998,"15min":2.8679999999999999},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":95.197999999999993,"5min":95.197999999999993,"15min":95.197999999999993},"last":0.22}]},{"osd":2,"last update":"Thu Feb 22 19:47:01 2024","interfaces":[{"interface":"back","average":{"1min":2.851,"5min":2.4409999999999998,"15min":2.5219999999999998},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":10.753,"5min":11.577,"15min":20.105},"last":2.6840000000000002},{"interface":"front","average":{"1min":3.5169999999999999,"5min":2.6549999999999998,"15min":2.5600000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.217},"max":{"1min":14.308999999999999,"5min":17.986000000000001,"15min":35.939999999999998},"last":0.998}]},{"osd":4,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.5840000000000001,"15min":1.925},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":2.7669999999999999,"5min":9.968,"15min":13.680999999999999},"last":0.496},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.613,"15min":1.8069999999999999},"min":{"1min":0.318,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":7.8719999999999999,"5min":7.8719999999999999,"15min":13.548},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:46:37 2024","interfaces":[{"interface":"back","average":{"1min":2.077,"5min":1.7729999999999999,"15min":1.758},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.718,"5min":9.3160000000000007,"15min":12.879},"last":0.35299999999999998},{"interface":"front","average":{"1min":1.385,"5min":1.8120000000000001,"15min":1.9590000000000001},"min":{"1min":0.32500000000000001,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":4.9279999999999999,"5min":10.041,"15min":13.707000000000001},"last":3.6379999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:47:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":1.7889999999999999,"15min":1.804},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":7.6859999999999999,"5min":9.423,"15min":12.898999999999999},"last":0.377},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.8560000000000001,"15min":2.0579999999999998},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.802,"5min":10.02,"15min":13.803000000000001},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:46:14 2024","interfaces":[{"interface":"back","average":{"1min":1.77,"5min":1.5720000000000001,"15min":1.821},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.907,"5min":8.0879999999999992,"15min":13.483000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":2.1659999999999999,"5min":1.978,"15min":1.8360000000000001},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":7.6580000000000004,"5min":11.539,"15min":12.846},"last":0.39600000000000002}]}]},{"osd":4,"up_from":0,"seq":137438953710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724776,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1210184,"kb_used_data":315200,"kb_used_omap":0,"kb_used_meta":894976,"kb_avail":92534968,"statfs":{"total":95995035648,"available":94755807232,"internally_reserved":0,"allocated":322764800,"data_stored":321035059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":916455424},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1619999999999999,"15min":1.45},"min":{"1min":0.26900000000000002,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":4.8810000000000002,"5min":8.4299999999999997,"15min":17.684000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.454,"5min":1.5249999999999999,"15min":1.6359999999999999},"min":{"1min":0.25600000000000001,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":5.2789999999999999,"5min":11.9,"15min":17.402999999999999},"last":1.958}]},{"osd":1,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.2010000000000001,"15min":1.498},"min":{"1min":0.245,"5min":0.19,"15min":0.19},"max":{"1min":4.9509999999999996,"5min":6.0750000000000002,"15min":29.390999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":2.0550000000000002,"5min":1.798,"15min":1.7310000000000001},"min":{"1min":0.29499999999999998,"5min":0.248,"15min":0.218},"max":{"1min":6.0970000000000004,"5min":9.0999999999999996,"15min":29.367000000000001},"last":1.76}]},{"osd":2,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":4.3620000000000001,"5min":2.2160000000000002,"15min":1.8919999999999999},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":47.859000000000002,"5min":47.859000000000002,"15min":47.859000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":3.8029999999999999,"5min":1.819,"15min":1.8109999999999999},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":45.183,"5min":45.183,"15min":45.183},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":3.6000000000000001,"5min":2.2120000000000002,"15min":2.0379999999999998},"min":{"1min":0.34799999999999998,"5min":0.254,"15min":0.254},"max":{"1min":43.616999999999997,"5min":43.616999999999997,"15min":43.616999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":4.0140000000000002,"5min":2.073,"15min":1.946},"min":{"1min":0.39500000000000002,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":43.918999999999997,"5min":43.918999999999997,"15min":43.918999999999997},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.93799999999999994,"15min":0.77500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25,"15min":0.223},"max":{"1min":2.3220000000000001,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1140000000000001,"15min":0.85099999999999998},"min":{"1min":0.28699999999999998,"5min":0.27100000000000002,"15min":0.25800000000000001},"max":{"1min":2.0169999999999999,"5min":10.407,"15min":10.407},"last":0.29899999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:47:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.63600000000000001,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.247},"max":{"1min":1.429,"5min":3.2599999999999998,"15min":5.1150000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":1.006,"15min":0.86799999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26400000000000001},"max":{"1min":1.125,"5min":10.436,"15min":10.436},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.98999999999999999,"15min":0.78000000000000003},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28299999999999997},"max":{"1min":2.5209999999999999,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.226,"15min":0.93400000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.27100000000000002},"max":{"1min":4.2560000000000002,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.35299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299433984,"data_stored":299433984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451219456,"data_stored":451215384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":342560768,"data_stored":342556696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":284950528,"data_stored":284950528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":323031040,"data_stored":323026968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:47:21.396 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:47:21.427 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:19.781857+0000 mgr.y (mgr.14535) 699 : cluster [DBG] pgmap v938: 138 pgs: 138 active+clean; 368 MiB data, 5.4 GiB used, 442 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.13k op/s 2024-02-22T19:47:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:20.066681+0000 osd.2 (osd.2) 272 : cluster [DBG] 10.3 scrub starts 2024-02-22T19:47:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:20.159410+0000 osd.2 (osd.2) 273 : cluster [DBG] 10.3 scrub ok 2024-02-22T19:47:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:20.454110+0000 osd.5 (osd.5) 181 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:47:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:20.457613+0000 osd.5 (osd.5) 182 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:47:21.598 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:19.781857+0000 mgr.y (mgr.14535) 699 : cluster [DBG] pgmap v938: 138 pgs: 138 active+clean; 368 MiB data, 5.4 GiB used, 442 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.13k op/s 2024-02-22T19:47:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:20.066681+0000 osd.2 (osd.2) 272 : cluster [DBG] 10.3 scrub starts 2024-02-22T19:47:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:20.159410+0000 osd.2 (osd.2) 273 : cluster [DBG] 10.3 scrub ok 2024-02-22T19:47:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:20.454110+0000 osd.5 (osd.5) 181 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:47:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:20.457613+0000 osd.5 (osd.5) 182 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:47:21.605 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:21.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:21.706 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:21.775 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:21.811 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:21.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:21.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:21.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:21.978 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:21.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:21.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:22.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:47:22.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:21.065565+0000 osd.2 (osd.2) 274 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:47:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:21.070724+0000 osd.2 (osd.2) 275 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:47:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:21.500573+0000 osd.7 (osd.7) 292 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:47:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:21.507005+0000 osd.7 (osd.7) 293 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:47:22.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:21.977287+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? 172.21.15.53:0/1651260537' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:22.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:21.065565+0000 osd.2 (osd.2) 274 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:47:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:21.070724+0000 osd.2 (osd.2) 275 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:47:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:21.500573+0000 osd.7 (osd.7) 292 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:47:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:21.507005+0000 osd.7 (osd.7) 293 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:47:22.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:21.977287+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? 172.21.15.53:0/1651260537' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:22.601 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Removed 113376 objects 2024-02-22T19:47:22.601 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Clean up completed and total clean up time :103.745 2024-02-22T19:47:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:21.065565+0000 osd.2 (osd.2) 274 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:47:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:21.070724+0000 osd.2 (osd.2) 275 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:47:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:21.500573+0000 osd.7 (osd.7) 292 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:47:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:21.507005+0000 osd.7 (osd.7) 293 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:47:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:21.977287+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? 172.21.15.53:0/1651260537' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:22.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:47:22.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:47:22.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:47:22.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:47:22.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:47:22.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:47:22.928 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:22.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:22.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:22.933 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:23.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:23.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:23.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:23.316 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:23.348 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:23.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:21.782500+0000 mgr.y (mgr.14535) 700 : cluster [DBG] pgmap v939: 138 pgs: 138 active+clean; 196 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-02-22T19:47:23.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:22.084846+0000 osd.2 (osd.2) 276 : cluster [DBG] 10.e scrub starts 2024-02-22T19:47:23.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:22.192710+0000 osd.2 (osd.2) 277 : cluster [DBG] 10.e scrub ok 2024-02-22T19:47:23.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:21.782500+0000 mgr.y (mgr.14535) 700 : cluster [DBG] pgmap v939: 138 pgs: 138 active+clean; 196 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-02-22T19:47:23.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:22.084846+0000 osd.2 (osd.2) 276 : cluster [DBG] 10.e scrub starts 2024-02-22T19:47:23.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:22.192710+0000 osd.2 (osd.2) 277 : cluster [DBG] 10.e scrub ok 2024-02-22T19:47:23.546 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:21.782500+0000 mgr.y (mgr.14535) 700 : cluster [DBG] pgmap v939: 138 pgs: 138 active+clean; 196 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-02-22T19:47:23.546 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:22.084846+0000 osd.2 (osd.2) 276 : cluster [DBG] 10.e scrub starts 2024-02-22T19:47:23.546 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:22.192710+0000 osd.2 (osd.2) 277 : cluster [DBG] 10.e scrub ok 2024-02-22T19:47:23.546 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:23.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:23.549 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:23.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:23.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:23.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:23.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:23.929 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-02-22T19:47:23.929 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-02-22T19:47:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:24.013 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:24.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:24.058 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:47:24.058 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:47:24.115 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:24.143 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:24.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:23.004287+0000 osd.2 (osd.2) 278 : cluster [DBG] 4.b scrub starts 2024-02-22T19:47:24.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:23.007845+0000 osd.2 (osd.2) 279 : cluster [DBG] 4.b scrub ok 2024-02-22T19:47:24.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:23.387387+0000 mon.a (mon.0) 1688 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:24.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:23.004287+0000 osd.2 (osd.2) 278 : cluster [DBG] 4.b scrub starts 2024-02-22T19:47:24.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:23.007845+0000 osd.2 (osd.2) 279 : cluster [DBG] 4.b scrub ok 2024-02-22T19:47:24.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:23.387387+0000 mon.a (mon.0) 1688 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:24.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:23.004287+0000 osd.2 (osd.2) 278 : cluster [DBG] 4.b scrub starts 2024-02-22T19:47:24.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:23.007845+0000 osd.2 (osd.2) 279 : cluster [DBG] 4.b scrub ok 2024-02-22T19:47:24.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:23.387387+0000 mon.a (mon.0) 1688 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:24.624 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:24.627 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:24.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:24.711 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:47:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:47:24.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:24.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:47:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:24.884 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:24.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:24.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:24.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:25.011 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:25.020 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:25.040 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:25.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:25.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:25.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:25.173 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:25.246 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:25.250 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:25.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:23.782863+0000 mgr.y (mgr.14535) 701 : cluster [DBG] pgmap v940: 138 pgs: 138 active+clean; 196 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 594 B/s rd, 0 B/s wr, 641 op/s 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.039235+0000 osd.2 (osd.2) 280 : cluster [DBG] 10.5 scrub starts 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.060059+0000 osd.2 (osd.2) 281 : cluster [DBG] 10.5 scrub ok 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.202959+0000 osd.0 (osd.0) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.203027+0000 osd.0 (osd.0) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.392520+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e463: 8 total, 8 up, 5 in 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.466009+0000 osd.5 (osd.5) 183 : cluster [DBG] 4.3 deep-scrub starts 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.474446+0000 osd.5 (osd.5) 184 : cluster [DBG] 4.3 deep-scrub ok 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.506308+0000 osd.7 (osd.7) 294 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:47:25.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:24.509267+0000 osd.7 (osd.7) 295 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:47:25.351 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:25.381 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:25.401 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:25.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:25.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:25.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:25.534 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:23.782863+0000 mgr.y (mgr.14535) 701 : cluster [DBG] pgmap v940: 138 pgs: 138 active+clean; 196 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 594 B/s rd, 0 B/s wr, 641 op/s 2024-02-22T19:47:25.534 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.039235+0000 osd.2 (osd.2) 280 : cluster [DBG] 10.5 scrub starts 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.060059+0000 osd.2 (osd.2) 281 : cluster [DBG] 10.5 scrub ok 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.202959+0000 osd.0 (osd.0) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.203027+0000 osd.0 (osd.0) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.392520+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e463: 8 total, 8 up, 5 in 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.466009+0000 osd.5 (osd.5) 183 : cluster [DBG] 4.3 deep-scrub starts 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.474446+0000 osd.5 (osd.5) 184 : cluster [DBG] 4.3 deep-scrub ok 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.506308+0000 osd.7 (osd.7) 294 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:24.509267+0000 osd.7 (osd.7) 295 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:23.782863+0000 mgr.y (mgr.14535) 701 : cluster [DBG] pgmap v940: 138 pgs: 138 active+clean; 196 MiB data, 5.0 GiB used, 442 GiB / 447 GiB avail; 594 B/s rd, 0 B/s wr, 641 op/s 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.039235+0000 osd.2 (osd.2) 280 : cluster [DBG] 10.5 scrub starts 2024-02-22T19:47:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.060059+0000 osd.2 (osd.2) 281 : cluster [DBG] 10.5 scrub ok 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.202959+0000 osd.0 (osd.0) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.203027+0000 osd.0 (osd.0) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.392520+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e463: 8 total, 8 up, 5 in 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.466009+0000 osd.5 (osd.5) 183 : cluster [DBG] 4.3 deep-scrub starts 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.474446+0000 osd.5 (osd.5) 184 : cluster [DBG] 4.3 deep-scrub ok 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.506308+0000 osd.7 (osd.7) 294 : cluster [DBG] 6.16 scrub starts 2024-02-22T19:47:25.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:24.509267+0000 osd.7 (osd.7) 295 : cluster [DBG] 6.16 scrub ok 2024-02-22T19:47:25.536 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:25.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:47:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:47:26.065 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:26.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:26.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:26.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:26.203 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:26.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:26.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:47:26.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:26.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:25.120506+0000 osd.3 (osd.3) 316 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:47:26.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:25.123650+0000 osd.3 (osd.3) 317 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:47:26.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:25.343487+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? 172.21.15.53:0/3025576352' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:47:26.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:25.374292+0000 mgr.y (mgr.14535) 702 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:26.360 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:25.120506+0000 osd.3 (osd.3) 316 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:47:26.360 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:25.123650+0000 osd.3 (osd.3) 317 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:47:26.360 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:25.343487+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? 172.21.15.53:0/3025576352' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:47:26.360 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:25.374292+0000 mgr.y (mgr.14535) 702 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:26.362 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:26.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:26.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:26.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:25.120506+0000 osd.3 (osd.3) 316 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:47:26.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:25.123650+0000 osd.3 (osd.3) 317 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:47:26.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:25.343487+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? 172.21.15.53:0/3025576352' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:47:26.518 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:25.374292+0000 mgr.y (mgr.14535) 702 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:26.593 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:26.621 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:26.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:47:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:47:26.795 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:47:27.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:47:27.372 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:25.783540+0000 mgr.y (mgr.14535) 703 : cluster [DBG] pgmap v942: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:27.372 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:26.038624+0000 osd.2 (osd.2) 282 : cluster [DBG] 4.d scrub starts 2024-02-22T19:47:27.372 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:26.042096+0000 osd.2 (osd.2) 283 : cluster [DBG] 4.d scrub ok 2024-02-22T19:47:27.372 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:26.261144+0000 mon.a (mon.0) 1691 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:27.372 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:26.267301+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? 172.21.15.53:0/3025576352' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:47:27.372 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:26.267358+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e464: 8 total, 8 up, 5 in 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:26.526084+0000 osd.7 (osd.7) 296 : cluster [DBG] 1.0 deep-scrub starts 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:26.529477+0000 osd.7 (osd.7) 297 : cluster [DBG] 1.0 deep-scrub ok 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:25.783540+0000 mgr.y (mgr.14535) 703 : cluster [DBG] pgmap v942: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:26.038624+0000 osd.2 (osd.2) 282 : cluster [DBG] 4.d scrub starts 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:26.042096+0000 osd.2 (osd.2) 283 : cluster [DBG] 4.d scrub ok 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:26.261144+0000 mon.a (mon.0) 1691 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:26.267301+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? 172.21.15.53:0/3025576352' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:26.267358+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e464: 8 total, 8 up, 5 in 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:26.526084+0000 osd.7 (osd.7) 296 : cluster [DBG] 1.0 deep-scrub starts 2024-02-22T19:47:27.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:26.529477+0000 osd.7 (osd.7) 297 : cluster [DBG] 1.0 deep-scrub ok 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:25.783540+0000 mgr.y (mgr.14535) 703 : cluster [DBG] pgmap v942: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:26.038624+0000 osd.2 (osd.2) 282 : cluster [DBG] 4.d scrub starts 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:26.042096+0000 osd.2 (osd.2) 283 : cluster [DBG] 4.d scrub ok 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:26.261144+0000 mon.a (mon.0) 1691 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:26.267301+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? 172.21.15.53:0/3025576352' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:26.267358+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e464: 8 total, 8 up, 5 in 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:26.526084+0000 osd.7 (osd.7) 296 : cluster [DBG] 1.0 deep-scrub starts 2024-02-22T19:47:27.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:26.529477+0000 osd.7 (osd.7) 297 : cluster [DBG] 1.0 deep-scrub ok 2024-02-22T19:47:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:28.071 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:28.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:28.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:28.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:28.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:27.783876+0000 mgr.y (mgr.14535) 704 : cluster [DBG] pgmap v944: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 736 op/s 2024-02-22T19:47:28.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:27.783876+0000 mgr.y (mgr.14535) 704 : cluster [DBG] pgmap v944: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 736 op/s 2024-02-22T19:47:28.781 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:27.783876+0000 mgr.y (mgr.14535) 704 : cluster [DBG] pgmap v944: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 B/s wr, 736 op/s 2024-02-22T19:47:29.056 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:29.059 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:29.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:29.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:29.194 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:29.215 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:29.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:29.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:29.297 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:47:29.297 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:47:29.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:29.349 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:29.425 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:47:29.427 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:47:29.443 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:47:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:29] "GET /metrics HTTP/1.1" 200 176494 "" "Prometheus/2.33.4" 2024-02-22T19:47:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:29.552 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:29.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:29.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:29.409910+0000 mgr.y (mgr.14535) 705 : audit [DBG] from='client.25595 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:47:29.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:29.682 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:29.704 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:29.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:29.409910+0000 mgr.y (mgr.14535) 705 : audit [DBG] from='client.25595 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:47:29.706 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:29.409910+0000 mgr.y (mgr.14535) 705 : audit [DBG] from='client.25595 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:47:29.707 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:29.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:29.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:29.935 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:29.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:29.937 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:29.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:47:30.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:30.065 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:30.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:47:30.085 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:30.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:30.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:30.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:30.206 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":944,"stamp":"2024-02-22T19:47:27.783705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":2207,"num_read_kb":1959,"num_write":227468,"num_write_kb":7256106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":452,"num_bytes_recovered":15766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59217,"ondisk_log_size":59217,"up":414,"acting":414,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":414,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":4509808,"kb_used_data":18952,"kb_used_omap":0,"kb_used_meta":4490816,"kb_avail":464215952,"statfs":{"total":479975178240,"available":475357134848,"internally_reserved":0,"allocated":19406848,"data_stored":10754463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4598595584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-385810456,"num_objects":-5888,"num_object_clones":0,"num_object_copies":-17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5888,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5888,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002345"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":425,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478200+0000","last_change":"2024-02-22T19:45:56.598157+0000","last_active":"2024-02-22T19:47:17.478200+0000","last_peered":"2024-02-22T19:47:17.478200+0000","last_clean":"2024-02-22T19:47:17.478200+0000","last_became_active":"2024-02-22T19:38:47.157827+0000","last_became_peered":"2024-02-22T19:38:47.157827+0000","last_unstale":"2024-02-22T19:47:17.478200+0000","last_undegraded":"2024-02-22T19:47:17.478200+0000","last_fullsized":"2024-02-22T19:47:17.478200+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:56.598068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:56.598068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":460,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:20.457748+0000","last_change":"2024-02-22T19:47:20.457714+0000","last_active":"2024-02-22T19:47:20.457748+0000","last_peered":"2024-02-22T19:47:20.457748+0000","last_clean":"2024-02-22T19:47:20.457748+0000","last_became_active":"2024-02-22T19:41:52.114324+0000","last_became_peered":"2024-02-22T19:41:52.114324+0000","last_unstale":"2024-02-22T19:47:20.457748+0000","last_undegraded":"2024-02-22T19:47:20.457748+0000","last_fullsized":"2024-02-22T19:47:20.457748+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:20.457620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:20.457620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":240,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.179620+0000","last_change":"2024-02-22T19:46:09.901457+0000","last_active":"2024-02-22T19:47:15.179620+0000","last_peered":"2024-02-22T19:47:15.179620+0000","last_clean":"2024-02-22T19:47:15.179620+0000","last_became_active":"2024-02-22T19:41:52.109052+0000","last_became_peered":"2024-02-22T19:41:52.109052+0000","last_unstale":"2024-02-22T19:47:15.179620+0000","last_undegraded":"2024-02-22T19:47:15.179620+0000","last_fullsized":"2024-02-22T19:47:15.179620+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:46:09.901351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.901351+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":440,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488466+0000","last_change":"2024-02-22T19:46:14.159107+0000","last_active":"2024-02-22T19:47:17.488466+0000","last_peered":"2024-02-22T19:47:17.488466+0000","last_clean":"2024-02-22T19:47:17.488466+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:47:17.488466+0000","last_undegraded":"2024-02-22T19:47:17.488466+0000","last_fullsized":"2024-02-22T19:47:17.488466+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.159012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.159012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":441,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487526+0000","last_change":"2024-02-22T19:44:56.776815+0000","last_active":"2024-02-22T19:47:17.487526+0000","last_peered":"2024-02-22T19:47:17.487526+0000","last_clean":"2024-02-22T19:47:17.487526+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:47:17.487526+0000","last_undegraded":"2024-02-22T19:47:17.487526+0000","last_fullsized":"2024-02-22T19:47:17.487526+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:56.776721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.776721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":548,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.486471+0000","last_change":"2024-02-22T19:44:07.984802+0000","last_active":"2024-02-22T19:47:17.486471+0000","last_peered":"2024-02-22T19:47:17.486471+0000","last_clean":"2024-02-22T19:47:17.486471+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:47:17.486471+0000","last_undegraded":"2024-02-22T19:47:17.486471+0000","last_fullsized":"2024-02-22T19:47:17.486471+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:44:07.984710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.984710+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":268,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.179119+0000","last_change":"2024-02-22T19:46:13.905945+0000","last_active":"2024-02-22T19:47:15.179119+0000","last_peered":"2024-02-22T19:47:15.179119+0000","last_clean":"2024-02-22T19:47:15.179119+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:47:15.179119+0000","last_undegraded":"2024-02-22T19:47:15.179119+0000","last_fullsized":"2024-02-22T19:47:15.179119+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.905878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.905878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":278,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478269+0000","last_change":"2024-02-22T19:44:07.795043+0000","last_active":"2024-02-22T19:47:17.478269+0000","last_peered":"2024-02-22T19:47:17.478269+0000","last_clean":"2024-02-22T19:47:17.478269+0000","last_became_active":"2024-02-22T19:41:52.113142+0000","last_became_peered":"2024-02-22T19:41:52.113142+0000","last_unstale":"2024-02-22T19:47:17.478269+0000","last_undegraded":"2024-02-22T19:47:17.478269+0000","last_fullsized":"2024-02-22T19:47:17.478269+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:07.794943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:07.794943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":345,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487290+0000","last_change":"2024-02-22T19:45:02.829140+0000","last_active":"2024-02-22T19:47:17.487290+0000","last_peered":"2024-02-22T19:47:17.487290+0000","last_clean":"2024-02-22T19:47:17.487290+0000","last_became_active":"2024-02-22T19:41:52.111423+0000","last_became_peered":"2024-02-22T19:41:52.111423+0000","last_unstale":"2024-02-22T19:47:17.487290+0000","last_undegraded":"2024-02-22T19:47:17.487290+0000","last_fullsized":"2024-02-22T19:47:17.487290+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:45:02.829064+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:45:02.829064+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":265,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477215+0000","last_change":"2024-02-22T19:45:03.818156+0000","last_active":"2024-02-22T19:47:17.477215+0000","last_peered":"2024-02-22T19:47:17.477215+0000","last_clean":"2024-02-22T19:47:17.477215+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:47:17.477215+0000","last_undegraded":"2024-02-22T19:47:17.477215+0000","last_fullsized":"2024-02-22T19:47:17.477215+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.818051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":295,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477888+0000","last_change":"2024-02-22T19:44:44.719650+0000","last_active":"2024-02-22T19:47:17.477888+0000","last_peered":"2024-02-22T19:47:17.477888+0000","last_clean":"2024-02-22T19:47:17.477888+0000","last_became_active":"2024-02-22T19:41:52.098999+0000","last_became_peered":"2024-02-22T19:41:52.098999+0000","last_unstale":"2024-02-22T19:47:17.477888+0000","last_undegraded":"2024-02-22T19:47:17.477888+0000","last_fullsized":"2024-02-22T19:47:17.477888+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:44:44.719546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:44.719546+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":268,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488295+0000","last_change":"2024-02-22T19:44:04.949624+0000","last_active":"2024-02-22T19:47:17.488295+0000","last_peered":"2024-02-22T19:47:17.488295+0000","last_clean":"2024-02-22T19:47:17.488295+0000","last_became_active":"2024-02-22T19:38:47.095754+0000","last_became_peered":"2024-02-22T19:38:47.095754+0000","last_unstale":"2024-02-22T19:47:17.488295+0000","last_undegraded":"2024-02-22T19:47:17.488295+0000","last_fullsized":"2024-02-22T19:47:17.488295+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:44:04.949558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:04.949558+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":343,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478356+0000","last_change":"2024-02-22T19:46:18.486367+0000","last_active":"2024-02-22T19:47:17.478356+0000","last_peered":"2024-02-22T19:47:17.478356+0000","last_clean":"2024-02-22T19:47:17.478356+0000","last_became_active":"2024-02-22T19:41:52.111380+0000","last_became_peered":"2024-02-22T19:41:52.111380+0000","last_unstale":"2024-02-22T19:47:17.478356+0000","last_undegraded":"2024-02-22T19:47:17.478356+0000","last_fullsized":"2024-02-22T19:47:17.478356+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:18.486267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.486267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":246,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480737+0000","last_change":"2024-02-22T19:45:52.252022+0000","last_active":"2024-02-22T19:47:17.480737+0000","last_peered":"2024-02-22T19:47:17.480737+0000","last_clean":"2024-02-22T19:47:17.480737+0000","last_became_active":"2024-02-22T19:39:22.245194+0000","last_became_peered":"2024-02-22T19:39:22.245194+0000","last_unstale":"2024-02-22T19:47:17.480737+0000","last_undegraded":"2024-02-22T19:47:17.480737+0000","last_fullsized":"2024-02-22T19:47:17.480737+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.251960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.251960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":500,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.491525+0000","last_change":"2024-02-22T19:46:16.806927+0000","last_active":"2024-02-22T19:47:17.491525+0000","last_peered":"2024-02-22T19:47:17.491525+0000","last_clean":"2024-02-22T19:47:17.491525+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:47:17.491525+0000","last_undegraded":"2024-02-22T19:47:17.491525+0000","last_fullsized":"2024-02-22T19:47:17.491525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:46:16.806791+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:46:16.806791+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":458,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488103+0000","last_change":"2024-02-22T19:45:14.832581+0000","last_active":"2024-02-22T19:47:17.488103+0000","last_peered":"2024-02-22T19:47:17.488103+0000","last_clean":"2024-02-22T19:47:17.488103+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:47:17.488103+0000","last_undegraded":"2024-02-22T19:47:17.488103+0000","last_fullsized":"2024-02-22T19:47:17.488103+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.832482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.832482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":254,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178196+0000","last_change":"2024-02-22T19:45:03.679204+0000","last_active":"2024-02-22T19:47:15.178196+0000","last_peered":"2024-02-22T19:47:15.178196+0000","last_clean":"2024-02-22T19:47:15.178196+0000","last_became_active":"2024-02-22T19:39:22.252109+0000","last_became_peered":"2024-02-22T19:39:22.252109+0000","last_unstale":"2024-02-22T19:47:15.178196+0000","last_undegraded":"2024-02-22T19:47:15.178196+0000","last_fullsized":"2024-02-22T19:47:15.178196+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.679103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":509,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.489089+0000","last_change":"2024-02-22T19:44:43.774884+0000","last_active":"2024-02-22T19:47:17.489089+0000","last_peered":"2024-02-22T19:47:17.489089+0000","last_clean":"2024-02-22T19:47:17.489089+0000","last_became_active":"2024-02-22T19:41:52.112977+0000","last_became_peered":"2024-02-22T19:41:52.112977+0000","last_unstale":"2024-02-22T19:47:17.489089+0000","last_undegraded":"2024-02-22T19:47:17.489089+0000","last_fullsized":"2024-02-22T19:47:17.489089+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:44:43.774778+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:44:43.774778+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":479,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481144+0000","last_change":"2024-02-22T19:47:05.294685+0000","last_active":"2024-02-22T19:47:17.481144+0000","last_peered":"2024-02-22T19:47:17.481144+0000","last_clean":"2024-02-22T19:47:17.481144+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:47:17.481144+0000","last_undegraded":"2024-02-22T19:47:17.481144+0000","last_fullsized":"2024-02-22T19:47:17.481144+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:47:05.294613+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.294613+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":462,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178044+0000","last_change":"2024-02-22T19:45:09.724649+0000","last_active":"2024-02-22T19:47:15.178044+0000","last_peered":"2024-02-22T19:47:15.178044+0000","last_clean":"2024-02-22T19:47:15.178044+0000","last_became_active":"2024-02-22T19:41:52.109691+0000","last_became_peered":"2024-02-22T19:41:52.109691+0000","last_unstale":"2024-02-22T19:47:15.178044+0000","last_undegraded":"2024-02-22T19:47:15.178044+0000","last_fullsized":"2024-02-22T19:47:15.178044+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:09.724557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.724557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":282,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481329+0000","last_change":"2024-02-22T19:47:12.324026+0000","last_active":"2024-02-22T19:47:17.481329+0000","last_peered":"2024-02-22T19:47:17.481329+0000","last_clean":"2024-02-22T19:47:17.481329+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:47:17.481329+0000","last_undegraded":"2024-02-22T19:47:17.481329+0000","last_fullsized":"2024-02-22T19:47:17.481329+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:12.323915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.323915+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":462,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178170+0000","last_change":"2024-02-22T19:44:48.653802+0000","last_active":"2024-02-22T19:47:15.178170+0000","last_peered":"2024-02-22T19:47:15.178170+0000","last_clean":"2024-02-22T19:47:15.178170+0000","last_became_active":"2024-02-22T19:38:47.150095+0000","last_became_peered":"2024-02-22T19:38:47.150095+0000","last_unstale":"2024-02-22T19:47:15.178170+0000","last_undegraded":"2024-02-22T19:47:15.178170+0000","last_fullsized":"2024-02-22T19:47:15.178170+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.653696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.653696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":461,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.485544+0000","last_change":"2024-02-22T19:44:10.017728+0000","last_active":"2024-02-22T19:47:17.485544+0000","last_peered":"2024-02-22T19:47:17.485544+0000","last_clean":"2024-02-22T19:47:17.485544+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:47:17.485544+0000","last_undegraded":"2024-02-22T19:47:17.485544+0000","last_fullsized":"2024-02-22T19:47:17.485544+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:10.017666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:44:10.017666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":503,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.479211+0000","last_change":"2024-02-22T19:45:09.809874+0000","last_active":"2024-02-22T19:47:17.479211+0000","last_peered":"2024-02-22T19:47:17.479211+0000","last_clean":"2024-02-22T19:47:17.479211+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:47:17.479211+0000","last_undegraded":"2024-02-22T19:47:17.479211+0000","last_fullsized":"2024-02-22T19:47:17.479211+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:09.809806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.809806+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":448,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481460+0000","last_change":"2024-02-22T19:47:06.262224+0000","last_active":"2024-02-22T19:47:17.481460+0000","last_peered":"2024-02-22T19:47:17.481460+0000","last_clean":"2024-02-22T19:47:17.481460+0000","last_became_active":"2024-02-22T19:38:47.147705+0000","last_became_peered":"2024-02-22T19:38:47.147705+0000","last_unstale":"2024-02-22T19:47:17.481460+0000","last_undegraded":"2024-02-22T19:47:17.481460+0000","last_fullsized":"2024-02-22T19:47:17.481460+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.262153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.262153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":455,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478892+0000","last_change":"2024-02-22T19:44:13.694618+0000","last_active":"2024-02-22T19:47:17.478892+0000","last_peered":"2024-02-22T19:47:17.478892+0000","last_clean":"2024-02-22T19:47:17.478892+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:47:17.478892+0000","last_undegraded":"2024-02-22T19:47:17.478892+0000","last_fullsized":"2024-02-22T19:47:17.478892+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.694552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.694552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":462,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.485565+0000","last_change":"2024-02-22T19:46:09.018391+0000","last_active":"2024-02-22T19:47:17.485565+0000","last_peered":"2024-02-22T19:47:17.485565+0000","last_clean":"2024-02-22T19:47:17.485565+0000","last_became_active":"2024-02-22T19:41:52.110169+0000","last_became_peered":"2024-02-22T19:41:52.110169+0000","last_unstale":"2024-02-22T19:47:17.485565+0000","last_undegraded":"2024-02-22T19:47:17.485565+0000","last_fullsized":"2024-02-22T19:47:17.485565+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:46:09.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.018318+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":469,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488620+0000","last_change":"2024-02-22T19:47:12.136077+0000","last_active":"2024-02-22T19:47:17.488620+0000","last_peered":"2024-02-22T19:47:17.488620+0000","last_clean":"2024-02-22T19:47:17.488620+0000","last_became_active":"2024-02-22T19:41:52.118627+0000","last_became_peered":"2024-02-22T19:41:52.118627+0000","last_unstale":"2024-02-22T19:47:17.488620+0000","last_undegraded":"2024-02-22T19:47:17.488620+0000","last_fullsized":"2024-02-22T19:47:17.488620+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:12.135980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.135980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":260,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478283+0000","last_change":"2024-02-22T19:44:12.674961+0000","last_active":"2024-02-22T19:47:17.478283+0000","last_peered":"2024-02-22T19:47:17.478283+0000","last_clean":"2024-02-22T19:47:17.478283+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:47:17.478283+0000","last_undegraded":"2024-02-22T19:47:17.478283+0000","last_fullsized":"2024-02-22T19:47:17.478283+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.674856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.674856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":482,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.486316+0000","last_change":"2024-02-22T19:44:11.041545+0000","last_active":"2024-02-22T19:47:17.486316+0000","last_peered":"2024-02-22T19:47:17.486316+0000","last_clean":"2024-02-22T19:47:17.486316+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:47:17.486316+0000","last_undegraded":"2024-02-22T19:47:17.486316+0000","last_fullsized":"2024-02-22T19:47:17.486316+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:44:11.041458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:11.041458+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":446,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477274+0000","last_change":"2024-02-22T19:44:14.661384+0000","last_active":"2024-02-22T19:47:17.477274+0000","last_peered":"2024-02-22T19:47:17.477274+0000","last_clean":"2024-02-22T19:47:17.477274+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:47:17.477274+0000","last_undegraded":"2024-02-22T19:47:17.477274+0000","last_fullsized":"2024-02-22T19:47:17.477274+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:14.661322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:14.661322+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":483,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481015+0000","last_change":"2024-02-22T19:46:20.359190+0000","last_active":"2024-02-22T19:47:17.481015+0000","last_peered":"2024-02-22T19:47:17.481015+0000","last_clean":"2024-02-22T19:47:17.481015+0000","last_became_active":"2024-02-22T19:41:52.110802+0000","last_became_peered":"2024-02-22T19:41:52.110802+0000","last_unstale":"2024-02-22T19:47:17.481015+0000","last_undegraded":"2024-02-22T19:47:17.481015+0000","last_fullsized":"2024-02-22T19:47:17.481015+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.359130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.359130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":486,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178434+0000","last_change":"2024-02-22T19:45:53.878055+0000","last_active":"2024-02-22T19:47:15.178434+0000","last_peered":"2024-02-22T19:47:15.178434+0000","last_clean":"2024-02-22T19:47:15.178434+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:47:15.178434+0000","last_undegraded":"2024-02-22T19:47:15.178434+0000","last_fullsized":"2024-02-22T19:47:15.178434+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:45:53.877957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:53.877957+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":476,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.485968+0000","last_change":"2024-02-22T19:44:52.796416+0000","last_active":"2024-02-22T19:47:17.485968+0000","last_peered":"2024-02-22T19:47:17.485968+0000","last_clean":"2024-02-22T19:47:17.485968+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:47:17.485968+0000","last_undegraded":"2024-02-22T19:47:17.485968+0000","last_fullsized":"2024-02-22T19:47:17.485968+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:52.796310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.796310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":248,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481147+0000","last_change":"2024-02-22T19:45:04.416774+0000","last_active":"2024-02-22T19:47:17.481147+0000","last_peered":"2024-02-22T19:47:17.481147+0000","last_clean":"2024-02-22T19:47:17.481147+0000","last_became_active":"2024-02-22T19:41:52.103785+0000","last_became_peered":"2024-02-22T19:41:52.103785+0000","last_unstale":"2024-02-22T19:47:17.481147+0000","last_undegraded":"2024-02-22T19:47:17.481147+0000","last_fullsized":"2024-02-22T19:47:17.481147+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:04.416671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:45:04.416671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":249,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.180534+0000","last_change":"2024-02-22T19:45:57.920895+0000","last_active":"2024-02-22T19:47:15.180534+0000","last_peered":"2024-02-22T19:47:15.180534+0000","last_clean":"2024-02-22T19:47:15.180534+0000","last_became_active":"2024-02-22T19:41:52.095822+0000","last_became_peered":"2024-02-22T19:41:52.095822+0000","last_unstale":"2024-02-22T19:47:15.180534+0000","last_undegraded":"2024-02-22T19:47:15.180534+0000","last_fullsized":"2024-02-22T19:47:15.180534+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:57.920791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:57.920791+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":263,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481138+0000","last_change":"2024-02-22T19:44:41.261006+0000","last_active":"2024-02-22T19:47:17.481138+0000","last_peered":"2024-02-22T19:47:17.481138+0000","last_clean":"2024-02-22T19:47:17.481138+0000","last_became_active":"2024-02-22T19:39:22.248569+0000","last_became_peered":"2024-02-22T19:39:22.248569+0000","last_unstale":"2024-02-22T19:47:17.481138+0000","last_undegraded":"2024-02-22T19:47:17.481138+0000","last_fullsized":"2024-02-22T19:47:17.481138+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:41.260945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:41.260945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":456,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480850+0000","last_change":"2024-02-22T19:45:09.368245+0000","last_active":"2024-02-22T19:47:17.480850+0000","last_peered":"2024-02-22T19:47:17.480850+0000","last_clean":"2024-02-22T19:47:17.480850+0000","last_became_active":"2024-02-22T19:41:52.103526+0000","last_became_peered":"2024-02-22T19:41:52.103526+0000","last_unstale":"2024-02-22T19:47:17.480850+0000","last_undegraded":"2024-02-22T19:47:17.480850+0000","last_fullsized":"2024-02-22T19:47:17.480850+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:09.368133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.368133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":470,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487364+0000","last_change":"2024-02-22T19:46:10.060876+0000","last_active":"2024-02-22T19:47:17.487364+0000","last_peered":"2024-02-22T19:47:17.487364+0000","last_clean":"2024-02-22T19:47:17.487364+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:47:17.487364+0000","last_undegraded":"2024-02-22T19:47:17.487364+0000","last_fullsized":"2024-02-22T19:47:17.487364+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:46:10.060783+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:46:10.060783+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":453,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.494488+0000","last_change":"2024-02-22T19:47:08.073243+0000","last_active":"2024-02-22T19:47:17.494488+0000","last_peered":"2024-02-22T19:47:17.494488+0000","last_clean":"2024-02-22T19:47:17.494488+0000","last_became_active":"2024-02-22T19:39:22.261853+0000","last_became_peered":"2024-02-22T19:39:22.261853+0000","last_unstale":"2024-02-22T19:47:17.494488+0000","last_undegraded":"2024-02-22T19:47:17.494488+0000","last_fullsized":"2024-02-22T19:47:17.494488+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:08.073147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:47:08.073147+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":254,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481284+0000","last_change":"2024-02-22T19:46:17.328336+0000","last_active":"2024-02-22T19:47:17.481284+0000","last_peered":"2024-02-22T19:47:17.481284+0000","last_clean":"2024-02-22T19:47:17.481284+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:47:17.481284+0000","last_undegraded":"2024-02-22T19:47:17.481284+0000","last_fullsized":"2024-02-22T19:47:17.481284+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.328243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":284,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478435+0000","last_change":"2024-02-22T19:47:07.646871+0000","last_active":"2024-02-22T19:47:17.478435+0000","last_peered":"2024-02-22T19:47:17.478435+0000","last_clean":"2024-02-22T19:47:17.478435+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:47:17.478435+0000","last_undegraded":"2024-02-22T19:47:17.478435+0000","last_fullsized":"2024-02-22T19:47:17.478435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:07.646805+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:47:07.646805+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":476,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.491710+0000","last_change":"2024-02-22T19:47:05.985304+0000","last_active":"2024-02-22T19:47:17.491710+0000","last_peered":"2024-02-22T19:47:17.491710+0000","last_clean":"2024-02-22T19:47:17.491710+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:47:17.491710+0000","last_undegraded":"2024-02-22T19:47:17.491710+0000","last_fullsized":"2024-02-22T19:47:17.491710+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:05.985178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.985178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":483,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480715+0000","last_change":"2024-02-22T19:46:11.272398+0000","last_active":"2024-02-22T19:47:17.480715+0000","last_peered":"2024-02-22T19:47:17.480715+0000","last_clean":"2024-02-22T19:47:17.480715+0000","last_became_active":"2024-02-22T19:41:52.111213+0000","last_became_peered":"2024-02-22T19:41:52.111213+0000","last_unstale":"2024-02-22T19:47:17.480715+0000","last_undegraded":"2024-02-22T19:47:17.480715+0000","last_fullsized":"2024-02-22T19:47:17.480715+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.272326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.272326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":256,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487672+0000","last_change":"2024-02-22T19:44:47.735006+0000","last_active":"2024-02-22T19:47:17.487672+0000","last_peered":"2024-02-22T19:47:17.487672+0000","last_clean":"2024-02-22T19:47:17.487672+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:47:17.487672+0000","last_undegraded":"2024-02-22T19:47:17.487672+0000","last_fullsized":"2024-02-22T19:47:17.487672+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:44:47.734940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.734940+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":483,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480919+0000","last_change":"2024-02-22T19:47:03.314628+0000","last_active":"2024-02-22T19:47:17.480919+0000","last_peered":"2024-02-22T19:47:17.480919+0000","last_clean":"2024-02-22T19:47:17.480919+0000","last_became_active":"2024-02-22T19:41:52.110540+0000","last_became_peered":"2024-02-22T19:41:52.110540+0000","last_unstale":"2024-02-22T19:47:17.480919+0000","last_undegraded":"2024-02-22T19:47:17.480919+0000","last_fullsized":"2024-02-22T19:47:17.480919+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.314522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.314522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":432,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477144+0000","last_change":"2024-02-22T19:44:58.799472+0000","last_active":"2024-02-22T19:47:17.477144+0000","last_peered":"2024-02-22T19:47:17.477144+0000","last_clean":"2024-02-22T19:47:17.477144+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:47:17.477144+0000","last_undegraded":"2024-02-22T19:47:17.477144+0000","last_fullsized":"2024-02-22T19:47:17.477144+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:58.799397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:44:58.799397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":470,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488168+0000","last_change":"2024-02-22T19:45:01.840336+0000","last_active":"2024-02-22T19:47:17.488168+0000","last_peered":"2024-02-22T19:47:17.488168+0000","last_clean":"2024-02-22T19:47:17.488168+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:47:17.488168+0000","last_undegraded":"2024-02-22T19:47:17.488168+0000","last_fullsized":"2024-02-22T19:47:17.488168+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:45:01.840227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.840227+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"462'14234","reported_seq":14340,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.590314+0000","last_change":"2024-02-22T19:43:57.347864+0000","last_active":"2024-02-22T19:47:22.590314+0000","last_peered":"2024-02-22T19:47:22.590314+0000","last_clean":"2024-02-22T19:47:22.590314+0000","last_became_active":"2024-02-22T19:43:57.347537+0000","last_became_peered":"2024-02-22T19:43:57.347537+0000","last_unstale":"2024-02-22T19:47:22.590314+0000","last_undegraded":"2024-02-22T19:47:22.590314+0000","last_fullsized":"2024-02-22T19:47:22.590314+0000","mapping_epoch":365,"log_start":"447'10700","ondisk_log_start":"447'10700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3534,"ondisk_log_size":3534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14234,"num_write_kb":455488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":301,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478396+0000","last_change":"2024-02-22T19:45:52.702554+0000","last_active":"2024-02-22T19:47:17.478396+0000","last_peered":"2024-02-22T19:47:17.478396+0000","last_clean":"2024-02-22T19:47:17.478396+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:47:17.478396+0000","last_undegraded":"2024-02-22T19:47:17.478396+0000","last_fullsized":"2024-02-22T19:47:17.478396+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.702473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":533,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481130+0000","last_change":"2024-02-22T19:47:16.375471+0000","last_active":"2024-02-22T19:47:17.481130+0000","last_peered":"2024-02-22T19:47:17.481130+0000","last_clean":"2024-02-22T19:47:17.481130+0000","last_became_active":"2024-02-22T19:43:26.940589+0000","last_became_peered":"2024-02-22T19:43:26.940589+0000","last_unstale":"2024-02-22T19:47:17.481130+0000","last_undegraded":"2024-02-22T19:47:17.481130+0000","last_fullsized":"2024-02-22T19:47:17.481130+0000","mapping_epoch":357,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.375370+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":324,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477062+0000","last_change":"2024-02-22T19:46:03.485142+0000","last_active":"2024-02-22T19:47:17.477062+0000","last_peered":"2024-02-22T19:47:17.477062+0000","last_clean":"2024-02-22T19:47:17.477062+0000","last_became_active":"2024-02-22T19:41:52.112292+0000","last_became_peered":"2024-02-22T19:41:52.112292+0000","last_unstale":"2024-02-22T19:47:17.477062+0000","last_undegraded":"2024-02-22T19:47:17.477062+0000","last_fullsized":"2024-02-22T19:47:17.477062+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:03.485065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:03.485065+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"326'16","reported_seq":739,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.414794+0000","last_change":"2024-02-22T19:46:15.312716+0000","last_active":"2024-02-22T19:47:24.414794+0000","last_peered":"2024-02-22T19:47:24.414794+0000","last_clean":"2024-02-22T19:47:24.414794+0000","last_became_active":"2024-02-22T19:41:54.317554+0000","last_became_peered":"2024-02-22T19:41:54.317554+0000","last_unstale":"2024-02-22T19:47:24.414794+0000","last_undegraded":"2024-02-22T19:47:24.414794+0000","last_fullsized":"2024-02-22T19:47:24.414794+0000","mapping_epoch":325,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'16","last_scrub_stamp":"2024-02-22T19:46:15.312651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.312651+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"462'14246","reported_seq":14357,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.589042+0000","last_change":"2024-02-22T19:47:11.595137+0000","last_active":"2024-02-22T19:47:22.589042+0000","last_peered":"2024-02-22T19:47:22.589042+0000","last_clean":"2024-02-22T19:47:22.589042+0000","last_became_active":"2024-02-22T19:43:57.350467+0000","last_became_peered":"2024-02-22T19:43:57.350467+0000","last_unstale":"2024-02-22T19:47:22.589042+0000","last_undegraded":"2024-02-22T19:47:22.589042+0000","last_fullsized":"2024-02-22T19:47:22.589042+0000","mapping_epoch":365,"log_start":"448'10900","ondisk_log_start":"448'10900","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"459'13519","last_scrub_stamp":"2024-02-22T19:47:11.595037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:11.595037+0000","log_size":3346,"ondisk_log_size":3346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14246,"num_write_kb":455872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":426,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488542+0000","last_change":"2024-02-22T19:44:57.962949+0000","last_active":"2024-02-22T19:47:17.488542+0000","last_peered":"2024-02-22T19:47:17.488542+0000","last_clean":"2024-02-22T19:47:17.488542+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:47:17.488542+0000","last_undegraded":"2024-02-22T19:47:17.488542+0000","last_fullsized":"2024-02-22T19:47:17.488542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.962855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.962855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":175,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.496997+0000","last_change":"2024-02-22T19:47:14.069558+0000","last_active":"2024-02-22T19:47:17.496997+0000","last_peered":"2024-02-22T19:47:17.496997+0000","last_clean":"2024-02-22T19:47:17.496997+0000","last_became_active":"2024-02-22T19:41:26.804658+0000","last_became_peered":"2024-02-22T19:41:26.804658+0000","last_unstale":"2024-02-22T19:47:17.496997+0000","last_undegraded":"2024-02-22T19:47:17.496997+0000","last_fullsized":"2024-02-22T19:47:17.496997+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.069465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.069465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":313,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.177815+0000","last_change":"2024-02-22T19:45:08.766029+0000","last_active":"2024-02-22T19:47:15.177815+0000","last_peered":"2024-02-22T19:47:15.177815+0000","last_clean":"2024-02-22T19:47:15.177815+0000","last_became_active":"2024-02-22T19:41:52.109232+0000","last_became_peered":"2024-02-22T19:41:52.109232+0000","last_unstale":"2024-02-22T19:47:15.177815+0000","last_undegraded":"2024-02-22T19:47:15.177815+0000","last_fullsized":"2024-02-22T19:47:15.177815+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:45:08.765921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:08.765921+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":491,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.496684+0000","last_change":"2024-02-22T19:47:09.025632+0000","last_active":"2024-02-22T19:47:17.496684+0000","last_peered":"2024-02-22T19:47:17.496684+0000","last_clean":"2024-02-22T19:47:17.496684+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:47:17.496684+0000","last_undegraded":"2024-02-22T19:47:17.496684+0000","last_fullsized":"2024-02-22T19:47:17.496684+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:09.025536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.025536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"462'14034","reported_seq":14148,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.600393+0000","last_change":"2024-02-22T19:47:00.151647+0000","last_active":"2024-02-22T19:47:22.600393+0000","last_peered":"2024-02-22T19:47:22.600393+0000","last_clean":"2024-02-22T19:47:22.600393+0000","last_became_active":"2024-02-22T19:43:57.349636+0000","last_became_peered":"2024-02-22T19:43:57.349636+0000","last_unstale":"2024-02-22T19:47:22.600393+0000","last_undegraded":"2024-02-22T19:47:22.600393+0000","last_fullsized":"2024-02-22T19:47:22.600393+0000","mapping_epoch":365,"log_start":"448'10700","ondisk_log_start":"448'10700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12423","last_scrub_stamp":"2024-02-22T19:47:00.151543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.151543+0000","log_size":3334,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14034,"num_write_kb":449088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":432,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480379+0000","last_change":"2024-02-22T19:45:55.202582+0000","last_active":"2024-02-22T19:47:17.480379+0000","last_peered":"2024-02-22T19:47:17.480379+0000","last_clean":"2024-02-22T19:47:17.480379+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:47:17.480379+0000","last_undegraded":"2024-02-22T19:47:17.480379+0000","last_fullsized":"2024-02-22T19:47:17.480379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.202515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.202515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":517,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.495016+0000","last_change":"2024-02-22T19:47:06.968685+0000","last_active":"2024-02-22T19:47:17.495016+0000","last_peered":"2024-02-22T19:47:17.495016+0000","last_clean":"2024-02-22T19:47:17.495016+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:47:17.495016+0000","last_undegraded":"2024-02-22T19:47:17.495016+0000","last_fullsized":"2024-02-22T19:47:17.495016+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.968586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.968586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":470,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481397+0000","last_change":"2024-02-22T19:46:14.318040+0000","last_active":"2024-02-22T19:47:17.481397+0000","last_peered":"2024-02-22T19:47:17.481397+0000","last_clean":"2024-02-22T19:47:17.481397+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:47:17.481397+0000","last_undegraded":"2024-02-22T19:47:17.481397+0000","last_fullsized":"2024-02-22T19:47:17.481397+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.317978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.317978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":460,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178344+0000","last_change":"2024-02-22T19:44:19.749503+0000","last_active":"2024-02-22T19:47:15.178344+0000","last_peered":"2024-02-22T19:47:15.178344+0000","last_clean":"2024-02-22T19:47:15.178344+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:47:15.178344+0000","last_undegraded":"2024-02-22T19:47:15.178344+0000","last_fullsized":"2024-02-22T19:47:15.178344+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:19.749430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:19.749430+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"462'13926","reported_seq":14042,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.060200+0000","last_change":"2024-02-22T19:47:24.060164+0000","last_active":"2024-02-22T19:47:24.060200+0000","last_peered":"2024-02-22T19:47:24.060200+0000","last_clean":"2024-02-22T19:47:24.060200+0000","last_became_active":"2024-02-22T19:43:57.347657+0000","last_became_peered":"2024-02-22T19:43:57.347657+0000","last_unstale":"2024-02-22T19:47:24.060200+0000","last_undegraded":"2024-02-22T19:47:24.060200+0000","last_fullsized":"2024-02-22T19:47:24.060200+0000","mapping_epoch":365,"log_start":"445'10100","ondisk_log_start":"445'10100","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"462'13926","last_scrub_stamp":"2024-02-22T19:47:24.060064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.060064+0000","log_size":3826,"ondisk_log_size":3826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13926,"num_write_kb":445632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":254,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481017+0000","last_change":"2024-02-22T19:44:57.360296+0000","last_active":"2024-02-22T19:47:17.481017+0000","last_peered":"2024-02-22T19:47:17.481017+0000","last_clean":"2024-02-22T19:47:17.481017+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:47:17.481017+0000","last_undegraded":"2024-02-22T19:47:17.481017+0000","last_fullsized":"2024-02-22T19:47:17.481017+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.360215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.360215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":496,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.492876+0000","last_change":"2024-02-22T19:47:00.864299+0000","last_active":"2024-02-22T19:47:17.492876+0000","last_peered":"2024-02-22T19:47:17.492876+0000","last_clean":"2024-02-22T19:47:17.492876+0000","last_became_active":"2024-02-22T19:39:22.267242+0000","last_became_peered":"2024-02-22T19:39:22.267242+0000","last_unstale":"2024-02-22T19:47:17.492876+0000","last_undegraded":"2024-02-22T19:47:17.492876+0000","last_fullsized":"2024-02-22T19:47:17.492876+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:00.864208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.864208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":278,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477785+0000","last_change":"2024-02-22T19:47:03.617591+0000","last_active":"2024-02-22T19:47:17.477785+0000","last_peered":"2024-02-22T19:47:17.477785+0000","last_clean":"2024-02-22T19:47:17.477785+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:47:17.477785+0000","last_undegraded":"2024-02-22T19:47:17.477785+0000","last_fullsized":"2024-02-22T19:47:17.477785+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.617526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.617526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":276,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:23.007964+0000","last_change":"2024-02-22T19:47:23.007925+0000","last_active":"2024-02-22T19:47:23.007964+0000","last_peered":"2024-02-22T19:47:23.007964+0000","last_clean":"2024-02-22T19:47:23.007964+0000","last_became_active":"2024-02-22T19:39:22.262313+0000","last_became_peered":"2024-02-22T19:39:22.262313+0000","last_unstale":"2024-02-22T19:47:23.007964+0000","last_undegraded":"2024-02-22T19:47:23.007964+0000","last_fullsized":"2024-02-22T19:47:23.007964+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:47:23.007849+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:47:23.007849+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"462'14324","reported_seq":14430,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.598079+0000","last_change":"2024-02-22T19:43:57.351947+0000","last_active":"2024-02-22T19:47:22.598079+0000","last_peered":"2024-02-22T19:47:22.598079+0000","last_clean":"2024-02-22T19:47:22.598079+0000","last_became_active":"2024-02-22T19:43:57.351647+0000","last_became_peered":"2024-02-22T19:43:57.351647+0000","last_unstale":"2024-02-22T19:47:22.598079+0000","last_undegraded":"2024-02-22T19:47:22.598079+0000","last_fullsized":"2024-02-22T19:47:22.598079+0000","mapping_epoch":365,"log_start":"447'10800","ondisk_log_start":"447'10800","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:43:57.131561+0000","log_size":3524,"ondisk_log_size":3524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14324,"num_write_kb":458368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":303,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477152+0000","last_change":"2024-02-22T19:46:11.428955+0000","last_active":"2024-02-22T19:47:17.477152+0000","last_peered":"2024-02-22T19:47:17.477152+0000","last_clean":"2024-02-22T19:47:17.477152+0000","last_became_active":"2024-02-22T19:39:22.261667+0000","last_became_peered":"2024-02-22T19:39:22.261667+0000","last_unstale":"2024-02-22T19:47:17.477152+0000","last_undegraded":"2024-02-22T19:47:17.477152+0000","last_fullsized":"2024-02-22T19:47:17.477152+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.428898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.428898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":449,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487761+0000","last_change":"2024-02-22T19:46:13.030959+0000","last_active":"2024-02-22T19:47:17.487761+0000","last_peered":"2024-02-22T19:47:17.487761+0000","last_clean":"2024-02-22T19:47:17.487761+0000","last_became_active":"2024-02-22T19:41:52.103880+0000","last_became_peered":"2024-02-22T19:41:52.103880+0000","last_unstale":"2024-02-22T19:47:17.487761+0000","last_undegraded":"2024-02-22T19:47:17.487761+0000","last_fullsized":"2024-02-22T19:47:17.487761+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.030866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.030866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":274,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477768+0000","last_change":"2024-02-22T19:47:15.553391+0000","last_active":"2024-02-22T19:47:17.477768+0000","last_peered":"2024-02-22T19:47:17.477768+0000","last_clean":"2024-02-22T19:47:17.477768+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:47:17.477768+0000","last_undegraded":"2024-02-22T19:47:17.477768+0000","last_fullsized":"2024-02-22T19:47:17.477768+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.553324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.553324+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":388,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.718400+0000","last_change":"2024-02-22T19:47:17.718400+0000","last_active":"2024-02-22T19:47:17.718400+0000","last_peered":"2024-02-22T19:47:17.718400+0000","last_clean":"2024-02-22T19:47:17.718400+0000","last_became_active":"2024-02-22T19:41:52.110405+0000","last_became_peered":"2024-02-22T19:41:52.110405+0000","last_unstale":"2024-02-22T19:47:17.718400+0000","last_undegraded":"2024-02-22T19:47:17.718400+0000","last_fullsized":"2024-02-22T19:47:17.718400+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.718312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.718312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":736,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.494699+0000","last_change":"2024-02-22T19:46:14.876056+0000","last_active":"2024-02-22T19:47:17.494699+0000","last_peered":"2024-02-22T19:47:17.494699+0000","last_clean":"2024-02-22T19:47:17.494699+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:47:17.494699+0000","last_undegraded":"2024-02-22T19:47:17.494699+0000","last_fullsized":"2024-02-22T19:47:17.494699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:46:14.875950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.875950+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"419'20","reported_seq":371,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:19.415420+0000","last_change":"2024-02-22T19:45:05.413797+0000","last_active":"2024-02-22T19:47:19.415420+0000","last_peered":"2024-02-22T19:47:19.415420+0000","last_clean":"2024-02-22T19:47:19.415420+0000","last_became_active":"2024-02-22T19:45:05.413407+0000","last_became_peered":"2024-02-22T19:45:05.413407+0000","last_unstale":"2024-02-22T19:47:19.415420+0000","last_undegraded":"2024-02-22T19:47:19.415420+0000","last_fullsized":"2024-02-22T19:47:19.415420+0000","mapping_epoch":322,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.782332+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"462'14348","reported_seq":14459,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.586401+0000","last_change":"2024-02-22T19:47:09.916480+0000","last_active":"2024-02-22T19:47:22.586401+0000","last_peered":"2024-02-22T19:47:22.586401+0000","last_clean":"2024-02-22T19:47:22.586401+0000","last_became_active":"2024-02-22T19:43:57.346743+0000","last_became_peered":"2024-02-22T19:43:57.346743+0000","last_unstale":"2024-02-22T19:47:22.586401+0000","last_undegraded":"2024-02-22T19:47:22.586401+0000","last_fullsized":"2024-02-22T19:47:22.586401+0000","mapping_epoch":365,"log_start":"447'10800","ondisk_log_start":"447'10800","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"458'13446","last_scrub_stamp":"2024-02-22T19:47:09.916392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.916392+0000","log_size":3548,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14348,"num_write_kb":459136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":496,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:21.507150+0000","last_change":"2024-02-22T19:47:21.507116+0000","last_active":"2024-02-22T19:47:21.507150+0000","last_peered":"2024-02-22T19:47:21.507150+0000","last_clean":"2024-02-22T19:47:21.507150+0000","last_became_active":"2024-02-22T19:39:22.253054+0000","last_became_peered":"2024-02-22T19:39:22.253054+0000","last_unstale":"2024-02-22T19:47:21.507150+0000","last_undegraded":"2024-02-22T19:47:21.507150+0000","last_fullsized":"2024-02-22T19:47:21.507150+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.507020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.507020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":446,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481536+0000","last_change":"2024-02-22T19:47:15.447527+0000","last_active":"2024-02-22T19:47:17.481536+0000","last_peered":"2024-02-22T19:47:17.481536+0000","last_clean":"2024-02-22T19:47:17.481536+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:47:17.481536+0000","last_undegraded":"2024-02-22T19:47:17.481536+0000","last_fullsized":"2024-02-22T19:47:17.481536+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.447427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.447427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":453,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.179427+0000","last_change":"2024-02-22T19:46:58.646300+0000","last_active":"2024-02-22T19:47:15.179427+0000","last_peered":"2024-02-22T19:47:15.179427+0000","last_clean":"2024-02-22T19:47:15.179427+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:47:15.179427+0000","last_undegraded":"2024-02-22T19:47:15.179427+0000","last_fullsized":"2024-02-22T19:47:15.179427+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:58.646210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.646210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":263,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.490926+0000","last_change":"2024-02-22T19:47:17.081986+0000","last_active":"2024-02-22T19:47:17.490926+0000","last_peered":"2024-02-22T19:47:17.490926+0000","last_clean":"2024-02-22T19:47:17.490926+0000","last_became_active":"2024-02-22T19:39:22.312767+0000","last_became_peered":"2024-02-22T19:39:22.312767+0000","last_unstale":"2024-02-22T19:47:17.490926+0000","last_undegraded":"2024-02-22T19:47:17.490926+0000","last_fullsized":"2024-02-22T19:47:17.490926+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.081880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.081880+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":547,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.494569+0000","last_change":"2024-02-22T19:46:58.874444+0000","last_active":"2024-02-22T19:47:17.494569+0000","last_peered":"2024-02-22T19:47:17.494569+0000","last_clean":"2024-02-22T19:47:17.494569+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:47:17.494569+0000","last_undegraded":"2024-02-22T19:47:17.494569+0000","last_fullsized":"2024-02-22T19:47:17.494569+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:46:58.874350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.874350+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"415'30","reported_seq":538,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.414816+0000","last_change":"2024-02-22T19:47:10.006289+0000","last_active":"2024-02-22T19:47:24.414816+0000","last_peered":"2024-02-22T19:47:24.414816+0000","last_clean":"2024-02-22T19:47:24.414816+0000","last_became_active":"2024-02-22T19:45:01.732679+0000","last_became_peered":"2024-02-22T19:45:01.732679+0000","last_unstale":"2024-02-22T19:47:24.414816+0000","last_undegraded":"2024-02-22T19:47:24.414816+0000","last_fullsized":"2024-02-22T19:47:24.414816+0000","mapping_epoch":217,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":415,"parent":"0.0","parent_split_bits":0,"last_scrub":"415'30","last_scrub_stamp":"2024-02-22T19:47:10.006180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.006180+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"462'14512","reported_seq":14622,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.601953+0000","last_change":"2024-02-22T19:47:15.918738+0000","last_active":"2024-02-22T19:47:22.601953+0000","last_peered":"2024-02-22T19:47:22.601953+0000","last_clean":"2024-02-22T19:47:22.601953+0000","last_became_active":"2024-02-22T19:43:57.344303+0000","last_became_peered":"2024-02-22T19:43:57.344303+0000","last_unstale":"2024-02-22T19:47:22.601953+0000","last_undegraded":"2024-02-22T19:47:22.601953+0000","last_fullsized":"2024-02-22T19:47:22.601953+0000","mapping_epoch":365,"log_start":"445'10700","ondisk_log_start":"445'10700","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"461'14078","last_scrub_stamp":"2024-02-22T19:47:15.918632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.918632+0000","log_size":3812,"ondisk_log_size":3812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14512,"num_write_kb":464384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":274,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.479259+0000","last_change":"2024-02-22T19:44:17.656581+0000","last_active":"2024-02-22T19:47:17.479259+0000","last_peered":"2024-02-22T19:47:17.479259+0000","last_clean":"2024-02-22T19:47:17.479259+0000","last_became_active":"2024-02-22T19:38:47.158165+0000","last_became_peered":"2024-02-22T19:38:47.158165+0000","last_unstale":"2024-02-22T19:47:17.479259+0000","last_undegraded":"2024-02-22T19:47:17.479259+0000","last_fullsized":"2024-02-22T19:47:17.479259+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.656518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":491,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480648+0000","last_change":"2024-02-22T19:44:49.368004+0000","last_active":"2024-02-22T19:47:17.480648+0000","last_peered":"2024-02-22T19:47:17.480648+0000","last_clean":"2024-02-22T19:47:17.480648+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:47:17.480648+0000","last_undegraded":"2024-02-22T19:47:17.480648+0000","last_fullsized":"2024-02-22T19:47:17.480648+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:49.367906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:49.367906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":459,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487145+0000","last_change":"2024-02-22T19:44:09.027439+0000","last_active":"2024-02-22T19:47:17.487145+0000","last_peered":"2024-02-22T19:47:17.487145+0000","last_clean":"2024-02-22T19:47:17.487145+0000","last_became_active":"2024-02-22T19:41:52.110935+0000","last_became_peered":"2024-02-22T19:41:52.110935+0000","last_unstale":"2024-02-22T19:47:17.487145+0000","last_undegraded":"2024-02-22T19:47:17.487145+0000","last_fullsized":"2024-02-22T19:47:17.487145+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:44:09.027320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":499,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481318+0000","last_change":"2024-02-22T19:46:12.320420+0000","last_active":"2024-02-22T19:47:17.481318+0000","last_peered":"2024-02-22T19:47:17.481318+0000","last_clean":"2024-02-22T19:47:17.481318+0000","last_became_active":"2024-02-22T19:41:52.110026+0000","last_became_peered":"2024-02-22T19:41:52.110026+0000","last_unstale":"2024-02-22T19:47:17.481318+0000","last_undegraded":"2024-02-22T19:47:17.481318+0000","last_fullsized":"2024-02-22T19:47:17.481318+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:46:12.320306+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.320306+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"423'13","reported_seq":524,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.414781+0000","last_change":"2024-02-22T19:45:09.548942+0000","last_active":"2024-02-22T19:47:24.414781+0000","last_peered":"2024-02-22T19:47:24.414781+0000","last_clean":"2024-02-22T19:47:24.414781+0000","last_became_active":"2024-02-22T19:45:09.547565+0000","last_became_peered":"2024-02-22T19:45:09.547565+0000","last_unstale":"2024-02-22T19:47:24.414781+0000","last_undegraded":"2024-02-22T19:47:24.414781+0000","last_fullsized":"2024-02-22T19:47:24.414781+0000","mapping_epoch":217,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'11","last_scrub_stamp":"2024-02-22T19:44:52.364238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.364238+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"462'14022","reported_seq":14133,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.591532+0000","last_change":"2024-02-22T19:46:08.745923+0000","last_active":"2024-02-22T19:47:22.591532+0000","last_peered":"2024-02-22T19:47:22.591532+0000","last_clean":"2024-02-22T19:47:22.591532+0000","last_became_active":"2024-02-22T19:43:57.346554+0000","last_became_peered":"2024-02-22T19:43:57.346554+0000","last_unstale":"2024-02-22T19:47:22.591532+0000","last_undegraded":"2024-02-22T19:47:22.591532+0000","last_fullsized":"2024-02-22T19:47:22.591532+0000","mapping_epoch":365,"log_start":"445'10200","ondisk_log_start":"445'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'8994","last_scrub_stamp":"2024-02-22T19:46:08.745819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:46:08.745819+0000","log_size":3822,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14022,"num_write_kb":448704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":344,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.180336+0000","last_change":"2024-02-22T19:44:55.655563+0000","last_active":"2024-02-22T19:47:15.180336+0000","last_peered":"2024-02-22T19:47:15.180336+0000","last_clean":"2024-02-22T19:47:15.180336+0000","last_became_active":"2024-02-22T19:41:52.110897+0000","last_became_peered":"2024-02-22T19:41:52.110897+0000","last_unstale":"2024-02-22T19:47:15.180336+0000","last_undegraded":"2024-02-22T19:47:15.180336+0000","last_fullsized":"2024-02-22T19:47:15.180336+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:55.655474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:55.655474+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":456,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.181513+0000","last_change":"2024-02-22T19:46:17.792837+0000","last_active":"2024-02-22T19:47:15.181513+0000","last_peered":"2024-02-22T19:47:15.181513+0000","last_clean":"2024-02-22T19:47:15.181513+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:47:15.181513+0000","last_undegraded":"2024-02-22T19:47:15.181513+0000","last_fullsized":"2024-02-22T19:47:15.181513+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.792736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.792736+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":247,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.486101+0000","last_change":"2024-02-22T19:47:16.015960+0000","last_active":"2024-02-22T19:47:17.486101+0000","last_peered":"2024-02-22T19:47:17.486101+0000","last_clean":"2024-02-22T19:47:17.486101+0000","last_became_active":"2024-02-22T19:39:22.246637+0000","last_became_peered":"2024-02-22T19:39:22.246637+0000","last_unstale":"2024-02-22T19:47:17.486101+0000","last_undegraded":"2024-02-22T19:47:17.486101+0000","last_fullsized":"2024-02-22T19:47:17.486101+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:16.015884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.015884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1586,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.157828+0000","last_change":"2024-02-22T19:47:19.438721+0000","last_active":"2024-02-22T19:47:24.157828+0000","last_peered":"2024-02-22T19:47:24.157828+0000","last_clean":"2024-02-22T19:47:24.157828+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:47:24.157828+0000","last_undegraded":"2024-02-22T19:47:24.157828+0000","last_fullsized":"2024-02-22T19:47:24.157828+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:47:19.438615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:19.438615+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1086,"num_read_kb":1081,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":273,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178775+0000","last_change":"2024-02-22T19:44:56.653960+0000","last_active":"2024-02-22T19:47:15.178775+0000","last_peered":"2024-02-22T19:47:15.178775+0000","last_clean":"2024-02-22T19:47:15.178775+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:47:15.178775+0000","last_undegraded":"2024-02-22T19:47:15.178775+0000","last_fullsized":"2024-02-22T19:47:15.178775+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:44:56.653848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.653848+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"412'25","reported_seq":368,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.414850+0000","last_change":"2024-02-22T19:45:06.780716+0000","last_active":"2024-02-22T19:47:24.414850+0000","last_peered":"2024-02-22T19:47:24.414850+0000","last_clean":"2024-02-22T19:47:24.414850+0000","last_became_active":"2024-02-22T19:44:57.599424+0000","last_became_peered":"2024-02-22T19:44:57.599424+0000","last_unstale":"2024-02-22T19:47:24.414850+0000","last_undegraded":"2024-02-22T19:47:24.414850+0000","last_fullsized":"2024-02-22T19:47:24.414850+0000","mapping_epoch":267,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":411,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'25","last_scrub_stamp":"2024-02-22T19:45:06.780605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:45:06.780605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"462'13848","reported_seq":13962,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.605409+0000","last_change":"2024-02-22T19:47:10.418043+0000","last_active":"2024-02-22T19:47:22.605409+0000","last_peered":"2024-02-22T19:47:22.605409+0000","last_clean":"2024-02-22T19:47:22.605409+0000","last_became_active":"2024-02-22T19:43:57.347455+0000","last_became_peered":"2024-02-22T19:43:57.347455+0000","last_unstale":"2024-02-22T19:47:22.605409+0000","last_undegraded":"2024-02-22T19:47:22.605409+0000","last_fullsized":"2024-02-22T19:47:22.605409+0000","mapping_epoch":365,"log_start":"448'10500","ondisk_log_start":"448'10500","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"459'13065","last_scrub_stamp":"2024-02-22T19:47:10.417940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.417940+0000","log_size":3348,"ondisk_log_size":3348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13848,"num_write_kb":443073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":321,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487446+0000","last_change":"2024-02-22T19:45:55.024849+0000","last_active":"2024-02-22T19:47:17.487446+0000","last_peered":"2024-02-22T19:47:17.487446+0000","last_clean":"2024-02-22T19:47:17.487446+0000","last_became_active":"2024-02-22T19:41:52.103543+0000","last_became_peered":"2024-02-22T19:41:52.103543+0000","last_unstale":"2024-02-22T19:47:17.487446+0000","last_undegraded":"2024-02-22T19:47:17.487446+0000","last_fullsized":"2024-02-22T19:47:17.487446+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.024753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.024753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":442,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.177906+0000","last_change":"2024-02-22T19:44:53.631267+0000","last_active":"2024-02-22T19:47:15.177906+0000","last_peered":"2024-02-22T19:47:15.177906+0000","last_clean":"2024-02-22T19:47:15.177906+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:47:15.177906+0000","last_undegraded":"2024-02-22T19:47:15.177906+0000","last_fullsized":"2024-02-22T19:47:15.177906+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:53.631188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.631188+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":457,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.181682+0000","last_change":"2024-02-22T19:44:13.731072+0000","last_active":"2024-02-22T19:47:15.181682+0000","last_peered":"2024-02-22T19:47:15.181682+0000","last_clean":"2024-02-22T19:47:15.181682+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:47:15.181682+0000","last_undegraded":"2024-02-22T19:47:15.181682+0000","last_fullsized":"2024-02-22T19:47:15.181682+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.730975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.730975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"463'37","reported_seq":292,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.474522+0000","last_change":"2024-02-22T19:47:24.474522+0000","last_active":"2024-02-22T19:47:24.474522+0000","last_peered":"2024-02-22T19:47:24.474522+0000","last_clean":"2024-02-22T19:47:24.474522+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:47:24.474522+0000","last_undegraded":"2024-02-22T19:47:24.474522+0000","last_fullsized":"2024-02-22T19:47:24.474522+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"463'37","last_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_deep_scrub":"463'37","last_deep_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.474452+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":446,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477500+0000","last_change":"2024-02-22T19:44:54.804568+0000","last_active":"2024-02-22T19:47:17.477500+0000","last_peered":"2024-02-22T19:47:17.477500+0000","last_clean":"2024-02-22T19:47:17.477500+0000","last_became_active":"2024-02-22T19:44:45.398192+0000","last_became_peered":"2024-02-22T19:44:45.398192+0000","last_unstale":"2024-02-22T19:47:17.477500+0000","last_undegraded":"2024-02-22T19:47:17.477500+0000","last_fullsized":"2024-02-22T19:47:17.477500+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":399,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:54.804469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:54.804469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"462'14286","reported_seq":14397,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.567338+0000","last_change":"2024-02-22T19:46:57.953017+0000","last_active":"2024-02-22T19:47:22.567338+0000","last_peered":"2024-02-22T19:47:22.567338+0000","last_clean":"2024-02-22T19:47:22.567338+0000","last_became_active":"2024-02-22T19:43:57.351385+0000","last_became_peered":"2024-02-22T19:43:57.351385+0000","last_unstale":"2024-02-22T19:47:22.567338+0000","last_undegraded":"2024-02-22T19:47:22.567338+0000","last_fullsized":"2024-02-22T19:47:22.567338+0000","mapping_epoch":365,"log_start":"445'10400","ondisk_log_start":"445'10400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12580","last_scrub_stamp":"2024-02-22T19:46:57.952946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:46:57.952946+0000","log_size":3886,"ondisk_log_size":3886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14286,"num_write_kb":457152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":173,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487844+0000","last_change":"2024-02-22T19:44:46.735448+0000","last_active":"2024-02-22T19:47:17.487844+0000","last_peered":"2024-02-22T19:47:17.487844+0000","last_clean":"2024-02-22T19:47:17.487844+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:47:17.487844+0000","last_undegraded":"2024-02-22T19:47:17.487844+0000","last_fullsized":"2024-02-22T19:47:17.487844+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:46.735345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:44:46.735345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":254,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.485512+0000","last_change":"2024-02-22T19:44:49.777279+0000","last_active":"2024-02-22T19:47:17.485512+0000","last_peered":"2024-02-22T19:47:17.485512+0000","last_clean":"2024-02-22T19:47:17.485512+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:47:17.485512+0000","last_undegraded":"2024-02-22T19:47:17.485512+0000","last_fullsized":"2024-02-22T19:47:17.485512+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:49.777178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:44:49.777178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":265,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478350+0000","last_change":"2024-02-22T19:46:19.538051+0000","last_active":"2024-02-22T19:47:17.478350+0000","last_peered":"2024-02-22T19:47:17.478350+0000","last_clean":"2024-02-22T19:47:17.478350+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:47:17.478350+0000","last_undegraded":"2024-02-22T19:47:17.478350+0000","last_fullsized":"2024-02-22T19:47:17.478350+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:19.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:46:19.537988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":577,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178383+0000","last_change":"2024-02-22T19:46:18.828987+0000","last_active":"2024-02-22T19:47:15.178383+0000","last_peered":"2024-02-22T19:47:15.178383+0000","last_clean":"2024-02-22T19:47:15.178383+0000","last_became_active":"2024-02-22T19:39:22.252589+0000","last_became_peered":"2024-02-22T19:39:22.252589+0000","last_unstale":"2024-02-22T19:47:15.178383+0000","last_undegraded":"2024-02-22T19:47:15.178383+0000","last_fullsized":"2024-02-22T19:47:15.178383+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:46:18.828884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.828884+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"395'16","reported_seq":556,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.414843+0000","last_change":"2024-02-22T19:46:58.513539+0000","last_active":"2024-02-22T19:47:24.414843+0000","last_peered":"2024-02-22T19:47:24.414843+0000","last_clean":"2024-02-22T19:47:24.414843+0000","last_became_active":"2024-02-22T19:44:41.682349+0000","last_became_peered":"2024-02-22T19:44:41.682349+0000","last_unstale":"2024-02-22T19:47:24.414843+0000","last_undegraded":"2024-02-22T19:47:24.414843+0000","last_fullsized":"2024-02-22T19:47:24.414843+0000","mapping_epoch":322,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'16","last_scrub_stamp":"2024-02-22T19:46:58.513473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.513473+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"462'14108","reported_seq":14219,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.589592+0000","last_change":"2024-02-22T19:47:02.888460+0000","last_active":"2024-02-22T19:47:22.589592+0000","last_peered":"2024-02-22T19:47:22.589592+0000","last_clean":"2024-02-22T19:47:22.589592+0000","last_became_active":"2024-02-22T19:43:57.348228+0000","last_became_peered":"2024-02-22T19:43:57.348228+0000","last_unstale":"2024-02-22T19:47:22.589592+0000","last_undegraded":"2024-02-22T19:47:22.589592+0000","last_fullsized":"2024-02-22T19:47:22.589592+0000","mapping_epoch":365,"log_start":"448'10800","ondisk_log_start":"448'10800","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12733","last_scrub_stamp":"2024-02-22T19:47:02.888397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:02.888397+0000","log_size":3308,"ondisk_log_size":3308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14108,"num_write_kb":451456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":350,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:18.018324+0000","last_change":"2024-02-22T19:47:18.018282+0000","last_active":"2024-02-22T19:47:18.018324+0000","last_peered":"2024-02-22T19:47:18.018324+0000","last_clean":"2024-02-22T19:47:18.018324+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:47:18.018324+0000","last_undegraded":"2024-02-22T19:47:18.018324+0000","last_fullsized":"2024-02-22T19:47:18.018324+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:18.018184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:18.018184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":444,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.480912+0000","last_change":"2024-02-22T19:44:45.370939+0000","last_active":"2024-02-22T19:47:17.480912+0000","last_peered":"2024-02-22T19:47:17.480912+0000","last_clean":"2024-02-22T19:47:17.480912+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:47:17.480912+0000","last_undegraded":"2024-02-22T19:47:17.480912+0000","last_fullsized":"2024-02-22T19:47:17.480912+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:45.370834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.370834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":262,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178569+0000","last_change":"2024-02-22T19:44:47.633310+0000","last_active":"2024-02-22T19:47:15.178569+0000","last_peered":"2024-02-22T19:47:15.178569+0000","last_clean":"2024-02-22T19:47:15.178569+0000","last_became_active":"2024-02-22T19:39:22.245223+0000","last_became_peered":"2024-02-22T19:39:22.245223+0000","last_unstale":"2024-02-22T19:47:15.178569+0000","last_undegraded":"2024-02-22T19:47:15.178569+0000","last_fullsized":"2024-02-22T19:47:15.178569+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:47.633242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.633242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":580,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477445+0000","last_change":"2024-02-22T19:44:45.716073+0000","last_active":"2024-02-22T19:47:17.477445+0000","last_peered":"2024-02-22T19:47:17.477445+0000","last_clean":"2024-02-22T19:47:17.477445+0000","last_became_active":"2024-02-22T19:41:52.106548+0000","last_became_peered":"2024-02-22T19:41:52.106548+0000","last_unstale":"2024-02-22T19:47:17.477445+0000","last_undegraded":"2024-02-22T19:47:17.477445+0000","last_fullsized":"2024-02-22T19:47:17.477445+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:44:45.715955+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:44:45.715955+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"407'13","reported_seq":717,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-02-22T19:47:24.392791+0000","last_change":"2024-02-22T19:44:53.757698+0000","last_active":"2024-02-22T19:47:24.392791+0000","last_peered":"2024-02-22T19:47:24.392791+0000","last_clean":"2024-02-22T19:47:24.392791+0000","last_became_active":"2024-02-22T19:44:53.415371+0000","last_became_peered":"2024-02-22T19:44:53.415371+0000","last_unstale":"2024-02-22T19:47:24.392791+0000","last_undegraded":"2024-02-22T19:47:24.392791+0000","last_fullsized":"2024-02-22T19:47:24.392791+0000","mapping_epoch":267,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":407,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:44:53.757594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.757594+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"462'14288","reported_seq":14407,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.598819+0000","last_change":"2024-02-22T19:47:22.192784+0000","last_active":"2024-02-22T19:47:22.598819+0000","last_peered":"2024-02-22T19:47:22.598819+0000","last_clean":"2024-02-22T19:47:22.598819+0000","last_became_active":"2024-02-22T19:43:57.348032+0000","last_became_peered":"2024-02-22T19:43:57.348032+0000","last_unstale":"2024-02-22T19:47:22.598819+0000","last_undegraded":"2024-02-22T19:47:22.598819+0000","last_fullsized":"2024-02-22T19:47:22.598819+0000","mapping_epoch":365,"log_start":"445'10400","ondisk_log_start":"445'10400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"462'14264","last_scrub_stamp":"2024-02-22T19:47:22.192715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:22.192715+0000","log_size":3888,"ondisk_log_size":3888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14288,"num_write_kb":457216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":257,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:21.070846+0000","last_change":"2024-02-22T19:47:21.070821+0000","last_active":"2024-02-22T19:47:21.070846+0000","last_peered":"2024-02-22T19:47:21.070846+0000","last_clean":"2024-02-22T19:47:21.070846+0000","last_became_active":"2024-02-22T19:41:52.102302+0000","last_became_peered":"2024-02-22T19:41:52.102302+0000","last_unstale":"2024-02-22T19:47:21.070846+0000","last_undegraded":"2024-02-22T19:47:21.070846+0000","last_fullsized":"2024-02-22T19:47:21.070846+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.070731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.070731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":246,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487374+0000","last_change":"2024-02-22T19:46:20.075957+0000","last_active":"2024-02-22T19:47:17.487374+0000","last_peered":"2024-02-22T19:47:17.487374+0000","last_clean":"2024-02-22T19:47:17.487374+0000","last_became_active":"2024-02-22T19:41:52.103448+0000","last_became_peered":"2024-02-22T19:41:52.103448+0000","last_unstale":"2024-02-22T19:47:17.487374+0000","last_undegraded":"2024-02-22T19:47:17.487374+0000","last_fullsized":"2024-02-22T19:47:17.487374+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.075867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.075867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":464,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487951+0000","last_change":"2024-02-22T19:45:58.995918+0000","last_active":"2024-02-22T19:47:17.487951+0000","last_peered":"2024-02-22T19:47:17.487951+0000","last_clean":"2024-02-22T19:47:17.487951+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:47:17.487951+0000","last_undegraded":"2024-02-22T19:47:17.487951+0000","last_fullsized":"2024-02-22T19:47:17.487951+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:58.995851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:45:58.995851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":568,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.485667+0000","last_change":"2024-02-22T19:45:09.692752+0000","last_active":"2024-02-22T19:47:17.485667+0000","last_peered":"2024-02-22T19:47:17.485667+0000","last_clean":"2024-02-22T19:47:17.485667+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:47:17.485667+0000","last_undegraded":"2024-02-22T19:47:17.485667+0000","last_fullsized":"2024-02-22T19:47:17.485667+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:45:09.692661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.692661+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":273,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.486231+0000","last_change":"2024-02-22T19:44:51.773621+0000","last_active":"2024-02-22T19:47:17.486231+0000","last_peered":"2024-02-22T19:47:17.486231+0000","last_clean":"2024-02-22T19:47:17.486231+0000","last_became_active":"2024-02-22T19:44:49.491617+0000","last_became_peered":"2024-02-22T19:44:49.491617+0000","last_unstale":"2024-02-22T19:47:17.486231+0000","last_undegraded":"2024-02-22T19:47:17.486231+0000","last_fullsized":"2024-02-22T19:47:17.486231+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":403,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:51.773527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:51.773527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"462'14042","reported_seq":14153,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.594729+0000","last_change":"2024-02-22T19:47:01.724088+0000","last_active":"2024-02-22T19:47:22.594729+0000","last_peered":"2024-02-22T19:47:22.594729+0000","last_clean":"2024-02-22T19:47:22.594729+0000","last_became_active":"2024-02-22T19:43:57.347224+0000","last_became_peered":"2024-02-22T19:43:57.347224+0000","last_unstale":"2024-02-22T19:47:22.594729+0000","last_undegraded":"2024-02-22T19:47:22.594729+0000","last_fullsized":"2024-02-22T19:47:22.594729+0000","mapping_epoch":365,"log_start":"445'10200","ondisk_log_start":"445'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"456'12645","last_scrub_stamp":"2024-02-22T19:47:01.723982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:01.723982+0000","log_size":3842,"ondisk_log_size":3842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14042,"num_write_kb":449344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":277,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478285+0000","last_change":"2024-02-22T19:45:00.729978+0000","last_active":"2024-02-22T19:47:17.478285+0000","last_peered":"2024-02-22T19:47:17.478285+0000","last_clean":"2024-02-22T19:47:17.478285+0000","last_became_active":"2024-02-22T19:41:52.107464+0000","last_became_peered":"2024-02-22T19:41:52.107464+0000","last_unstale":"2024-02-22T19:47:17.478285+0000","last_undegraded":"2024-02-22T19:47:17.478285+0000","last_fullsized":"2024-02-22T19:47:17.478285+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.729907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.729907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":292,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478190+0000","last_change":"2024-02-22T19:46:17.513957+0000","last_active":"2024-02-22T19:47:17.478190+0000","last_peered":"2024-02-22T19:47:17.478190+0000","last_clean":"2024-02-22T19:47:17.478190+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:47:17.478190+0000","last_undegraded":"2024-02-22T19:47:17.478190+0000","last_fullsized":"2024-02-22T19:47:17.478190+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.513841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.513841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":437,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.485617+0000","last_change":"2024-02-22T19:44:12.048204+0000","last_active":"2024-02-22T19:47:17.485617+0000","last_peered":"2024-02-22T19:47:17.485617+0000","last_clean":"2024-02-22T19:47:17.485617+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:47:17.485617+0000","last_undegraded":"2024-02-22T19:47:17.485617+0000","last_fullsized":"2024-02-22T19:47:17.485617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.048068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.048068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":181,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.478966+0000","last_change":"2024-02-22T19:46:15.453110+0000","last_active":"2024-02-22T19:47:17.478966+0000","last_peered":"2024-02-22T19:47:17.478966+0000","last_clean":"2024-02-22T19:47:17.478966+0000","last_became_active":"2024-02-22T19:41:26.814931+0000","last_became_peered":"2024-02-22T19:41:26.814931+0000","last_unstale":"2024-02-22T19:47:17.478966+0000","last_undegraded":"2024-02-22T19:47:17.478966+0000","last_fullsized":"2024-02-22T19:47:17.478966+0000","mapping_epoch":312,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":313,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:15.453047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.453047+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"462'14000","reported_seq":14110,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.447582+0000","last_change":"2024-02-22T19:47:04.139059+0000","last_active":"2024-02-22T19:47:22.447582+0000","last_peered":"2024-02-22T19:47:22.447582+0000","last_clean":"2024-02-22T19:47:22.447582+0000","last_became_active":"2024-02-22T19:43:57.345992+0000","last_became_peered":"2024-02-22T19:43:57.345992+0000","last_unstale":"2024-02-22T19:47:22.447582+0000","last_undegraded":"2024-02-22T19:47:22.447582+0000","last_fullsized":"2024-02-22T19:47:22.447582+0000","mapping_epoch":365,"log_start":"445'10200","ondisk_log_start":"445'10200","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'12751","last_scrub_stamp":"2024-02-22T19:47:04.138958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:04.138958+0000","log_size":3800,"ondisk_log_size":3800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14000,"num_write_kb":448000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":453,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487272+0000","last_change":"2024-02-22T19:44:42.822533+0000","last_active":"2024-02-22T19:47:17.487272+0000","last_peered":"2024-02-22T19:47:17.487272+0000","last_clean":"2024-02-22T19:47:17.487272+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:47:17.487272+0000","last_undegraded":"2024-02-22T19:47:17.487272+0000","last_fullsized":"2024-02-22T19:47:17.487272+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:42.822463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:42.822463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":247,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.491005+0000","last_change":"2024-02-22T19:46:20.790826+0000","last_active":"2024-02-22T19:47:17.491005+0000","last_peered":"2024-02-22T19:47:17.491005+0000","last_clean":"2024-02-22T19:47:17.491005+0000","last_became_active":"2024-02-22T19:38:47.083224+0000","last_became_peered":"2024-02-22T19:38:47.083224+0000","last_unstale":"2024-02-22T19:47:17.491005+0000","last_undegraded":"2024-02-22T19:47:17.491005+0000","last_fullsized":"2024-02-22T19:47:17.491005+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.790735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.790735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":256,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178694+0000","last_change":"2024-02-22T19:47:10.692993+0000","last_active":"2024-02-22T19:47:15.178694+0000","last_peered":"2024-02-22T19:47:15.178694+0000","last_clean":"2024-02-22T19:47:15.178694+0000","last_became_active":"2024-02-22T19:41:52.100768+0000","last_became_peered":"2024-02-22T19:41:52.100768+0000","last_unstale":"2024-02-22T19:47:15.178694+0000","last_undegraded":"2024-02-22T19:47:15.178694+0000","last_fullsized":"2024-02-22T19:47:15.178694+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:10.692928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.692928+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3","version":"462'14306","reported_seq":14422,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.591862+0000","last_change":"2024-02-22T19:47:20.159513+0000","last_active":"2024-02-22T19:47:22.591862+0000","last_peered":"2024-02-22T19:47:22.591862+0000","last_clean":"2024-02-22T19:47:22.591862+0000","last_became_active":"2024-02-22T19:43:57.347832+0000","last_became_peered":"2024-02-22T19:43:57.347832+0000","last_unstale":"2024-02-22T19:47:22.591862+0000","last_undegraded":"2024-02-22T19:47:22.591862+0000","last_fullsized":"2024-02-22T19:47:22.591862+0000","mapping_epoch":365,"log_start":"445'10500","ondisk_log_start":"445'10500","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"462'14150","last_scrub_stamp":"2024-02-22T19:47:20.159417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:20.159417+0000","log_size":3806,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14306,"num_write_kb":457792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":265,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.477238+0000","last_change":"2024-02-22T19:44:48.769697+0000","last_active":"2024-02-22T19:47:17.477238+0000","last_peered":"2024-02-22T19:47:17.477238+0000","last_clean":"2024-02-22T19:47:17.477238+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:47:17.477238+0000","last_undegraded":"2024-02-22T19:47:17.477238+0000","last_fullsized":"2024-02-22T19:47:17.477238+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.769602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.769602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":462,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.494203+0000","last_change":"2024-02-22T19:46:17.949817+0000","last_active":"2024-02-22T19:47:17.494203+0000","last_peered":"2024-02-22T19:47:17.494203+0000","last_clean":"2024-02-22T19:47:17.494203+0000","last_became_active":"2024-02-22T19:41:52.107300+0000","last_became_peered":"2024-02-22T19:41:52.107300+0000","last_unstale":"2024-02-22T19:47:17.494203+0000","last_undegraded":"2024-02-22T19:47:17.494203+0000","last_fullsized":"2024-02-22T19:47:17.494203+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.949728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.949728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":378,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178225+0000","last_change":"2024-02-22T19:47:14.714599+0000","last_active":"2024-02-22T19:47:15.178225+0000","last_peered":"2024-02-22T19:47:15.178225+0000","last_clean":"2024-02-22T19:47:15.178225+0000","last_became_active":"2024-02-22T19:41:52.109797+0000","last_became_peered":"2024-02-22T19:41:52.109797+0000","last_unstale":"2024-02-22T19:47:15.178225+0000","last_undegraded":"2024-02-22T19:47:15.178225+0000","last_fullsized":"2024-02-22T19:47:15.178225+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.714507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.714507+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":348,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.497552+0000","last_change":"2024-02-22T19:46:09.851700+0000","last_active":"2024-02-22T19:47:17.497552+0000","last_peered":"2024-02-22T19:47:17.497552+0000","last_clean":"2024-02-22T19:47:17.497552+0000","last_became_active":"2024-02-22T19:41:52.106476+0000","last_became_peered":"2024-02-22T19:41:52.106476+0000","last_unstale":"2024-02-22T19:47:17.497552+0000","last_undegraded":"2024-02-22T19:47:17.497552+0000","last_fullsized":"2024-02-22T19:47:17.497552+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:46:09.851636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.851636+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"462'14230","reported_seq":14340,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:22.542066+0000","last_change":"2024-02-22T19:47:13.940276+0000","last_active":"2024-02-22T19:47:22.542066+0000","last_peered":"2024-02-22T19:47:22.542066+0000","last_clean":"2024-02-22T19:47:22.542066+0000","last_became_active":"2024-02-22T19:43:57.351420+0000","last_became_peered":"2024-02-22T19:43:57.351420+0000","last_unstale":"2024-02-22T19:47:22.542066+0000","last_undegraded":"2024-02-22T19:47:22.542066+0000","last_fullsized":"2024-02-22T19:47:22.542066+0000","mapping_epoch":365,"log_start":"445'10400","ondisk_log_start":"445'10400","created":365,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"460'13632","last_scrub_stamp":"2024-02-22T19:47:13.940207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:43:57.131561+0000","last_clean_scrub_stamp":"2024-02-22T19:47:13.940207+0000","log_size":3830,"ondisk_log_size":3830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14230,"num_write_kb":455360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":455,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.481253+0000","last_change":"2024-02-22T19:44:59.333051+0000","last_active":"2024-02-22T19:47:17.481253+0000","last_peered":"2024-02-22T19:47:17.481253+0000","last_clean":"2024-02-22T19:47:17.481253+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:47:17.481253+0000","last_undegraded":"2024-02-22T19:47:17.481253+0000","last_fullsized":"2024-02-22T19:47:17.481253+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:59.332960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:59.332960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":282,"reported_epoch":461,"state":"active+clean","last_fresh":"2024-02-22T19:47:15.178445+0000","last_change":"2024-02-22T19:45:15.816947+0000","last_active":"2024-02-22T19:47:15.178445+0000","last_peered":"2024-02-22T19:47:15.178445+0000","last_clean":"2024-02-22T19:47:15.178445+0000","last_became_active":"2024-02-22T19:38:47.170520+0000","last_became_peered":"2024-02-22T19:38:47.170520+0000","last_unstale":"2024-02-22T19:47:15.178445+0000","last_undegraded":"2024-02-22T19:47:15.178445+0000","last_fullsized":"2024-02-22T19:47:15.178445+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:45:15.816835+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:45:15.816835+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":498,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.488381+0000","last_change":"2024-02-22T19:46:12.047830+0000","last_active":"2024-02-22T19:47:17.488381+0000","last_peered":"2024-02-22T19:47:17.488381+0000","last_clean":"2024-02-22T19:47:17.488381+0000","last_became_active":"2024-02-22T19:39:22.244614+0000","last_became_peered":"2024-02-22T19:39:22.244614+0000","last_unstale":"2024-02-22T19:47:17.488381+0000","last_undegraded":"2024-02-22T19:47:17.488381+0000","last_fullsized":"2024-02-22T19:47:17.488381+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:46:12.047696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.047696+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":312,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-02-22T19:47:17.487509+0000","last_change":"2024-02-22T19:44:40.870790+0000","last_active":"2024-02-22T19:47:17.487509+0000","last_peered":"2024-02-22T19:47:17.487509+0000","last_clean":"2024-02-22T19:47:17.487509+0000","last_became_active":"2024-02-22T19:41:52.109615+0000","last_became_peered":"2024-02-22T19:41:52.109615+0000","last_unstale":"2024-02-22T19:47:17.487509+0000","last_undegraded":"2024-02-22T19:47:17.487509+0000","last_fullsized":"2024-02-22T19:47:17.487509+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:44:40.870681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:40.870681+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226754,"num_write_kb":7256065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58454,"ondisk_log_size":58454,"up":48,"acting":48,"num_store_stats":5},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1086,"num_read_kb":1081,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":6615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1103,"num_read_kb":860,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":8822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":728,"ondisk_log_size":728,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266912,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":896592,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":892800,"kb_avail":92848560,"statfs":{"total":95995035648,"available":95076925440,"internally_reserved":0,"allocated":3874816,"data_stored":2151410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":914227200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":1.252,"15min":1.5760000000000001},"min":{"1min":0.23100000000000001,"5min":0.219,"15min":0.19900000000000001},"max":{"1min":3.0390000000000001,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.98699999999999999,"15min":1.524},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.20699999999999999},"max":{"1min":3.7389999999999999,"5min":5.1349999999999998,"15min":42.155999999999999},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8580000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.20000000000000001},"max":{"1min":4.1180000000000003,"5min":36.509,"15min":54.951000000000001},"last":0.62},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.4830000000000001,"15min":2.133},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":6.1559999999999997,"5min":7.1449999999999996,"15min":57.012},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":2.698,"5min":1.6819999999999999,"15min":1.7050000000000001},"min":{"1min":0.33800000000000002,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":25.053999999999998,"5min":25.053999999999998,"15min":41.640999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":3.234,"5min":1.8979999999999999,"15min":2},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":24.956,"5min":24.956,"15min":41.722000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.97,"15min":2.0230000000000001},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":6.694,"5min":36.491999999999997,"15min":41.457000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.5029999999999999,"5min":2.157,"15min":2.0619999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.5,"5min":36.732999999999997,"15min":41.311999999999998},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.70399999999999996,"15min":0.73899999999999999},"min":{"1min":0.28100000000000003,"5min":0.25900000000000001,"15min":0.23999999999999999},"max":{"1min":4.0910000000000002,"5min":4.4219999999999997,"15min":6.4720000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.94599999999999995,"15min":0.77600000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":3.1120000000000001,"5min":5.0910000000000002,"15min":7.6219999999999999},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.0029999999999999,"15min":0.86799999999999999},"min":{"1min":0.28699999999999998,"5min":0.27200000000000002,"15min":0.27000000000000002},"max":{"1min":1.8069999999999999,"5min":6.625,"15min":6.9039999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.80700000000000005,"15min":0.81599999999999995},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":1.776,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.67400000000000004,"15min":0.84099999999999997},"min":{"1min":0.35199999999999998,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":1.9339999999999999,"5min":3.956,"15min":12.199999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":1.1040000000000001,"15min":0.93700000000000006},"min":{"1min":0.36099999999999999,"5min":0.317,"15min":0.28100000000000003},"max":{"1min":1.952,"5min":6.6719999999999997,"15min":12.067},"last":0.501}]}]},{"osd":6,"up_from":0,"seq":193273528549,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575102,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":903388,"kb_used_data":3796,"kb_used_omap":0,"kb_used_meta":899584,"kb_avail":92841764,"statfs":{"total":95995035648,"available":95069966336,"internally_reserved":0,"allocated":3887104,"data_stored":2151386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":921174016},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:10 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":2.4430000000000001,"15min":2.1869999999999998},"min":{"1min":0.20999999999999999,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":4.6769999999999996,"5min":51.091000000000001,"15min":51.091000000000001},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.8380000000000001,"5min":2.2999999999999998,"15min":2.2650000000000001},"min":{"1min":0.28499999999999998,"5min":0.215,"15min":0.17499999999999999},"max":{"1min":5.8630000000000004,"5min":51.289999999999999,"15min":51.289999999999999},"last":3.512}]},{"osd":2,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":2.4369999999999998,"5min":2.9049999999999998,"15min":2.387},"min":{"1min":0.23000000000000001,"5min":0.191,"15min":0.191},"max":{"1min":7.0430000000000001,"5min":51.027000000000001,"15min":51.027000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":3.1339999999999999,"5min":3.0659999999999998,"15min":2.528},"min":{"1min":0.29899999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":12.957000000000001,"5min":51.369999999999997,"15min":51.369999999999997},"last":5.8479999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:46:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":2.4380000000000002,"15min":2.3919999999999999},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.214},"max":{"1min":6.2670000000000003,"5min":51.755000000000003,"15min":51.755000000000003},"last":3.2250000000000001},{"interface":"front","average":{"1min":2.9710000000000001,"5min":2.895,"15min":2.6320000000000001},"min":{"1min":0.44500000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":9.6820000000000004,"5min":51.741,"15min":51.741},"last":4.1360000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:47:15 2024","interfaces":[{"interface":"back","average":{"1min":1.7609999999999999,"5min":2.5390000000000001,"15min":2},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.24399999999999999},"max":{"1min":4.6379999999999999,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":1.597,"15min":1.518},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":2.4020000000000001,"5min":51.276000000000003,"15min":51.276000000000003},"last":2.5510000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:46:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":2.6680000000000001,"15min":2.0209999999999999},"min":{"1min":0.34399999999999997,"5min":0.32800000000000001,"15min":0.28100000000000003},"max":{"1min":6.2210000000000001,"5min":51.183999999999997,"15min":51.183999999999997},"last":0.438},{"interface":"front","average":{"1min":1.591,"5min":2.3999999999999999,"15min":1.7989999999999999},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":6.3129999999999997,"5min":51.212000000000003,"15min":51.212000000000003},"last":3.452}]},{"osd":6,"last update":"Thu Feb 22 19:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.8560000000000001,"15min":1.6319999999999999},"min":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.28599999999999998},"max":{"1min":2.8130000000000002,"5min":51.079999999999998,"15min":51.079999999999998},"last":2.593},{"interface":"front","average":{"1min":1.859,"5min":2.8109999999999999,"15min":2.056},"min":{"1min":0.51100000000000001,"5min":0.44,"15min":0.34999999999999998},"max":{"1min":4.657,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":2.5640000000000001,"15min":2.0019999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":6.327,"5min":51.244999999999997,"15min":51.244999999999997},"last":3.484},{"interface":"front","average":{"1min":1.1459999999999999,"5min":2.028,"15min":1.641},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.3040000000000003,"5min":51.317999999999998,"15min":51.317999999999998},"last":2.5779999999999998}]}]},{"osd":0,"up_from":0,"seq":34359738627,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411576,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":899208,"kb_used_data":3776,"kb_used_omap":0,"kb_used_meta":895424,"kb_avail":92845944,"statfs":{"total":95995035648,"available":95074246656,"internally_reserved":0,"allocated":3866624,"data_stored":2149677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":916914176},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:22 2024","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":1.6399999999999999,"15min":1.9850000000000001},"min":{"1min":0.222,"5min":0.217,"15min":0.18099999999999999},"max":{"1min":8.7409999999999997,"5min":8.7409999999999997,"15min":22.338999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":2.1579999999999999,"5min":2.1230000000000002,"15min":2.4390000000000001},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.17599999999999999},"max":{"1min":5.5030000000000001,"5min":14.603999999999999,"15min":30.427},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:47:22 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":1.907,"15min":2.0750000000000002},"min":{"1min":0.34599999999999997,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":15.996,"5min":15.996,"15min":15.996},"last":0.24399999999999999},{"interface":"front","average":{"1min":3.8599999999999999,"5min":2.218,"15min":2.1040000000000001},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.193},"max":{"1min":11.986000000000001,"5min":11.986000000000001,"15min":13.782},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:46:55 2024","interfaces":[{"interface":"back","average":{"1min":2.2320000000000002,"5min":2.0939999999999999,"15min":2.379},"min":{"1min":0.312,"5min":0.27000000000000002,"15min":0.20999999999999999},"max":{"1min":6.5049999999999999,"5min":12.151999999999999,"15min":30.071000000000002},"last":0.308},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.3540000000000001,"15min":2.6309999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":9.2829999999999995,"5min":28.263999999999999,"15min":32.119},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4850000000000001,"15min":1.579},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.26400000000000001},"max":{"1min":4.0789999999999997,"5min":8.2289999999999992,"15min":12.43},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.742,"5min":1.375,"15min":1.4670000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.253},"max":{"1min":5.5039999999999996,"5min":5.7999999999999998,"15min":10.34},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.6160000000000001,"15min":1.673},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.5190000000000001,"5min":14.045,"15min":40.633000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.9750000000000001,"15min":1.9379999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.26600000000000001},"max":{"1min":7.9450000000000003,"5min":14.513999999999999,"15min":40.997999999999998},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.532,"15min":1.696},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.311},"max":{"1min":2.4649999999999999,"5min":8.2530000000000001,"15min":21.640000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.583,"15min":1.552},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":5.5380000000000003,"5min":5.8869999999999996,"15min":10.369999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:47:22 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.7270000000000001,"15min":1.7989999999999999},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.6820000000000004,"5min":14.199,"15min":14.199},"last":0.38500000000000001},{"interface":"front","average":{"1min":2.8879999999999999,"5min":2.0379999999999998,"15min":1.7949999999999999},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.29899999999999999},"max":{"1min":8.8520000000000003,"5min":18.021999999999998,"15min":18.021999999999998},"last":0.46000000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149938,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":907740,"kb_used_data":3796,"kb_used_omap":0,"kb_used_meta":903936,"kb_avail":92837412,"statfs":{"total":95995035648,"available":95065509888,"internally_reserved":0,"allocated":3887104,"data_stored":2151279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":925630464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:01 2024","interfaces":[{"interface":"back","average":{"1min":1.976,"5min":1.911,"15min":2.1819999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":6.6440000000000001,"5min":9.3599999999999994,"15min":23.48},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.97,"5min":2.141,"15min":2.5329999999999999},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.20799999999999999},"max":{"1min":6.7350000000000003,"5min":10.297000000000001,"15min":23.411999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:47:01 2024","interfaces":[{"interface":"back","average":{"1min":8.9220000000000006,"5min":3.839,"15min":3.383},"min":{"1min":0.214,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":94.454999999999998,"5min":94.454999999999998,"15min":94.454999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":9.5449999999999999,"5min":3.5539999999999998,"15min":2.8679999999999999},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":95.197999999999993,"5min":95.197999999999993,"15min":95.197999999999993},"last":0.31}]},{"osd":2,"last update":"Thu Feb 22 19:47:01 2024","interfaces":[{"interface":"back","average":{"1min":2.851,"5min":2.4409999999999998,"15min":2.5219999999999998},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":10.753,"5min":11.577,"15min":20.105},"last":0.36099999999999999},{"interface":"front","average":{"1min":3.5169999999999999,"5min":2.6549999999999998,"15min":2.5600000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.217},"max":{"1min":14.308999999999999,"5min":17.986000000000001,"15min":35.939999999999998},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.5840000000000001,"15min":1.925},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":2.7669999999999999,"5min":9.968,"15min":13.680999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.613,"15min":1.8069999999999999},"min":{"1min":0.318,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":7.8719999999999999,"5min":7.8719999999999999,"15min":13.548},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:46:37 2024","interfaces":[{"interface":"back","average":{"1min":2.077,"5min":1.7729999999999999,"15min":1.758},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.718,"5min":9.3160000000000007,"15min":12.879},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.385,"5min":1.8120000000000001,"15min":1.9590000000000001},"min":{"1min":0.32500000000000001,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":4.9279999999999999,"5min":10.041,"15min":13.707000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:47:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":1.7889999999999999,"15min":1.804},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":7.6859999999999999,"5min":9.423,"15min":12.898999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.8560000000000001,"15min":2.0579999999999998},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.802,"5min":10.02,"15min":13.803000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:47:18 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.6459999999999999,"15min":1.8500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":6.8049999999999997,"5min":8.0879999999999992,"15min":13.483000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.913,"15min":1.9219999999999999},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":6.9710000000000001,"5min":9.3930000000000007,"15min":12.846},"last":0.63200000000000001}]}]},{"osd":4,"up_from":0,"seq":137438953712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":39,"seq":167503724778,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":902880,"kb_used_data":3800,"kb_used_omap":0,"kb_used_meta":899072,"kb_avail":92842272,"statfs":{"total":95995035648,"available":95070486528,"internally_reserved":0,"allocated":3891200,"data_stored":2150711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":920649728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1619999999999999,"15min":1.45},"min":{"1min":0.26900000000000002,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":4.8810000000000002,"5min":8.4299999999999997,"15min":17.684000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.454,"5min":1.5249999999999999,"15min":1.6359999999999999},"min":{"1min":0.25600000000000001,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":5.2789999999999999,"5min":11.9,"15min":17.402999999999999},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.2010000000000001,"15min":1.498},"min":{"1min":0.245,"5min":0.19,"15min":0.19},"max":{"1min":4.9509999999999996,"5min":6.0750000000000002,"15min":29.390999999999998},"last":0.622},{"interface":"front","average":{"1min":2.0550000000000002,"5min":1.798,"15min":1.7310000000000001},"min":{"1min":0.29499999999999998,"5min":0.248,"15min":0.218},"max":{"1min":6.0970000000000004,"5min":9.0999999999999996,"15min":29.367000000000001},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":4.3620000000000001,"5min":2.2160000000000002,"15min":1.8919999999999999},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":47.859000000000002,"5min":47.859000000000002,"15min":47.859000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":3.8029999999999999,"5min":1.819,"15min":1.8109999999999999},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":45.183,"5min":45.183,"15min":45.183},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":3.6000000000000001,"5min":2.2120000000000002,"15min":2.0379999999999998},"min":{"1min":0.34799999999999998,"5min":0.254,"15min":0.254},"max":{"1min":43.616999999999997,"5min":43.616999999999997,"15min":43.616999999999997},"last":2.1600000000000001},{"interface":"front","average":{"1min":4.0140000000000002,"5min":2.073,"15min":1.946},"min":{"1min":0.39500000000000002,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":43.918999999999997,"5min":43.918999999999997,"15min":43.918999999999997},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Feb 22 19:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.93799999999999994,"15min":0.77500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25,"15min":0.223},"max":{"1min":2.3220000000000001,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1140000000000001,"15min":0.85099999999999998},"min":{"1min":0.28699999999999998,"5min":0.27100000000000002,"15min":0.25800000000000001},"max":{"1min":2.0169999999999999,"5min":10.407,"15min":10.407},"last":0.98399999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:47:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.63600000000000001,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.247},"max":{"1min":1.429,"5min":3.2599999999999998,"15min":5.1150000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":1.006,"15min":0.86799999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26400000000000001},"max":{"1min":1.125,"5min":10.436,"15min":10.436},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.98999999999999999,"15min":0.78000000000000003},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28299999999999997},"max":{"1min":2.5209999999999999,"5min":6.7119999999999997,"15min":6.7119999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.226,"15min":0.93400000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.27100000000000002},"max":{"1min":4.2560000000000002,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.44500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:47:30.212 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-22T19:47:30.212 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:47:30.213 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-02-22T19:47:30.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 4 2024-02-22T19:47:30.218 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:30.242 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:30.245 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:30.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:30.264 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:30.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:30.372 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:30.376 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:30.403 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:30.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:30.408 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:30.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:30.518 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_3' removed 2024-02-22T19:47:30.546 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:30.569 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:30.572 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:30.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:30.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:29.784401+0000 mgr.y (mgr.14535) 706 : cluster [DBG] pgmap v945: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 392 op/s 2024-02-22T19:47:30.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:29.820145+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.53:0/1867210441' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:47:30.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:29.820472+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:47:30.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:29.784401+0000 mgr.y (mgr.14535) 706 : cluster [DBG] pgmap v945: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 392 op/s 2024-02-22T19:47:30.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:29.820145+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.53:0/1867210441' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:47:30.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:29.820472+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:47:30.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:30.704 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:30.769 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:29.784401+0000 mgr.y (mgr.14535) 706 : cluster [DBG] pgmap v945: 138 pgs: 138 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 392 op/s 2024-02-22T19:47:30.769 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:29.820145+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.53:0/1867210441' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:47:30.769 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:29.820472+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:47:31.419 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-02-22T19:47:31.420 INFO:tasks.radosbench:Beginning radosbench... 2024-02-22T19:47:31.420 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-02-22T19:47:31.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool create unique_pool_4 16 2024-02-22T19:47:31.458 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:31.461 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:31.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:31.562 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:31.669 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:31.671 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:30.492893+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:47:31.671 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:30.492997+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e465: 8 total, 8 up, 5 in 2024-02-22T19:47:31.672 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:31.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:31.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:30.492893+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:47:31.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:30.492997+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e465: 8 total, 8 up, 5 in 2024-02-22T19:47:31.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:30.492893+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:47:31.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:30.492997+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e465: 8 total, 8 up, 5 in 2024-02-22T19:47:31.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:31.848 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:31.927 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:31.929 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:31.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:31.938 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:47:31.938 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:47:32.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:32.158 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:32.161 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:32.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:32.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:32.326 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:32.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:32.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:32.454 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:31.785147+0000 mgr.y (mgr.14535) 707 : cluster [DBG] pgmap v947: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:47:32.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:32.482704+0000 mon.a (mon.0) 1697 : cluster [DBG] osdmap e466: 8 total, 8 up, 5 in 2024-02-22T19:47:32.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:47:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:32.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:32.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:32.753 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:31.785147+0000 mgr.y (mgr.14535) 707 : cluster [DBG] pgmap v947: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:47:32.753 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:32.482704+0000 mon.a (mon.0) 1697 : cluster [DBG] osdmap e466: 8 total, 8 up, 5 in 2024-02-22T19:47:32.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:31.785147+0000 mgr.y (mgr.14535) 707 : cluster [DBG] pgmap v947: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:47:32.753 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:32.482704+0000 mon.a (mon.0) 1697 : cluster [DBG] osdmap e466: 8 total, 8 up, 5 in 2024-02-22T19:47:32.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:32.784 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:32.804 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:32.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:32.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:32.871 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:32.904 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:47:32.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:32.922 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:47:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:32.940 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:47:32.959 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:47:32.977 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:32.981 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:32.982 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:32.982 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:47:33.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:47:33.018 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:47:33.036 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:47:33.037 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.038 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:47:33.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:33.183 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:33.187 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:33.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:33.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:33.569 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:33.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:32.886479+0000 mon.a (mon.0) 1698 : audit [DBG] from='client.? 172.21.15.53:0/3326063618' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:33.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:32.886479+0000 mon.a (mon.0) 1698 : audit [DBG] from='client.? 172.21.15.53:0/3326063618' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:33.821 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:32.886479+0000 mon.a (mon.0) 1698 : audit [DBG] from='client.? 172.21.15.53:0/3326063618' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:33.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:33.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:33.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:33.968 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:34.167 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:34.169 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:34.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:34.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:34.298 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:34.421 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:34.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:34.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:34.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:33.785575+0000 mgr.y (mgr.14535) 708 : cluster [DBG] pgmap v949: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 544 B/s rd, 0 op/s 2024-02-22T19:47:34.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:34.553 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:34.576 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:34.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:34.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:34.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:34.707 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:47:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:47:34.707 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:33.785575+0000 mgr.y (mgr.14535) 708 : cluster [DBG] pgmap v949: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 544 B/s rd, 0 op/s 2024-02-22T19:47:34.709 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165025 8.9 8:9a152755:::benchmark_data_smithi053_130774_object53291:head [delete] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.658911+0000", 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 578.051613309, 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6659777019999999, 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:34.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165025 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.658911+0000", 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.658911+0000", 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.658909+0000", 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8040000000000007e-06 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.658918+0000", 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6376999999999999e-05 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.658934+0000", 2024-02-22T19:47:34.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8039000000000001e-05 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.658952+0000", 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3319000000000006e-05 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659045+0000", 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000138827 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659184+0000", 2024-02-22T19:47:34.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146763 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3", 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.659331+0000", 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0095584829999999996 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.668889+0000", 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2690999999999999e-05 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.668902+0000", 2024-02-22T19:47:34.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.50222874100000003 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171131+0000", 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1099e-05 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171142+0000", 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.13097662 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.302119+0000", 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000153187 2024-02-22T19:47:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for missing object", 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.302272+0000", 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022486778999999998 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.324759+0000", 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000129777 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.324888+0000", 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6659777019999999 2024-02-22T19:47:34.720 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165031 8.9 8:977e7f43:::benchmark_data_smithi053_130774_object53297:head [delete] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.691333+0000", 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 578.01919096500001, 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.633769864, 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165031 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691333+0000", 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691333+0000", 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691331+0000", 2024-02-22T19:47:34.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0820000000000004e-06 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691337+0000", 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0695e-05 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691348+0000", 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1660000000000002e-06 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691353+0000", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1025000000000001e-05 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691394+0000", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5050000000000004e-06 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.691400+0000", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47976554100000002 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171166+0000", 2024-02-22T19:47:34.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1699999999999997e-06 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171174+0000", 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1311366789999999 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.302311+0000", 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5228999999999993e-05 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.302396+0000", 2024-02-22T19:47:34.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013974199999999999 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.302536+0000", 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0080303600000000003 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.310566+0000", 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00038456299999999998 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.310951+0000", 2024-02-22T19:47:34.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007758193 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.318709+0000", 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062578759999999999 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.324967+0000", 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3031000000000001e-05 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.325020+0000", 2024-02-22T19:47:34.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2954999999999994e-05 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.325103+0000", 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.633769864 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165038 8.9 8:91cb6ae5:::benchmark_data_smithi053_130774_object53304:head [delete] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.696327+0000", 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 578.01419653400001, 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.629047285, 2024-02-22T19:47:34.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165038 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696327+0000", 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696327+0000", 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696326+0000", 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6999999999999996e-06 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696332+0000", 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1518000000000001e-05 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696343+0000", 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9610000000000002e-06 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696349+0000", 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4209000000000002e-05 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696404+0000", 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2379e-05 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.696416+0000", 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.47477892100000002 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171195+0000", 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5700000000000004e-06 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171202+0000", 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.133922396 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.305125+0000", 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000160068 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.305285+0000", 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018438499999999999 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.305469+0000", 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00510575 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.310575+0000", 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000422812 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.310998+0000", 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007774726 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.318773+0000", 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0063949530000000001 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.325168+0000", 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6765000000000003e-05 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.325224+0000", 2024-02-22T19:47:34.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015033700000000001 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.325375+0000", 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.629047285 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165043 8.9 8:928244a8:::benchmark_data_smithi053_130774_object53309:head [delete] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.711204+0000", 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 577.99931965099995, 2024-02-22T19:47:34.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6167141389999999, 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165043 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711204+0000", 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711204+0000", 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711203+0000", 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0710000000000002e-06 2024-02-22T19:47:34.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711208+0000", 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3920000000000003e-06 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711217+0000", 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8060000000000002e-06 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711222+0000", 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3544000000000002e-05 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711265+0000", 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1790000000000002e-06 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.711272+0000", 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.459950474 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171223+0000", 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6699999999999994e-06 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171230+0000", 2024-02-22T19:47:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1364133670000001 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.307644+0000", 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011972900000000001 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.307763+0000", 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5437000000000002e-05 2024-02-22T19:47:34.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.307849+0000", 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030056499999999999 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.310855+0000", 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0010063990000000001 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.311861+0000", 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007287773 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.319149+0000", 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0086366749999999999 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327785+0000", 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1097999999999999e-05 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327827+0000", 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1896000000000002e-05 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327918+0000", 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6167141389999999 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:165045 8.9 8:909c9cae:::benchmark_data_smithi053_130774_object53311:head [delete] snapc 0=[] ondisk+write+known_if_redirected e217)", 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:56.713378+0000", 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 577.99714542499999, 2024-02-22T19:47:34.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.614762931, 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165045 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713378+0000", 2024-02-22T19:47:34.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713378+0000", 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713377+0000", 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5289999999999997e-06 2024-02-22T19:47:34.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713384+0000", 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1623999999999999e-05 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713395+0000", 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8739999999999997e-06 2024-02-22T19:47:34.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713402+0000", 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9042999999999999e-05 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713441+0000", 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3989999999999998e-06 2024-02-22T19:47:34.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.713449+0000", 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.457802443 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171251+0000", 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5750000000000003e-06 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:57.171259+0000", 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.137967613 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.309226+0000", 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116018 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.309342+0000", 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116198 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:47:34.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.309459+0000", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015506859999999999 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.311009+0000", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000975464 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.311985+0000", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072705499999999998 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.319255+0000", 2024-02-22T19:47:34.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087261579999999995 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.327982+0000", 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5121999999999998e-05 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328037+0000", 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010477800000000001 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:58.328141+0000", 2024-02-22T19:47:34.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.614762931 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:183844 8.6 8:6529e3f7:::benchmark_data_smithi053_130774_object72110:head [delete] snapc 0=[] ondisk+write+known_if_redirected e238)", 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:24.993093+0000", 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 549.71743048799999, 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.079676040000000004, 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 183844 2024-02-22T19:47:34.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993093+0000", 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993093+0000", 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993093+0000", 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8119999999999998e-06 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993098+0000", 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1369999999999996e-06 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993105+0000", 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.982e-06 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993108+0000", 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8918000000000002e-05 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993147+0000", 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0136999999999994e-05 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993237+0000", 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016817000000000001 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2,7", 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.993405+0000", 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048537839999999999 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.998259+0000", 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00067657600000000002 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:24.998935+0000", 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024487810000000001 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:25.001384+0000", 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.071223543 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:25.072608+0000", 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2592e-05 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:25.072670+0000", 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9298999999999999e-05 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:25.072769+0000", 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.079676040000000004 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194901 8.6 8:689782a8:::benchmark_data_smithi053_130774_object83167:head [delete] snapc 0=[] ondisk+write+known_if_redirected e244)", 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.032458+0000", 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.67806573500002, 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.25089750799999999, 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194901 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032458+0000", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032458+0000", 2024-02-22T19:47:34.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032458+0000", 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.563e-06 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032461+0000", 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7149999999999997e-06 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032466+0000", 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.965e-06 2024-02-22T19:47:34.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032469+0000", 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0704000000000002e-05 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032510+0000", 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4480000000000005e-05 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032584+0000", 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012855200000000001 2024-02-22T19:47:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,2,7", 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.032713+0000", 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031619740000000001 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.035875+0000", 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033235349999999999 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.039198+0000", 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053784920000000003 2024-02-22T19:47:34.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.044577+0000", 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6630000000000004e-06 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.044583+0000", 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.23872807400000001 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.283311+0000", 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4388000000000003e-05 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.283356+0000", 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.25089750799999999 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194909 8.6 8:62fdc315:::benchmark_data_smithi053_130774_object83175:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.050371+0000", 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.66015337500005, 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.26232413700000001, 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.767 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194909 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050371+0000", 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050371+0000", 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050369+0000", 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6790000000000005e-06 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050376+0000", 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3325e-05 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050390+0000", 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8360000000000003e-06 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050397+0000", 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0568999999999997e-05 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050437+0000", 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8560000000000007e-06 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.050445+0000", 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.23326866800000001 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.283714+0000", 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010534100000000001 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.283819+0000", 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015440500000000001 2024-02-22T19:47:34.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.283973+0000", 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021638733 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305612+0000", 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6390999999999995e-05 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305678+0000", 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0069303050000000003 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312609+0000", 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4737999999999998e-05 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312654+0000", 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1137000000000003e-05 2024-02-22T19:47:34.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312695+0000", 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.26232413700000001 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.774 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194928 8.6 8:6ae1a0f2:::benchmark_data_smithi053_130774_object83194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.084161+0000", 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.62636273999999, 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.22864633500000001, 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194928 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084161+0000", 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084161+0000", 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084160+0000", 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2490000000000001e-06 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084168+0000", 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.698e-05 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084185+0000", 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8430000000000004e-06 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084192+0000", 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2516999999999999e-05 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084244+0000", 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5010000000000004e-06 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.084253+0000", 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.20247991400000001 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.286733+0000", 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110023 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.286843+0000", 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014825 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.286991+0000", 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018625462999999998 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305616+0000", 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104802 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305721+0000", 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070109250000000003 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312732+0000", 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1946999999999997e-05 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312774+0000", 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3441000000000001e-05 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312808+0000", 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.22864633500000001 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194930 8.6 8:6ae8e912:::benchmark_data_smithi053_130774_object83196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.782 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.088426+0000", 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.62209763999999, 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.22449294, 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194930 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088426+0000", 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088426+0000", 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088424+0000", 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1999e-05 2024-02-22T19:47:34.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088436+0000", 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8088999999999999e-05 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088454+0000", 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1905999999999999e-05 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088466+0000", 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1137999999999996e-05 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088528+0000", 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7999999999999993e-06 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.088537+0000", 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.20022020200000001 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.288758+0000", 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013035 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.288888+0000", 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000149197 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.289037+0000", 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016584022 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305621+0000", 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000157606 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305779+0000", 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070595099999999997 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312838+0000", 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2839000000000001e-05 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312881+0000", 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8170000000000002e-05 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312919+0000", 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.22449294 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.790 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194940 8.6 8:6cd7e562:::benchmark_data_smithi053_130774_object83206:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.105428+0000", 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.605096215, 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.20759551700000001, 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194940 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105428+0000", 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105428+0000", 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105426+0000", 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5250000000000002e-06 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105432+0000", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0316000000000001e-05 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105443+0000", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9280000000000001e-06 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105448+0000", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6979e-05 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105485+0000", 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0849999999999996e-06 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.105490+0000", 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.185447738 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.290937+0000", 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9025999999999999e-05 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.291006+0000", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9025999999999999e-05 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.291085+0000", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014538752 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305624+0000", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019589100000000001 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305820+0000", 2024-02-22T19:47:34.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0071311760000000004 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312951+0000", 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8217999999999999e-05 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.312990+0000", 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3705999999999999e-05 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313023+0000", 2024-02-22T19:47:34.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.20759551700000001 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194948 8.6 8:68bd8890:::benchmark_data_smithi053_130774_object83214:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.124582+0000", 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.58594187899996, 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.18855049199999999, 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194948 2024-02-22T19:47:34.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124582+0000", 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124582+0000", 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124581+0000", 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3050000000000003e-06 2024-02-22T19:47:34.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124586+0000", 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4830000000000002e-06 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124593+0000", 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6369999999999998e-06 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124598+0000", 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2759000000000003e-05 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124631+0000", 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-06 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.124636+0000", 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.167646452 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.292282+0000", 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116618 2024-02-22T19:47:34.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.292399+0000", 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012798400000000001 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.292527+0000", 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013101487 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305628+0000", 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00023279799999999999 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305861+0000", 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0071955700000000001 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313057+0000", 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6764999999999998e-05 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313093+0000", 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9029000000000002e-05 2024-02-22T19:47:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313133+0000", 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.18855049199999999 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194965 8.6 8:65aea436:::benchmark_data_smithi053_130774_object83231:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.159819+0000", 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.550704838, 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.153416525, 2024-02-22T19:47:34.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194965 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159819+0000", 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159819+0000", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159818+0000", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9640000000000003e-06 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159826+0000", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2666999999999999e-05 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159838+0000", 2024-02-22T19:47:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8820000000000003e-06 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159845+0000", 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7667999999999998e-05 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159893+0000", 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1559999999999995e-06 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.159901+0000", 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.133844879 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.293746+0000", 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107995 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.293854+0000", 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0822000000000006e-05 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.293945+0000", 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01178122 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305726+0000", 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017157399999999999 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305897+0000", 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072676010000000003 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313165+0000", 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6865e-05 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313202+0000", 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3733999999999999e-05 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313236+0000", 2024-02-22T19:47:34.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.153416525 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194977 8.6 8:670e9c13:::benchmark_data_smithi053_130774_object83243:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.188966+0000", 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.52155784900003, 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.12437103200000001, 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194977 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.188966+0000", 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.188966+0000", 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.188964+0000", 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0475000000000001e-05 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.188974+0000", 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7654000000000001e-05 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.188992+0000", 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6519999999999995e-06 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.189001+0000", 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2070999999999998e-05 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.189043+0000", 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2779999999999999e-06 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.189049+0000", 2024-02-22T19:47:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10684442299999999 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.295893+0000", 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1279000000000006e-05 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.295975+0000", 2024-02-22T19:47:34.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013888799999999999 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.296114+0000", 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096166819999999997 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305730+0000", 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020198500000000001 2024-02-22T19:47:34.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305932+0000", 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073342010000000003 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313266+0000", 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7048999999999997e-05 2024-02-22T19:47:34.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313303+0000", 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3670000000000001e-05 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313337+0000", 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.12437103200000001 2024-02-22T19:47:34.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194988 8.6 8:6772ae97:::benchmark_data_smithi053_130774_object83254:head [delete] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.230189+0000", 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.48033521699995, 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.083248167999999997, 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194988 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230189+0000", 2024-02-22T19:47:34.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230189+0000", 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230187+0000", 2024-02-22T19:47:34.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2020000000000001e-06 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230193+0000", 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0088000000000001e-05 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230204+0000", 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1540000000000004e-06 2024-02-22T19:47:34.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230208+0000", 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0304999999999999e-05 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230268+0000", 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7270000000000007e-06 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.230276+0000", 2024-02-22T19:47:34.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.067957418000000006 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.298233+0000", 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010082299999999999 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.298334+0000", 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011743099999999999 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.298451+0000", 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072826749999999997 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305734+0000", 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001651247 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.307385+0000", 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005980478 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313366+0000", 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8534000000000001e-05 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313404+0000", 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2486e-05 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.313437+0000", 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.083248167999999997 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.826 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:194992 8.6 8:66aa9c03:::benchmark_data_smithi053_130774_object83258:head [delete] snapc 0=[] ondisk+write+known_if_redirected e246)", 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:42.240511+0000", 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 532.47001252999996, 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.084977134999999995, 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 194992 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240511+0000", 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240511+0000", 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240510+0000", 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6849999999999998e-06 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240516+0000", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4660000000000005e-06 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240525+0000", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1529999999999996e-06 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240531+0000", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8534000000000001e-05 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240569+0000", 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7180000000000001e-06 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.240575+0000", 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.060312669999999999 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.300887+0000", 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000106858 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.300994+0000", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000128548 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.301123+0000", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004615381 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.305738+0000", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042100389999999996 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.309948+0000", 2024-02-22T19:47:34.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015451316999999999 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.325400+0000", 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8492000000000003e-05 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.325448+0000", 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0450000000000001e-05 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:42.325489+0000", 2024-02-22T19:47:34.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.084977134999999995 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:197535 8.5 8:a9138953:::benchmark_data_smithi053_130774_object85801:head [delete] snapc 0=[] ondisk+write+known_if_redirected e247)", 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:46.107270+0000", 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 528.60325387800003, 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.131895745, 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 197535 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107270+0000", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107270+0000", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107269+0000", 2024-02-22T19:47:34.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4800000000000001e-06 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107273+0000", 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3489999999999997e-06 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107279+0000", 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-06 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107284+0000", 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3967e-05 2024-02-22T19:47:34.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107338+0000", 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:46.107349+0000", 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0527787980000001 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.160128+0000", 2024-02-22T19:47:34.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2772000000000001e-05 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.160140+0000", 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0582943629999999 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.218435+0000", 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000165558 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.218600+0000", 2024-02-22T19:47:34.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00054328199999999997 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,4,7", 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.219144+0000", 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010539366999999999 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.229683+0000", 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3749000000000003e-05 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.229737+0000", 2024-02-22T19:47:34.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027101220000000001 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.232447+0000", 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065139730000000002 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.238961+0000", 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2920000000000001e-05 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.239024+0000", 2024-02-22T19:47:34.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014212299999999999 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.239166+0000", 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.131895745 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:197600 8.5 8:a1a9087c:::benchmark_data_smithi053_130774_object85866:head [delete] snapc 0=[] ondisk+write+known_if_redirected e248)", 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:38:47.350433+0000", 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 527.36009070299997, 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.893416349, 2024-02-22T19:47:34.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 197600 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350433+0000", 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350433+0000", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350432+0000", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2320000000000004e-06 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350439+0000", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2014999999999999e-05 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350451+0000", 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9410000000000001e-06 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350458+0000", 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4783999999999997e-05 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350502+0000", 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2859999999999999e-06 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:47.350511+0000", 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87924771099999999 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.229758+0000", 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013996200000000001 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.229898+0000", 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016705499999999999 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,4,7", 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.230065+0000", 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023942939999999999 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.232460+0000", 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00068001300000000004 2024-02-22T19:47:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.233140+0000", 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8038000000000003e-05 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.233188+0000", 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010418851999999999 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.243607+0000", 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2529999999999998e-05 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.243679+0000", 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017033599999999999 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:38:48.243850+0000", 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.893416349 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15054.0:209458 8.9 8:9e8dda5d:::benchmark_data_smithi053_130774_object97724:head [delete] snapc 0=[] ondisk+write+known_if_redirected e260)", 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:39:07.018404+0000", 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 507.69211976000003, 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345352911, 2024-02-22T19:47:34.848 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15054", 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 209458 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018404+0000", 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:34.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018404+0000", 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018402+0000", 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0056e-05 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018412+0000", 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1965e-05 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018424+0000", 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4029999999999998e-06 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018430+0000", 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3554999999999997e-05 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018493+0000", 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3560999999999999e-05 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.018507+0000", 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.20128049200000001 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.219787+0000", 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5189999999999992e-06 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:07.219796+0000", 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0619253120000001 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.281721+0000", 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9071000000000002e-05 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.281800+0000", 2024-02-22T19:47:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000106376 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0,3,5", 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.281906+0000", 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001893905 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.283800+0000", 2024-02-22T19:47:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8689999999999997e-05 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.283839+0000", 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012450147 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.296289+0000", 2024-02-22T19:47:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.067296442999999997 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.363586+0000", 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5528999999999997e-05 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.363631+0000", 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012589 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:39:08.363757+0000", 2024-02-22T19:47:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345352911 2024-02-22T19:47:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:34.858 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:34.859 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:33.785575+0000 mgr.y (mgr.14535) 708 : cluster [DBG] pgmap v949: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 544 B/s rd, 0 op/s 2024-02-22T19:47:34.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:34.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:34.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:34.995 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:35.374 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:47:35.445 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:35.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:35.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:35.512 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:47:35.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:35.578 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:35.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:47:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:47:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:35.384203+0000 mgr.y (mgr.14535) 709 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:35.384312+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.53:0/3176855765' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:35.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:35.384694+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:35.604 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:35.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:35.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:35.688 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:35.384203+0000 mgr.y (mgr.14535) 709 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:35.689 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:35.384312+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.53:0/3176855765' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:35.689 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:35.384694+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:35.689 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:35.384203+0000 mgr.y (mgr.14535) 709 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:35.689 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:35.384312+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.53:0/3176855765' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:35.689 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:35.384694+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:47:35.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:35.935 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:35.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:35.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:36.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:36.063 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:47:36.067 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:35.502980+0000 mon.a (mon.0) 1700 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:35.509465+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:47:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:35.509542+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e467: 8 total, 8 up, 5 in 2024-02-22T19:47:36.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:35.785990+0000 mgr.y (mgr.14535) 710 : cluster [DBG] pgmap v951: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:47:36.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:35.502980+0000 mon.a (mon.0) 1700 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:36.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:35.509465+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:47:36.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:35.509542+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e467: 8 total, 8 up, 5 in 2024-02-22T19:47:36.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:35.785990+0000 mgr.y (mgr.14535) 710 : cluster [DBG] pgmap v951: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:47:36.824 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:35.502980+0000 mon.a (mon.0) 1700 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:36.824 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:35.509465+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:47:36.825 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:35.509542+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e467: 8 total, 8 up, 5 in 2024-02-22T19:47:36.825 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:35.785990+0000 mgr.y (mgr.14535) 710 : cluster [DBG] pgmap v951: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 682 B/s rd, 0 op/s 2024-02-22T19:47:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:36.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:36.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:36.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:37.033 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:37.036 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:37.036 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:37.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:37.166 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:37.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:37.188 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:37.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:37.304 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:37.304 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:47:37.304 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-02-22T19:47:37.304 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:37.339 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:37.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:37.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:37.517 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:37.536 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:37.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:37.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:37.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:37.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:47:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:37.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:37.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:38.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:38.052 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:38.072 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:38.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:38.075 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:38.176 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:38.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:38.204 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:38.405 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:38.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:38.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:38.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:38.540 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:38.638 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:38.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:38.642 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:38.697 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:38.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:37.484478+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e468: 8 total, 8 up, 5 in 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:37.786293+0000 mgr.y (mgr.14535) 711 : cluster [DBG] pgmap v953: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:38.388604+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:37.484478+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e468: 8 total, 8 up, 5 in 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:37.786293+0000 mgr.y (mgr.14535) 711 : cluster [DBG] pgmap v953: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:38.388604+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:38.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:47:38.591+0000 7fe2d794e700 -1 mon.a@0(leader).osd e468 definitely_dead 0 2024-02-22T19:47:38.770 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:37.484478+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e468: 8 total, 8 up, 5 in 2024-02-22T19:47:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:37.786293+0000 mgr.y (mgr.14535) 711 : cluster [DBG] pgmap v953: 122 pgs: 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-02-22T19:47:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:38.388604+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:38.969 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:38.972 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:38.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:39.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:39.103 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:39.124 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:39.126 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:39.127 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:39.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:47:39.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:39.254 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:39.490 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:39] "GET /metrics HTTP/1.1" 200 174119 "" "Prometheus/2.33.4" 2024-02-22T19:47:39.518 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.4. 2024-02-22T19:47:39.518 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_4' created 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:38.592560+0000 mon.c (mon.2) 82 : audit [DBG] from='client.? 172.21.15.53:0/373296184' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:38.592931+0000 mon.a (mon.0) 1705 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:38.872681+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.53:0/2046286212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:38.873009+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:38.592560+0000 mon.c (mon.2) 82 : audit [DBG] from='client.? 172.21.15.53:0/373296184' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:38.592931+0000 mon.a (mon.0) 1705 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:47:39.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:38.872681+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.53:0/2046286212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-02-22T19:47:39.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:38.873009+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-02-22T19:47:39.803 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:39.804 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:38.592560+0000 mon.c (mon.2) 82 : audit [DBG] from='client.? 172.21.15.53:0/373296184' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:47:39.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:38.592931+0000 mon.a (mon.0) 1705 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:47:39.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:38.872681+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.53:0/2046286212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-02-22T19:47:39.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:38.873009+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-02-22T19:47:39.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:39.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:39.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:39.937 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:39.990 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:47:39.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:39.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:47:40.071 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:40.071 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:47:40.071 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:47:40.071 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:47:40.072 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:40.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:40.122 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:40.280 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:40.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:40.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:40.411 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:40.444 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-02-22T19:47:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:40.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:40.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:39.496929+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-02-22T19:47:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:39.497017+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-02-22T19:47:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:39.497092+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e469: 8 total, 8 up, 6 in 2024-02-22T19:47:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:39.786683+0000 mgr.y (mgr.14535) 712 : cluster [DBG] pgmap v955: 138 pgs: 16 unknown, 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:47:40.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:40.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:47:40.677 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:39.496929+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-02-22T19:47:40.677 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:39.497017+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-02-22T19:47:40.677 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:39.497092+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e469: 8 total, 8 up, 6 in 2024-02-22T19:47:40.677 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:39.786683+0000 mgr.y (mgr.14535) 712 : cluster [DBG] pgmap v955: 138 pgs: 16 unknown, 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:47:40.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:39.496929+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-02-22T19:47:40.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:39.497017+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-02-22T19:47:40.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:39.497092+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e469: 8 total, 8 up, 6 in 2024-02-22T19:47:40.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:39.786683+0000 mgr.y (mgr.14535) 712 : cluster [DBG] pgmap v955: 138 pgs: 16 unknown, 122 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-22T19:47:40.679 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:40.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-02-22T19:47:40.878 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:40.881 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:40.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:40.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:40.997 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:47:41.031 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:41.289 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:41.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:41.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:41.540 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:40.501216+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e470: 8 total, 8 up, 6 in 2024-02-22T19:47:41.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:40.501216+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e470: 8 total, 8 up, 6 in 2024-02-22T19:47:41.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:40.501216+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e470: 8 total, 8 up, 6 in 2024-02-22T19:47:42.122 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:42.614 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:41.501810+0000 mon.a (mon.0) 1711 : cluster [DBG] osdmap e471: 8 total, 8 up, 6 in 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:41.502542+0000 osd.1 (osd.1) 276 : cluster [DBG] 5.5 starting backfill to osd.4 from (0'0,0'0] MAX to 419'20 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:41.502605+0000 osd.3 (osd.3) 318 : cluster [DBG] 5.0 starting backfill to osd.4 from (0'0,0'0] MAX to 407'13 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:41.503565+0000 osd.7 (osd.7) 298 : cluster [DBG] 5.9 starting backfill to osd.4 from (0'0,0'0] MAX to 326'16 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:41.508601+0000 osd.5 (osd.5) 185 : cluster [DBG] 5.7 starting backfill to osd.4 from (0'0,0'0] MAX to 412'25 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:41.787055+0000 mgr.y (mgr.14535) 713 : cluster [DBG] pgmap v958: 138 pgs: 8 creating+peering, 13 peering, 8 unknown, 109 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:42.485638+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e472: 8 total, 8 up, 6 in 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:41.501810+0000 mon.a (mon.0) 1711 : cluster [DBG] osdmap e471: 8 total, 8 up, 6 in 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:41.502542+0000 osd.1 (osd.1) 276 : cluster [DBG] 5.5 starting backfill to osd.4 from (0'0,0'0] MAX to 419'20 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:41.502605+0000 osd.3 (osd.3) 318 : cluster [DBG] 5.0 starting backfill to osd.4 from (0'0,0'0] MAX to 407'13 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:41.503565+0000 osd.7 (osd.7) 298 : cluster [DBG] 5.9 starting backfill to osd.4 from (0'0,0'0] MAX to 326'16 2024-02-22T19:47:42.615 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:41.508601+0000 osd.5 (osd.5) 185 : cluster [DBG] 5.7 starting backfill to osd.4 from (0'0,0'0] MAX to 412'25 2024-02-22T19:47:42.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:41.787055+0000 mgr.y (mgr.14535) 713 : cluster [DBG] pgmap v958: 138 pgs: 8 creating+peering, 13 peering, 8 unknown, 109 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:47:42.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:42.485638+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e472: 8 total, 8 up, 6 in 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1365 5.9 5.84ada7c9 (undecoded) ondisk+write+known_if_redirected e209)", 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:54.260684+0000", 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 588.50277388500001, 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.37209318499999999, 2024-02-22T19:47:42.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1365 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260684+0000", 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260684+0000", 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260683+0000", 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9210000000000001e-06 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260686+0000", 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4490000000000001e-06 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260690+0000", 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1849999999999997e-06 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260695+0000", 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7299999999999996e-05 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260762+0000", 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1106e-05 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.260773+0000", 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.37197053000000002 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.632744+0000", 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3179000000000002e-05 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.632777+0000", 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.37209318499999999 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.770 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1366 5.9 5.84ada7c9 (undecoded) ondisk+write+known_if_redirected e214)", 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:54.261398+0000", 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 588.50205952099998, 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.37160578300000002, 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1366 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261398+0000", 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261398+0000", 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261397+0000", 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999998e-06 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261402+0000", 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4689999999999997e-06 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261410+0000", 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9779999999999996e-06 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261415+0000", 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00035657100000000003 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261772+0000", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5780000000000004e-06 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.261778+0000", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.37121785299999999 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.632996+0000", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6539999999999999e-06 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:54.633004+0000", 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.37160578300000002 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1365 5.9 5:93e5b521:::notify.7:head [watch ping cookie 94129597104000] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e215)", 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:55.656678+0000", 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 587.10677948299997, 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.014532542, 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:42.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1365 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656678+0000", 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656678+0000", 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656676+0000", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.674e-06 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656683+0000", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9450000000000005e-06 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656693+0000", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6469999999999998e-06 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656701+0000", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00073733499999999999 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.657438+0000", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1280999999999999e-05 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.657449+0000", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.013525724 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.670975+0000", 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179691 2024-02-22T19:47:42.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671155+0000", 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5872999999999999e-05 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671210+0000", 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.014532542 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.779 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1366 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94129597104000 gen 1] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e215)", 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:55.656709+0000", 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 587.10674794600004, 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0157923150000001, 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1366 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656709+0000", 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656709+0000", 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656709+0000", 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4590000000000001e-06 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656712+0000", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6029999999999998e-06 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656716+0000", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.656720+0000", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00076279400000000002 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.657483+0000", 2024-02-22T19:47:42.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3579999999999997e-06 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.657490+0000", 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.013756324 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671246+0000", 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8673999999999999e-05 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671325+0000", 2024-02-22T19:47:42.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001176603 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672502+0000", 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0157923150000001 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:1367 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94129597104000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e216)", 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:37:55.662828+0000", 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 587.10062939500006, 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0099097779999999, 2024-02-22T19:47:42.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 1367 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662828+0000", 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662828+0000", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662827+0000", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3199999999999996e-06 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662833+0000", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0668e-05 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662844+0000", 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.933e-06 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662849+0000", 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2947000000000001e-05 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662892+0000", 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0580000000000003e-06 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:47:42.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:55.662898+0000", 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0087340410000001 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671632+0000", 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6531000000000004e-05 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.671698+0000", 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0010396209999999999 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:37:56.672738+0000", 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0099097779999999 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:109863 10.1 10:84647948:::benchmark_data_smithi053_180506_object109862:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e430)", 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:35.348874+0000", 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 127.41458303, 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.094411825000000005, 2024-02-22T19:47:42.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 109863 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348874+0000", 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348874+0000", 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348873+0000", 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2573e-05 2024-02-22T19:47:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348925+0000", 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2632999999999999e-05 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348938+0000", 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7950000000000002e-06 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348945+0000", 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6623999999999999e-05 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.348992+0000", 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109474 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.349101+0000", 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016898999999999999 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.349270+0000", 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067805180000000001 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.356050+0000", 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029645219999999998 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359015+0000", 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.084168091 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.443183+0000", 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4904000000000002e-05 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.443238+0000", 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8185e-05 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.443286+0000", 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.094411825000000005 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:109876 10.c 10:32d43208:::benchmark_data_smithi053_180506_object109875:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e430)", 2024-02-22T19:47:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:35.359249+0000", 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 127.404208895, 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.089882264000000003, 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 109876 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359249+0000", 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359249+0000", 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359247+0000", 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5667999999999997e-05 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359293+0000", 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0873000000000001e-05 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359304+0000", 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4829999999999999e-06 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359309+0000", 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6245000000000003e-05 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359345+0000", 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000154399 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359500+0000", 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000191576 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.359691+0000", 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018498939999999999 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.361541+0000", 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00129462 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.362836+0000", 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.086189476000000001 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.449025+0000", 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2885999999999994e-05 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.449088+0000", 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2669000000000001e-05 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:35.449131+0000", 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.089882264000000003 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:113453 10.1 10:867150f9:::benchmark_data_smithi053_180506_object74:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:38.910491+0000", 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.852966454, 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.131719736, 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.802 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 113453 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910491+0000", 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910491+0000", 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910489+0000", 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6499999999999996e-06 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910497+0000", 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3371e-05 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910510+0000", 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1879999999999999e-06 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910516+0000", 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9400999999999997e-05 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910566+0000", 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013909999999999999 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910705+0000", 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014411299999999999 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.910849+0000", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021457529999999998 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.912995+0000", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0091584240000000001 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.922153+0000", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.11996707600000001 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.042120+0000", 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6174999999999996e-05 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.042186+0000", 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4474e-05 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.042211+0000", 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.131719736 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:113480 10.c 10:3dbd0673:::benchmark_data_smithi053_180506_object101:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:38.929270+0000", 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.834187892, 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.118273319, 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 113480 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929270+0000", 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929270+0000", 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929268+0000", 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7350000000000008e-06 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929278+0000", 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1651000000000001e-05 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929289+0000", 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8340000000000004e-06 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929295+0000", 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8133000000000001e-05 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929343+0000", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3741999999999998e-05 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929437+0000", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2154e-05 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929519+0000", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003865444 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.933385+0000", 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1105e-05 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.933426+0000", 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.113975326 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047401+0000", 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3582999999999998e-05 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047455+0000", 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8055999999999999e-05 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047543+0000", 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.118273319 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:113481 10.c 10:3d1fd70e:::benchmark_data_smithi053_180506_object102:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:38.929382+0000", 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.834075349, 2024-02-22T19:47:42.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.118290004, 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 113481 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929382+0000", 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929382+0000", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929381+0000", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7120000000000002e-06 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929385+0000", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.905e-06 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929392+0000", 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4000000000000001e-06 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929395+0000", 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022548300000000002 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.931650+0000", 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012858999999999999 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.931779+0000", 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016358799999999999 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.931942+0000", 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001671044 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.933613+0000", 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0024e-05 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.933633+0000", 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.113957187 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047591+0000", 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3339e-05 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047634+0000", 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8096000000000003e-05 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047672+0000", 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.118290004 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:139498 10.c 10:3261a395:::benchmark_data_smithi053_180506_object26119:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:02.374475+0000", 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.388982031, 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.11499174299999999, 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 139498 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374475+0000", 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374475+0000", 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374474+0000", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1149999999999999e-06 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374480+0000", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8220000000000002e-06 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374490+0000", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-06 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374495+0000", 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3157999999999997e-05 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374528+0000", 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108768 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374637+0000", 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015968699999999999 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.374797+0000", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016444140000000001 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.376441+0000", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00317958 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.379621+0000", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.109782431 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.489403+0000", 2024-02-22T19:47:42.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5970999999999998e-05 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.489439+0000", 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7987000000000002e-05 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.489467+0000", 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.11499174299999999 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:139502 10.1 10:870c78aa:::benchmark_data_smithi053_180506_object26123:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:02.381201+0000", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.382256187, 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.114032805, 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 139502 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381201+0000", 2024-02-22T19:47:42.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381201+0000", 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381200+0000", 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5300000000000002e-06 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381207+0000", 2024-02-22T19:47:42.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2706e-05 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381219+0000", 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5610000000000004e-06 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381226+0000", 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8148e-05 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381274+0000", 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011754199999999999 2024-02-22T19:47:42.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381392+0000", 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014272800000000001 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.381534+0000", 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034220280000000001 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.384956+0000", 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018690169999999999 2024-02-22T19:47:42.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.386825+0000", 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10831639899999999 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495142+0000", 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9274000000000003e-05 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495191+0000", 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3069999999999999e-05 2024-02-22T19:47:42.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495234+0000", 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.114032805 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:139518 10.1 10:86842b6e:::benchmark_data_smithi053_180506_object26139:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:02.399287+0000", 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.36417089, 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.096097736000000003, 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.830 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 139518 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399287+0000", 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399287+0000", 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:42.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399285+0000", 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8639999999999996e-06 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399293+0000", 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3334e-05 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399306+0000", 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6390000000000003e-06 2024-02-22T19:47:42.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399313+0000", 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1377e-05 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399354+0000", 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2175000000000005e-05 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399436+0000", 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1388000000000001e-05 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.399517+0000", 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001767221 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.401285+0000", 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00070584200000000004 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.401991+0000", 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.093297808999999995 2024-02-22T19:47:42.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495288+0000", 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8749999999999999e-05 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495337+0000", 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7166000000000002e-05 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495384+0000", 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.096097736000000003 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.835 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:139521 10.1 10:86ffa283:::benchmark_data_smithi053_180506_object26142:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:02.405360+0000", 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.358097304, 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.090146405999999998, 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 139521 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.836 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405360+0000", 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405360+0000", 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999905 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405350+0000", 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0945e-05 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405371+0000", 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6659999999999999e-05 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405398+0000", 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2850000000000001e-05 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405411+0000", 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3084999999999994e-05 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405504+0000", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020626499999999999 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405710+0000", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8245999999999996e-05 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.405789+0000", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001161076 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.406950+0000", 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0011766159999999999 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.408126+0000", 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.087301847000000002 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495428+0000", 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5024000000000001e-05 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495473+0000", 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3401000000000001e-05 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:02.495507+0000", 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.090146405999999998 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:144808 10.c 10:38589f13:::benchmark_data_smithi053_180506_object31429:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:07.499656+0000", 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 95.263801818999994, 2024-02-22T19:47:42.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.111374404, 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 144808 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499656+0000", 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499656+0000", 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499654+0000", 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7449999999999995e-06 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499662+0000", 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3097e-05 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499675+0000", 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6449999999999999e-06 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499681+0000", 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3068999999999997e-05 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499724+0000", 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1676999999999997e-05 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499816+0000", 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000121798 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.499938+0000", 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002135432 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.502073+0000", 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017406170000000001 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.503814+0000", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.107153128 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.610967+0000", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1146999999999998e-05 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.610998+0000", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1779999999999997e-05 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611030+0000", 2024-02-22T19:47:42.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.111374404 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:144817 10.c 10:3379b085:::benchmark_data_smithi053_180506_object31438:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:07.505517+0000", 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 95.257940180000006, 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560871300000001, 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 144817 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505517+0000", 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505517+0000", 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505516+0000", 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.409e-06 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505522+0000", 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0502e-05 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505533+0000", 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1030000000000001e-06 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505538+0000", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1409999999999999e-05 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505569+0000", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5816999999999996e-05 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505645+0000", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7416999999999994e-05 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505723+0000", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026535249999999999 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.508376+0000", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031051899999999999 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.508687+0000", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.102390883 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611078+0000", 2024-02-22T19:47:42.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6710999999999999e-05 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611104+0000", 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1580999999999999e-05 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611126+0000", 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560871300000001 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:144818 10.c 10:37a103a1:::benchmark_data_smithi053_180506_object31439:head [delete] snapc 0=[] ondisk+write+known_if_redirected e440)", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:07.505605+0000", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 95.257852174999996, 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10561321, 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 144818 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505605+0000", 2024-02-22T19:47:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505605+0000", 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505605+0000", 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.444e-06 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505608+0000", 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0159999999999997e-06 2024-02-22T19:47:42.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505614+0000", 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5080000000000001e-06 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.505618+0000", 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016160460000000001 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.507234+0000", 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0059000000000001e-05 2024-02-22T19:47:42.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.507304+0000", 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4635000000000005e-05 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.507388+0000", 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0011208780000000001 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.508509+0000", 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028251099999999999 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.508792+0000", 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.102380048 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611172+0000", 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6054000000000002e-05 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611198+0000", 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0625999999999999e-05 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:07.611218+0000", 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10561321 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:181094 10.c 10:30b7c0a1:::benchmark_data_smithi053_180506_object67715:head [delete] snapc 0=[] ondisk+write+known_if_redirected e449)", 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:40.750086+0000", 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 62.013371466000002, 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.16312349800000001, 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.858 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 181094 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750086+0000", 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750086+0000", 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750085+0000", 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.127e-06 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750091+0000", 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.221e-05 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750103+0000", 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5120000000000008e-06 2024-02-22T19:47:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750112+0000", 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7141999999999997e-05 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750159+0000", 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5073000000000004e-05 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750244+0000", 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1638999999999995e-05 2024-02-22T19:47:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.750325+0000", 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020783749999999999 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.752404+0000", 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023098070000000001 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.754714+0000", 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.158420907 2024-02-22T19:47:42.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.913135+0000", 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2261000000000002e-05 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.913167+0000", 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2607000000000001e-05 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.913209+0000", 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.16312349800000001 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.863 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:181096 10.c 10:31d9587f:::benchmark_data_smithi053_180506_object67717:head [delete] snapc 0=[] ondisk+write+known_if_redirected e449)", 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:40.753003+0000", 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 62.010454312999997, 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.16052426, 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 181096 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.864 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753003+0000", 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753003+0000", 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753001+0000", 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7700000000000004e-06 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753008+0000", 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2942000000000001e-05 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753021+0000", 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8410000000000002e-06 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753028+0000", 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4159999999999998e-05 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753062+0000", 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8708999999999994e-05 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753151+0000", 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0428999999999998e-05 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.753241+0000", 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001554478 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.754796+0000", 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00034210399999999998 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.755138+0000", 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.15834376999999999 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.913482+0000", 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5828000000000001e-05 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.913508+0000", 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9820999999999999e-05 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.913527+0000", 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.16052426 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:223085 10.c 10:322cb96e:::benchmark_data_smithi053_180506_object109706:head [delete] snapc 0=[] ondisk+write+known_if_redirected e462)", 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:47:19.046275+0000", 2024-02-22T19:47:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.717182611999998, 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.114623023, 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 223085 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046275+0000", 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046275+0000", 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046274+0000", 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0429999999999998e-06 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046279+0000", 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1599999999999998e-06 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046287+0000", 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4000000000000002e-06 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046291+0000", 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5639000000000001e-05 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046337+0000", 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012403300000000001 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046461+0000", 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016090599999999999 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,3", 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.046622+0000", 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021209990000000001 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.048743+0000", 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026595519999999999 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.051402+0000", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.109422133 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.160825+0000", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3460999999999997e-05 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.160858+0000", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9765000000000003e-05 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:47:42.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.160898+0000", 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.114623023 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:47:42.875 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:47:42.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:41.501810+0000 mon.a (mon.0) 1711 : cluster [DBG] osdmap e471: 8 total, 8 up, 6 in 2024-02-22T19:47:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:41.502542+0000 osd.1 (osd.1) 276 : cluster [DBG] 5.5 starting backfill to osd.4 from (0'0,0'0] MAX to 419'20 2024-02-22T19:47:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:41.502605+0000 osd.3 (osd.3) 318 : cluster [DBG] 5.0 starting backfill to osd.4 from (0'0,0'0] MAX to 407'13 2024-02-22T19:47:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:41.503565+0000 osd.7 (osd.7) 298 : cluster [DBG] 5.9 starting backfill to osd.4 from (0'0,0'0] MAX to 326'16 2024-02-22T19:47:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:41.508601+0000 osd.5 (osd.5) 185 : cluster [DBG] 5.7 starting backfill to osd.4 from (0'0,0'0] MAX to 412'25 2024-02-22T19:47:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:41.787055+0000 mgr.y (mgr.14535) 713 : cluster [DBG] pgmap v958: 138 pgs: 8 creating+peering, 13 peering, 8 unknown, 109 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-22T19:47:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:42.485638+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e472: 8 total, 8 up, 6 in 2024-02-22T19:47:43.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:47:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:42.869112+0000 mon.a (mon.0) 1713 : audit [DBG] from='client.? 172.21.15.53:0/274712995' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:43.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:43.487461+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e473: 8 total, 8 up, 6 in 2024-02-22T19:47:43.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:42.869112+0000 mon.a (mon.0) 1713 : audit [DBG] from='client.? 172.21.15.53:0/274712995' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:43.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:43.487461+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e473: 8 total, 8 up, 6 in 2024-02-22T19:47:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:42.869112+0000 mon.a (mon.0) 1713 : audit [DBG] from='client.? 172.21.15.53:0/274712995' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:47:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:43.487461+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e473: 8 total, 8 up, 6 in 2024-02-22T19:47:44.400 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:44.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:44.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:47:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:47:44.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:44.528 INFO:teuthology.orchestra.run.smithi053.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-02-22T19:47:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:43.787423+0000 mgr.y (mgr.14535) 714 : cluster [DBG] pgmap v961: 138 pgs: 8 creating+peering, 13 peering, 8 unknown, 109 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail 2024-02-22T19:47:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:44.461233+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.53:0/2556022702' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:47:44.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:44.461524+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:47:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:43.787423+0000 mgr.y (mgr.14535) 714 : cluster [DBG] pgmap v961: 138 pgs: 8 creating+peering, 13 peering, 8 unknown, 109 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail 2024-02-22T19:47:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:44.461233+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.53:0/2556022702' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:47:44.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:44.461524+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:47:44.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:47:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:47:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:43.787423+0000 mgr.y (mgr.14535) 714 : cluster [DBG] pgmap v961: 138 pgs: 8 creating+peering, 13 peering, 8 unknown, 109 active+clean; 5.7 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail 2024-02-22T19:47:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:44.461233+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.53:0/2556022702' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:47:44.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:44.461524+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T19:47:45.445 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5, 4] out_osds: [0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:47:45.445 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:47:45.445 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-02-22T19:47:45.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-22T19:47:45.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:44.523379+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:47:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:45.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:45.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:45.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:45.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:47:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:47:45.681 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:44.523379+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:47:45.681 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:44.523457+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e474: 8 total, 8 up, 6 in 2024-02-22T19:47:45.681 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:45.395073+0000 mgr.y (mgr.14535) 715 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:45.685 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:45.708 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:45.711 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:45.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:45.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:45.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:44.523379+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-02-22T19:47:45.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:44.523457+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e474: 8 total, 8 up, 6 in 2024-02-22T19:47:45.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:45.395073+0000 mgr.y (mgr.14535) 715 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:45.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:44.523457+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e474: 8 total, 8 up, 6 in 2024-02-22T19:47:45.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:45.395073+0000 mgr.y (mgr.14535) 715 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:45.845 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:45.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:45.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:45.974 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:46.005 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:46.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:46.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:46.156 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:46.161 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:46.182 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:46.210 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:46.300 DEBUG:teuthology.orchestra.run.smithi053:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-02-22T19:47:46.302 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-02-22T19:47:46.303 INFO:teuthology.orchestra.run:waiting for 3000 2024-02-22T19:47:46.427 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:hints = 1 2024-02-22T19:47:46.427 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-02-22T19:47:46.427 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object prefix: benchmark_data_smithi053_199129 2024-02-22T19:47:46.428 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:47:46.428 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 0 0 0 0 0 0 - 0 2024-02-22T19:47:46.560 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:46.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:46.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:46.569 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:47:46.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:46.796 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:45.787969+0000 mgr.y (mgr.14535) 716 : cluster [DBG] pgmap v963: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 5.7 KiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 716 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 593 B/s, 13 objects/s recovering 2024-02-22T19:47:46.796 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:46.299094+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.53:0/222446807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:47:46.796 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:46.299414+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:47:46.797 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:45.787969+0000 mgr.y (mgr.14535) 716 : cluster [DBG] pgmap v963: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 5.7 KiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 716 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 593 B/s, 13 objects/s recovering 2024-02-22T19:47:46.797 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:46.299094+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.53:0/222446807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:47:46.797 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:46.299414+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:47:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:45.787969+0000 mgr.y (mgr.14535) 716 : cluster [DBG] pgmap v963: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 5.7 KiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 716 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 593 B/s, 13 objects/s recovering 2024-02-22T19:47:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:46.299094+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.53:0/222446807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:47:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:46.299414+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:47:47.428 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 1 16 2698 2682 167.615 167.625 0.00532849 0.00594804 2024-02-22T19:47:49.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:46.522261+0000 mon.a (mon.0) 1719 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:46.522521+0000 mon.a (mon.0) 1720 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:46.540130+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:46.540194+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e475: 8 total, 8 up, 6 in 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.202049+0000 osd.2 (osd.2) 284 : cluster [DBG] 6.f scrub starts 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.490034+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e476: 8 total, 8 up, 6 in 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:46.522261+0000 mon.a (mon.0) 1719 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:47:49.075 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:46.522521+0000 mon.a (mon.0) 1720 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:46.540130+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:46.540194+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e475: 8 total, 8 up, 6 in 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.202049+0000 osd.2 (osd.2) 284 : cluster [DBG] 6.f scrub starts 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.490034+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e476: 8 total, 8 up, 6 in 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:46.842453+0000 osd.1 (osd.1) 277 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:46.844811+0000 osd.1 (osd.1) 278 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.396419+0000 osd.2 (osd.2) 285 : cluster [DBG] 6.f scrub ok 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.788414+0000 mgr.y (mgr.14535) 717 : cluster [DBG] pgmap v966: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 5.7 KiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 713 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 591 B/s, 13 objects/s recovering 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:48.158189+0000 osd.2 (osd.2) 286 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:48.162893+0000 osd.2 (osd.2) 287 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:46.522261+0000 mon.a (mon.0) 1719 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:46.522521+0000 mon.a (mon.0) 1720 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:47:49.076 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:46.540130+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:47:49.077 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:46.540194+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e475: 8 total, 8 up, 6 in 2024-02-22T19:47:49.077 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.202049+0000 osd.2 (osd.2) 284 : cluster [DBG] 6.f scrub starts 2024-02-22T19:47:49.077 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.490034+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e476: 8 total, 8 up, 6 in 2024-02-22T19:47:49.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:49.078 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:49.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:49.078 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 2 16 5664 5648 176.487 185.375 0.00538658 0.00565266 2024-02-22T19:47:49.078 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:46.842453+0000 osd.1 (osd.1) 277 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:47:49.078 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:46.844811+0000 osd.1 (osd.1) 278 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.396419+0000 osd.2 (osd.2) 285 : cluster [DBG] 6.f scrub ok 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.788414+0000 mgr.y (mgr.14535) 717 : cluster [DBG] pgmap v966: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 5.7 KiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 713 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 591 B/s, 13 objects/s recovering 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:48.158189+0000 osd.2 (osd.2) 286 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:48.162893+0000 osd.2 (osd.2) 287 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:46.842453+0000 osd.1 (osd.1) 277 : cluster [DBG] 3.4 scrub starts 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:46.844811+0000 osd.1 (osd.1) 278 : cluster [DBG] 3.4 scrub ok 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.396419+0000 osd.2 (osd.2) 285 : cluster [DBG] 6.f scrub ok 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.788414+0000 mgr.y (mgr.14535) 717 : cluster [DBG] pgmap v966: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 5.7 KiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 713 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 591 B/s, 13 objects/s recovering 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:48.158189+0000 osd.2 (osd.2) 286 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:47:49.079 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:48.162893+0000 osd.2 (osd.2) 287 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:47:49.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:49.217 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:49.242 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:49.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:49.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:49.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:49.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:49.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:49.428 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 3 16 8443 8427 175.549 173.688 0.00333785 0.00568289 2024-02-22T19:47:49.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:49] "GET /metrics HTTP/1.1" 200 174119 "" "Prometheus/2.33.4" 2024-02-22T19:47:49.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:49.574 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.731717+0000 osd.7 (osd.7) 299 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:47:49.574 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.829388+0000 osd.7 (osd.7) 300 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:47:49.574 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.884375+0000 osd.1 (osd.1) 279 : cluster [DBG] 4.a scrub starts 2024-02-22T19:47:49.574 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:47.887857+0000 osd.1 (osd.1) 280 : cluster [DBG] 4.a scrub ok 2024-02-22T19:47:49.578 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:50.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.731717+0000 osd.7 (osd.7) 299 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:47:50.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.829388+0000 osd.7 (osd.7) 300 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:47:50.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.884375+0000 osd.1 (osd.1) 279 : cluster [DBG] 4.a scrub starts 2024-02-22T19:47:50.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:47.887857+0000 osd.1 (osd.1) 280 : cluster [DBG] 4.a scrub ok 2024-02-22T19:47:50.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.731717+0000 osd.7 (osd.7) 299 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:47:50.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.829388+0000 osd.7 (osd.7) 300 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:47:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.884375+0000 osd.1 (osd.1) 279 : cluster [DBG] 4.a scrub starts 2024-02-22T19:47:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:47.887857+0000 osd.1 (osd.1) 280 : cluster [DBG] 4.a scrub ok 2024-02-22T19:47:50.429 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 4 16 10928 10912 170.432 155.312 0.00477257 0.00585946 2024-02-22T19:47:50.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:48.772872+0000 osd.7 (osd.7) 301 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:48.776945+0000 osd.7 (osd.7) 302 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:49.118588+0000 osd.3 (osd.3) 319 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:49.123976+0000 osd.3 (osd.3) 320 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:49.704225+0000 mon.a (mon.0) 1724 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:49.790745+0000 mgr.y (mgr.14535) 718 : cluster [DBG] pgmap v967: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 261 MiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 1.3 KiB/s rd, 44 MiB/s wr, 698 op/s; 3/13125 objects misplaced (0.023%); 424 B/s, 9 objects/s recovering 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:48.772872+0000 osd.7 (osd.7) 301 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:48.776945+0000 osd.7 (osd.7) 302 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:49.118588+0000 osd.3 (osd.3) 319 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:49.123976+0000 osd.3 (osd.3) 320 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:47:50.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:49.704225+0000 mon.a (mon.0) 1724 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:47:50.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:49.790745+0000 mgr.y (mgr.14535) 718 : cluster [DBG] pgmap v967: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 261 MiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 1.3 KiB/s rd, 44 MiB/s wr, 698 op/s; 3/13125 objects misplaced (0.023%); 424 B/s, 9 objects/s recovering 2024-02-22T19:47:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:48.772872+0000 osd.7 (osd.7) 301 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:47:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:48.776945+0000 osd.7 (osd.7) 302 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:47:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:49.118588+0000 osd.3 (osd.3) 319 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:47:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:49.123976+0000 osd.3 (osd.3) 320 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:47:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:49.704225+0000 mon.a (mon.0) 1724 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:47:50.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:49.790745+0000 mgr.y (mgr.14535) 718 : cluster [DBG] pgmap v967: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 261 MiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 1.3 KiB/s rd, 44 MiB/s wr, 698 op/s; 3/13125 objects misplaced (0.023%); 424 B/s, 9 objects/s recovering 2024-02-22T19:47:51.429 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 5 15 13489 13474 168.368 160.125 0.00431383 0.00592967 2024-02-22T19:47:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:49.901234+0000 osd.1 (osd.1) 281 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:47:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:49.907131+0000 osd.1 (osd.1) 282 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:47:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:50.216368+0000 osd.4 (osd.4) 161 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:47:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:50.219849+0000 osd.4 (osd.4) 162 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:47:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:49.901234+0000 osd.1 (osd.1) 281 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:47:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:49.907131+0000 osd.1 (osd.1) 282 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:47:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:50.216368+0000 osd.4 (osd.4) 161 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:47:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:50.219849+0000 osd.4 (osd.4) 162 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:47:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:49.901234+0000 osd.1 (osd.1) 281 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:47:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:49.907131+0000 osd.1 (osd.1) 282 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:47:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:50.216368+0000 osd.4 (osd.4) 161 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:47:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:50.219849+0000 osd.4 (osd.4) 162 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:47:52.430 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 6 16 15096 15080 157.023 100.375 0.00407181 0.00636284 2024-02-22T19:47:52.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:47:52.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:51.221493+0000 osd.4 (osd.4) 163 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:47:52.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:51.225125+0000 osd.4 (osd.4) 164 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:47:52.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:51.791450+0000 mgr.y (mgr.14535) 719 : cluster [DBG] pgmap v968: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 557 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 704 B/s rd, 77 MiB/s wr, 1.23k op/s; 3/27333 objects misplaced (0.011%); 165 B/s, 1 objects/s recovering 2024-02-22T19:47:52.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:51.221493+0000 osd.4 (osd.4) 163 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:47:52.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:51.225125+0000 osd.4 (osd.4) 164 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:47:52.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:51.791450+0000 mgr.y (mgr.14535) 719 : cluster [DBG] pgmap v968: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 557 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 704 B/s rd, 77 MiB/s wr, 1.23k op/s; 3/27333 objects misplaced (0.011%); 165 B/s, 1 objects/s recovering 2024-02-22T19:47:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:51.221493+0000 osd.4 (osd.4) 163 : cluster [DBG] 4.1 scrub starts 2024-02-22T19:47:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:51.225125+0000 osd.4 (osd.4) 164 : cluster [DBG] 4.1 scrub ok 2024-02-22T19:47:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:51.791450+0000 mgr.y (mgr.14535) 719 : cluster [DBG] pgmap v968: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 557 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 704 B/s rd, 77 MiB/s wr, 1.23k op/s; 3/27333 objects misplaced (0.011%); 165 B/s, 1 objects/s recovering 2024-02-22T19:47:53.364 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:53.374 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:53.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:53.430 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 7 16 16998 16982 151.573 118.875 0.00763829 0.00658937 2024-02-22T19:47:53.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:53.544 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:53.548 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:53.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:53.649 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:53.687 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:53.711 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:53.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:53.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:52.698363+0000 osd.5 (osd.5) 186 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:47:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:52.699334+0000 osd.5 (osd.5) 187 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:47:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:52.844134+0000 mon.a (mon.0) 1725 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:47:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:52.845246+0000 mon.a (mon.0) 1726 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:47:53.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:52.852008+0000 mon.a (mon.0) 1727 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:53.389868+0000 mon.a (mon.0) 1728 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:52.698363+0000 osd.5 (osd.5) 186 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:52.699334+0000 osd.5 (osd.5) 187 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:52.844134+0000 mon.a (mon.0) 1725 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:52.845246+0000 mon.a (mon.0) 1726 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:52.852008+0000 mon.a (mon.0) 1727 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:53.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:53.389868+0000 mon.a (mon.0) 1728 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:53.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:53.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:52.698363+0000 osd.5 (osd.5) 186 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:47:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:52.699334+0000 osd.5 (osd.5) 187 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:47:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:52.844134+0000 mon.a (mon.0) 1725 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:47:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:52.845246+0000 mon.a (mon.0) 1726 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:47:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:52.852008+0000 mon.a (mon.0) 1727 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:53.389868+0000 mon.a (mon.0) 1728 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:53.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:53.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:53.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:54.030 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:54.252 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:54.255 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:54.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:54.287 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:47:54.287 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:47:54.287 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 476, 2024-02-22T19:47:54.287 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-02-22T19:22:59.746063+0000", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-02-22T19:47:47.482913+0000", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-02-22T19:28:53.274045+0000", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-02-22T19:47:39.489272+0000", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5804032, 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 18, 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-02-22T19:47:54.288 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 11, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "pacific", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "device_health_metrics", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:26:54.342689+0000", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.289 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 11, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 11, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 11, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 11, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 11, 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:47:54.290 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "164", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "163", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.291 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.292 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-02-22T19:47:54.293 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_devicehealth": {} 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "iscsi", 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.555543+0000", 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 3, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 3, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 3, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 3, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 3, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:47:54.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "242", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.296 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.298 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".rgw.root", 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.986190+0000", 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.299 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.300 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "58", 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.301 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.302 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 4, 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.log", 2024-02-22T19:47:54.303 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:01.494051+0000", 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:47:54.304 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "297", 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.305 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.306 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.307 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 5, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.control", 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:03.518927+0000", 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.308 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 12, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 12, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 12, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 12, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 12, 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "5.c", 2024-02-22T19:47:54.309 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 422, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 421, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 421, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "392'11" 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "422", 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "422", 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "420", 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.310 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.311 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:47:54.312 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 6, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.meta", 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:05.596341+0000", 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.313 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.314 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "64", 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.315 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.316 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_bias": 4 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:47:54.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 11, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "unique_pool_4", 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:47:39.311188+0000", 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:47:54.318 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 16, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 16, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 16, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 16, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 16, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.319 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "474", 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:47:54.320 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:47:54.321 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "rados": {} 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.322 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab", 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 8, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 276, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.325 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/162271081", 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6802/162271081", 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6804/162271081", 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6803/162271081", 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2", 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:47:54.326 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 470, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-02-22T19:47:54.328 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6805/2973403044", 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6806/2973403044", 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6808/2973403044", 2024-02-22T19:47:54.329 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6807/2973403044", 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83", 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 18, 2024-02-22T19:47:54.330 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 469, 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.331 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.332 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/936980935", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6810/936980935", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6812/936980935", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6811/936980935", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993", 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 26, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 472, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.334 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-02-22T19:47:54.336 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6813/3878043918", 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6814/3878043918", 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6816/3878043918", 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6815/3878043918", 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:47:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211", 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 32, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 469, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.338 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6800", 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6801", 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6803", 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6802", 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.340 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6800/2373040976", 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6801/2373040976", 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6803/2373040976", 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6802/2373040976", 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07", 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.341 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 39, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 472, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6804", 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.342 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6805", 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6807", 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.343 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6806", 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6804/2413553331", 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6805/2413553331", 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6807/2413553331", 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6806/2413553331", 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.345 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e", 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 45, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 285, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.351 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6808", 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6809", 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:47:54.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6811", 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6810", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6808/3043274732", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6809/3043274732", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6811/3043274732", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6810/3043274732", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a", 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 53, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 470, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:47:54.355 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6812", 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6813", 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:47:54.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6815", 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6814", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6812/2605082260", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6813/2605082260", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6815/2605082260", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6814/2605082260", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:47:24.203029+0000", 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:47:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:47:03.902857+0000", 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.360 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:47:03.690994+0000", 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:46:45.955058+0000", 2024-02-22T19:47:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:47:06.788636+0000", 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:46:28.816657+0000", 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:46:46.714977+0000", 2024-02-22T19:47:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:46:43.605473+0000", 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [ 2024-02-22T19:47:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "1.4", 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: 6 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.1", 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.e", 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 7, 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.f", 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.7", 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.b", 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.f", 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 5 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "4.17", 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.8", 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 1, 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [ 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.5", 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: 3 2024-02-22T19:47:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.9", 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: 3 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": { 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4161439976": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2578736337": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:47:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/762516580": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3676085495": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3516945073": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/828421682": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4127851809": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/1906929179": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2640420245": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/2854906057": "2024-02-23T19:23:32.485117+0000" 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-02-22T19:47:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "ruleset-failure-domain": "osd", 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-02-22T19:47:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-02-22T19:47:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-02-22T19:47:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-02-22T19:47:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:47:54.374 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:47:54.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:47:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:54.430 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 8 15 18905 18890 147.53 119.25 0.0144186 0.00677151 2024-02-22T19:47:54.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:53.126511+0000 osd.4 (osd.4) 165 : cluster [DBG] 4.c scrub starts 2024-02-22T19:47:54.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:53.131518+0000 osd.4 (osd.4) 166 : cluster [DBG] 4.c scrub ok 2024-02-22T19:47:54.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:53.791917+0000 mgr.y (mgr.14535) 720 : cluster [DBG] pgmap v969: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 557 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 70 MiB/s wr, 1.11k op/s; 3/27333 objects misplaced (0.011%) 2024-02-22T19:47:54.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:54.181405+0000 mon.a (mon.0) 1729 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:54.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:54.283896+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? 172.21.15.53:0/3238284317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:47:55.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:53.126511+0000 osd.4 (osd.4) 165 : cluster [DBG] 4.c scrub starts 2024-02-22T19:47:55.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:53.131518+0000 osd.4 (osd.4) 166 : cluster [DBG] 4.c scrub ok 2024-02-22T19:47:55.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:53.791917+0000 mgr.y (mgr.14535) 720 : cluster [DBG] pgmap v969: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 557 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 70 MiB/s wr, 1.11k op/s; 3/27333 objects misplaced (0.011%) 2024-02-22T19:47:55.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:54.181405+0000 mon.a (mon.0) 1729 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:55.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:54.283896+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? 172.21.15.53:0/3238284317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:47:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:47:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:47:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:53.126511+0000 osd.4 (osd.4) 165 : cluster [DBG] 4.c scrub starts 2024-02-22T19:47:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:53.131518+0000 osd.4 (osd.4) 166 : cluster [DBG] 4.c scrub ok 2024-02-22T19:47:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:53.791917+0000 mgr.y (mgr.14535) 720 : cluster [DBG] pgmap v969: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 557 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 70 MiB/s wr, 1.11k op/s; 3/27333 objects misplaced (0.011%) 2024-02-22T19:47:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:54.181405+0000 mon.a (mon.0) 1729 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:47:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:54.283896+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? 172.21.15.53:0/3238284317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:47:55.285 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:47:55.285 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:47:55.430 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 9 16 20967 20951 145.449 128.812 0.00625267 0.00686833 2024-02-22T19:47:55.855 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:47:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:47:55.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:53.889534+0000 osd.1 (osd.1) 283 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:47:55.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:53.891402+0000 osd.1 (osd.1) 284 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:47:55.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:54.737282+0000 osd.5 (osd.5) 188 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:47:55.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:54.743143+0000 osd.5 (osd.5) 189 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:47:55.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:55.405217+0000 mgr.y (mgr.14535) 721 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:53.889534+0000 osd.1 (osd.1) 283 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:47:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:53.891402+0000 osd.1 (osd.1) 284 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:47:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:54.737282+0000 osd.5 (osd.5) 188 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:47:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:54.743143+0000 osd.5 (osd.5) 189 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:47:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:55.405217+0000 mgr.y (mgr.14535) 721 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:53.889534+0000 osd.1 (osd.1) 283 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:47:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:53.891402+0000 osd.1 (osd.1) 284 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:47:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:54.737282+0000 osd.5 (osd.5) 188 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:47:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:54.743143+0000 osd.5 (osd.5) 189 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:47:56.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:55.405217+0000 mgr.y (mgr.14535) 721 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:47:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:47:56.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:56.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:47:56.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:56.367 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:56.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:56.370 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:56.431 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 10 16 22876 22860 142.827 119.312 0.00528391 0.00699643 2024-02-22T19:47:56.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:54.794638+0000 osd.7 (osd.7) 303 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:54.799089+0000 osd.7 (osd.7) 304 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:54.922810+0000 osd.1 (osd.1) 285 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:54.924166+0000 osd.1 (osd.1) 286 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.128937+0000 osd.4 (osd.4) 167 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.133162+0000 osd.4 (osd.4) 168 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:47:57.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.792853+0000 mgr.y (mgr.14535) 722 : cluster [DBG] pgmap v970: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.2 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 130 MiB/s wr, 2.09k op/s; 3/58521 objects misplaced (0.005%) 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:54.794638+0000 osd.7 (osd.7) 303 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:54.799089+0000 osd.7 (osd.7) 304 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:54.922810+0000 osd.1 (osd.1) 285 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:54.924166+0000 osd.1 (osd.1) 286 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.128937+0000 osd.4 (osd.4) 167 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.133162+0000 osd.4 (osd.4) 168 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:47:57.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.792853+0000 mgr.y (mgr.14535) 722 : cluster [DBG] pgmap v970: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.2 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 130 MiB/s wr, 2.09k op/s; 3/58521 objects misplaced (0.005%) 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:54.794638+0000 osd.7 (osd.7) 303 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:54.799089+0000 osd.7 (osd.7) 304 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:54.922810+0000 osd.1 (osd.1) 285 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:54.924166+0000 osd.1 (osd.1) 286 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.128937+0000 osd.4 (osd.4) 167 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.133162+0000 osd.4 (osd.4) 168 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:47:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.792853+0000 mgr.y (mgr.14535) 722 : cluster [DBG] pgmap v970: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.2 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 130 MiB/s wr, 2.09k op/s; 3/58521 objects misplaced (0.005%) 2024-02-22T19:47:57.194 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:57.194 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:47:57.231 INFO:tasks.thrashosds.thrasher:j is {'epoch': 476, 'fsid': 'b6783888-d1b7-11ee-95bf-87774f69a715', 'created': '2024-02-22T19:22:59.746063+0000', 'modified': '2024-02-22T19:47:47.482913+0000', 'last_up_change': '2024-02-22T19:28:53.274045+0000', 'last_in_change': '2024-02-22T19:47:39.489272+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '163', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '297', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 12, 'pg_placement_num': 12, 'pg_placement_num_target': 12, 'pg_num_target': 12, 'pg_num_pending': 12, 'last_pg_merge_meta': {'source_pgid': '5.c', 'ready_epoch': 422, 'last_epoch_started': 421, 'last_epoch_clean': 421, 'source_version': "0'0", 'target_version': "392'11"}, 'last_change': '422', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '422', 'last_force_op_resend_preluminous': '420', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-02-22T19:47:39.311188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '474', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '4326a76a-6144-4859-9b40-5ff69edbbcab', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 276, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6801', 'nonce': 162271081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 162271081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 162271081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6803', 'nonce': 162271081}]}, 'public_addr': '172.21.15.53:6801/162271081', 'cluster_addr': '172.21.15.53:6802/162271081', 'heartbeat_back_addr': '172.21.15.53:6804/162271081', 'heartbeat_front_addr': '172.21.15.53:6803/162271081', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 470, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6805', 'nonce': 2973403044}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2973403044}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2973403044}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6807', 'nonce': 2973403044}]}, 'public_addr': '172.21.15.53:6805/2973403044', 'cluster_addr': '172.21.15.53:6806/2973403044', 'heartbeat_back_addr': '172.21.15.53:6808/2973403044', 'heartbeat_front_addr': '172.21.15.53:6807/2973403044', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 469, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6809', 'nonce': 936980935}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 936980935}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 936980935}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6811', 'nonce': 936980935}]}, 'public_addr': '172.21.15.53:6809/936980935', 'cluster_addr': '172.21.15.53:6810/936980935', 'heartbeat_back_addr': '172.21.15.53:6812/936980935', 'heartbeat_front_addr': '172.21.15.53:6811/936980935', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3f285320-aa28-4fc1-b1c7-456c9aecd993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 472, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6813', 'nonce': 3878043918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 3878043918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 3878043918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6815', 'nonce': 3878043918}]}, 'public_addr': '172.21.15.53:6813/3878043918', 'cluster_addr': '172.21.15.53:6814/3878043918', 'heartbeat_back_addr': '172.21.15.53:6816/3878043918', 'heartbeat_front_addr': '172.21.15.53:6815/3878043918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '773e8080-1028-48fe-8f6e-ed6beca60211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 469, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2373040976}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6801', 'nonce': 2373040976}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 2373040976}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2373040976}]}, 'public_addr': '172.21.15.64:6800/2373040976', 'cluster_addr': '172.21.15.64:6801/2373040976', 'heartbeat_back_addr': '172.21.15.64:6803/2373040976', 'heartbeat_front_addr': '172.21.15.64:6802/2373040976', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 472, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6804', 'nonce': 2413553331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6805', 'nonce': 2413553331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 2413553331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2413553331}]}, 'public_addr': '172.21.15.64:6804/2413553331', 'cluster_addr': '172.21.15.64:6805/2413553331', 'heartbeat_back_addr': '172.21.15.64:6807/2413553331', 'heartbeat_front_addr': '172.21.15.64:6806/2413553331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 285, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6808', 'nonce': 3043274732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6809', 'nonce': 3043274732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 3043274732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 3043274732}]}, 'public_addr': '172.21.15.64:6808/3043274732', 'cluster_addr': '172.21.15.64:6809/3043274732', 'heartbeat_back_addr': '172.21.15.64:6811/3043274732', 'heartbeat_front_addr': '172.21.15.64:6810/3043274732', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'cde36201-cb77-4967-9402-6fdbe633b40a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 470, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6812', 'nonce': 2605082260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6813', 'nonce': 2605082260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6815', 'nonce': 2605082260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 2605082260}]}, 'public_addr': '172.21.15.64:6812/2605082260', 'cluster_addr': '172.21.15.64:6813/2605082260', 'heartbeat_back_addr': '172.21.15.64:6815/2605082260', 'heartbeat_front_addr': '172.21.15.64:6814/2605082260', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:47:24.203029+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:47:03.902857+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:47:03.690994+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:46:45.955058+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:47:06.788636+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:46:28.816657+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:46:46.714977+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:46:43.605473+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.4', 'osds': [3, 5, 6]}, {'pgid': '2.1', 'osds': [5, 3, 4]}], 'pg_upmap_items': [{'pgid': '3.e', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '3.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.8', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [{'pgid': '5.5', 'osds': [1, 7, 3]}, {'pgid': '5.9', 'osds': [7, 1, 3]}], 'primary_temp': [], 'blocklist': {'172.21.15.53:0/4161439976': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/2578736337': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/762516580': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:6800/3676085495': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:6800/3516945073': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/828421682': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/4127851809': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/1906929179': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:0/2640420245': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:6800/2854906057': '2024-02-23T19:23:32.485117+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-22T19:47:57.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:47:57.287 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:57.305 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:57.305 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:57.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:47:57.431 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 11 16 24826 24810 140.921 121.875 0.00415943 0.00709019 2024-02-22T19:47:57.442 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:57.466 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:47:57.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:57.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:47:57.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:47:57.609 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:47:57.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:57.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:47:57.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:47:57.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:47:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.811582+0000 osd.7 (osd.7) 305 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:47:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.877239+0000 osd.7 (osd.7) 306 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:47:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.923218+0000 osd.1 (osd.1) 287 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:47:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:55.926633+0000 osd.1 (osd.1) 288 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:47:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:56.157532+0000 osd.3 (osd.3) 321 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:47:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:56.165267+0000 osd.3 (osd.3) 322 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:47:57.962 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:47:57.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:57.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.811582+0000 osd.7 (osd.7) 305 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.877239+0000 osd.7 (osd.7) 306 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.923218+0000 osd.1 (osd.1) 287 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:55.926633+0000 osd.1 (osd.1) 288 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:56.157532+0000 osd.3 (osd.3) 321 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:56.165267+0000 osd.3 (osd.3) 322 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.811582+0000 osd.7 (osd.7) 305 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.877239+0000 osd.7 (osd.7) 306 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:47:57.967 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.923218+0000 osd.1 (osd.1) 287 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:47:57.968 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:55.926633+0000 osd.1 (osd.1) 288 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:47:57.968 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:56.157532+0000 osd.3 (osd.3) 321 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:47:57.968 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:56.165267+0000 osd.3 (osd.3) 322 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:47:58.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:47:58.106 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:47:58.138 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:58.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:47:58.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:47:58.294 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:58.432 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 12 16 26440 26424 137.573 100.875 0.00651633 0.00726192 2024-02-22T19:47:59.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:57.127908+0000 osd.3 (osd.3) 323 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:47:59.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:57.136962+0000 osd.3 (osd.3) 324 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:47:59.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:57.777642+0000 osd.5 (osd.5) 190 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:59.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:57.777687+0000 osd.5 (osd.5) 191 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:59.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:57.796076+0000 mgr.y (mgr.14535) 723 : cluster [DBG] pgmap v971: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.2 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 993 B/s rd, 117 MiB/s wr, 1.87k op/s; 3/58521 objects misplaced (0.005%) 2024-02-22T19:47:59.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:57.127908+0000 osd.3 (osd.3) 323 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:47:59.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:57.136962+0000 osd.3 (osd.3) 324 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:47:59.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:57.777642+0000 osd.5 (osd.5) 190 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:59.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:57.777687+0000 osd.5 (osd.5) 191 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:59.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:57.796076+0000 mgr.y (mgr.14535) 723 : cluster [DBG] pgmap v971: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.2 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 993 B/s rd, 117 MiB/s wr, 1.87k op/s; 3/58521 objects misplaced (0.005%) 2024-02-22T19:47:59.069 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:47:59.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:59.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:47:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:57.127908+0000 osd.3 (osd.3) 323 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:47:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:57.136962+0000 osd.3 (osd.3) 324 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:47:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:57.777642+0000 osd.5 (osd.5) 190 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:47:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:57.777687+0000 osd.5 (osd.5) 191 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:47:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:57.796076+0000 mgr.y (mgr.14535) 723 : cluster [DBG] pgmap v971: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.2 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 993 B/s rd, 117 MiB/s wr, 1.87k op/s; 3/58521 objects misplaced (0.005%) 2024-02-22T19:47:59.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:47:59.214 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:59.243 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:47:59.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:47:59.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:47:59.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:47:59.385 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:47:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:47:59] "GET /metrics HTTP/1.1" 200 176640 "" "Prometheus/2.33.4" 2024-02-22T19:47:59.388 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:47:59.432 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 13 16 28271 28255 135.793 114.438 0.00194004 0.00735589 2024-02-22T19:48:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:58.667416+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e477: 8 total, 8 up, 6 in 2024-02-22T19:48:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:47:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:47:59.198191+0000 mon.a (mon.0) 1731 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:58.667416+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e477: 8 total, 8 up, 6 in 2024-02-22T19:48:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:47:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:47:59.198191+0000 mon.a (mon.0) 1731 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:58.667416+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e477: 8 total, 8 up, 6 in 2024-02-22T19:48:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:47:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:47:59.198191+0000 mon.a (mon.0) 1731 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:00.433 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 14 16 30043 30027 134.003 110.75 0.00550311 0.0074562 2024-02-22T19:48:00.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:58.759603+0000 osd.7 (osd.7) 307 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:48:00.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:58.764396+0000 osd.7 (osd.7) 308 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:48:00.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:59.799041+0000 mgr.y (mgr.14535) 724 : cluster [DBG] pgmap v973: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.5 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 124 MiB/s wr, 1.98k op/s; 3/72654 objects misplaced (0.004%) 2024-02-22T19:48:00.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:59.929913+0000 osd.1 (osd.1) 289 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:48:00.948 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:47:59.995371+0000 osd.1 (osd.1) 290 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:48:00.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:58.759603+0000 osd.7 (osd.7) 307 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:48:00.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:58.764396+0000 osd.7 (osd.7) 308 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:48:00.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:59.799041+0000 mgr.y (mgr.14535) 724 : cluster [DBG] pgmap v973: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.5 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 124 MiB/s wr, 1.98k op/s; 3/72654 objects misplaced (0.004%) 2024-02-22T19:48:00.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:59.929913+0000 osd.1 (osd.1) 289 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:48:00.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:47:59.995371+0000 osd.1 (osd.1) 290 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:48:00.949 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:00.952 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:00.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:01.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:01.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:58.759603+0000 osd.7 (osd.7) 307 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:48:01.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:58.764396+0000 osd.7 (osd.7) 308 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:48:01.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:59.799041+0000 mgr.y (mgr.14535) 724 : cluster [DBG] pgmap v973: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.5 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 124 MiB/s wr, 1.98k op/s; 3/72654 objects misplaced (0.004%) 2024-02-22T19:48:01.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:59.929913+0000 osd.1 (osd.1) 289 : cluster [DBG] 6.1d scrub starts 2024-02-22T19:48:01.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:47:59.995371+0000 osd.1 (osd.1) 290 : cluster [DBG] 6.1d scrub ok 2024-02-22T19:48:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:01.121 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:01.125 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:01.125 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:01.226 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:01.283 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:01.321 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:01.327 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:01.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:01.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:01.434 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 15 16 31658 31642 131.782 100.938 0.00532221 0.00758331 2024-02-22T19:48:01.461 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:01.488 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:01.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:01.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:01.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:48:01.651 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:01.735 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:48:01.738 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:01.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:48:01.760 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:01.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:01.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:00.089509+0000 osd.4 (osd.4) 169 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:48:01.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:00.092441+0000 osd.4 (osd.4) 170 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:48:01.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:00.851136+0000 osd.3 (osd.3) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:01.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:00.851188+0000 osd.3 (osd.3) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:01.904 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:01.909 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:01.909 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:02.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:00.089509+0000 osd.4 (osd.4) 169 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:48:02.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:00.092441+0000 osd.4 (osd.4) 170 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:48:02.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:00.851136+0000 osd.3 (osd.3) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:02.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:00.851188+0000 osd.3 (osd.3) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:02.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:00.089509+0000 osd.4 (osd.4) 169 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:48:02.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:00.092441+0000 osd.4 (osd.4) 170 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:48:02.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:00.851136+0000 osd.3 (osd.3) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:02.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:00.851188+0000 osd.3 (osd.3) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:02.010 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:02.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:02.087 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:02.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:02.230 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:02.261 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:02.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:02.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:02.429 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:02.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:02.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:02.436 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 16 16 33253 33237 129.766 99.6875 0.00725148 0.00770013 2024-02-22T19:48:02.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:02.599 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:02.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:02.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:01.125082+0000 osd.4 (osd.4) 171 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:48:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:01.129949+0000 osd.4 (osd.4) 172 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:48:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:01.388695+0000 osd.7 (osd.7) 309 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:01.388746+0000 osd.7 (osd.7) 310 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:01.695898+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e478: 8 total, 8 up, 6 in 2024-02-22T19:48:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:01.799617+0000 mgr.y (mgr.14535) 725 : cluster [DBG] pgmap v975: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.8 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 154 MiB/s wr, 2.47k op/s; 3/86592 objects misplaced (0.003%) 2024-02-22T19:48:02.873 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:02.878 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:01.125082+0000 osd.4 (osd.4) 171 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:48:02.878 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:01.129949+0000 osd.4 (osd.4) 172 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:01.388695+0000 osd.7 (osd.7) 309 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:01.388746+0000 osd.7 (osd.7) 310 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:01.695898+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e478: 8 total, 8 up, 6 in 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:01.799617+0000 mgr.y (mgr.14535) 725 : cluster [DBG] pgmap v975: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.8 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 154 MiB/s wr, 2.47k op/s; 3/86592 objects misplaced (0.003%) 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:01.125082+0000 osd.4 (osd.4) 171 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:01.129949+0000 osd.4 (osd.4) 172 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:01.388695+0000 osd.7 (osd.7) 309 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:01.388746+0000 osd.7 (osd.7) 310 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:01.695898+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e478: 8 total, 8 up, 6 in 2024-02-22T19:48:02.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:01.799617+0000 mgr.y (mgr.14535) 725 : cluster [DBG] pgmap v975: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.8 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 154 MiB/s wr, 2.47k op/s; 3/86592 objects misplaced (0.003%) 2024-02-22T19:48:02.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:02.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:02.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:03.043 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:03.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:03.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:03.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:03.256 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:03.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:03.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:03.436 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 17 16 34677 34661 127.368 89 0.0206548 0.00784496 2024-02-22T19:48:03.741 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:04.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.092504+0000 osd.3 (osd.3) 327 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:48:04.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.098330+0000 osd.3 (osd.3) 328 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:48:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.133793+0000 osd.4 (osd.4) 173 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:48:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.141467+0000 osd.4 (osd.4) 174 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:48:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.963272+0000 osd.1 (osd.1) 291 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:48:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.967300+0000 osd.1 (osd.1) 292 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:48:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:03.495603+0000 mon.b (mon.1) 99 : audit [DBG] from='client.? 172.21.15.53:0/3382951928' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.092504+0000 osd.3 (osd.3) 327 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.098330+0000 osd.3 (osd.3) 328 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.133793+0000 osd.4 (osd.4) 173 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.141467+0000 osd.4 (osd.4) 174 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.963272+0000 osd.1 (osd.1) 291 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.967300+0000 osd.1 (osd.1) 292 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:48:04.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:03.495603+0000 mon.b (mon.1) 99 : audit [DBG] from='client.? 172.21.15.53:0/3382951928' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.092504+0000 osd.3 (osd.3) 327 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:48:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.098330+0000 osd.3 (osd.3) 328 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:48:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.133793+0000 osd.4 (osd.4) 173 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:48:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.141467+0000 osd.4 (osd.4) 174 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:48:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.963272+0000 osd.1 (osd.1) 291 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:48:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.967300+0000 osd.1 (osd.1) 292 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:48:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:03.495603+0000 mon.b (mon.1) 99 : audit [DBG] from='client.? 172.21.15.53:0/3382951928' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:04.436 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 18 16 36259 36243 125.786 98.875 0.0176841 0.00794377 2024-02-22T19:48:05.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.830473+0000 osd.7 (osd.7) 311 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:48:05.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:02.833549+0000 osd.7 (osd.7) 312 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:48:05.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:03.802303+0000 mgr.y (mgr.14535) 726 : cluster [DBG] pgmap v976: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.8 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s; 3/86592 objects misplaced (0.003%) 2024-02-22T19:48:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.830473+0000 osd.7 (osd.7) 311 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:48:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:02.833549+0000 osd.7 (osd.7) 312 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:48:05.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:03.802303+0000 mgr.y (mgr.14535) 726 : cluster [DBG] pgmap v976: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.8 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s; 3/86592 objects misplaced (0.003%) 2024-02-22T19:48:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:48:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:48:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.830473+0000 osd.7 (osd.7) 311 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:48:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:02.833549+0000 osd.7 (osd.7) 312 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:48:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:03.802303+0000 mgr.y (mgr.14535) 726 : cluster [DBG] pgmap v976: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 1.8 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s; 3/86592 objects misplaced (0.003%) 2024-02-22T19:48:05.438 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 19 16 37724 37708 123.969 91.5625 0.0117607 0.00805981 2024-02-22T19:48:05.729 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:48:05.756 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:03.831253+0000 osd.7 (osd.7) 313 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:03.835419+0000 osd.7 (osd.7) 314 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:04.063257+0000 osd.3 (osd.3) 329 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:04.068577+0000 osd.3 (osd.3) 330 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:05.415216+0000 mgr.y (mgr.14535) 727 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:05.426387+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.53:0/13334379' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:05.427556+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:03.831253+0000 osd.7 (osd.7) 313 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:03.835419+0000 osd.7 (osd.7) 314 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:04.063257+0000 osd.3 (osd.3) 329 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:04.068577+0000 osd.3 (osd.3) 330 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:05.415216+0000 mgr.y (mgr.14535) 727 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:05.426387+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.53:0/13334379' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:06.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:05.427556+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:03.831253+0000 osd.7 (osd.7) 313 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:48:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:03.835419+0000 osd.7 (osd.7) 314 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:48:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:04.063257+0000 osd.3 (osd.3) 329 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:48:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:04.068577+0000 osd.3 (osd.3) 330 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:48:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:05.415216+0000 mgr.y (mgr.14535) 727 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:06.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:05.426387+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.53:0/13334379' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:06.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:05.427556+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:06.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:48:06.439687+0000 min lat: 0.00115108 max lat: 0.165391 avg lat: 0.00813992 2024-02-22T19:48:06.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:48:06.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 20 16 39330 39314 122.789 100.375 0.0049583 0.00813992 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.124437+0000 osd.4 (osd.4) 175 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.127955+0000 osd.4 (osd.4) 176 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.723768+0000 mon.a (mon.0) 1734 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:05.740383+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.740464+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e479: 8 total, 8 up, 6 in 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.803632+0000 mgr.y (mgr.14535) 728 : cluster [DBG] pgmap v978: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.4 KiB/s rd, 151 MiB/s wr, 2.42k op/s; 3/110235 objects misplaced (0.003%) 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.815787+0000 osd.7 (osd.7) 315 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:48:06.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:05.818749+0000 osd.7 (osd.7) 316 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.124437+0000 osd.4 (osd.4) 175 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.127955+0000 osd.4 (osd.4) 176 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.723768+0000 mon.a (mon.0) 1734 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:05.740383+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.740464+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e479: 8 total, 8 up, 6 in 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.803632+0000 mgr.y (mgr.14535) 728 : cluster [DBG] pgmap v978: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.4 KiB/s rd, 151 MiB/s wr, 2.42k op/s; 3/110235 objects misplaced (0.003%) 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.815787+0000 osd.7 (osd.7) 315 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:48:06.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:05.818749+0000 osd.7 (osd.7) 316 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.124437+0000 osd.4 (osd.4) 175 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.127955+0000 osd.4 (osd.4) 176 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.723768+0000 mon.a (mon.0) 1734 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:05.740383+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.740464+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e479: 8 total, 8 up, 6 in 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.803632+0000 mgr.y (mgr.14535) 728 : cluster [DBG] pgmap v978: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.4 KiB/s rd, 151 MiB/s wr, 2.42k op/s; 3/110235 objects misplaced (0.003%) 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.815787+0000 osd.7 (osd.7) 315 : cluster [DBG] 3.11 deep-scrub starts 2024-02-22T19:48:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:05.818749+0000 osd.7 (osd.7) 316 : cluster [DBG] 3.11 deep-scrub ok 2024-02-22T19:48:07.230 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:07.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:07.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:07.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:07.374 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:07.398 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:07.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:07.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:07.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 21 16 40854 40838 121.478 95.25 0.00661676 0.0082267 2024-02-22T19:48:07.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:07.538 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:07.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:07.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:07.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:07.731 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:07.825 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:48:07.837 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:48:08.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:06.048375+0000 osd.3 (osd.3) 331 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:48:08.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:06.055122+0000 osd.3 (osd.3) 332 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:48:08.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:07.514564+0000 mon.a (mon.0) 1737 : cluster [DBG] osdmap e480: 8 total, 8 up, 6 in 2024-02-22T19:48:08.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:06.048375+0000 osd.3 (osd.3) 331 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:48:08.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:06.055122+0000 osd.3 (osd.3) 332 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:48:08.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:07.514564+0000 mon.a (mon.0) 1737 : cluster [DBG] osdmap e480: 8 total, 8 up, 6 in 2024-02-22T19:48:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:06.048375+0000 osd.3 (osd.3) 331 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:48:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:06.055122+0000 osd.3 (osd.3) 332 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:48:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:07.514564+0000 mon.a (mon.0) 1737 : cluster [DBG] osdmap e480: 8 total, 8 up, 6 in 2024-02-22T19:48:08.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 22 15 42175 42160 119.712 82.625 0.0155807 0.00834771 2024-02-22T19:48:08.802 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:07.092722+0000 osd.4 (osd.4) 177 : cluster [DBG] 6.e deep-scrub starts 2024-02-22T19:48:08.802 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:07.111942+0000 osd.4 (osd.4) 178 : cluster [DBG] 6.e deep-scrub ok 2024-02-22T19:48:08.802 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:07.779644+0000 mgr.y (mgr.14535) 729 : audit [DBG] from='client.25654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:48:08.802 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:07.804036+0000 mgr.y (mgr.14535) 730 : cluster [DBG] pgmap v980: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 838 B/s rd, 81 MiB/s wr, 1.29k op/s; 3/110235 objects misplaced (0.003%) 2024-02-22T19:48:08.802 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:08.391858+0000 mon.a (mon.0) 1738 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:08.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:07.092722+0000 osd.4 (osd.4) 177 : cluster [DBG] 6.e deep-scrub starts 2024-02-22T19:48:08.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:07.111942+0000 osd.4 (osd.4) 178 : cluster [DBG] 6.e deep-scrub ok 2024-02-22T19:48:08.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:07.779644+0000 mgr.y (mgr.14535) 729 : audit [DBG] from='client.25654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:48:08.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:07.804036+0000 mgr.y (mgr.14535) 730 : cluster [DBG] pgmap v980: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 838 B/s rd, 81 MiB/s wr, 1.29k op/s; 3/110235 objects misplaced (0.003%) 2024-02-22T19:48:08.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:08.391858+0000 mon.a (mon.0) 1738 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:08.865 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:48:08.865 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:48:08.865 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:48:08.866 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:48:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:07.092722+0000 osd.4 (osd.4) 177 : cluster [DBG] 6.e deep-scrub starts 2024-02-22T19:48:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:07.111942+0000 osd.4 (osd.4) 178 : cluster [DBG] 6.e deep-scrub ok 2024-02-22T19:48:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:07.779644+0000 mgr.y (mgr.14535) 729 : audit [DBG] from='client.25654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:48:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:07.804036+0000 mgr.y (mgr.14535) 730 : cluster [DBG] pgmap v980: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 838 B/s rd, 81 MiB/s wr, 1.29k op/s; 3/110235 objects misplaced (0.003%) 2024-02-22T19:48:09.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:08.391858+0000 mon.a (mon.0) 1738 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:09.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:48:09.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 23 16 43577 43561 118.315 87.5625 0.0116501 0.00844593 2024-02-22T19:48:09.708 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:09.712 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:09.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:09.714 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:48:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:09] "GET /metrics HTTP/1.1" 200 176657 "" "Prometheus/2.33.4" 2024-02-22T19:48:09.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:48:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:08.710962+0000 osd.5 (osd.5) 192 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:48:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:08.833373+0000 osd.5 (osd.5) 193 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:48:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:08.710962+0000 osd.5 (osd.5) 192 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:48:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:08.833373+0000 osd.5 (osd.5) 193 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:48:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:08.710962+0000 osd.5 (osd.5) 192 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:48:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:08.833373+0000 osd.5 (osd.5) 193 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:48:10.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 24 16 44952 44936 116.966 85.9375 0.00453641 0.00854576 2024-02-22T19:48:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:09.761929+0000 osd.5 (osd.5) 194 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:48:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:09.768841+0000 osd.5 (osd.5) 195 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:48:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:09.804718+0000 mgr.y (mgr.14535) 731 : cluster [DBG] pgmap v981: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s; 3/121488 objects misplaced (0.002%) 2024-02-22T19:48:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:09.761929+0000 osd.5 (osd.5) 194 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:48:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:09.768841+0000 osd.5 (osd.5) 195 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:48:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:09.804718+0000 mgr.y (mgr.14535) 731 : cluster [DBG] pgmap v981: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s; 3/121488 objects misplaced (0.002%) 2024-02-22T19:48:11.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:09.761929+0000 osd.5 (osd.5) 194 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:48:11.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:09.768841+0000 osd.5 (osd.5) 195 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:48:11.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:09.804718+0000 mgr.y (mgr.14535) 731 : cluster [DBG] pgmap v981: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s; 3/121488 objects misplaced (0.002%) 2024-02-22T19:48:11.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 25 16 46406 46390 115.923 90.875 0.0056593 0.00861995 2024-02-22T19:48:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.046507+0000 osd.4 (osd.4) 179 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:48:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.057578+0000 osd.4 (osd.4) 180 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:48:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.137450+0000 osd.3 (osd.3) 333 : cluster [DBG] 6.c scrub starts 2024-02-22T19:48:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.140852+0000 osd.3 (osd.3) 334 : cluster [DBG] 6.c scrub ok 2024-02-22T19:48:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.635127+0000 osd.4 (osd.4) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:12.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.635193+0000 osd.4 (osd.4) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:12.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:10.791763+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e481: 8 total, 8 up, 6 in 2024-02-22T19:48:12.180 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.046507+0000 osd.4 (osd.4) 179 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.057578+0000 osd.4 (osd.4) 180 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.137450+0000 osd.3 (osd.3) 333 : cluster [DBG] 6.c scrub starts 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.140852+0000 osd.3 (osd.3) 334 : cluster [DBG] 6.c scrub ok 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.635127+0000 osd.4 (osd.4) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.635193+0000 osd.4 (osd.4) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:10.791763+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e481: 8 total, 8 up, 6 in 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.046507+0000 osd.4 (osd.4) 179 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.057578+0000 osd.4 (osd.4) 180 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.137450+0000 osd.3 (osd.3) 333 : cluster [DBG] 6.c scrub starts 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.140852+0000 osd.3 (osd.3) 334 : cluster [DBG] 6.c scrub ok 2024-02-22T19:48:12.181 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.635127+0000 osd.4 (osd.4) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:12.182 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.635193+0000 osd.4 (osd.4) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:12.182 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:10.791763+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e481: 8 total, 8 up, 6 in 2024-02-22T19:48:12.194 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:48:12.200 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:12.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:48:12.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:12.353 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":979,"stamp":"2024-02-22T19:48:07.514634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2395543204,"num_objects":36745,"num_object_clones":0,"num_object_copies":110235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":36745,"num_whiteouts":0,"num_read":2244,"num_read_kb":1996,"num_write":37267,"num_write_kb":2339433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":510,"num_bytes_recovered":18312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37328,"ondisk_log_size":37328,"up":414,"acting":414,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":416,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":11608060,"kb_used_data":6836364,"kb_used_omap":0,"kb_used_meta":4771648,"kb_avail":550862852,"statfs":{"total":575970213888,"available":564083560448,"internally_reserved":0,"allocated":7000436736,"data_stored":6989625432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4886167552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":820969472,"num_objects":12527,"num_object_clones":0,"num_object_copies":37581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12527,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":12527,"num_write_kb":801728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.716260"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666467+0000","last_change":"2024-02-22T19:47:40.529625+0000","last_active":"2024-02-22T19:47:58.666467+0000","last_peered":"2024-02-22T19:47:58.666467+0000","last_clean":"2024-02-22T19:47:58.666467+0000","last_became_active":"2024-02-22T19:47:40.529014+0000","last_became_peered":"2024-02-22T19:47:40.529014+0000","last_unstale":"2024-02-22T19:47:58.666467+0000","last_undegraded":"2024-02-22T19:47:58.666467+0000","last_fullsized":"2024-02-22T19:47:58.666467+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:56.598068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:56.598068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":484,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674282+0000","last_change":"2024-02-22T19:47:40.523932+0000","last_active":"2024-02-22T19:47:58.674282+0000","last_peered":"2024-02-22T19:47:58.674282+0000","last_clean":"2024-02-22T19:47:58.674282+0000","last_became_active":"2024-02-22T19:47:40.523572+0000","last_became_peered":"2024-02-22T19:47:40.523572+0000","last_unstale":"2024-02-22T19:47:58.674282+0000","last_undegraded":"2024-02-22T19:47:58.674282+0000","last_fullsized":"2024-02-22T19:47:58.674282+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:20.457620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:20.457620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":28,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.665845+0000","last_change":"2024-02-22T19:47:42.959147+0000","last_active":"2024-02-22T19:47:58.665845+0000","last_peered":"2024-02-22T19:47:58.665845+0000","last_clean":"2024-02-22T19:47:58.665845+0000","last_became_active":"2024-02-22T19:47:40.534653+0000","last_became_peered":"2024-02-22T19:47:40.534653+0000","last_unstale":"2024-02-22T19:47:58.665845+0000","last_undegraded":"2024-02-22T19:47:58.665845+0000","last_fullsized":"2024-02-22T19:47:58.665845+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":472,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:46:09.901351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.901351+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":455,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.675889+0000","last_change":"2024-02-22T19:46:14.159107+0000","last_active":"2024-02-22T19:47:58.675889+0000","last_peered":"2024-02-22T19:47:58.675889+0000","last_clean":"2024-02-22T19:47:58.675889+0000","last_became_active":"2024-02-22T19:41:52.115896+0000","last_became_peered":"2024-02-22T19:41:52.115896+0000","last_unstale":"2024-02-22T19:47:58.675889+0000","last_undegraded":"2024-02-22T19:47:58.675889+0000","last_fullsized":"2024-02-22T19:47:58.675889+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.159012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.159012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":456,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673250+0000","last_change":"2024-02-22T19:44:56.776815+0000","last_active":"2024-02-22T19:47:58.673250+0000","last_peered":"2024-02-22T19:47:58.673250+0000","last_clean":"2024-02-22T19:47:58.673250+0000","last_became_active":"2024-02-22T19:41:52.112449+0000","last_became_peered":"2024-02-22T19:41:52.112449+0000","last_unstale":"2024-02-22T19:47:58.673250+0000","last_undegraded":"2024-02-22T19:47:58.673250+0000","last_fullsized":"2024-02-22T19:47:58.673250+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:56.776721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:44:56.776721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":568,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:02.098450+0000","last_change":"2024-02-22T19:48:02.098450+0000","last_active":"2024-02-22T19:48:02.098450+0000","last_peered":"2024-02-22T19:48:02.098450+0000","last_clean":"2024-02-22T19:48:02.098450+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:48:02.098450+0000","last_undegraded":"2024-02-22T19:48:02.098450+0000","last_fullsized":"2024-02-22T19:48:02.098450+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:48:02.098338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.098338+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":284,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.671436+0000","last_change":"2024-02-22T19:46:13.905945+0000","last_active":"2024-02-22T19:47:58.671436+0000","last_peered":"2024-02-22T19:47:58.671436+0000","last_clean":"2024-02-22T19:47:58.671436+0000","last_became_active":"2024-02-22T19:39:22.265936+0000","last_became_peered":"2024-02-22T19:39:22.265936+0000","last_unstale":"2024-02-22T19:47:58.671436+0000","last_undegraded":"2024-02-22T19:47:58.671436+0000","last_fullsized":"2024-02-22T19:47:58.671436+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.905878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.905878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":307,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674474+0000","last_change":"2024-02-22T19:47:52.699438+0000","last_active":"2024-02-22T19:47:58.674474+0000","last_peered":"2024-02-22T19:47:58.674474+0000","last_clean":"2024-02-22T19:47:58.674474+0000","last_became_active":"2024-02-22T19:47:40.523380+0000","last_became_peered":"2024-02-22T19:47:40.523380+0000","last_unstale":"2024-02-22T19:47:58.674474+0000","last_undegraded":"2024-02-22T19:47:58.674474+0000","last_fullsized":"2024-02-22T19:47:58.674474+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:52.699342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:52.699342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":33,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:02.141583+0000","last_change":"2024-02-22T19:48:02.141583+0000","last_active":"2024-02-22T19:48:02.141583+0000","last_peered":"2024-02-22T19:48:02.141583+0000","last_clean":"2024-02-22T19:48:02.141583+0000","last_became_active":"2024-02-22T19:47:40.534475+0000","last_became_peered":"2024-02-22T19:47:40.534475+0000","last_unstale":"2024-02-22T19:48:02.141583+0000","last_undegraded":"2024-02-22T19:48:02.141583+0000","last_fullsized":"2024-02-22T19:48:02.141583+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:48:02.141476+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.141476+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":280,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673242+0000","last_change":"2024-02-22T19:45:03.818156+0000","last_active":"2024-02-22T19:47:58.673242+0000","last_peered":"2024-02-22T19:47:58.673242+0000","last_clean":"2024-02-22T19:47:58.673242+0000","last_became_active":"2024-02-22T19:37:57.182875+0000","last_became_peered":"2024-02-22T19:37:57.182875+0000","last_unstale":"2024-02-22T19:47:58.673242+0000","last_undegraded":"2024-02-22T19:47:58.673242+0000","last_fullsized":"2024-02-22T19:47:58.673242+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.818051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'1","reported_seq":323,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673747+0000","last_change":"2024-02-22T19:47:40.541038+0000","last_active":"2024-02-22T19:47:58.673747+0000","last_peered":"2024-02-22T19:47:58.673747+0000","last_clean":"2024-02-22T19:47:58.673747+0000","last_became_active":"2024-02-22T19:47:40.525513+0000","last_became_peered":"2024-02-22T19:47:40.525513+0000","last_unstale":"2024-02-22T19:47:58.673747+0000","last_undegraded":"2024-02-22T19:47:58.673747+0000","last_fullsized":"2024-02-22T19:47:58.673747+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:44:44.719546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:44.719546+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":29,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666406+0000","last_change":"2024-02-22T19:47:50.219982+0000","last_active":"2024-02-22T19:47:58.666406+0000","last_peered":"2024-02-22T19:47:58.666406+0000","last_clean":"2024-02-22T19:47:58.666406+0000","last_became_active":"2024-02-22T19:47:40.535314+0000","last_became_peered":"2024-02-22T19:47:40.535314+0000","last_unstale":"2024-02-22T19:47:58.666406+0000","last_undegraded":"2024-02-22T19:47:58.666406+0000","last_fullsized":"2024-02-22T19:47:58.666406+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":472,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:47:50.219857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:50.219857+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666424+0000","last_change":"2024-02-22T19:47:40.535662+0000","last_active":"2024-02-22T19:47:58.666424+0000","last_peered":"2024-02-22T19:47:58.666424+0000","last_clean":"2024-02-22T19:47:58.666424+0000","last_became_active":"2024-02-22T19:47:40.535327+0000","last_became_peered":"2024-02-22T19:47:40.535327+0000","last_unstale":"2024-02-22T19:47:58.666424+0000","last_undegraded":"2024-02-22T19:47:58.666424+0000","last_fullsized":"2024-02-22T19:47:58.666424+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:18.486267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.486267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666618+0000","last_change":"2024-02-22T19:47:40.531038+0000","last_active":"2024-02-22T19:47:58.666618+0000","last_peered":"2024-02-22T19:47:58.666618+0000","last_clean":"2024-02-22T19:47:58.666618+0000","last_became_active":"2024-02-22T19:47:40.529638+0000","last_became_peered":"2024-02-22T19:47:40.529638+0000","last_unstale":"2024-02-22T19:47:58.666618+0000","last_undegraded":"2024-02-22T19:47:58.666618+0000","last_fullsized":"2024-02-22T19:47:58.666618+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.251960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.251960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":520,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.678031+0000","last_change":"2024-02-22T19:47:48.163004+0000","last_active":"2024-02-22T19:47:58.678031+0000","last_peered":"2024-02-22T19:47:58.678031+0000","last_clean":"2024-02-22T19:47:58.678031+0000","last_became_active":"2024-02-22T19:29:03.080833+0000","last_became_peered":"2024-02-22T19:29:03.080833+0000","last_unstale":"2024-02-22T19:47:58.678031+0000","last_undegraded":"2024-02-22T19:47:58.678031+0000","last_fullsized":"2024-02-22T19:47:58.678031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:47:48.162902+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:47:48.162902+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":473,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674120+0000","last_change":"2024-02-22T19:45:14.832581+0000","last_active":"2024-02-22T19:47:58.674120+0000","last_peered":"2024-02-22T19:47:58.674120+0000","last_clean":"2024-02-22T19:47:58.674120+0000","last_became_active":"2024-02-22T19:41:52.121182+0000","last_became_peered":"2024-02-22T19:41:52.121182+0000","last_unstale":"2024-02-22T19:47:58.674120+0000","last_undegraded":"2024-02-22T19:47:58.674120+0000","last_fullsized":"2024-02-22T19:47:58.674120+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.832482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.832482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:48:01.130102+0000","last_change":"2024-02-22T19:48:01.130061+0000","last_active":"2024-02-22T19:48:01.130102+0000","last_peered":"2024-02-22T19:48:01.130102+0000","last_clean":"2024-02-22T19:48:01.130102+0000","last_became_active":"2024-02-22T19:47:40.530915+0000","last_became_peered":"2024-02-22T19:47:40.530915+0000","last_unstale":"2024-02-22T19:48:01.130102+0000","last_undegraded":"2024-02-22T19:48:01.130102+0000","last_fullsized":"2024-02-22T19:48:01.130102+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:01.129956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:48:01.129956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":545,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674920+0000","last_change":"2024-02-22T19:47:41.273825+0000","last_active":"2024-02-22T19:47:58.674920+0000","last_peered":"2024-02-22T19:47:58.674920+0000","last_clean":"2024-02-22T19:47:58.674920+0000","last_became_active":"2024-02-22T19:47:40.537193+0000","last_became_peered":"2024-02-22T19:47:40.537193+0000","last_unstale":"2024-02-22T19:47:58.674920+0000","last_undegraded":"2024-02-22T19:47:58.674920+0000","last_fullsized":"2024-02-22T19:47:58.674920+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:44:43.774778+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:44:43.774778+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":494,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677054+0000","last_change":"2024-02-22T19:47:05.294685+0000","last_active":"2024-02-22T19:47:58.677054+0000","last_peered":"2024-02-22T19:47:58.677054+0000","last_clean":"2024-02-22T19:47:58.677054+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:47:58.677054+0000","last_undegraded":"2024-02-22T19:47:58.677054+0000","last_fullsized":"2024-02-22T19:47:58.677054+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:47:05.294613+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.294613+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":492,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:02.967377+0000","last_change":"2024-02-22T19:48:02.967377+0000","last_active":"2024-02-22T19:48:02.967377+0000","last_peered":"2024-02-22T19:48:02.967377+0000","last_clean":"2024-02-22T19:48:02.967377+0000","last_became_active":"2024-02-22T19:47:40.530203+0000","last_became_peered":"2024-02-22T19:47:40.530203+0000","last_unstale":"2024-02-22T19:48:02.967377+0000","last_undegraded":"2024-02-22T19:48:02.967377+0000","last_fullsized":"2024-02-22T19:48:02.967377+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:02.967304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.967304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":297,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677457+0000","last_change":"2024-02-22T19:47:12.324026+0000","last_active":"2024-02-22T19:47:58.677457+0000","last_peered":"2024-02-22T19:47:58.677457+0000","last_clean":"2024-02-22T19:47:58.677457+0000","last_became_active":"2024-02-22T19:39:22.249841+0000","last_became_peered":"2024-02-22T19:39:22.249841+0000","last_unstale":"2024-02-22T19:47:58.677457+0000","last_undegraded":"2024-02-22T19:47:58.677457+0000","last_fullsized":"2024-02-22T19:47:58.677457+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:12.323915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.323915+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":492,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:59.995515+0000","last_change":"2024-02-22T19:47:59.995479+0000","last_active":"2024-02-22T19:47:59.995515+0000","last_peered":"2024-02-22T19:47:59.995515+0000","last_clean":"2024-02-22T19:47:59.995515+0000","last_became_active":"2024-02-22T19:47:40.525659+0000","last_became_peered":"2024-02-22T19:47:40.525659+0000","last_unstale":"2024-02-22T19:47:59.995515+0000","last_undegraded":"2024-02-22T19:47:59.995515+0000","last_fullsized":"2024-02-22T19:47:59.995515+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:59.995375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:59.995375+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":481,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.668513+0000","last_change":"2024-02-22T19:47:56.165358+0000","last_active":"2024-02-22T19:47:58.668513+0000","last_peered":"2024-02-22T19:47:58.668513+0000","last_clean":"2024-02-22T19:47:58.668513+0000","last_became_active":"2024-02-22T19:39:22.270157+0000","last_became_peered":"2024-02-22T19:39:22.270157+0000","last_unstale":"2024-02-22T19:47:58.668513+0000","last_undegraded":"2024-02-22T19:47:58.668513+0000","last_fullsized":"2024-02-22T19:47:58.668513+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:56.165274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:47:56.165274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":518,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674401+0000","last_change":"2024-02-22T19:45:09.809874+0000","last_active":"2024-02-22T19:47:58.674401+0000","last_peered":"2024-02-22T19:47:58.674401+0000","last_clean":"2024-02-22T19:47:58.674401+0000","last_became_active":"2024-02-22T19:41:52.114084+0000","last_became_peered":"2024-02-22T19:41:52.114084+0000","last_unstale":"2024-02-22T19:47:58.674401+0000","last_undegraded":"2024-02-22T19:47:58.674401+0000","last_fullsized":"2024-02-22T19:47:58.674401+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:09.809806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.809806+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":472,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676653+0000","last_change":"2024-02-22T19:47:40.527176+0000","last_active":"2024-02-22T19:47:58.676653+0000","last_peered":"2024-02-22T19:47:58.676653+0000","last_clean":"2024-02-22T19:47:58.676653+0000","last_became_active":"2024-02-22T19:47:40.526670+0000","last_became_peered":"2024-02-22T19:47:40.526670+0000","last_unstale":"2024-02-22T19:47:58.676653+0000","last_undegraded":"2024-02-22T19:47:58.676653+0000","last_fullsized":"2024-02-22T19:47:58.676653+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.262153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.262153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":470,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674538+0000","last_change":"2024-02-22T19:44:13.694618+0000","last_active":"2024-02-22T19:47:58.674538+0000","last_peered":"2024-02-22T19:47:58.674538+0000","last_clean":"2024-02-22T19:47:58.674538+0000","last_became_active":"2024-02-22T19:41:52.114006+0000","last_became_peered":"2024-02-22T19:41:52.114006+0000","last_unstale":"2024-02-22T19:47:58.674538+0000","last_undegraded":"2024-02-22T19:47:58.674538+0000","last_fullsized":"2024-02-22T19:47:58.674538+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:13.694552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:13.694552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":492,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.668624+0000","last_change":"2024-02-22T19:47:40.843176+0000","last_active":"2024-02-22T19:47:58.668624+0000","last_peered":"2024-02-22T19:47:58.668624+0000","last_clean":"2024-02-22T19:47:58.668624+0000","last_became_active":"2024-02-22T19:47:40.532966+0000","last_became_peered":"2024-02-22T19:47:40.532966+0000","last_unstale":"2024-02-22T19:47:58.668624+0000","last_undegraded":"2024-02-22T19:47:58.668624+0000","last_fullsized":"2024-02-22T19:47:58.668624+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:46:09.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.018318+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":493,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676061+0000","last_change":"2024-02-22T19:47:40.535567+0000","last_active":"2024-02-22T19:47:58.676061+0000","last_peered":"2024-02-22T19:47:58.676061+0000","last_clean":"2024-02-22T19:47:58.676061+0000","last_became_active":"2024-02-22T19:47:40.535155+0000","last_became_peered":"2024-02-22T19:47:40.535155+0000","last_unstale":"2024-02-22T19:47:58.676061+0000","last_undegraded":"2024-02-22T19:47:58.676061+0000","last_fullsized":"2024-02-22T19:47:58.676061+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:12.135980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.135980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":275,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673837+0000","last_change":"2024-02-22T19:44:12.674961+0000","last_active":"2024-02-22T19:47:58.673837+0000","last_peered":"2024-02-22T19:47:58.673837+0000","last_clean":"2024-02-22T19:47:58.673837+0000","last_became_active":"2024-02-22T19:37:57.183053+0000","last_became_peered":"2024-02-22T19:37:57.183053+0000","last_unstale":"2024-02-22T19:47:58.673837+0000","last_undegraded":"2024-02-22T19:47:58.673837+0000","last_fullsized":"2024-02-22T19:47:58.673837+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:12.674856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:44:12.674856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":502,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672484+0000","last_change":"2024-02-22T19:47:57.137077+0000","last_active":"2024-02-22T19:47:58.672484+0000","last_peered":"2024-02-22T19:47:58.672484+0000","last_clean":"2024-02-22T19:47:58.672484+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:47:58.672484+0000","last_undegraded":"2024-02-22T19:47:58.672484+0000","last_fullsized":"2024-02-22T19:47:58.672484+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:57.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:57.136969+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":466,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672164+0000","last_change":"2024-02-22T19:47:54.743244+0000","last_active":"2024-02-22T19:47:58.672164+0000","last_peered":"2024-02-22T19:47:58.672164+0000","last_clean":"2024-02-22T19:47:58.672164+0000","last_became_active":"2024-02-22T19:41:52.111630+0000","last_became_peered":"2024-02-22T19:41:52.111630+0000","last_unstale":"2024-02-22T19:47:58.672164+0000","last_undegraded":"2024-02-22T19:47:58.672164+0000","last_fullsized":"2024-02-22T19:47:58.672164+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:54.743150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.743150+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":512,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:03.835497+0000","last_change":"2024-02-22T19:48:03.835497+0000","last_active":"2024-02-22T19:48:03.835497+0000","last_peered":"2024-02-22T19:48:03.835497+0000","last_clean":"2024-02-22T19:48:03.835497+0000","last_became_active":"2024-02-22T19:47:40.538414+0000","last_became_peered":"2024-02-22T19:47:40.538414+0000","last_unstale":"2024-02-22T19:48:03.835497+0000","last_undegraded":"2024-02-22T19:48:03.835497+0000","last_fullsized":"2024-02-22T19:48:03.835497+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:03.835423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:48:03.835423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":502,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.671856+0000","last_change":"2024-02-22T19:45:53.878055+0000","last_active":"2024-02-22T19:47:58.671856+0000","last_peered":"2024-02-22T19:47:58.671856+0000","last_clean":"2024-02-22T19:47:58.671856+0000","last_became_active":"2024-02-22T19:41:52.096289+0000","last_became_peered":"2024-02-22T19:41:52.096289+0000","last_unstale":"2024-02-22T19:47:58.671856+0000","last_undegraded":"2024-02-22T19:47:58.671856+0000","last_fullsized":"2024-02-22T19:47:58.671856+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:45:53.877957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:53.877957+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":491,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.668738+0000","last_change":"2024-02-22T19:44:52.796416+0000","last_active":"2024-02-22T19:47:58.668738+0000","last_peered":"2024-02-22T19:47:58.668738+0000","last_clean":"2024-02-22T19:47:58.668738+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:47:58.668738+0000","last_undegraded":"2024-02-22T19:47:58.668738+0000","last_fullsized":"2024-02-22T19:47:58.668738+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:52.796310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.796310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:48:00.092591+0000","last_change":"2024-02-22T19:48:00.092551+0000","last_active":"2024-02-22T19:48:00.092591+0000","last_peered":"2024-02-22T19:48:00.092591+0000","last_clean":"2024-02-22T19:48:00.092591+0000","last_became_active":"2024-02-22T19:47:40.530829+0000","last_became_peered":"2024-02-22T19:47:40.530829+0000","last_unstale":"2024-02-22T19:48:00.092591+0000","last_undegraded":"2024-02-22T19:48:00.092591+0000","last_fullsized":"2024-02-22T19:48:00.092591+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:00.092448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:00.092448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":30,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666311+0000","last_change":"2024-02-22T19:47:41.401565+0000","last_active":"2024-02-22T19:47:58.666311+0000","last_peered":"2024-02-22T19:47:58.666311+0000","last_clean":"2024-02-22T19:47:58.666311+0000","last_became_active":"2024-02-22T19:47:40.528979+0000","last_became_peered":"2024-02-22T19:47:40.528979+0000","last_unstale":"2024-02-22T19:47:58.666311+0000","last_undegraded":"2024-02-22T19:47:58.666311+0000","last_fullsized":"2024-02-22T19:47:58.666311+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:57.920791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:57.920791+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":292,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677265+0000","last_change":"2024-02-22T19:47:40.541670+0000","last_active":"2024-02-22T19:47:58.677265+0000","last_peered":"2024-02-22T19:47:58.677265+0000","last_clean":"2024-02-22T19:47:58.677265+0000","last_became_active":"2024-02-22T19:47:40.541359+0000","last_became_peered":"2024-02-22T19:47:40.541359+0000","last_unstale":"2024-02-22T19:47:58.677265+0000","last_undegraded":"2024-02-22T19:47:58.677265+0000","last_fullsized":"2024-02-22T19:47:58.677265+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:24.509271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.509271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":485,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.764542+0000","last_change":"2024-02-22T19:47:58.764502+0000","last_active":"2024-02-22T19:47:58.764542+0000","last_peered":"2024-02-22T19:47:58.764542+0000","last_clean":"2024-02-22T19:47:58.764542+0000","last_became_active":"2024-02-22T19:47:40.526305+0000","last_became_peered":"2024-02-22T19:47:40.526305+0000","last_unstale":"2024-02-22T19:47:58.764542+0000","last_undegraded":"2024-02-22T19:47:58.764542+0000","last_fullsized":"2024-02-22T19:47:58.764542+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:58.764403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:47:58.764403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":485,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672739+0000","last_change":"2024-02-22T19:46:10.060876+0000","last_active":"2024-02-22T19:47:58.672739+0000","last_peered":"2024-02-22T19:47:58.672739+0000","last_clean":"2024-02-22T19:47:58.672739+0000","last_became_active":"2024-02-22T19:29:03.446785+0000","last_became_peered":"2024-02-22T19:29:03.446785+0000","last_unstale":"2024-02-22T19:47:58.672739+0000","last_undegraded":"2024-02-22T19:47:58.672739+0000","last_fullsized":"2024-02-22T19:47:58.672739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:46:10.060783+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:46:10.060783+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666985+0000","last_change":"2024-02-22T19:47:40.532181+0000","last_active":"2024-02-22T19:47:58.666985+0000","last_peered":"2024-02-22T19:47:58.666985+0000","last_clean":"2024-02-22T19:47:58.666985+0000","last_became_active":"2024-02-22T19:47:40.531839+0000","last_became_peered":"2024-02-22T19:47:40.531839+0000","last_unstale":"2024-02-22T19:47:58.666985+0000","last_undegraded":"2024-02-22T19:47:58.666985+0000","last_fullsized":"2024-02-22T19:47:58.666985+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:08.073147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:47:08.073147+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":269,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676500+0000","last_change":"2024-02-22T19:46:17.328336+0000","last_active":"2024-02-22T19:47:58.676500+0000","last_peered":"2024-02-22T19:47:58.676500+0000","last_clean":"2024-02-22T19:47:58.676500+0000","last_became_active":"2024-02-22T19:39:22.245719+0000","last_became_peered":"2024-02-22T19:39:22.245719+0000","last_unstale":"2024-02-22T19:47:58.676500+0000","last_undegraded":"2024-02-22T19:47:58.676500+0000","last_fullsized":"2024-02-22T19:47:58.676500+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.328243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":299,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674636+0000","last_change":"2024-02-22T19:47:07.646871+0000","last_active":"2024-02-22T19:47:58.674636+0000","last_peered":"2024-02-22T19:47:58.674636+0000","last_clean":"2024-02-22T19:47:58.674636+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:47:58.674636+0000","last_undegraded":"2024-02-22T19:47:58.674636+0000","last_fullsized":"2024-02-22T19:47:58.674636+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:07.646805+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:47:07.646805+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":500,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.678202+0000","last_change":"2024-02-22T19:47:40.542335+0000","last_active":"2024-02-22T19:47:58.678202+0000","last_peered":"2024-02-22T19:47:58.678202+0000","last_clean":"2024-02-22T19:47:58.678202+0000","last_became_active":"2024-02-22T19:47:40.542055+0000","last_became_peered":"2024-02-22T19:47:40.542055+0000","last_unstale":"2024-02-22T19:47:58.678202+0000","last_undegraded":"2024-02-22T19:47:58.678202+0000","last_fullsized":"2024-02-22T19:47:58.678202+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:05.985178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.985178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":507,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676398+0000","last_change":"2024-02-22T19:47:40.539584+0000","last_active":"2024-02-22T19:47:58.676398+0000","last_peered":"2024-02-22T19:47:58.676398+0000","last_clean":"2024-02-22T19:47:58.676398+0000","last_became_active":"2024-02-22T19:47:40.539063+0000","last_became_peered":"2024-02-22T19:47:40.539063+0000","last_unstale":"2024-02-22T19:47:58.676398+0000","last_undegraded":"2024-02-22T19:47:58.676398+0000","last_fullsized":"2024-02-22T19:47:58.676398+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.272326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:30.619969+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.272326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":271,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673416+0000","last_change":"2024-02-22T19:44:47.735006+0000","last_active":"2024-02-22T19:47:58.673416+0000","last_peered":"2024-02-22T19:47:58.673416+0000","last_clean":"2024-02-22T19:47:58.673416+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:47:58.673416+0000","last_undegraded":"2024-02-22T19:47:58.673416+0000","last_fullsized":"2024-02-22T19:47:58.673416+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:44:47.734940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:44:47.734940+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":507,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676864+0000","last_change":"2024-02-22T19:47:40.519744+0000","last_active":"2024-02-22T19:47:58.676864+0000","last_peered":"2024-02-22T19:47:58.676864+0000","last_clean":"2024-02-22T19:47:58.676864+0000","last_became_active":"2024-02-22T19:47:40.519396+0000","last_became_peered":"2024-02-22T19:47:40.519396+0000","last_unstale":"2024-02-22T19:47:58.676864+0000","last_undegraded":"2024-02-22T19:47:58.676864+0000","last_fullsized":"2024-02-22T19:47:58.676864+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.314522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.314522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":447,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672054+0000","last_change":"2024-02-22T19:44:58.799472+0000","last_active":"2024-02-22T19:47:58.672054+0000","last_peered":"2024-02-22T19:47:58.672054+0000","last_clean":"2024-02-22T19:47:58.672054+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:47:58.672054+0000","last_undegraded":"2024-02-22T19:47:58.672054+0000","last_fullsized":"2024-02-22T19:47:58.672054+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:58.799397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:44:58.799397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":485,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674204+0000","last_change":"2024-02-22T19:45:01.840336+0000","last_active":"2024-02-22T19:47:58.674204+0000","last_peered":"2024-02-22T19:47:58.674204+0000","last_clean":"2024-02-22T19:47:58.674204+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:47:58.674204+0000","last_undegraded":"2024-02-22T19:47:58.674204+0000","last_fullsized":"2024-02-22T19:47:58.674204+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:45:01.840227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.840227+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"478'2259","reported_seq":2276,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.407066+0000","last_change":"2024-02-22T19:47:40.525749+0000","last_active":"2024-02-22T19:48:04.407066+0000","last_peered":"2024-02-22T19:48:04.407066+0000","last_clean":"2024-02-22T19:48:04.407066+0000","last_became_active":"2024-02-22T19:47:40.525436+0000","last_became_peered":"2024-02-22T19:47:40.525436+0000","last_unstale":"2024-02-22T19:48:04.407066+0000","last_undegraded":"2024-02-22T19:48:04.407066+0000","last_fullsized":"2024-02-22T19:48:04.407066+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2259,"ondisk_log_size":2259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":148045824,"num_objects":2259,"num_object_clones":0,"num_object_copies":6777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2259,"num_write_kb":144576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":316,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674579+0000","last_change":"2024-02-22T19:45:52.702554+0000","last_active":"2024-02-22T19:47:58.674579+0000","last_peered":"2024-02-22T19:47:58.674579+0000","last_clean":"2024-02-22T19:47:58.674579+0000","last_became_active":"2024-02-22T19:41:52.107184+0000","last_became_peered":"2024-02-22T19:41:52.107184+0000","last_unstale":"2024-02-22T19:47:58.674579+0000","last_undegraded":"2024-02-22T19:47:58.674579+0000","last_fullsized":"2024-02-22T19:47:58.674579+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.702473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":563,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.678466+0000","last_change":"2024-02-22T19:47:40.571808+0000","last_active":"2024-02-22T19:47:58.678466+0000","last_peered":"2024-02-22T19:47:58.678466+0000","last_clean":"2024-02-22T19:47:58.678466+0000","last_became_active":"2024-02-22T19:47:40.527578+0000","last_became_peered":"2024-02-22T19:47:40.527578+0000","last_unstale":"2024-02-22T19:47:58.678466+0000","last_undegraded":"2024-02-22T19:47:58.678466+0000","last_fullsized":"2024-02-22T19:47:58.678466+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.375370+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":364,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672055+0000","last_change":"2024-02-22T19:47:41.126913+0000","last_active":"2024-02-22T19:47:58.672055+0000","last_peered":"2024-02-22T19:47:58.672055+0000","last_clean":"2024-02-22T19:47:58.672055+0000","last_became_active":"2024-02-22T19:47:40.521680+0000","last_became_peered":"2024-02-22T19:47:40.521680+0000","last_unstale":"2024-02-22T19:47:58.672055+0000","last_undegraded":"2024-02-22T19:47:58.672055+0000","last_fullsized":"2024-02-22T19:47:58.672055+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:03.485065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:03.485065+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"473'18","reported_seq":775,"reported_epoch":478,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:48:01.702615+0000","last_change":"2024-02-22T19:47:41.562244+0000","last_active":"2024-02-22T19:48:01.702615+0000","last_peered":"2024-02-22T19:48:01.702615+0000","last_clean":"2024-02-22T19:47:39.417805+0000","last_became_active":"2024-02-22T19:47:41.560455+0000","last_became_peered":"2024-02-22T19:47:41.560455+0000","last_unstale":"2024-02-22T19:48:01.702615+0000","last_undegraded":"2024-02-22T19:48:01.702615+0000","last_fullsized":"2024-02-22T19:48:01.702615+0000","mapping_epoch":470,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'16","last_scrub_stamp":"2024-02-22T19:46:15.312651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.312651+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,1,3],"avail_no_missing":["7","1","3"],"object_location_counts":[{"shards":"1,3,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"478'2356","reported_seq":2373,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.783427+0000","last_change":"2024-02-22T19:47:40.527623+0000","last_active":"2024-02-22T19:48:04.783427+0000","last_peered":"2024-02-22T19:48:04.783427+0000","last_clean":"2024-02-22T19:48:04.783427+0000","last_became_active":"2024-02-22T19:47:40.526780+0000","last_became_peered":"2024-02-22T19:47:40.526780+0000","last_unstale":"2024-02-22T19:48:04.783427+0000","last_undegraded":"2024-02-22T19:48:04.783427+0000","last_fullsized":"2024-02-22T19:48:04.783427+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2356,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154402816,"num_objects":2356,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":150784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":441,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.675975+0000","last_change":"2024-02-22T19:44:57.962949+0000","last_active":"2024-02-22T19:47:58.675975+0000","last_peered":"2024-02-22T19:47:58.675975+0000","last_clean":"2024-02-22T19:47:58.675975+0000","last_became_active":"2024-02-22T19:29:07.101546+0000","last_became_peered":"2024-02-22T19:29:07.101546+0000","last_unstale":"2024-02-22T19:47:58.675975+0000","last_undegraded":"2024-02-22T19:47:58.675975+0000","last_fullsized":"2024-02-22T19:47:58.675975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:57.962855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:57.962855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":199,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677334+0000","last_change":"2024-02-22T19:47:40.527811+0000","last_active":"2024-02-22T19:47:58.677334+0000","last_peered":"2024-02-22T19:47:58.677334+0000","last_clean":"2024-02-22T19:47:58.677334+0000","last_became_active":"2024-02-22T19:47:40.527298+0000","last_became_peered":"2024-02-22T19:47:40.527298+0000","last_unstale":"2024-02-22T19:47:58.677334+0000","last_undegraded":"2024-02-22T19:47:58.677334+0000","last_fullsized":"2024-02-22T19:47:58.677334+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.069465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.069465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":30,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666280+0000","last_change":"2024-02-22T19:47:43.621896+0000","last_active":"2024-02-22T19:47:58.666280+0000","last_peered":"2024-02-22T19:47:58.666280+0000","last_clean":"2024-02-22T19:47:58.666280+0000","last_became_active":"2024-02-22T19:47:40.536339+0000","last_became_peered":"2024-02-22T19:47:40.536339+0000","last_unstale":"2024-02-22T19:47:58.666280+0000","last_undegraded":"2024-02-22T19:47:58.666280+0000","last_fullsized":"2024-02-22T19:47:58.666280+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:45:08.765921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:08.765921+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":506,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677135+0000","last_change":"2024-02-22T19:47:09.025632+0000","last_active":"2024-02-22T19:47:58.677135+0000","last_peered":"2024-02-22T19:47:58.677135+0000","last_clean":"2024-02-22T19:47:58.677135+0000","last_became_active":"2024-02-22T19:39:22.262297+0000","last_became_peered":"2024-02-22T19:39:22.262297+0000","last_unstale":"2024-02-22T19:47:58.677135+0000","last_undegraded":"2024-02-22T19:47:58.677135+0000","last_fullsized":"2024-02-22T19:47:58.677135+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:09.025536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.025536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"478'2306","reported_seq":2323,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.434725+0000","last_change":"2024-02-22T19:47:40.538749+0000","last_active":"2024-02-22T19:48:04.434725+0000","last_peered":"2024-02-22T19:48:04.434725+0000","last_clean":"2024-02-22T19:48:04.434725+0000","last_became_active":"2024-02-22T19:47:40.538446+0000","last_became_peered":"2024-02-22T19:47:40.538446+0000","last_unstale":"2024-02-22T19:48:04.434725+0000","last_undegraded":"2024-02-22T19:48:04.434725+0000","last_fullsized":"2024-02-22T19:48:04.434725+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2306,"ondisk_log_size":2306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":151126016,"num_objects":2306,"num_object_clones":0,"num_object_copies":6918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":147584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":452,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:02.833667+0000","last_change":"2024-02-22T19:48:02.833667+0000","last_active":"2024-02-22T19:48:02.833667+0000","last_peered":"2024-02-22T19:48:02.833667+0000","last_clean":"2024-02-22T19:48:02.833667+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:48:02.833667+0000","last_undegraded":"2024-02-22T19:48:02.833667+0000","last_fullsized":"2024-02-22T19:48:02.833667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:02.833563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.833563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":532,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677264+0000","last_change":"2024-02-22T19:47:06.968685+0000","last_active":"2024-02-22T19:47:58.677264+0000","last_peered":"2024-02-22T19:47:58.677264+0000","last_clean":"2024-02-22T19:47:58.677264+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:47:58.677264+0000","last_undegraded":"2024-02-22T19:47:58.677264+0000","last_fullsized":"2024-02-22T19:47:58.677264+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.968586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.968586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":485,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677175+0000","last_change":"2024-02-22T19:46:14.318040+0000","last_active":"2024-02-22T19:47:58.677175+0000","last_peered":"2024-02-22T19:47:58.677175+0000","last_clean":"2024-02-22T19:47:58.677175+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:47:58.677175+0000","last_undegraded":"2024-02-22T19:47:58.677175+0000","last_fullsized":"2024-02-22T19:47:58.677175+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.317978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.317978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":481,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.671540+0000","last_change":"2024-02-22T19:47:47.887966+0000","last_active":"2024-02-22T19:47:58.671540+0000","last_peered":"2024-02-22T19:47:58.671540+0000","last_clean":"2024-02-22T19:47:58.671540+0000","last_became_active":"2024-02-22T19:41:52.095757+0000","last_became_peered":"2024-02-22T19:41:52.095757+0000","last_unstale":"2024-02-22T19:47:58.671540+0000","last_undegraded":"2024-02-22T19:47:58.671540+0000","last_fullsized":"2024-02-22T19:47:58.671540+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:47:47.887865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.887865+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"478'2266","reported_seq":2283,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.416368+0000","last_change":"2024-02-22T19:47:40.526664+0000","last_active":"2024-02-22T19:48:04.416368+0000","last_peered":"2024-02-22T19:48:04.416368+0000","last_clean":"2024-02-22T19:48:04.416368+0000","last_became_active":"2024-02-22T19:47:40.526359+0000","last_became_peered":"2024-02-22T19:47:40.526359+0000","last_unstale":"2024-02-22T19:48:04.416368+0000","last_undegraded":"2024-02-22T19:48:04.416368+0000","last_fullsized":"2024-02-22T19:48:04.416368+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2266,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":148504576,"num_objects":2266,"num_object_clones":0,"num_object_copies":6798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":145024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":274,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676514+0000","last_change":"2024-02-22T19:47:55.877350+0000","last_active":"2024-02-22T19:47:58.676514+0000","last_peered":"2024-02-22T19:47:58.676514+0000","last_clean":"2024-02-22T19:47:58.676514+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:47:58.676514+0000","last_undegraded":"2024-02-22T19:47:58.676514+0000","last_fullsized":"2024-02-22T19:47:58.676514+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.877246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.877246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":520,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677708+0000","last_change":"2024-02-22T19:47:40.540971+0000","last_active":"2024-02-22T19:47:58.677708+0000","last_peered":"2024-02-22T19:47:58.677708+0000","last_clean":"2024-02-22T19:47:58.677708+0000","last_became_active":"2024-02-22T19:47:40.540572+0000","last_became_peered":"2024-02-22T19:47:40.540572+0000","last_unstale":"2024-02-22T19:47:58.677708+0000","last_undegraded":"2024-02-22T19:47:58.677708+0000","last_fullsized":"2024-02-22T19:47:58.677708+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:00.864208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.864208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":293,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673672+0000","last_change":"2024-02-22T19:47:03.617591+0000","last_active":"2024-02-22T19:47:58.673672+0000","last_peered":"2024-02-22T19:47:58.673672+0000","last_clean":"2024-02-22T19:47:58.673672+0000","last_became_active":"2024-02-22T19:41:52.097037+0000","last_became_peered":"2024-02-22T19:41:52.097037+0000","last_unstale":"2024-02-22T19:47:58.673672+0000","last_undegraded":"2024-02-22T19:47:58.673672+0000","last_fullsized":"2024-02-22T19:47:58.673672+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.617526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.617526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":310,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677409+0000","last_change":"2024-02-22T19:47:40.919236+0000","last_active":"2024-02-22T19:47:58.677409+0000","last_peered":"2024-02-22T19:47:58.677409+0000","last_clean":"2024-02-22T19:47:58.677409+0000","last_became_active":"2024-02-22T19:47:40.532341+0000","last_became_peered":"2024-02-22T19:47:40.532341+0000","last_unstale":"2024-02-22T19:47:58.677409+0000","last_undegraded":"2024-02-22T19:47:58.677409+0000","last_fullsized":"2024-02-22T19:47:58.677409+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:47:23.007849+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:47:23.007849+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"478'2289","reported_seq":2306,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.939264+0000","last_change":"2024-02-22T19:47:40.530828+0000","last_active":"2024-02-22T19:48:04.939264+0000","last_peered":"2024-02-22T19:48:04.939264+0000","last_clean":"2024-02-22T19:48:04.939264+0000","last_became_active":"2024-02-22T19:47:40.530414+0000","last_became_peered":"2024-02-22T19:47:40.530414+0000","last_unstale":"2024-02-22T19:48:04.939264+0000","last_undegraded":"2024-02-22T19:48:04.939264+0000","last_fullsized":"2024-02-22T19:48:04.939264+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2289,"ondisk_log_size":2289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":150011904,"num_objects":2289,"num_object_clones":0,"num_object_copies":6867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2289,"num_write_kb":146496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":327,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673192+0000","last_change":"2024-02-22T19:47:40.522595+0000","last_active":"2024-02-22T19:47:58.673192+0000","last_peered":"2024-02-22T19:47:58.673192+0000","last_clean":"2024-02-22T19:47:58.673192+0000","last_became_active":"2024-02-22T19:47:40.522250+0000","last_became_peered":"2024-02-22T19:47:40.522250+0000","last_unstale":"2024-02-22T19:47:58.673192+0000","last_undegraded":"2024-02-22T19:47:58.673192+0000","last_fullsized":"2024-02-22T19:47:58.673192+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.428898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.428898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":473,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673502+0000","last_change":"2024-02-22T19:47:40.545351+0000","last_active":"2024-02-22T19:47:58.673502+0000","last_peered":"2024-02-22T19:47:58.673502+0000","last_clean":"2024-02-22T19:47:58.673502+0000","last_became_active":"2024-02-22T19:47:40.544519+0000","last_became_peered":"2024-02-22T19:47:40.544519+0000","last_unstale":"2024-02-22T19:47:58.673502+0000","last_undegraded":"2024-02-22T19:47:58.673502+0000","last_fullsized":"2024-02-22T19:47:58.673502+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.030866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.030866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":289,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674498+0000","last_change":"2024-02-22T19:47:15.553391+0000","last_active":"2024-02-22T19:47:58.674498+0000","last_peered":"2024-02-22T19:47:58.674498+0000","last_clean":"2024-02-22T19:47:58.674498+0000","last_became_active":"2024-02-22T19:41:52.111611+0000","last_became_peered":"2024-02-22T19:41:52.111611+0000","last_unstale":"2024-02-22T19:47:58.674498+0000","last_undegraded":"2024-02-22T19:47:58.674498+0000","last_fullsized":"2024-02-22T19:47:58.674498+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.553324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.553324+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666650+0000","last_change":"2024-02-22T19:47:40.531223+0000","last_active":"2024-02-22T19:47:58.666650+0000","last_peered":"2024-02-22T19:47:58.666650+0000","last_clean":"2024-02-22T19:47:58.666650+0000","last_became_active":"2024-02-22T19:47:40.530109+0000","last_became_peered":"2024-02-22T19:47:40.530109+0000","last_unstale":"2024-02-22T19:47:58.666650+0000","last_undegraded":"2024-02-22T19:47:58.666650+0000","last_fullsized":"2024-02-22T19:47:58.666650+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.718312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.718312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"264'40","reported_seq":760,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:59.423257+0000","last_change":"2024-02-22T19:46:14.876056+0000","last_active":"2024-02-22T19:47:59.423257+0000","last_peered":"2024-02-22T19:47:59.423257+0000","last_clean":"2024-02-22T19:47:59.423257+0000","last_became_active":"2024-02-22T19:29:03.082962+0000","last_became_peered":"2024-02-22T19:29:03.082962+0000","last_unstale":"2024-02-22T19:47:59.423257+0000","last_undegraded":"2024-02-22T19:47:59.423257+0000","last_fullsized":"2024-02-22T19:47:59.423257+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:46:14.875950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.875950+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"473'24","reported_seq":417,"reported_epoch":477,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:47:59.422035+0000","last_change":"2024-02-22T19:47:41.562264+0000","last_active":"2024-02-22T19:47:59.422035+0000","last_peered":"2024-02-22T19:47:59.422035+0000","last_clean":"2024-02-22T19:47:39.417862+0000","last_became_active":"2024-02-22T19:47:41.560520+0000","last_became_peered":"2024-02-22T19:47:41.560520+0000","last_unstale":"2024-02-22T19:47:59.422035+0000","last_undegraded":"2024-02-22T19:47:59.422035+0000","last_fullsized":"2024-02-22T19:47:59.422035+0000","mapping_epoch":470,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:15.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:15.782332+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,7,3],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"1,3,7","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.8","version":"478'2335","reported_seq":2352,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.942035+0000","last_change":"2024-02-22T19:47:40.532408+0000","last_active":"2024-02-22T19:48:04.942035+0000","last_peered":"2024-02-22T19:48:04.942035+0000","last_clean":"2024-02-22T19:48:04.942035+0000","last_became_active":"2024-02-22T19:47:40.532111+0000","last_became_peered":"2024-02-22T19:47:40.532111+0000","last_unstale":"2024-02-22T19:48:04.942035+0000","last_undegraded":"2024-02-22T19:48:04.942035+0000","last_fullsized":"2024-02-22T19:48:04.942035+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2335,"ondisk_log_size":2335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":153026560,"num_objects":2335,"num_object_clones":0,"num_object_copies":7005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2335,"num_write_kb":149440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":520,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676612+0000","last_change":"2024-02-22T19:47:40.526995+0000","last_active":"2024-02-22T19:47:58.676612+0000","last_peered":"2024-02-22T19:47:58.676612+0000","last_clean":"2024-02-22T19:47:58.676612+0000","last_became_active":"2024-02-22T19:47:40.526255+0000","last_became_peered":"2024-02-22T19:47:40.526255+0000","last_unstale":"2024-02-22T19:47:58.676612+0000","last_undegraded":"2024-02-22T19:47:58.676612+0000","last_fullsized":"2024-02-22T19:47:58.676612+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.507020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.507020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":461,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677308+0000","last_change":"2024-02-22T19:47:15.447527+0000","last_active":"2024-02-22T19:47:58.677308+0000","last_peered":"2024-02-22T19:47:58.677308+0000","last_clean":"2024-02-22T19:47:58.677308+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:47:58.677308+0000","last_undegraded":"2024-02-22T19:47:58.677308+0000","last_fullsized":"2024-02-22T19:47:58.677308+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.447427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.447427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":469,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.671628+0000","last_change":"2024-02-22T19:46:58.646300+0000","last_active":"2024-02-22T19:47:58.671628+0000","last_peered":"2024-02-22T19:47:58.671628+0000","last_clean":"2024-02-22T19:47:58.671628+0000","last_became_active":"2024-02-22T19:41:52.096750+0000","last_became_peered":"2024-02-22T19:41:52.096750+0000","last_unstale":"2024-02-22T19:47:58.671628+0000","last_undegraded":"2024-02-22T19:47:58.671628+0000","last_fullsized":"2024-02-22T19:47:58.671628+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:58.646210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.646210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672081+0000","last_change":"2024-02-22T19:47:40.533498+0000","last_active":"2024-02-22T19:47:58.672081+0000","last_peered":"2024-02-22T19:47:58.672081+0000","last_clean":"2024-02-22T19:47:58.672081+0000","last_became_active":"2024-02-22T19:47:40.533130+0000","last_became_peered":"2024-02-22T19:47:40.533130+0000","last_unstale":"2024-02-22T19:47:58.672081+0000","last_undegraded":"2024-02-22T19:47:58.672081+0000","last_fullsized":"2024-02-22T19:47:58.672081+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.081880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.081880+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":562,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.679523+0000","last_change":"2024-02-22T19:46:58.874444+0000","last_active":"2024-02-22T19:47:58.679523+0000","last_peered":"2024-02-22T19:47:58.679523+0000","last_clean":"2024-02-22T19:47:58.679523+0000","last_became_active":"2024-02-22T19:39:22.259207+0000","last_became_peered":"2024-02-22T19:39:22.259207+0000","last_unstale":"2024-02-22T19:47:58.679523+0000","last_undegraded":"2024-02-22T19:47:58.679523+0000","last_fullsized":"2024-02-22T19:47:58.679523+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:46:58.874350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.874350+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"415'30","reported_seq":561,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:01.714615+0000","last_change":"2024-02-22T19:47:10.006289+0000","last_active":"2024-02-22T19:48:01.714615+0000","last_peered":"2024-02-22T19:48:01.714615+0000","last_clean":"2024-02-22T19:48:01.714615+0000","last_became_active":"2024-02-22T19:45:01.732679+0000","last_became_peered":"2024-02-22T19:45:01.732679+0000","last_unstale":"2024-02-22T19:48:01.714615+0000","last_undegraded":"2024-02-22T19:48:01.714615+0000","last_fullsized":"2024-02-22T19:48:01.714615+0000","mapping_epoch":217,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":415,"parent":"0.0","parent_split_bits":0,"last_scrub":"415'30","last_scrub_stamp":"2024-02-22T19:47:10.006180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.006180+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"478'2271","reported_seq":2288,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.944609+0000","last_change":"2024-02-22T19:47:40.535414+0000","last_active":"2024-02-22T19:48:04.944609+0000","last_peered":"2024-02-22T19:48:04.944609+0000","last_clean":"2024-02-22T19:48:04.944609+0000","last_became_active":"2024-02-22T19:47:40.535133+0000","last_became_peered":"2024-02-22T19:47:40.535133+0000","last_unstale":"2024-02-22T19:48:04.944609+0000","last_undegraded":"2024-02-22T19:48:04.944609+0000","last_fullsized":"2024-02-22T19:48:04.944609+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2271,"ondisk_log_size":2271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":148832256,"num_objects":2271,"num_object_clones":0,"num_object_copies":6813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2271,"num_write_kb":145344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":298,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674449+0000","last_change":"2024-02-22T19:47:40.532548+0000","last_active":"2024-02-22T19:47:58.674449+0000","last_peered":"2024-02-22T19:47:58.674449+0000","last_clean":"2024-02-22T19:47:58.674449+0000","last_became_active":"2024-02-22T19:47:40.531699+0000","last_became_peered":"2024-02-22T19:47:40.531699+0000","last_unstale":"2024-02-22T19:47:58.674449+0000","last_undegraded":"2024-02-22T19:47:58.674449+0000","last_fullsized":"2024-02-22T19:47:58.674449+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:44:17.656518+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":511,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676375+0000","last_change":"2024-02-22T19:47:26.529553+0000","last_active":"2024-02-22T19:47:58.676375+0000","last_peered":"2024-02-22T19:47:58.676375+0000","last_clean":"2024-02-22T19:47:58.676375+0000","last_became_active":"2024-02-22T19:41:52.103851+0000","last_became_peered":"2024-02-22T19:41:52.103851+0000","last_unstale":"2024-02-22T19:47:58.676375+0000","last_undegraded":"2024-02-22T19:47:58.676375+0000","last_fullsized":"2024-02-22T19:47:58.676375+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_clean_scrub_stamp":"2024-02-22T19:47:26.529483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":488,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673220+0000","last_change":"2024-02-22T19:47:40.534850+0000","last_active":"2024-02-22T19:47:58.673220+0000","last_peered":"2024-02-22T19:47:58.673220+0000","last_clean":"2024-02-22T19:47:58.673220+0000","last_became_active":"2024-02-22T19:47:40.534548+0000","last_became_peered":"2024-02-22T19:47:40.534548+0000","last_unstale":"2024-02-22T19:47:58.673220+0000","last_undegraded":"2024-02-22T19:47:58.673220+0000","last_fullsized":"2024-02-22T19:47:58.673220+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:25.123659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:47:25.123659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":34,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666832+0000","last_change":"2024-02-22T19:47:43.296238+0000","last_active":"2024-02-22T19:47:58.666832+0000","last_peered":"2024-02-22T19:47:58.666832+0000","last_clean":"2024-02-22T19:47:58.666832+0000","last_became_active":"2024-02-22T19:47:40.534650+0000","last_became_peered":"2024-02-22T19:47:40.534650+0000","last_unstale":"2024-02-22T19:47:58.666832+0000","last_undegraded":"2024-02-22T19:47:58.666832+0000","last_fullsized":"2024-02-22T19:47:58.666832+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":472,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:46:12.320306+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.320306+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"423'13","reported_seq":552,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:01.703011+0000","last_change":"2024-02-22T19:47:54.799210+0000","last_active":"2024-02-22T19:48:01.703011+0000","last_peered":"2024-02-22T19:48:01.703011+0000","last_clean":"2024-02-22T19:48:01.703011+0000","last_became_active":"2024-02-22T19:45:09.547565+0000","last_became_peered":"2024-02-22T19:45:09.547565+0000","last_unstale":"2024-02-22T19:48:01.703011+0000","last_undegraded":"2024-02-22T19:48:01.703011+0000","last_fullsized":"2024-02-22T19:48:01.703011+0000","mapping_epoch":217,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"423'13","last_scrub_stamp":"2024-02-22T19:47:54.799098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.799098+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"478'2282","reported_seq":2299,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.917114+0000","last_change":"2024-02-22T19:47:40.531644+0000","last_active":"2024-02-22T19:48:04.917114+0000","last_peered":"2024-02-22T19:48:04.917114+0000","last_clean":"2024-02-22T19:48:04.917114+0000","last_became_active":"2024-02-22T19:47:40.531132+0000","last_became_peered":"2024-02-22T19:47:40.531132+0000","last_unstale":"2024-02-22T19:48:04.917114+0000","last_undegraded":"2024-02-22T19:48:04.917114+0000","last_fullsized":"2024-02-22T19:48:04.917114+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2282,"ondisk_log_size":2282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":149553152,"num_objects":2282,"num_object_clones":0,"num_object_copies":6846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2282,"num_write_kb":146048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":374,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673364+0000","last_change":"2024-02-22T19:47:55.926732+0000","last_active":"2024-02-22T19:47:58.673364+0000","last_peered":"2024-02-22T19:47:58.673364+0000","last_clean":"2024-02-22T19:47:58.673364+0000","last_became_active":"2024-02-22T19:47:40.525803+0000","last_became_peered":"2024-02-22T19:47:40.525803+0000","last_unstale":"2024-02-22T19:47:58.673364+0000","last_undegraded":"2024-02-22T19:47:58.673364+0000","last_fullsized":"2024-02-22T19:47:58.673364+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.926641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.926641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":472,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673148+0000","last_change":"2024-02-22T19:46:17.792837+0000","last_active":"2024-02-22T19:47:58.673148+0000","last_peered":"2024-02-22T19:47:58.673148+0000","last_clean":"2024-02-22T19:47:58.673148+0000","last_became_active":"2024-02-22T19:37:57.179030+0000","last_became_peered":"2024-02-22T19:37:57.179030+0000","last_unstale":"2024-02-22T19:47:58.673148+0000","last_undegraded":"2024-02-22T19:47:58.673148+0000","last_fullsized":"2024-02-22T19:47:58.673148+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.792736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.792736+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666101+0000","last_change":"2024-02-22T19:47:40.536647+0000","last_active":"2024-02-22T19:47:58.666101+0000","last_peered":"2024-02-22T19:47:58.666101+0000","last_clean":"2024-02-22T19:47:58.666101+0000","last_became_active":"2024-02-22T19:47:40.535376+0000","last_became_peered":"2024-02-22T19:47:40.535376+0000","last_unstale":"2024-02-22T19:47:58.666101+0000","last_undegraded":"2024-02-22T19:47:58.666101+0000","last_fullsized":"2024-02-22T19:47:58.666101+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:16.015884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.015884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1641,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:03.209268+0000","last_change":"2024-02-22T19:47:19.438721+0000","last_active":"2024-02-22T19:48:03.209268+0000","last_peered":"2024-02-22T19:48:03.209268+0000","last_clean":"2024-02-22T19:48:03.209268+0000","last_became_active":"2024-02-22T19:39:22.240347+0000","last_became_peered":"2024-02-22T19:39:22.240347+0000","last_unstale":"2024-02-22T19:48:03.209268+0000","last_undegraded":"2024-02-22T19:48:03.209268+0000","last_fullsized":"2024-02-22T19:48:03.209268+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:47:19.438615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:19.438615+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1126,"num_read_kb":1121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":294,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.675429+0000","last_change":"2024-02-22T19:47:49.907254+0000","last_active":"2024-02-22T19:47:58.675429+0000","last_peered":"2024-02-22T19:47:58.675429+0000","last_clean":"2024-02-22T19:47:58.675429+0000","last_became_active":"2024-02-22T19:39:22.253509+0000","last_became_peered":"2024-02-22T19:39:22.253509+0000","last_unstale":"2024-02-22T19:47:58.675429+0000","last_undegraded":"2024-02-22T19:47:58.675429+0000","last_fullsized":"2024-02-22T19:47:58.675429+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:47:49.907139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:49.907139+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"473'28","reported_seq":418,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:01.700289+0000","last_change":"2024-02-22T19:47:43.495969+0000","last_active":"2024-02-22T19:48:01.700289+0000","last_peered":"2024-02-22T19:48:01.700289+0000","last_clean":"2024-02-22T19:48:01.700289+0000","last_became_active":"2024-02-22T19:47:43.495664+0000","last_became_peered":"2024-02-22T19:47:43.495664+0000","last_unstale":"2024-02-22T19:48:01.700289+0000","last_undegraded":"2024-02-22T19:48:01.700289+0000","last_fullsized":"2024-02-22T19:48:01.700289+0000","mapping_epoch":472,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'25","last_scrub_stamp":"2024-02-22T19:45:06.780605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:45:06.780605+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"478'2333","reported_seq":2350,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.271742+0000","last_change":"2024-02-22T19:47:40.526558+0000","last_active":"2024-02-22T19:48:04.271742+0000","last_peered":"2024-02-22T19:48:04.271742+0000","last_clean":"2024-02-22T19:48:04.271742+0000","last_became_active":"2024-02-22T19:47:40.526262+0000","last_became_peered":"2024-02-22T19:47:40.526262+0000","last_unstale":"2024-02-22T19:48:04.271742+0000","last_undegraded":"2024-02-22T19:48:04.271742+0000","last_fullsized":"2024-02-22T19:48:04.271742+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2333,"ondisk_log_size":2333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":152895488,"num_objects":2333,"num_object_clones":0,"num_object_copies":6999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":149312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":345,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673166+0000","last_change":"2024-02-22T19:47:40.545400+0000","last_active":"2024-02-22T19:47:58.673166+0000","last_peered":"2024-02-22T19:47:58.673166+0000","last_clean":"2024-02-22T19:47:58.673166+0000","last_became_active":"2024-02-22T19:47:40.544234+0000","last_became_peered":"2024-02-22T19:47:40.544234+0000","last_unstale":"2024-02-22T19:47:58.673166+0000","last_undegraded":"2024-02-22T19:47:58.673166+0000","last_fullsized":"2024-02-22T19:47:58.673166+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.024753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.024753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":463,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.671323+0000","last_change":"2024-02-22T19:47:53.891515+0000","last_active":"2024-02-22T19:47:58.671323+0000","last_peered":"2024-02-22T19:47:58.671323+0000","last_clean":"2024-02-22T19:47:58.671323+0000","last_became_active":"2024-02-22T19:41:52.094688+0000","last_became_peered":"2024-02-22T19:41:52.094688+0000","last_unstale":"2024-02-22T19:47:58.671323+0000","last_undegraded":"2024-02-22T19:47:58.671323+0000","last_fullsized":"2024-02-22T19:47:58.671323+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:53.891414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:47:53.891414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":478,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673434+0000","last_change":"2024-02-22T19:47:46.844912+0000","last_active":"2024-02-22T19:47:58.673434+0000","last_peered":"2024-02-22T19:47:58.673434+0000","last_clean":"2024-02-22T19:47:58.673434+0000","last_became_active":"2024-02-22T19:37:57.178664+0000","last_became_peered":"2024-02-22T19:37:57.178664+0000","last_unstale":"2024-02-22T19:47:58.673434+0000","last_undegraded":"2024-02-22T19:47:58.673434+0000","last_fullsized":"2024-02-22T19:47:58.673434+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:46.844819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:47:46.844819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"463'37","reported_seq":307,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673939+0000","last_change":"2024-02-22T19:47:24.474522+0000","last_active":"2024-02-22T19:47:58.673939+0000","last_peered":"2024-02-22T19:47:58.673939+0000","last_clean":"2024-02-22T19:47:58.673939+0000","last_became_active":"2024-02-22T19:39:22.247351+0000","last_became_peered":"2024-02-22T19:39:22.247351+0000","last_unstale":"2024-02-22T19:47:58.673939+0000","last_undegraded":"2024-02-22T19:47:58.673939+0000","last_fullsized":"2024-02-22T19:47:58.673939+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"463'37","last_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_deep_scrub":"463'37","last_deep_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.474452+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":461,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674295+0000","last_change":"2024-02-22T19:44:54.804568+0000","last_active":"2024-02-22T19:47:58.674295+0000","last_peered":"2024-02-22T19:47:58.674295+0000","last_clean":"2024-02-22T19:47:58.674295+0000","last_became_active":"2024-02-22T19:44:45.398192+0000","last_became_peered":"2024-02-22T19:44:45.398192+0000","last_unstale":"2024-02-22T19:47:58.674295+0000","last_undegraded":"2024-02-22T19:47:58.674295+0000","last_fullsized":"2024-02-22T19:47:58.674295+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":399,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:54.804469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:54.804469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"478'2253","reported_seq":2270,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.778379+0000","last_change":"2024-02-22T19:47:40.525763+0000","last_active":"2024-02-22T19:48:04.778379+0000","last_peered":"2024-02-22T19:48:04.778379+0000","last_clean":"2024-02-22T19:48:04.778379+0000","last_became_active":"2024-02-22T19:47:40.525470+0000","last_became_peered":"2024-02-22T19:47:40.525470+0000","last_unstale":"2024-02-22T19:48:04.778379+0000","last_undegraded":"2024-02-22T19:48:04.778379+0000","last_fullsized":"2024-02-22T19:48:04.778379+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2253,"ondisk_log_size":2253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":147652608,"num_objects":2253,"num_object_clones":0,"num_object_copies":6759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2253,"num_write_kb":144192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":188,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673728+0000","last_change":"2024-02-22T19:44:46.735448+0000","last_active":"2024-02-22T19:47:58.673728+0000","last_peered":"2024-02-22T19:47:58.673728+0000","last_clean":"2024-02-22T19:47:58.673728+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:47:58.673728+0000","last_undegraded":"2024-02-22T19:47:58.673728+0000","last_fullsized":"2024-02-22T19:47:58.673728+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:46.735345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:36.670303+0000","last_clean_scrub_stamp":"2024-02-22T19:44:46.735345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":274,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.068652+0000","last_change":"2024-02-22T19:48:04.068652+0000","last_active":"2024-02-22T19:48:04.068652+0000","last_peered":"2024-02-22T19:48:04.068652+0000","last_clean":"2024-02-22T19:48:04.068652+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:48:04.068652+0000","last_undegraded":"2024-02-22T19:48:04.068652+0000","last_fullsized":"2024-02-22T19:48:04.068652+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:04.068584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:48:04.068584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":280,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674552+0000","last_change":"2024-02-22T19:46:19.538051+0000","last_active":"2024-02-22T19:47:58.674552+0000","last_peered":"2024-02-22T19:47:58.674552+0000","last_clean":"2024-02-22T19:47:58.674552+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:47:58.674552+0000","last_undegraded":"2024-02-22T19:47:58.674552+0000","last_fullsized":"2024-02-22T19:47:58.674552+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:19.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:46:19.537988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":614,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.671624+0000","last_change":"2024-02-22T19:47:40.563981+0000","last_active":"2024-02-22T19:47:58.671624+0000","last_peered":"2024-02-22T19:47:58.671624+0000","last_clean":"2024-02-22T19:47:58.671624+0000","last_became_active":"2024-02-22T19:47:40.526691+0000","last_became_peered":"2024-02-22T19:47:40.526691+0000","last_unstale":"2024-02-22T19:47:58.671624+0000","last_undegraded":"2024-02-22T19:47:58.671624+0000","last_fullsized":"2024-02-22T19:47:58.671624+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:46:18.828884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.828884+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"395'16","reported_seq":579,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:01.699675+0000","last_change":"2024-02-22T19:46:58.513539+0000","last_active":"2024-02-22T19:48:01.699675+0000","last_peered":"2024-02-22T19:48:01.699675+0000","last_clean":"2024-02-22T19:48:01.699675+0000","last_became_active":"2024-02-22T19:44:41.682349+0000","last_became_peered":"2024-02-22T19:44:41.682349+0000","last_unstale":"2024-02-22T19:48:01.699675+0000","last_undegraded":"2024-02-22T19:48:01.699675+0000","last_fullsized":"2024-02-22T19:48:01.699675+0000","mapping_epoch":322,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'16","last_scrub_stamp":"2024-02-22T19:46:58.513473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.513473+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.e","version":"478'2180","reported_seq":2197,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.314040+0000","last_change":"2024-02-22T19:47:40.529634+0000","last_active":"2024-02-22T19:48:04.314040+0000","last_peered":"2024-02-22T19:48:04.314040+0000","last_clean":"2024-02-22T19:48:04.314040+0000","last_became_active":"2024-02-22T19:47:40.529239+0000","last_became_peered":"2024-02-22T19:47:40.529239+0000","last_unstale":"2024-02-22T19:48:04.314040+0000","last_undegraded":"2024-02-22T19:48:04.314040+0000","last_fullsized":"2024-02-22T19:48:04.314040+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2180,"ondisk_log_size":2180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":142868480,"num_objects":2180,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2180,"num_write_kb":139520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":365,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672525+0000","last_change":"2024-02-22T19:47:18.018282+0000","last_active":"2024-02-22T19:47:58.672525+0000","last_peered":"2024-02-22T19:47:58.672525+0000","last_clean":"2024-02-22T19:47:58.672525+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:47:58.672525+0000","last_undegraded":"2024-02-22T19:47:58.672525+0000","last_fullsized":"2024-02-22T19:47:58.672525+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:18.018184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:18.018184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":464,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.677060+0000","last_change":"2024-02-22T19:47:47.829499+0000","last_active":"2024-02-22T19:47:58.677060+0000","last_peered":"2024-02-22T19:47:58.677060+0000","last_clean":"2024-02-22T19:47:58.677060+0000","last_became_active":"2024-02-22T19:41:52.103577+0000","last_became_peered":"2024-02-22T19:41:52.103577+0000","last_unstale":"2024-02-22T19:47:58.677060+0000","last_undegraded":"2024-02-22T19:47:58.677060+0000","last_fullsized":"2024-02-22T19:47:58.677060+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:47.829395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.829395+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":292,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.675281+0000","last_change":"2024-02-22T19:47:54.924281+0000","last_active":"2024-02-22T19:47:58.675281+0000","last_peered":"2024-02-22T19:47:58.675281+0000","last_clean":"2024-02-22T19:47:58.675281+0000","last_became_active":"2024-02-22T19:47:40.525251+0000","last_became_peered":"2024-02-22T19:47:40.525251+0000","last_unstale":"2024-02-22T19:47:58.675281+0000","last_undegraded":"2024-02-22T19:47:58.675281+0000","last_fullsized":"2024-02-22T19:47:58.675281+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:54.924175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.924175+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":39,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666195+0000","last_change":"2024-02-22T19:47:51.225194+0000","last_active":"2024-02-22T19:47:58.666195+0000","last_peered":"2024-02-22T19:47:58.666195+0000","last_clean":"2024-02-22T19:47:58.666195+0000","last_became_active":"2024-02-22T19:47:40.521013+0000","last_became_peered":"2024-02-22T19:47:40.521013+0000","last_unstale":"2024-02-22T19:47:58.666195+0000","last_undegraded":"2024-02-22T19:47:58.666195+0000","last_fullsized":"2024-02-22T19:47:58.666195+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:47:51.225129+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:47:51.225129+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"473'16","reported_seq":767,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:01.702147+0000","last_change":"2024-02-22T19:47:43.496019+0000","last_active":"2024-02-22T19:48:01.702147+0000","last_peered":"2024-02-22T19:48:01.702147+0000","last_clean":"2024-02-22T19:48:01.702147+0000","last_became_active":"2024-02-22T19:47:43.495566+0000","last_became_peered":"2024-02-22T19:47:43.495566+0000","last_unstale":"2024-02-22T19:48:01.702147+0000","last_undegraded":"2024-02-22T19:48:01.702147+0000","last_fullsized":"2024-02-22T19:48:01.702147+0000","mapping_epoch":472,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:44:53.757594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.757594+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"478'2327","reported_seq":2344,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.412340+0000","last_change":"2024-02-22T19:47:40.530401+0000","last_active":"2024-02-22T19:48:04.412340+0000","last_peered":"2024-02-22T19:48:04.412340+0000","last_clean":"2024-02-22T19:48:04.412340+0000","last_became_active":"2024-02-22T19:47:40.530110+0000","last_became_peered":"2024-02-22T19:47:40.530110+0000","last_unstale":"2024-02-22T19:48:04.412340+0000","last_undegraded":"2024-02-22T19:48:04.412340+0000","last_fullsized":"2024-02-22T19:48:04.412340+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2327,"ondisk_log_size":2327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":152502272,"num_objects":2327,"num_object_clones":0,"num_object_copies":6981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2327,"num_write_kb":148928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666750+0000","last_change":"2024-02-22T19:47:40.532231+0000","last_active":"2024-02-22T19:47:58.666750+0000","last_peered":"2024-02-22T19:47:58.666750+0000","last_clean":"2024-02-22T19:47:58.666750+0000","last_became_active":"2024-02-22T19:47:40.531852+0000","last_became_peered":"2024-02-22T19:47:40.531852+0000","last_unstale":"2024-02-22T19:47:58.666750+0000","last_undegraded":"2024-02-22T19:47:58.666750+0000","last_fullsized":"2024-02-22T19:47:58.666750+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.070731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.070731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666637+0000","last_change":"2024-02-22T19:47:40.535183+0000","last_active":"2024-02-22T19:47:58.666637+0000","last_peered":"2024-02-22T19:47:58.666637+0000","last_clean":"2024-02-22T19:47:58.666637+0000","last_became_active":"2024-02-22T19:47:40.534711+0000","last_became_peered":"2024-02-22T19:47:40.534711+0000","last_unstale":"2024-02-22T19:47:58.666637+0000","last_undegraded":"2024-02-22T19:47:58.666637+0000","last_fullsized":"2024-02-22T19:47:58.666637+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.075867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.075867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":479,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.673922+0000","last_change":"2024-02-22T19:45:58.995918+0000","last_active":"2024-02-22T19:47:58.673922+0000","last_peered":"2024-02-22T19:47:58.673922+0000","last_clean":"2024-02-22T19:47:58.673922+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:47:58.673922+0000","last_undegraded":"2024-02-22T19:47:58.673922+0000","last_fullsized":"2024-02-22T19:47:58.673922+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:58.995851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:45:58.995851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":583,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.668786+0000","last_change":"2024-02-22T19:45:09.692752+0000","last_active":"2024-02-22T19:47:58.668786+0000","last_peered":"2024-02-22T19:47:58.668786+0000","last_clean":"2024-02-22T19:47:58.668786+0000","last_became_active":"2024-02-22T19:39:22.244716+0000","last_became_peered":"2024-02-22T19:39:22.244716+0000","last_unstale":"2024-02-22T19:47:58.668786+0000","last_undegraded":"2024-02-22T19:47:58.668786+0000","last_fullsized":"2024-02-22T19:47:58.668786+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:45:09.692661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.692661+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":23,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666389+0000","last_change":"2024-02-22T19:47:55.133259+0000","last_active":"2024-02-22T19:47:58.666389+0000","last_peered":"2024-02-22T19:47:58.666389+0000","last_clean":"2024-02-22T19:47:58.666389+0000","last_became_active":"2024-02-22T19:47:40.535824+0000","last_became_peered":"2024-02-22T19:47:40.535824+0000","last_unstale":"2024-02-22T19:47:58.666389+0000","last_undegraded":"2024-02-22T19:47:58.666389+0000","last_fullsized":"2024-02-22T19:47:58.666389+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.133168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.133168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"478'2281","reported_seq":2298,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.273877+0000","last_change":"2024-02-22T19:47:40.530634+0000","last_active":"2024-02-22T19:48:04.273877+0000","last_peered":"2024-02-22T19:48:04.273877+0000","last_clean":"2024-02-22T19:48:04.273877+0000","last_became_active":"2024-02-22T19:47:40.530437+0000","last_became_peered":"2024-02-22T19:47:40.530437+0000","last_unstale":"2024-02-22T19:48:04.273877+0000","last_undegraded":"2024-02-22T19:48:04.273877+0000","last_fullsized":"2024-02-22T19:48:04.273877+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2281,"ondisk_log_size":2281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":149487616,"num_objects":2281,"num_object_clones":0,"num_object_copies":6843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2281,"num_write_kb":145984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":301,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674709+0000","last_change":"2024-02-22T19:47:40.521785+0000","last_active":"2024-02-22T19:47:58.674709+0000","last_peered":"2024-02-22T19:47:58.674709+0000","last_clean":"2024-02-22T19:47:58.674709+0000","last_became_active":"2024-02-22T19:47:40.521465+0000","last_became_peered":"2024-02-22T19:47:40.521465+0000","last_unstale":"2024-02-22T19:47:58.674709+0000","last_undegraded":"2024-02-22T19:47:58.674709+0000","last_fullsized":"2024-02-22T19:47:58.674709+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.729907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.729907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":307,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674393+0000","last_change":"2024-02-22T19:46:17.513957+0000","last_active":"2024-02-22T19:47:58.674393+0000","last_peered":"2024-02-22T19:47:58.674393+0000","last_clean":"2024-02-22T19:47:58.674393+0000","last_became_active":"2024-02-22T19:39:22.269003+0000","last_became_peered":"2024-02-22T19:39:22.269003+0000","last_unstale":"2024-02-22T19:47:58.674393+0000","last_undegraded":"2024-02-22T19:47:58.674393+0000","last_fullsized":"2024-02-22T19:47:58.674393+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.513841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.513841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":457,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.668746+0000","last_change":"2024-02-22T19:47:49.124103+0000","last_active":"2024-02-22T19:47:58.668746+0000","last_peered":"2024-02-22T19:47:58.668746+0000","last_clean":"2024-02-22T19:47:58.668746+0000","last_became_active":"2024-02-22T19:29:01.012435+0000","last_became_peered":"2024-02-22T19:29:01.012435+0000","last_unstale":"2024-02-22T19:47:58.668746+0000","last_undegraded":"2024-02-22T19:47:58.668746+0000","last_fullsized":"2024-02-22T19:47:58.668746+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:49.123984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:49.123984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":221,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.674226+0000","last_change":"2024-02-22T19:47:40.581212+0000","last_active":"2024-02-22T19:47:58.674226+0000","last_peered":"2024-02-22T19:47:58.674226+0000","last_clean":"2024-02-22T19:47:58.674226+0000","last_became_active":"2024-02-22T19:47:40.532193+0000","last_became_peered":"2024-02-22T19:47:40.532193+0000","last_unstale":"2024-02-22T19:47:58.674226+0000","last_undegraded":"2024-02-22T19:47:58.674226+0000","last_fullsized":"2024-02-22T19:47:58.674226+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:15.453047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.453047+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"478'2345","reported_seq":2362,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.947809+0000","last_change":"2024-02-22T19:47:40.526886+0000","last_active":"2024-02-22T19:48:04.947809+0000","last_peered":"2024-02-22T19:48:04.947809+0000","last_clean":"2024-02-22T19:48:04.947809+0000","last_became_active":"2024-02-22T19:47:40.526501+0000","last_became_peered":"2024-02-22T19:47:40.526501+0000","last_unstale":"2024-02-22T19:48:04.947809+0000","last_undegraded":"2024-02-22T19:48:04.947809+0000","last_fullsized":"2024-02-22T19:48:04.947809+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2345,"ondisk_log_size":2345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":153681920,"num_objects":2345,"num_object_clones":0,"num_object_copies":7035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2345,"num_write_kb":150080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":468,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672661+0000","last_change":"2024-02-22T19:44:42.822533+0000","last_active":"2024-02-22T19:47:58.672661+0000","last_peered":"2024-02-22T19:47:58.672661+0000","last_clean":"2024-02-22T19:47:58.672661+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:47:58.672661+0000","last_undegraded":"2024-02-22T19:47:58.672661+0000","last_fullsized":"2024-02-22T19:47:58.672661+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:42.822463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:42.822463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666523+0000","last_change":"2024-02-22T19:47:40.531490+0000","last_active":"2024-02-22T19:47:58.666523+0000","last_peered":"2024-02-22T19:47:58.666523+0000","last_clean":"2024-02-22T19:47:58.666523+0000","last_became_active":"2024-02-22T19:47:40.531155+0000","last_became_peered":"2024-02-22T19:47:40.531155+0000","last_unstale":"2024-02-22T19:47:58.666523+0000","last_undegraded":"2024-02-22T19:47:58.666523+0000","last_fullsized":"2024-02-22T19:47:58.666523+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:20.790735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:20.790735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":30,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666479+0000","last_change":"2024-02-22T19:47:42.270936+0000","last_active":"2024-02-22T19:47:58.666479+0000","last_peered":"2024-02-22T19:47:58.666479+0000","last_clean":"2024-02-22T19:47:58.666479+0000","last_became_active":"2024-02-22T19:47:40.535245+0000","last_became_peered":"2024-02-22T19:47:40.535245+0000","last_unstale":"2024-02-22T19:47:58.666479+0000","last_undegraded":"2024-02-22T19:47:58.666479+0000","last_fullsized":"2024-02-22T19:47:58.666479+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:10.692928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.692928+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"478'2275","reported_seq":2292,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.790115+0000","last_change":"2024-02-22T19:47:40.532576+0000","last_active":"2024-02-22T19:48:04.790115+0000","last_peered":"2024-02-22T19:48:04.790115+0000","last_clean":"2024-02-22T19:48:04.790115+0000","last_became_active":"2024-02-22T19:47:40.532176+0000","last_became_peered":"2024-02-22T19:47:40.532176+0000","last_unstale":"2024-02-22T19:48:04.790115+0000","last_undegraded":"2024-02-22T19:48:04.790115+0000","last_fullsized":"2024-02-22T19:48:04.790115+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2275,"ondisk_log_size":2275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":149094400,"num_objects":2275,"num_object_clones":0,"num_object_copies":6825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2275,"num_write_kb":145600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":280,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.672153+0000","last_change":"2024-02-22T19:44:48.769697+0000","last_active":"2024-02-22T19:47:58.672153+0000","last_peered":"2024-02-22T19:47:58.672153+0000","last_clean":"2024-02-22T19:47:58.672153+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:47:58.672153+0000","last_undegraded":"2024-02-22T19:47:58.672153+0000","last_fullsized":"2024-02-22T19:47:58.672153+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.769602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.769602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":491,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.679399+0000","last_change":"2024-02-22T19:47:47.396521+0000","last_active":"2024-02-22T19:47:58.679399+0000","last_peered":"2024-02-22T19:47:58.679399+0000","last_clean":"2024-02-22T19:47:58.679399+0000","last_became_active":"2024-02-22T19:47:40.532868+0000","last_became_peered":"2024-02-22T19:47:40.532868+0000","last_unstale":"2024-02-22T19:47:58.679399+0000","last_undegraded":"2024-02-22T19:47:58.679399+0000","last_fullsized":"2024-02-22T19:47:58.679399+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:47.396426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.396426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666306+0000","last_change":"2024-02-22T19:47:40.534857+0000","last_active":"2024-02-22T19:47:58.666306+0000","last_peered":"2024-02-22T19:47:58.666306+0000","last_clean":"2024-02-22T19:47:58.666306+0000","last_became_active":"2024-02-22T19:47:40.534261+0000","last_became_peered":"2024-02-22T19:47:40.534261+0000","last_unstale":"2024-02-22T19:47:58.666306+0000","last_undegraded":"2024-02-22T19:47:58.666306+0000","last_fullsized":"2024-02-22T19:47:58.666306+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.714507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.714507+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":36,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666813+0000","last_change":"2024-02-22T19:47:42.007286+0000","last_active":"2024-02-22T19:47:58.666813+0000","last_peered":"2024-02-22T19:47:58.666813+0000","last_clean":"2024-02-22T19:47:58.666813+0000","last_became_active":"2024-02-22T19:47:40.532252+0000","last_became_peered":"2024-02-22T19:47:40.532252+0000","last_unstale":"2024-02-22T19:47:58.666813+0000","last_undegraded":"2024-02-22T19:47:58.666813+0000","last_fullsized":"2024-02-22T19:47:58.666813+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:47:26.042105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:26.042105+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":38,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"478'2195","reported_seq":2212,"reported_epoch":478,"state":"active+clean","last_fresh":"2024-02-22T19:48:04.428588+0000","last_change":"2024-02-22T19:47:40.527105+0000","last_active":"2024-02-22T19:48:04.428588+0000","last_peered":"2024-02-22T19:48:04.428588+0000","last_clean":"2024-02-22T19:48:04.428588+0000","last_became_active":"2024-02-22T19:47:40.525823+0000","last_became_peered":"2024-02-22T19:47:40.525823+0000","last_unstale":"2024-02-22T19:48:04.428588+0000","last_undegraded":"2024-02-22T19:48:04.428588+0000","last_fullsized":"2024-02-22T19:48:04.428588+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":2195,"ondisk_log_size":2195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":143851520,"num_objects":2195,"num_object_clones":0,"num_object_copies":6585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2195,"num_write_kb":140480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":475,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.676613+0000","last_change":"2024-02-22T19:47:48.777053+0000","last_active":"2024-02-22T19:47:58.676613+0000","last_peered":"2024-02-22T19:47:58.676613+0000","last_clean":"2024-02-22T19:47:58.676613+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:47:58.676613+0000","last_undegraded":"2024-02-22T19:47:58.676613+0000","last_fullsized":"2024-02-22T19:47:58.676613+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:48.776953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:48.776953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":22,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666536+0000","last_change":"2024-02-22T19:47:41.403911+0000","last_active":"2024-02-22T19:47:58.666536+0000","last_peered":"2024-02-22T19:47:58.666536+0000","last_clean":"2024-02-22T19:47:58.666536+0000","last_became_active":"2024-02-22T19:47:40.532545+0000","last_became_peered":"2024-02-22T19:47:40.532545+0000","last_unstale":"2024-02-22T19:47:58.666536+0000","last_undegraded":"2024-02-22T19:47:58.666536+0000","last_fullsized":"2024-02-22T19:47:58.666536+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:45:15.816835+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:30:55.019989+0000","last_clean_scrub_stamp":"2024-02-22T19:45:15.816835+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":526,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.675819+0000","last_change":"2024-02-22T19:47:40.667997+0000","last_active":"2024-02-22T19:47:58.675819+0000","last_peered":"2024-02-22T19:47:58.675819+0000","last_clean":"2024-02-22T19:47:58.675819+0000","last_became_active":"2024-02-22T19:47:40.532988+0000","last_became_peered":"2024-02-22T19:47:40.532988+0000","last_unstale":"2024-02-22T19:47:58.675819+0000","last_undegraded":"2024-02-22T19:47:58.675819+0000","last_fullsized":"2024-02-22T19:47:58.675819+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:46:12.047696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.047696+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4365,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":35,"reported_epoch":477,"state":"active+clean","last_fresh":"2024-02-22T19:47:58.666309+0000","last_change":"2024-02-22T19:47:53.131640+0000","last_active":"2024-02-22T19:47:58.666309+0000","last_peered":"2024-02-22T19:47:58.666309+0000","last_clean":"2024-02-22T19:47:58.666309+0000","last_became_active":"2024-02-22T19:47:40.534364+0000","last_became_peered":"2024-02-22T19:47:40.534364+0000","last_unstale":"2024-02-22T19:47:58.666309+0000","last_undegraded":"2024-02-22T19:47:58.666309+0000","last_fullsized":"2024-02-22T19:47:58.666309+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:53.131527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:53.131527+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":2395537408,"num_objects":36553,"num_object_clones":0,"num_object_copies":109659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36553,"num_write_kb":2339392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7153319936,"data_stored":7153319936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36553,"ondisk_log_size":36553,"up":48,"acting":48,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1126,"num_read_kb":1121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":6349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1101,"num_read_kb":858,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":10032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":728,"ondisk_log_size":728,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266920,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2014336,"kb_used_data":1125624,"kb_used_omap":0,"kb_used_meta":888704,"kb_avail":91730816,"statfs":{"total":95995035648,"available":93932355584,"internally_reserved":0,"allocated":1152638976,"data_stored":1150846868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":910032896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":1.1539999999999999,"15min":1.5740000000000001},"min":{"1min":0.23899999999999999,"5min":0.219,"15min":0.19900000000000001},"max":{"1min":2.6230000000000002,"5min":7.9790000000000001,"15min":42.597999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.997,"15min":1.5369999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.20699999999999999},"max":{"1min":2.113,"5min":5.1349999999999998,"15min":42.155999999999999},"last":1.081}]},{"osd":1,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.3260000000000001,"15min":1.891},"min":{"1min":0.218,"5min":0.218,"15min":0.20000000000000001},"max":{"1min":4.1689999999999996,"5min":6.1200000000000001,"15min":54.951000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.452,"15min":2.1739999999999999},"min":{"1min":0.28799999999999998,"5min":0.23100000000000001,"15min":0.214},"max":{"1min":6.2190000000000003,"5min":7.1449999999999996,"15min":57.012},"last":1.53}]},{"osd":2,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":4.335,"5min":2.29,"15min":1.954},"min":{"1min":0.52000000000000002,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":42.377000000000002,"5min":42.377000000000002,"15min":42.377000000000002},"last":0.625},{"interface":"front","average":{"1min":3.5139999999999998,"5min":2.1949999999999998,"15min":2.1789999999999998},"min":{"1min":0.35499999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":39.520000000000003,"5min":39.520000000000003,"15min":41.722000000000001},"last":1.0589999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":3.2829999999999999,"5min":1.9159999999999999,"15min":2.1869999999999998},"min":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":37.905000000000001,"5min":37.905000000000001,"15min":41.457000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":3.105,"5min":2.0539999999999998,"15min":2.2290000000000001},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":38.25,"5min":38.25,"15min":41.311999999999998},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.67600000000000005,"15min":0.74399999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.23999999999999999},"max":{"1min":2.101,"5min":4.4219999999999997,"15min":6.4720000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.26100000000000001},"max":{"1min":3.9119999999999999,"5min":5.0910000000000002,"15min":7.6219999999999999},"last":0.39800000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.81899999999999995,"15min":0.88200000000000001},"min":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.04,"5min":6.625,"15min":6.9039999999999999},"last":0.374},{"interface":"front","average":{"1min":0.629,"5min":0.76900000000000002,"15min":0.82499999999999996},"min":{"1min":0.29299999999999998,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":2.2400000000000002,"5min":7.3239999999999998,"15min":7.3239999999999998},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.73599999999999999,"15min":0.82899999999999996},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":2.6840000000000002,"5min":3.956,"15min":6.5},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.88200000000000001,"15min":0.92800000000000005},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":3.9430000000000001,"5min":6.6719999999999997,"15min":7.6390000000000002},"last":0.34399999999999997}]}]},{"osd":6,"up_from":0,"seq":193273528557,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575110,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2600224,"kb_used_data":1700632,"kb_used_omap":0,"kb_used_meta":899584,"kb_avail":91144928,"statfs":{"total":95995035648,"available":93332406272,"internally_reserved":0,"allocated":1741447168,"data_stored":1739623829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":921174016},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:10 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":2.4430000000000001,"15min":2.1869999999999998},"min":{"1min":0.20999999999999999,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":4.6769999999999996,"5min":51.091000000000001,"15min":51.091000000000001},"last":2.387},{"interface":"front","average":{"1min":1.8380000000000001,"5min":2.2999999999999998,"15min":2.2650000000000001},"min":{"1min":0.28499999999999998,"5min":0.215,"15min":0.17499999999999999},"max":{"1min":5.8630000000000004,"5min":51.289999999999999,"15min":51.289999999999999},"last":1.9550000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:47:27 2024","interfaces":[{"interface":"back","average":{"1min":2.661,"5min":3.1219999999999999,"15min":2.5099999999999998},"min":{"1min":0.27000000000000002,"5min":0.191,"15min":0.191},"max":{"1min":9.6300000000000008,"5min":51.027000000000001,"15min":51.027000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":2.2349999999999999,"5min":2.9929999999999999,"15min":2.6400000000000001},"min":{"1min":0.32100000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":7.173,"5min":51.369999999999997,"15min":51.369999999999997},"last":3.6680000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:48:02 2024","interfaces":[{"interface":"back","average":{"1min":5.1180000000000003,"5min":2.3340000000000001,"15min":2.4820000000000002},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.214},"max":{"1min":82.034000000000006,"5min":82.034000000000006,"15min":82.034000000000006},"last":0.40600000000000003},{"interface":"front","average":{"1min":5.2670000000000003,"5min":2.919,"15min":2.698},"min":{"1min":0.26800000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":81.765000000000001,"5min":81.765000000000001,"15min":81.765000000000001},"last":2.6480000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:47:15 2024","interfaces":[{"interface":"back","average":{"1min":1.7609999999999999,"5min":2.5390000000000001,"15min":2},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.24399999999999999},"max":{"1min":4.6379999999999999,"5min":51.158999999999999,"15min":51.158999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":1.597,"15min":1.518},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":2.4020000000000001,"5min":51.276000000000003,"15min":51.276000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:47:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":2.2730000000000001,"15min":2.028},"min":{"1min":0.312,"5min":0.312,"15min":0.28100000000000003},"max":{"1min":4.6079999999999997,"5min":51.183999999999997,"15min":51.183999999999997},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.976,"15min":1.833},"min":{"1min":0.38900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.4580000000000002,"5min":51.212000000000003,"15min":51.212000000000003},"last":2.3740000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.8560000000000001,"15min":1.6319999999999999},"min":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.28599999999999998},"max":{"1min":2.8130000000000002,"5min":51.079999999999998,"15min":51.079999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.859,"5min":2.8109999999999999,"15min":2.056},"min":{"1min":0.51100000000000001,"5min":0.44,"15min":0.34999999999999998},"max":{"1min":4.657,"5min":51.119999999999997,"15min":51.119999999999997},"last":0.628}]},{"osd":7,"last update":"Thu Feb 22 19:47:27 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":2.1840000000000002,"15min":2.0449999999999999},"min":{"1min":0.36699999999999999,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":3.5430000000000001,"5min":51.244999999999997,"15min":51.244999999999997},"last":2.3319999999999999},{"interface":"front","average":{"1min":0.872,"5min":1.944,"15min":1.647},"min":{"1min":0.38200000000000001,"5min":0.36599999999999999,"15min":0.30399999999999999},"max":{"1min":3.0830000000000002,"5min":51.317999999999998,"15min":51.317999999999998},"last":0.46200000000000002}]}]},{"osd":0,"up_from":0,"seq":34359738635,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411584,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2058048,"kb_used_data":1158520,"kb_used_omap":0,"kb_used_meta":899520,"kb_avail":91687104,"statfs":{"total":95995035648,"available":93887594496,"internally_reserved":0,"allocated":1186324480,"data_stored":1184531870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":921108480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:22 2024","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":1.6399999999999999,"15min":1.9850000000000001},"min":{"1min":0.222,"5min":0.217,"15min":0.18099999999999999},"max":{"1min":8.7409999999999997,"5min":8.7409999999999997,"15min":22.338999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":2.1579999999999999,"5min":2.1230000000000002,"15min":2.4390000000000001},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.17599999999999999},"max":{"1min":5.5030000000000001,"5min":14.603999999999999,"15min":30.427},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Feb 22 19:47:22 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":1.907,"15min":2.0750000000000002},"min":{"1min":0.34599999999999997,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":15.996,"5min":15.996,"15min":15.996},"last":2.2160000000000002},{"interface":"front","average":{"1min":3.8599999999999999,"5min":2.218,"15min":2.1040000000000001},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.193},"max":{"1min":11.986000000000001,"5min":11.986000000000001,"15min":13.782},"last":1.026}]},{"osd":3,"last update":"Thu Feb 22 19:47:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0979999999999999,"5min":2.0870000000000002,"15min":2.3220000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":8.9469999999999992,"5min":8.9469999999999992,"15min":30.071000000000002},"last":4.0190000000000001},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.8149999999999999,"15min":2.5230000000000001},"min":{"1min":0.29299999999999998,"5min":0.183,"15min":0.183},"max":{"1min":6.9850000000000003,"5min":9.3149999999999995,"15min":32.119},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4850000000000001,"15min":1.579},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.26400000000000001},"max":{"1min":4.0789999999999997,"5min":8.2289999999999992,"15min":12.43},"last":1.9330000000000001},{"interface":"front","average":{"1min":1.742,"5min":1.375,"15min":1.4670000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.253},"max":{"1min":5.5039999999999996,"5min":5.7999999999999998,"15min":10.34},"last":1.4870000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:47:33 2024","interfaces":[{"interface":"back","average":{"1min":2.206,"5min":1.754,"15min":1.78},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.28000000000000003},"max":{"1min":8.7929999999999993,"5min":14.045,"15min":40.633000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7,"15min":1.9399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.26600000000000001},"max":{"1min":5.6580000000000004,"5min":14.513999999999999,"15min":40.997999999999998},"last":1.958}]},{"osd":6,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.532,"15min":1.696},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.311},"max":{"1min":2.4649999999999999,"5min":8.2530000000000001,"15min":21.640000000000001},"last":1.992},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.583,"15min":1.552},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":5.5380000000000003,"5min":5.8869999999999996,"15min":10.369999999999999},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:47:22 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.7270000000000001,"15min":1.7989999999999999},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.29099999999999998},"max":{"1min":5.6820000000000004,"5min":14.199,"15min":14.199},"last":1.974},{"interface":"front","average":{"1min":2.8879999999999999,"5min":2.0379999999999998,"15min":1.7949999999999999},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.29899999999999999},"max":{"1min":8.8520000000000003,"5min":18.021999999999998,"15min":18.021999999999998},"last":0.38900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149946,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2031132,"kb_used_data":1143572,"kb_used_omap":0,"kb_used_meta":887552,"kb_avail":91714020,"statfs":{"total":95995035648,"available":93915156480,"internally_reserved":0,"allocated":1171017728,"data_stored":1169198268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":908853248},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.6299999999999999,"15min":2.1000000000000001},"min":{"1min":0.24399999999999999,"5min":0.187,"15min":0.187},"max":{"1min":6.944,"5min":9.3599999999999994,"15min":23.48},"last":1.488},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.7629999999999999,"15min":2.4079999999999999},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.20799999999999999},"max":{"1min":4.8499999999999996,"5min":10.297000000000001,"15min":23.411999999999999},"last":1.2789999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":2.8740000000000001,"5min":3.778,"15min":3.3559999999999999},"min":{"1min":0.28599999999999998,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":17.734000000000002,"5min":94.454999999999998,"15min":94.454999999999998},"last":9.1999999999999993},{"interface":"front","average":{"1min":2.706,"5min":3.5510000000000002,"15min":2.8700000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.214},"max":{"1min":17.696999999999999,"5min":95.197999999999993,"15min":95.197999999999993},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":2.036,"5min":2.3450000000000002,"15min":2.4510000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":17.736000000000001,"5min":17.736000000000001,"15min":20.105},"last":3.1419999999999999},{"interface":"front","average":{"1min":1.5680000000000001,"5min":2.4039999999999999,"15min":2.46},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":7.1760000000000002,"5min":17.986000000000001,"15min":35.939999999999998},"last":2.012}]},{"osd":4,"last update":"Thu Feb 22 19:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.5840000000000001,"15min":1.925},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":2.7669999999999999,"5min":9.968,"15min":13.680999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.613,"15min":1.8069999999999999},"min":{"1min":0.318,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":7.8719999999999999,"5min":7.8719999999999999,"15min":13.548},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.9379999999999999,"5min":1.681,"15min":1.7869999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.0019999999999998,"5min":9.3160000000000007,"15min":12.879},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.6739999999999999,"15min":1.96},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.3529999999999998,"5min":10.041,"15min":13.707000000000001},"last":1.3500000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:47:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":1.7889999999999999,"15min":1.804},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32300000000000001},"max":{"1min":7.6859999999999999,"5min":9.423,"15min":12.898999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.8560000000000001,"15min":2.0579999999999998},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.802,"5min":10.02,"15min":13.803000000000001},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:47:18 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.6459999999999999,"15min":1.8500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":6.8049999999999997,"5min":8.0879999999999992,"15min":13.483000000000001},"last":1.175},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.913,"15min":1.9219999999999999},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":6.9710000000000001,"5min":9.3930000000000007,"15min":12.846},"last":1.238}]}]},{"osd":4,"up_from":32,"seq":137438953720,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1310788,"kb_used_data":1005372,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":92434364,"statfs":{"total":95995035648,"available":94652788736,"internally_reserved":0,"allocated":1029500928,"data_stored":1027705210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:24 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0189999999999999,"15min":1.6040000000000001},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.20499999999999999},"max":{"1min":3.1419999999999999,"5min":8.6920000000000002,"15min":69.251999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.333,"5min":1.419,"15min":1.7170000000000001},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":4.9509999999999996,"5min":9.4900000000000002,"15min":69.373000000000005},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:47:24 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4319999999999999,"15min":1.5960000000000001},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.215},"max":{"1min":3.2069999999999999,"5min":11.037000000000001,"15min":27.234999999999999},"last":1.04},{"interface":"front","average":{"1min":1.339,"5min":1.1599999999999999,"15min":1.3340000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.24299999999999999},"max":{"1min":5.8170000000000002,"5min":5.8170000000000002,"15min":28.879000000000001},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:47:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.379,"15min":1.4950000000000001},"min":{"1min":0.34300000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":4.4889999999999999,"5min":7.0739999999999998,"15min":23.082000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.389,"15min":1.6040000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24399999999999999},"max":{"1min":4.859,"5min":6.3550000000000004,"15min":23.138999999999999},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Feb 22 19:47:24 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.611,"15min":1.766},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23100000000000001},"max":{"1min":4.71,"5min":15.715,"15min":25.547000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.821,"5min":1.609,"15min":1.796},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.25900000000000001},"max":{"1min":5.3929999999999998,"5min":11.082000000000001,"15min":25.623000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.919,"5min":5.4850000000000003,"15min":5.4850000000000003},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.74299999999999999,"15min":0.79600000000000004},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.249},"max":{"1min":2.1800000000000002,"5min":4.9800000000000004,"15min":13.211},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61299999999999999,"15min":0.74399999999999999},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.25900000000000001},"max":{"1min":2.0150000000000001,"5min":2.4910000000000001,"15min":7.2670000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58599999999999997,"15min":0.78900000000000003},"min":{"1min":0.33200000000000002,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":1.0660000000000001,"5min":2.1179999999999999,"15min":13.353},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.746,"15min":0.77500000000000002},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.27900000000000003},"max":{"1min":2.2570000000000001,"5min":5.391,"15min":11.111000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.65600000000000003,"15min":0.73199999999999998},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.307},"max":{"1min":1.23,"5min":2.6299999999999999,"15min":3.8839999999999999},"last":0.60799999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724786,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1593532,"kb_used_data":702644,"kb_used_omap":0,"kb_used_meta":890880,"kb_avail":92151620,"statfs":{"total":95995035648,"available":94363258880,"internally_reserved":0,"allocated":719507456,"data_stored":717719387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912261120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:47:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":1.077,"15min":1.425},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":2.585,"5min":8.4299999999999997,"15min":17.684000000000001},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.103,"15min":1.607},"min":{"1min":0.26300000000000001,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":3.105,"5min":5.2789999999999999,"15min":17.402999999999999},"last":1.536}]},{"osd":1,"last update":"Thu Feb 22 19:47:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.1359999999999999,"15min":1.4990000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.19},"max":{"1min":3.3170000000000002,"5min":4.9509999999999996,"15min":29.390999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.5680000000000001,"15min":1.746},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.218},"max":{"1min":4.9500000000000002,"5min":9.0999999999999996,"15min":29.367000000000001},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Feb 22 19:47:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":2.02,"15min":1.899},"min":{"1min":0.32300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.085,"5min":47.859000000000002,"15min":47.859000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":1.665,"15min":1.798},"min":{"1min":0.34899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.4769999999999999,"5min":45.183,"15min":45.183},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:47:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":2.1080000000000001,"15min":2.0219999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.254},"max":{"1min":2.7719999999999998,"5min":43.616999999999997,"15min":43.616999999999997},"last":4.4349999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.9570000000000001,"15min":1.907},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":5.0069999999999997,"5min":43.918999999999997,"15min":43.918999999999997},"last":0.435}]},{"osd":4,"last update":"Thu Feb 22 19:47:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.223},"max":{"1min":1.0700000000000001,"5min":7.5419999999999998,"15min":7.5419999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.84499999999999997,"15min":0.85199999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":1.823,"5min":5.577,"15min":10.407},"last":0.30399999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:47:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.63600000000000001,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.247},"max":{"1min":1.429,"5min":3.2599999999999998,"15min":5.1150000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":1.006,"15min":0.86799999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26400000000000001},"max":{"1min":1.125,"5min":10.436,"15min":10.436},"last":0.34200000000000003}]},{"osd":7,"last update":"Thu Feb 22 19:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.67500000000000004,"15min":0.76900000000000002},"min":{"1min":0.318,"5min":0.28999999999999998,"15min":0.28299999999999997},"max":{"1min":0.83299999999999996,"5min":3.8410000000000002,"15min":6.7119999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.95899999999999996,"15min":0.92700000000000005},"min":{"1min":0.35299999999999998,"5min":0.29599999999999999,"15min":0.27100000000000002},"max":{"1min":0.63400000000000001,"5min":7.5069999999999997,"15min":7.6379999999999999},"last":0.33100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1765081088,"data_stored":1765081088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1216544768,"data_stored":1216544768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1213136896,"data_stored":1213136896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1035337728,"data_stored":1035337728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":738590720,"data_stored":738590720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1184628736,"data_stored":1184628736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:48:12.358 INFO:tasks.thrashosds.thrasher:Setting 11.2 to [1, 5, 7] 2024-02-22T19:48:12.358 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '11.2', '1', '5', '7'] 2024-02-22T19:48:12.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pg-upmap 11.2 1 5 7 2024-02-22T19:48:12.364 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:12.415 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:12.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:12.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:12.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 26 16 47730 47714 114.647 82.75 0.0192103 0.00871694 2024-02-22T19:48:12.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:12.559 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:12.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:12.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:12.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:12.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:11.794447+0000 mon.a (mon.0) 1740 : cluster [DBG] osdmap e482: 8 total, 8 up, 6 in 2024-02-22T19:48:12.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:11.805341+0000 mgr.y (mgr.14535) 732 : cluster [DBG] pgmap v984: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.22k op/s; 3/132135 objects misplaced (0.002%) 2024-02-22T19:48:12.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.080177+0000 osd.2 (osd.2) 288 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:12.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.080239+0000 osd.2 (osd.2) 289 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:12.996 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.518854+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e483: 8 total, 8 up, 6 in 2024-02-22T19:48:12.996 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:11.794447+0000 mon.a (mon.0) 1740 : cluster [DBG] osdmap e482: 8 total, 8 up, 6 in 2024-02-22T19:48:12.996 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:11.805341+0000 mgr.y (mgr.14535) 732 : cluster [DBG] pgmap v984: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.22k op/s; 3/132135 objects misplaced (0.002%) 2024-02-22T19:48:12.996 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.080177+0000 osd.2 (osd.2) 288 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:12.996 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.080239+0000 osd.2 (osd.2) 289 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:12.996 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.518854+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e483: 8 total, 8 up, 6 in 2024-02-22T19:48:13.023 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:13.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:13.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:13.076 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:11.794447+0000 mon.a (mon.0) 1740 : cluster [DBG] osdmap e482: 8 total, 8 up, 6 in 2024-02-22T19:48:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:11.805341+0000 mgr.y (mgr.14535) 732 : cluster [DBG] pgmap v984: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.22k op/s; 3/132135 objects misplaced (0.002%) 2024-02-22T19:48:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.080177+0000 osd.2 (osd.2) 288 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.080239+0000 osd.2 (osd.2) 289 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.518854+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e483: 8 total, 8 up, 6 in 2024-02-22T19:48:13.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:13.157 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:13.188 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:13.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:13.194 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:13.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:13.334 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:13.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:13.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:13.439 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 27 16 48887 48871 113.079 72.3125 0.00959538 0.00883826 2024-02-22T19:48:13.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:13.522 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:13.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:13.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:13.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:13.676 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:13.704 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:13.708 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:13.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:13.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.092964+0000 osd.4 (osd.4) 183 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.096424+0000 osd.4 (osd.4) 184 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.177941+0000 osd.3 (osd.3) 335 : cluster [DBG] 6.b scrub starts 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.345705+0000 osd.3 (osd.3) 336 : cluster [DBG] 6.b scrub ok 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.754084+0000 osd.5 (osd.5) 196 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:12.759358+0000 osd.5 (osd.5) 197 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.089040+0000 osd.4 (osd.4) 185 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:48:13.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.092050+0000 osd.4 (osd.4) 186 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:48:13.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.531231+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e484: 8 total, 8 up, 6 in 2024-02-22T19:48:13.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.698179+0000 osd.7 (osd.7) 317 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:48:13.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.706792+0000 osd.7 (osd.7) 318 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.092964+0000 osd.4 (osd.4) 183 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.096424+0000 osd.4 (osd.4) 184 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.177941+0000 osd.3 (osd.3) 335 : cluster [DBG] 6.b scrub starts 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.345705+0000 osd.3 (osd.3) 336 : cluster [DBG] 6.b scrub ok 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.754084+0000 osd.5 (osd.5) 196 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:12.759358+0000 osd.5 (osd.5) 197 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.089040+0000 osd.4 (osd.4) 185 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:48:13.884 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.092050+0000 osd.4 (osd.4) 186 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:48:13.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.531231+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e484: 8 total, 8 up, 6 in 2024-02-22T19:48:13.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.698179+0000 osd.7 (osd.7) 317 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:48:13.885 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.706792+0000 osd.7 (osd.7) 318 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:48:13.885 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:13.885 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:48:13.885 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:48:13.885 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:48:13.885 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:48:13.885 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.092964+0000 osd.4 (osd.4) 183 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.096424+0000 osd.4 (osd.4) 184 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.177941+0000 osd.3 (osd.3) 335 : cluster [DBG] 6.b scrub starts 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.345705+0000 osd.3 (osd.3) 336 : cluster [DBG] 6.b scrub ok 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.754084+0000 osd.5 (osd.5) 196 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:12.759358+0000 osd.5 (osd.5) 197 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.089040+0000 osd.4 (osd.4) 185 : cluster [DBG] 6.1a scrub starts 2024-02-22T19:48:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.092050+0000 osd.4 (osd.4) 186 : cluster [DBG] 6.1a scrub ok 2024-02-22T19:48:14.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.531231+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e484: 8 total, 8 up, 6 in 2024-02-22T19:48:14.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.698179+0000 osd.7 (osd.7) 317 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:48:14.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.706792+0000 osd.7 (osd.7) 318 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:48:14.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:48:14.442 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 28 16 50090 50074 111.714 75.1875 0.0136735 0.00894581 2024-02-22T19:48:14.754 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a[93504]: debug 2024-02-22T19:48:14.424+0000 7fb44d3ea700 -1 RGWWatcher::handle_error cookie 94129597104000 err (110) Connection timed out 2024-02-22T19:48:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.191652+0000 osd.3 (osd.3) 337 : cluster [DBG] 1.7 deep-scrub starts 2024-02-22T19:48:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.199915+0000 osd.3 (osd.3) 338 : cluster [DBG] 1.7 deep-scrub ok 2024-02-22T19:48:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.805777+0000 mgr.y (mgr.14535) 733 : cluster [DBG] pgmap v987: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 55 MiB/s wr, 886 op/s; 3/132135 objects misplaced (0.002%) 2024-02-22T19:48:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.956036+0000 osd.1 (osd.1) 293 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:48:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:13.962113+0000 osd.1 (osd.1) 294 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:48:15.103 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:48:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:48:15.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.191652+0000 osd.3 (osd.3) 337 : cluster [DBG] 1.7 deep-scrub starts 2024-02-22T19:48:15.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.199915+0000 osd.3 (osd.3) 338 : cluster [DBG] 1.7 deep-scrub ok 2024-02-22T19:48:15.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.805777+0000 mgr.y (mgr.14535) 733 : cluster [DBG] pgmap v987: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 55 MiB/s wr, 886 op/s; 3/132135 objects misplaced (0.002%) 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.956036+0000 osd.1 (osd.1) 293 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:13.962113+0000 osd.1 (osd.1) 294 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.191652+0000 osd.3 (osd.3) 337 : cluster [DBG] 1.7 deep-scrub starts 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.199915+0000 osd.3 (osd.3) 338 : cluster [DBG] 1.7 deep-scrub ok 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.805777+0000 mgr.y (mgr.14535) 733 : cluster [DBG] pgmap v987: 138 pgs: 2 active+remapped+backfill_toofull, 136 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 55 MiB/s wr, 886 op/s; 3/132135 objects misplaced (0.002%) 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.956036+0000 osd.1 (osd.1) 293 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:48:15.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:13.962113+0000 osd.1 (osd.1) 294 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:48:15.442 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 29 15 51433 51418 110.758 84 0.00719065 0.00902351 2024-02-22T19:48:15.835 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:14.188388+0000 osd.3 (osd.3) 339 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:14.195973+0000 osd.3 (osd.3) 340 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:14.739375+0000 osd.7 (osd.7) 319 : cluster [DBG] 3.d scrub starts 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:14.743646+0000 osd.7 (osd.7) 320 : cluster [DBG] 3.d scrub ok 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:14.816862+0000 osd.5 (osd.5) 198 : cluster [DBG] 3.1c deep-scrub starts 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:14.828247+0000 osd.5 (osd.5) 199 : cluster [DBG] 3.1c deep-scrub ok 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.008242+0000 osd.6 (osd.6) 162 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.008287+0000 osd.6 (osd.6) 163 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.056885+0000 osd.4 (osd.4) 187 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.060994+0000 osd.4 (osd.4) 188 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:48:16.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:15.427464+0000 mgr.y (mgr.14535) 734 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:14.188388+0000 osd.3 (osd.3) 339 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:14.195973+0000 osd.3 (osd.3) 340 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:14.739375+0000 osd.7 (osd.7) 319 : cluster [DBG] 3.d scrub starts 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:14.743646+0000 osd.7 (osd.7) 320 : cluster [DBG] 3.d scrub ok 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:14.816862+0000 osd.5 (osd.5) 198 : cluster [DBG] 3.1c deep-scrub starts 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:14.828247+0000 osd.5 (osd.5) 199 : cluster [DBG] 3.1c deep-scrub ok 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.008242+0000 osd.6 (osd.6) 162 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.008287+0000 osd.6 (osd.6) 163 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.056885+0000 osd.4 (osd.4) 187 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.060994+0000 osd.4 (osd.4) 188 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:48:16.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:15.427464+0000 mgr.y (mgr.14535) 734 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:14.188388+0000 osd.3 (osd.3) 339 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:14.195973+0000 osd.3 (osd.3) 340 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:14.739375+0000 osd.7 (osd.7) 319 : cluster [DBG] 3.d scrub starts 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:14.743646+0000 osd.7 (osd.7) 320 : cluster [DBG] 3.d scrub ok 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:14.816862+0000 osd.5 (osd.5) 198 : cluster [DBG] 3.1c deep-scrub starts 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:14.828247+0000 osd.5 (osd.5) 199 : cluster [DBG] 3.1c deep-scrub ok 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.008242+0000 osd.6 (osd.6) 162 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.008287+0000 osd.6 (osd.6) 163 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.056885+0000 osd.4 (osd.4) 187 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.060994+0000 osd.4 (osd.4) 188 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:48:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:15.427464+0000 mgr.y (mgr.14535) 734 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:16.446 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 30 16 52797 52781 109.894 85.1875 0.0108635 0.00909473 2024-02-22T19:48:17.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.199338+0000 osd.3 (osd.3) 341 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:48:17.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.211407+0000 osd.3 (osd.3) 342 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:48:17.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.806567+0000 mgr.y (mgr.14535) 735 : cluster [DBG] pgmap v988: 138 pgs: 138 active+clean; 3.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 1020 B/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:17.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.819769+0000 mon.a (mon.0) 1743 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-02-22T19:48:17.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.819815+0000 mon.a (mon.0) 1744 : cluster [INF] Cluster is now healthy 2024-02-22T19:48:17.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:15.836998+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e485: 8 total, 8 up, 6 in 2024-02-22T19:48:17.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.199338+0000 osd.3 (osd.3) 341 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:48:17.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.211407+0000 osd.3 (osd.3) 342 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:48:17.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.806567+0000 mgr.y (mgr.14535) 735 : cluster [DBG] pgmap v988: 138 pgs: 138 active+clean; 3.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 1020 B/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:17.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.819769+0000 mon.a (mon.0) 1743 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-02-22T19:48:17.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.819815+0000 mon.a (mon.0) 1744 : cluster [INF] Cluster is now healthy 2024-02-22T19:48:17.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:15.836998+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e485: 8 total, 8 up, 6 in 2024-02-22T19:48:17.036 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.199338+0000 osd.3 (osd.3) 341 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:48:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.211407+0000 osd.3 (osd.3) 342 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:48:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.806567+0000 mgr.y (mgr.14535) 735 : cluster [DBG] pgmap v988: 138 pgs: 138 active+clean; 3.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 1020 B/s rd, 81 MiB/s wr, 1.30k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.819769+0000 mon.a (mon.0) 1743 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-02-22T19:48:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.819815+0000 mon.a (mon.0) 1744 : cluster [INF] Cluster is now healthy 2024-02-22T19:48:17.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:15.836998+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e485: 8 total, 8 up, 6 in 2024-02-22T19:48:17.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:48:17.446 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 31 16 54041 54025 108.857 77.75 0.0250961 0.00917958 2024-02-22T19:48:17.589 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:17.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:17.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:17.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:17.752 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:17.755 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:17.756 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:17.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:17.898 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:17.924 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:17.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:17.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:18.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:18.086 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:18.449 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 32 16 55173 55157 107.664 70.75 0.0134772 0.00928379 2024-02-22T19:48:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:48:18.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:16.896102+0000 osd.5 (osd.5) 200 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:48:18.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:16.904272+0000 osd.5 (osd.5) 201 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:48:18.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:17.040537+0000 osd.4 (osd.4) 189 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:48:18.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:17.043434+0000 osd.4 (osd.4) 190 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:48:18.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:17.533306+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e486: 8 total, 8 up, 6 in 2024-02-22T19:48:18.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:17.807505+0000 mgr.y (mgr.14535) 736 : cluster [DBG] pgmap v991: 138 pgs: 138 active+clean; 3.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 967 B/s rd, 77 MiB/s wr, 1.23k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:18.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:18.174462+0000 osd.2 (osd.2) 290 : cluster [DBG] 5.b scrub starts 2024-02-22T19:48:18.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:18.180377+0000 osd.2 (osd.2) 291 : cluster [DBG] 5.b scrub ok 2024-02-22T19:48:18.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:18.315692+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.53:0/3400228562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:48:18.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:18.316309+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:16.896102+0000 osd.5 (osd.5) 200 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:16.904272+0000 osd.5 (osd.5) 201 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:17.040537+0000 osd.4 (osd.4) 189 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:17.043434+0000 osd.4 (osd.4) 190 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:17.533306+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e486: 8 total, 8 up, 6 in 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:17.807505+0000 mgr.y (mgr.14535) 736 : cluster [DBG] pgmap v991: 138 pgs: 138 active+clean; 3.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 967 B/s rd, 77 MiB/s wr, 1.23k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:18.174462+0000 osd.2 (osd.2) 290 : cluster [DBG] 5.b scrub starts 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:18.180377+0000 osd.2 (osd.2) 291 : cluster [DBG] 5.b scrub ok 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:18.315692+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.53:0/3400228562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:48:19.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:18.316309+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:16.896102+0000 osd.5 (osd.5) 200 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:16.904272+0000 osd.5 (osd.5) 201 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:17.040537+0000 osd.4 (osd.4) 189 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:17.043434+0000 osd.4 (osd.4) 190 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:17.533306+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e486: 8 total, 8 up, 6 in 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:17.807505+0000 mgr.y (mgr.14535) 736 : cluster [DBG] pgmap v991: 138 pgs: 138 active+clean; 3.1 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 967 B/s rd, 77 MiB/s wr, 1.23k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:18.174462+0000 osd.2 (osd.2) 290 : cluster [DBG] 5.b scrub starts 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:18.180377+0000 osd.2 (osd.2) 291 : cluster [DBG] 5.b scrub ok 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:18.315692+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.53:0/3400228562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:48:19.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:18.316309+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:48:19.448 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 33 16 56313 56297 106.559 71.25 0.0112184 0.00937953 2024-02-22T19:48:19.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:18.007025+0000 osd.1 (osd.1) 295 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:48:19.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:18.011860+0000 osd.1 (osd.1) 296 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:48:19.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:18.538834+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:19.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:18.551704+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:48:19.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:18.551765+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e487: 8 total, 8 up, 6 in 2024-02-22T19:48:19.559 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:19] "GET /metrics HTTP/1.1" 200 176657 "" "Prometheus/2.33.4" 2024-02-22T19:48:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:18.007025+0000 osd.1 (osd.1) 295 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:48:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:18.011860+0000 osd.1 (osd.1) 296 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:48:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:18.538834+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:18.551704+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:48:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:18.551765+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e487: 8 total, 8 up, 6 in 2024-02-22T19:48:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:18.007025+0000 osd.1 (osd.1) 295 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:48:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:18.011860+0000 osd.1 (osd.1) 296 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:48:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:18.538834+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:18.551704+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:48:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:18.551765+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e487: 8 total, 8 up, 6 in 2024-02-22T19:48:20.377 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:20.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:20.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:20.448 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 34 16 57550 57534 105.699 77.3125 0.0134313 0.00945566 2024-02-22T19:48:20.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:20.549 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:20.555 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:20.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:20.652 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:19.808231+0000 mgr.y (mgr.14535) 737 : cluster [DBG] pgmap v993: 138 pgs: 138 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:20.652 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:19.808231+0000 mgr.y (mgr.14535) 737 : cluster [DBG] pgmap v993: 138 pgs: 138 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:20.656 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:20.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:20.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:20.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:20.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:19.808231+0000 mgr.y (mgr.14535) 737 : cluster [DBG] pgmap v993: 138 pgs: 138 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:48:20.892 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:21.448 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 35 16 58765 58749 104.848 75.9375 0.00860319 0.00953212 2024-02-22T19:48:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:21.987 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:21.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:22.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:22.092 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:22.122 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:22.145 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:22.148 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:22.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:22.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:22.310 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:22.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:22.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:22.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:22.448 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 36 16 59883 59867 103.877 69.875 0.0111457 0.00962084 2024-02-22T19:48:22.449 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:22.476 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:22.479 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:22.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:22.545 INFO:teuthology.orchestra.run.smithi053.stderr:set 11.2 pg_upmap mapping to [1,5,7] 2024-02-22T19:48:22.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:22.637 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:22.994 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:21.809406+0000 mgr.y (mgr.14535) 738 : cluster [DBG] pgmap v994: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:48:22.994 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:21.937483+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.53:0/2431889745' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "11.2", "id": [1, 5, 7]}]: dispatch 2024-02-22T19:48:22.994 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:22.533378+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? 172.21.15.53:0/2431889745' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "11.2", "id": [1, 5, 7]}]': finished 2024-02-22T19:48:22.994 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:22.533447+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e488: 8 total, 8 up, 6 in 2024-02-22T19:48:22.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:21.809406+0000 mgr.y (mgr.14535) 738 : cluster [DBG] pgmap v994: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:48:22.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:21.937483+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.53:0/2431889745' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "11.2", "id": [1, 5, 7]}]: dispatch 2024-02-22T19:48:22.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:22.533378+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? 172.21.15.53:0/2431889745' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "11.2", "id": [1, 5, 7]}]': finished 2024-02-22T19:48:22.995 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:22.533447+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e488: 8 total, 8 up, 6 in 2024-02-22T19:48:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:21.809406+0000 mgr.y (mgr.14535) 738 : cluster [DBG] pgmap v994: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:48:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:21.937483+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.53:0/2431889745' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "11.2", "id": [1, 5, 7]}]: dispatch 2024-02-22T19:48:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:22.533378+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? 172.21.15.53:0/2431889745' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "11.2", "id": [1, 5, 7]}]': finished 2024-02-22T19:48:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:22.533447+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e488: 8 total, 8 up, 6 in 2024-02-22T19:48:23.449 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 37 16 60257 60241 101.702 23.375 0.00306068 0.00962678 2024-02-22T19:48:23.905 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:23.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:23.909 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:23.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:48:24.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:24.039 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:24.448 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 38 16 60257 60241 99.0273 0 - 0.00962678 2024-02-22T19:48:24.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:23.202484+0000 osd.1 (osd.1) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:23.202543+0000 osd.1 (osd.1) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:23.388449+0000 mon.a (mon.0) 1754 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:23.533710+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e489: 8 total, 8 up, 6 in 2024-02-22T19:48:24.661 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:24.219344+0000 mon.a (mon.0) 1756 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:23.202484+0000 osd.1 (osd.1) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:23.202543+0000 osd.1 (osd.1) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:23.388449+0000 mon.a (mon.0) 1754 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:23.533710+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e489: 8 total, 8 up, 6 in 2024-02-22T19:48:24.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:24.219344+0000 mon.a (mon.0) 1756 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:24.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:23.202484+0000 osd.1 (osd.1) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:23.202543+0000 osd.1 (osd.1) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:23.388449+0000 mon.a (mon.0) 1754 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:23.533710+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e489: 8 total, 8 up, 6 in 2024-02-22T19:48:24.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:24.219344+0000 mon.a (mon.0) 1756 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:48:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:48:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:25.411 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:25.411 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:25.451 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 39 16 60759 60743 97.2866 15.6875 0.0108061 0.0102744 2024-02-22T19:48:25.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:25.576 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:25.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:25.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:25.646 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:23.809748+0000 mgr.y (mgr.14535) 739 : cluster [DBG] pgmap v997: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:48:25.646 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:24.535783+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e490: 8 total, 8 up, 6 in 2024-02-22T19:48:25.646 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:23.809748+0000 mgr.y (mgr.14535) 739 : cluster [DBG] pgmap v997: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:48:25.646 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:24.535783+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e490: 8 total, 8 up, 6 in 2024-02-22T19:48:25.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:25.760 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:25.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:48:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:48:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:23.809748+0000 mgr.y (mgr.14535) 739 : cluster [DBG] pgmap v997: 138 pgs: 138 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-02-22T19:48:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:24.535783+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e490: 8 total, 8 up, 6 in 2024-02-22T19:48:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:25.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:25.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:26.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:26.077 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:26.106 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:26.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:26.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:26.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:26.246 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:26.269 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:26.274 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:26.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:26.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:26.442 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:26.446 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:26.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:26.456 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:48:26.456641+0000 min lat: 0.00115108 max lat: 2.39342 avg lat: 0.0103721 2024-02-22T19:48:26.456 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:48:26.456 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 40 16 61723 61707 96.3501 60.25 0.013122 0.0103721 2024-02-22T19:48:26.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:26.581 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209439 8.e 8:72194cde:::benchmark_data_smithi053_130774_object97705:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233144+0000", 2024-02-22T19:48:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.926303588, 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.154478621, 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209439 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233144+0000", 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233144+0000", 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233142+0000", 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2409999999999998e-06 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233149+0000", 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0718e-05 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233160+0000", 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6279999999999996e-06 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233168+0000", 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000542933 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233711+0000", 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6080000000000002e-06 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233720+0000", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.125119301 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.358839+0000", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017001000000000001 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359009+0000", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028542798000000001 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387552+0000", 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0285999999999998e-05 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387623+0000", 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.154478621 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209440 8.e 8:79838a5f:::benchmark_data_smithi053_130774_object97706:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233180+0000", 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.92626752299998, 2024-02-22T19:48:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1543472880000001, 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209440 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233180+0000", 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233180+0000", 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233179+0000", 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.641e-06 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233183+0000", 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233189+0000", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4549999999999997e-06 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233193+0000", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055031999999999998 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233744+0000", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.875e-06 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233752+0000", 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1252841280000001 2024-02-22T19:48:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359036+0000", 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014295099999999999 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for missing object", 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359179+0000", 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028263030000000001 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387442+0000", 2024-02-22T19:48:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5550999999999995e-05 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387527+0000", 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1543472880000001 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209447 8.e 8:7a1e666f:::benchmark_data_smithi053_130774_object97713:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.92624954500002, 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1516851020000001, 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209447 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233198+0000", 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233200+0000", 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0690000000000001e-06 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233205+0000", 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1099999999999996e-06 2024-02-22T19:48:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233209+0000", 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056241600000000004 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233772+0000", 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3029999999999996e-06 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233778+0000", 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1254261780000001 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359204+0000", 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013111000000000001 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359335+0000", 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000186489 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.359522+0000", 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0076211359999999997 2024-02-22T19:48:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367143+0000", 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070781389999999998 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.374221+0000", 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5577999999999997e-05 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.374257+0000", 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010431807 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384689+0000", 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1398999999999999e-05 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384750+0000", 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133077 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384883+0000", 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1516851020000001 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209448 8.e 8:7f230ff2:::benchmark_data_smithi053_130774_object97714:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233214+0000", 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.92623393300005, 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1523798379999999, 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209448 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233214+0000", 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233214+0000", 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233213+0000", 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6010000000000002e-06 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233216+0000", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1209999999999998e-06 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233221+0000", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9179999999999996e-06 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233225+0000", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057156999999999998 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233796+0000", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9750000000000004e-06 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233802+0000", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.128199044 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.362001+0000", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019163900000000001 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.362193+0000", 2024-02-22T19:48:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000404282 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.362597+0000", 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011586292 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.374184+0000", 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042569410000000002 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378441+0000", 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0064523219999999999 2024-02-22T19:48:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384893+0000", 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049842899999999999 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385391+0000", 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6817999999999997e-05 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385448+0000", 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000145289 2024-02-22T19:48:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385593+0000", 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1523798379999999 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209452 8.e 8:755e269c:::benchmark_data_smithi053_130774_object97718:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233229+0000", 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.92621873400003, 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153409022, 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209452 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233229+0000", 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233229+0000", 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233228+0000", 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.593e-06 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233231+0000", 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0420000000000002e-06 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233236+0000", 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8999999999999999e-06 2024-02-22T19:48:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233240+0000", 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058253799999999996 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233822+0000", 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7869999999999998e-06 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233828+0000", 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1309483170000001 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.364777+0000", 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110528 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.364887+0000", 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000164594 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.365052+0000", 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013431972 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378484+0000", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044846099999999999 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378932+0000", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005964886 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384897+0000", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001559932 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386457+0000", 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5113e-05 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386502+0000", 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386638+0000", 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153409022 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209454 8.2 8:480e384d:::benchmark_data_smithi053_130774_object97720:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233244+0000", 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.92620372299996, 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1015162540000001, 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209454 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233244+0000", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233244+0000", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233243+0000", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233246+0000", 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1050000000000003e-06 2024-02-22T19:48:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233251+0000", 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2749999999999997e-06 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233255+0000", 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044480709999999996 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.237704+0000", 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5820000000000008e-06 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.237713+0000", 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0658666889999999 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.303580+0000", 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031736720000000002 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.306753+0000", 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033246799999999999 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,4,5", 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.307086+0000", 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027269775 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334356+0000", 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7673999999999999e-05 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334393+0000", 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0490000000000003e-06 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334398+0000", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012682700000000001 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334525+0000", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6285999999999993e-05 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334612+0000", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014847300000000001 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.334760+0000", 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1015162540000001 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209465 8.e 8:7bc93309:::benchmark_data_smithi053_130774_object97731:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e261)", 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.92618255499997, 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153583528, 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209465 2024-02-22T19:48:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233265+0000", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5919999999999999e-06 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233267+0000", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233272+0000", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6940000000000001e-06 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233276+0000", 2024-02-22T19:48:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057307899999999999 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233849+0000", 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0809999999999999e-06 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.233855+0000", 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333234169999999 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367179+0000", 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000128818 2024-02-22T19:48:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367307+0000", 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000179359 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.367487+0000", 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011036155000000001 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378523+0000", 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017368010000000001 2024-02-22T19:48:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.380260+0000", 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046411279999999996 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384901+0000", 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776489 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386677+0000", 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4347000000000001e-05 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386722+0000", 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012679 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386848+0000", 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.153583528 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209470 8.2 8:4aca6c6d:::benchmark_data_smithi053_130774_object97736:head [delete] snapc 0=[] ondisk+write+known_if_redirected e261)", 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.315545+0000", 2024-02-22T19:48:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.84390246199996, 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0310337919999999, 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209470 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315545+0000", 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315545+0000", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315543+0000", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2250000000000003e-06 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315551+0000", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3710000000000006e-06 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315560+0000", 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.127e-06 2024-02-22T19:48:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315566+0000", 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1929999999999999e-05 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315618+0000", 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315624+0000", 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015601757 2024-02-22T19:48:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.331225+0000", 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125742 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.331351+0000", 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015536599999999999 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3,4,5", 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.331507+0000", 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003654034 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.335161+0000", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044777899999999998 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.335608+0000", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016139 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.335770+0000", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010615696000000001 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.346386+0000", 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9052999999999999e-05 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.346445+0000", 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013429100000000001 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.346579+0000", 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0310337919999999 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209471 8.e 8:7312cf00:::benchmark_data_smithi053_130774_object97737:head [delete] snapc 0=[] ondisk+write+known_if_redirected e261)", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 558.84387309600004, 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0714779350000001, 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209471 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:48:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315574+0000", 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2849999999999999e-06 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315577+0000", 2024-02-22T19:48:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7289999999999997e-06 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315583+0000", 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.216e-06 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315587+0000", 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9706000000000001e-05 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315647+0000", 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8909999999999997e-06 2024-02-22T19:48:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.315651+0000", 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0537560939999999 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.369407+0000", 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014177699999999999 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.369549+0000", 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000205603 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.369754+0000", 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091389360000000003 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.378893+0000", 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060116980000000002 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.384905+0000", 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107773 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.385013+0000", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001870015 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386883+0000", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3253000000000001e-05 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.386926+0000", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126515 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.387052+0000", 2024-02-22T19:48:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0714779350000001 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212901 8.e 8:7ce38e7a:::benchmark_data_smithi053_130774_object101167:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.238047+0000", 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 552.92140092700004, 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0423131109999999, 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212901 2024-02-22T19:48:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238047+0000", 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238047+0000", 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238045+0000", 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6779999999999995e-06 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238054+0000", 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.715e-05 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238072+0000", 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238079+0000", 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053799999999999996 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238617+0000", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014364600000000001 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238760+0000", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000162418 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238923+0000", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061704639999999996 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245093+0000", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027255399999999999 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245366+0000", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040326799999999998 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245769+0000", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091892870000000008 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254958+0000", 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3540000000000006e-06 2024-02-22T19:48:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254968+0000", 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0253166010000001 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280284+0000", 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5383000000000007e-05 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280360+0000", 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0423131109999999 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212902 8.e 8:7e8b2c88:::benchmark_data_smithi053_130774_object101168:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.238106+0000", 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 552.92134162100001, 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0426704659999999, 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212902 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238106+0000", 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238106+0000", 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238105+0000", 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5990000000000002e-06 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238109+0000", 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3079999999999998e-06 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238114+0000", 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0799999999999999e-06 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.238118+0000", 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033961669999999998 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.241514+0000", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117797 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.241632+0000", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142676 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,4,7", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.241775+0000", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033266820000000001 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.245102+0000", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090548199999999999 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.246007+0000", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049880870000000004 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.250995+0000", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039965390000000003 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254992+0000", 2024-02-22T19:48:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6800000000000004e-06 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.254998+0000", 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0257239949999999 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280722+0000", 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3993999999999998e-05 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280776+0000", 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0426704659999999 2024-02-22T19:48:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212933 8.e 8:7a346efd:::benchmark_data_smithi053_130774_object101199:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.308899+0000", 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 552.85054858800004, 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98675647300000002, 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212933 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308899+0000", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308899+0000", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308898+0000", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0759999999999996e-06 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308902+0000", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8299999999999998e-06 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308909+0000", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2849999999999996e-06 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.308913+0000", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000246085 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.309159+0000", 2024-02-22T19:48:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0141e-05 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.309170+0000", 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.97163018700000003 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280800+0000", 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5624999999999993e-05 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.280895+0000", 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013860200000000001 2024-02-22T19:48:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.281034+0000", 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0076832949999999997 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288717+0000", 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6510000000000001e-05 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288754+0000", 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0068142130000000004 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295568+0000", 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6344000000000002e-05 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295614+0000", 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1001999999999998e-05 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295655+0000", 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98675647300000002 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212962 8.e 8:7d2dab33:::benchmark_data_smithi053_130774_object101228:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.351104+0000", 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 552.80834356599996, 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95283383899999996, 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212962 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351104+0000", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351104+0000", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351102+0000", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4099999999999997e-06 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351111+0000", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8179e-05 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351129+0000", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1519999999999992e-06 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351138+0000", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016201900000000001 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351300+0000", 2024-02-22T19:48:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7909999999999999e-06 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.351309+0000", 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.93233665300000002 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.283646+0000", 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8783999999999995e-05 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.283745+0000", 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012550699999999999 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.283870+0000", 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01179499 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295665+0000", 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040818579999999998 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.299747+0000", 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041355860000000001 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303883+0000", 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9145999999999998e-05 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303912+0000", 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5894999999999999e-05 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303938+0000", 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.95283383899999996 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212973 8.e 8:7ea0c169:::benchmark_data_smithi053_130774_object101239:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.361988+0000", 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 552.79745981200006, 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94206076900000002, 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212973 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361988+0000", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361988+0000", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361987+0000", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361990+0000", 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9150000000000004e-06 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361995+0000", 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6550000000000002e-06 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.361999+0000", 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3893000000000006e-05 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.362062+0000", 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1496e-05 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.362074+0000", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92424356200000002 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.286318+0000", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9658e-05 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.286417+0000", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012773999999999999 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:48:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.286545+0000", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091270350000000004 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295672+0000", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041171259999999996 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.299789+0000", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041904109999999998 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.303979+0000", 2024-02-22T19:48:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3694e-05 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304013+0000", 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5299000000000002e-05 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304048+0000", 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94206076900000002 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:212988 8.e 8:7f780119:::benchmark_data_smithi053_130774_object101254:head [delete] snapc 0=[] ondisk+write+known_if_redirected e265)", 2024-02-22T19:48:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:13.392848+0000", 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 552.76659980399995, 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.911307753, 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212988 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392848+0000", 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392848+0000", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392847+0000", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0579999999999997e-06 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392852+0000", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4080000000000001e-06 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392859+0000", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.392864+0000", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090847599999999997 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.393772+0000", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0638999999999999e-05 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:13.393783+0000", 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89499468199999999 2024-02-22T19:48:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288778+0000", 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101211 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.288879+0000", 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127248 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,7", 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.289006+0000", 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066732939999999998 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.295679+0000", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041529660000000001 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.299832+0000", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042760970000000004 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304108+0000", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5398e-05 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304134+0000", 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1591999999999999e-05 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:14.304155+0000", 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.911307753 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:26.697 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:48:26.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:26.698 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:26.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:26.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:25.439265+0000 mgr.y (mgr.14535) 740 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:26.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:25.439265+0000 mgr.y (mgr.14535) 740 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:26.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:26.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:25.439265+0000 mgr.y (mgr.14535) 740 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:26.876 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:27.434 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:48:27.458 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 41 16 62720 62704 95.5147 62.3125 0.0242922 0.010463 2024-02-22T19:48:27.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:25.813259+0000 mgr.y (mgr.14535) 741 : cluster [DBG] pgmap v999: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.7 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.13k op/s; 3727/181299 objects degraded (2.056%); 3727/181299 objects misplaced (2.056%) 2024-02-22T19:48:27.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:26.401829+0000 mon.a (mon.0) 1758 : cluster [WRN] Health check failed: Degraded data redundancy: 3727/181299 objects degraded (2.056%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:27.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:25.813259+0000 mgr.y (mgr.14535) 741 : cluster [DBG] pgmap v999: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.7 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.13k op/s; 3727/181299 objects degraded (2.056%); 3727/181299 objects misplaced (2.056%) 2024-02-22T19:48:27.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:26.401829+0000 mon.a (mon.0) 1758 : cluster [WRN] Health check failed: Degraded data redundancy: 3727/181299 objects degraded (2.056%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:25.813259+0000 mgr.y (mgr.14535) 741 : cluster [DBG] pgmap v999: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.7 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.13k op/s; 3727/181299 objects degraded (2.056%); 3727/181299 objects misplaced (2.056%) 2024-02-22T19:48:27.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:26.401829+0000 mon.a (mon.0) 1758 : cluster [WRN] Health check failed: Degraded data redundancy: 3727/181299 objects degraded (2.056%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:28.461 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 42 16 63616 63600 94.5679 56 0.012604 0.0105695 2024-02-22T19:48:28.851 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:27.815352+0000 mgr.y (mgr.14535) 742 : cluster [DBG] pgmap v1000: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.7 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 38 MiB/s wr, 608 op/s; 3727/181299 objects degraded (2.056%); 3727/181299 objects misplaced (2.056%) 2024-02-22T19:48:28.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:27.815352+0000 mgr.y (mgr.14535) 742 : cluster [DBG] pgmap v1000: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.7 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 38 MiB/s wr, 608 op/s; 3727/181299 objects degraded (2.056%); 3727/181299 objects misplaced (2.056%) 2024-02-22T19:48:28.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:27.815352+0000 mgr.y (mgr.14535) 742 : cluster [DBG] pgmap v1000: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.7 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 38 MiB/s wr, 608 op/s; 3727/181299 objects degraded (2.056%); 3727/181299 objects misplaced (2.056%) 2024-02-22T19:48:28.880 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:28.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:28.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:28.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:29.054 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:29.434 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:29.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:29.440 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:29.464 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 43 16 64545 64529 93.713 58.0625 0.0137902 0.0106643 2024-02-22T19:48:29.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:29.606 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:48:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:29] "GET /metrics HTTP/1.1" 200 176661 "" "Prometheus/2.33.4" 2024-02-22T19:48:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:30.406 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 7, 5, 4] out_osds: [0, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:48:30.406 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:48:30.406 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 2, 3, 7, 5, 4] 2024-02-22T19:48:30.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 1 2024-02-22T19:48:30.467 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 44 16 65500 65484 92.9348 59.6875 0.00931733 0.0107561 2024-02-22T19:48:30.720 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:30.803 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:48:30.843 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:48:30.873 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:48:30.917 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:48:30.944 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:48:30.986 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:48:31.023 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:48:31.058 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:48:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:29.815971+0000 mgr.y (mgr.14535) 743 : cluster [DBG] pgmap v1001: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.4 KiB/s rd, 49 MiB/s wr, 785 op/s; 3727/187482 objects degraded (1.988%); 3727/187482 objects misplaced (1.988%) 2024-02-22T19:48:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:30.726875+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? 172.21.15.53:0/4201143169' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:31.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:29.815971+0000 mgr.y (mgr.14535) 743 : cluster [DBG] pgmap v1001: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.4 KiB/s rd, 49 MiB/s wr, 785 op/s; 3727/187482 objects degraded (1.988%); 3727/187482 objects misplaced (1.988%) 2024-02-22T19:48:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:30.726875+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? 172.21.15.53:0/4201143169' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:29.815971+0000 mgr.y (mgr.14535) 743 : cluster [DBG] pgmap v1001: 138 pgs: 1 activating+undersized+degraded+remapped, 137 active+clean; 3.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.4 KiB/s rd, 49 MiB/s wr, 785 op/s; 3727/187482 objects degraded (1.988%); 3727/187482 objects misplaced (1.988%) 2024-02-22T19:48:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:30.726875+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? 172.21.15.53:0/4201143169' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:31.468 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 45 16 66340 66324 92.0346 52.5 0.00849731 0.0108608 2024-02-22T19:48:32.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:31.871579+0000 mon.a (mon.0) 1759 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3727/187482 objects degraded (1.988%), 1 pg degraded) 2024-02-22T19:48:32.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:31.871579+0000 mon.a (mon.0) 1759 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3727/187482 objects degraded (1.988%), 1 pg degraded) 2024-02-22T19:48:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:31.871579+0000 mon.a (mon.0) 1759 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3727/187482 objects degraded (1.988%), 1 pg degraded) 2024-02-22T19:48:32.469 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 46 16 67287 67271 91.3181 59.1875 0.0125755 0.0109457 2024-02-22T19:48:32.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:31.816510+0000 mgr.y (mgr.14535) 744 : cluster [DBG] pgmap v1002: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 983 op/s; 7342/194772 objects misplaced (3.770%); 433 KiB/s, 6 objects/s recovering 2024-02-22T19:48:33.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:31.816510+0000 mgr.y (mgr.14535) 744 : cluster [DBG] pgmap v1002: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 983 op/s; 7342/194772 objects misplaced (3.770%); 433 KiB/s, 6 objects/s recovering 2024-02-22T19:48:33.324 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:33.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:33.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:31.816510+0000 mgr.y (mgr.14535) 744 : cluster [DBG] pgmap v1002: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 983 op/s; 7342/194772 objects misplaced (3.770%); 433 KiB/s, 6 objects/s recovering 2024-02-22T19:48:33.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:33.476 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 47 16 68144 68128 90.5046 53.5625 0.0292201 0.0110439 2024-02-22T19:48:33.509 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:33.973 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:48:34.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:33.731319+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.53:0/2230743856' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:34.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:33.731681+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:33.731319+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.53:0/2230743856' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:33.731681+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:34.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:34.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:33.731319+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.53:0/2230743856' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:33.731681+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:48:34.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:34.450 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:34.478 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 48 16 68953 68937 89.6677 50.5625 0.0286837 0.0111465 2024-02-22T19:48:34.478 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:34.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:34.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:34.615 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:34.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:34.648 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:34.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:34.809 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:34.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:48:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:48:34.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:33.816947+0000 mgr.y (mgr.14535) 745 : cluster [DBG] pgmap v1003: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1023 B/s rd, 51 MiB/s wr, 814 op/s; 7342/194772 objects misplaced (3.770%); 358 KiB/s, 5 objects/s recovering 2024-02-22T19:48:34.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:33.940454+0000 mon.a (mon.0) 1761 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:34.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:33.949238+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:48:34.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:33.949294+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e491: 8 total, 8 up, 6 in 2024-02-22T19:48:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:33.816947+0000 mgr.y (mgr.14535) 745 : cluster [DBG] pgmap v1003: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1023 B/s rd, 51 MiB/s wr, 814 op/s; 7342/194772 objects misplaced (3.770%); 358 KiB/s, 5 objects/s recovering 2024-02-22T19:48:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:33.940454+0000 mon.a (mon.0) 1761 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:33.949238+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:48:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:33.949294+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e491: 8 total, 8 up, 6 in 2024-02-22T19:48:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:33.816947+0000 mgr.y (mgr.14535) 745 : cluster [DBG] pgmap v1003: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1023 B/s rd, 51 MiB/s wr, 814 op/s; 7342/194772 objects misplaced (3.770%); 358 KiB/s, 5 objects/s recovering 2024-02-22T19:48:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:33.940454+0000 mon.a (mon.0) 1761 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:33.949238+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:48:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:33.949294+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e491: 8 total, 8 up, 6 in 2024-02-22T19:48:35.478 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 49 16 69911 69895 89.0601 59.875 0.00981282 0.0112221 2024-02-22T19:48:35.794 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:35.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:35.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:35.851 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:48:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:48:35.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:35.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:35.449211+0000 mgr.y (mgr.14535) 746 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:35.989 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:35.449211+0000 mgr.y (mgr.14535) 746 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:36.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:36.096 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:36.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:36.257 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:35.449211+0000 mgr.y (mgr.14535) 746 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:36.397 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:36.402 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:36.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:36.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:36.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:48:36.477 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:48:36.477 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:48:36.479 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 50 16 70802 70786 88.3918 55.6875 0.0402817 0.011307 2024-02-22T19:48:36.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:36.548 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:36.573 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:36.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:36.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:36.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:48:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:37.261 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:35.817697+0000 mgr.y (mgr.14535) 747 : cluster [DBG] pgmap v1005: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1023 B/s rd, 55 MiB/s wr, 887 op/s; 7228/207918 objects misplaced (3.476%); 723 KiB/s, 11 objects/s recovering 2024-02-22T19:48:37.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:35.817697+0000 mgr.y (mgr.14535) 747 : cluster [DBG] pgmap v1005: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1023 B/s rd, 55 MiB/s wr, 887 op/s; 7228/207918 objects misplaced (3.476%); 723 KiB/s, 11 objects/s recovering 2024-02-22T19:48:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:35.817697+0000 mgr.y (mgr.14535) 747 : cluster [DBG] pgmap v1005: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1023 B/s rd, 55 MiB/s wr, 887 op/s; 7228/207918 objects misplaced (3.476%); 723 KiB/s, 11 objects/s recovering 2024-02-22T19:48:37.445 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:48:37.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:37.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:48:37.482 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 51 16 71674 71658 87.7226 54.5 0.0096932 0.0113955 2024-02-22T19:48:37.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:48:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:37.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:48:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:48:38.350 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:38.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:48:38.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:38.482 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 52 16 72485 72469 87.0103 50.6875 0.0142968 0.0114879 2024-02-22T19:48:38.490 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:38.521 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:38.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:38.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:38.658 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:38.686 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:38.690 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:38.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:38.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:37.552322+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e492: 8 total, 8 up, 6 in 2024-02-22T19:48:38.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:37.818203+0000 mgr.y (mgr.14535) 748 : cluster [DBG] pgmap v1007: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 53 MiB/s wr, 851 op/s; 7228/207918 objects misplaced (3.476%); 904 KiB/s, 14 objects/s recovering 2024-02-22T19:48:38.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:37.552322+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e492: 8 total, 8 up, 6 in 2024-02-22T19:48:38.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:37.818203+0000 mgr.y (mgr.14535) 748 : cluster [DBG] pgmap v1007: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 53 MiB/s wr, 851 op/s; 7228/207918 objects misplaced (3.476%); 904 KiB/s, 14 objects/s recovering 2024-02-22T19:48:38.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:38.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:37.552322+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e492: 8 total, 8 up, 6 in 2024-02-22T19:48:38.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:37.818203+0000 mgr.y (mgr.14535) 748 : cluster [DBG] pgmap v1007: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 53 MiB/s wr, 851 op/s; 7228/207918 objects misplaced (3.476%); 904 KiB/s, 14 objects/s recovering 2024-02-22T19:48:38.860 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:38.930 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:38.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:38.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:39.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:39.067 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:39.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:39.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:39.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:39.216 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:39.489 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 53 16 73252 73236 86.2732 47.9375 0.00411661 0.0115865 2024-02-22T19:48:39.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:48:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:39] "GET /metrics HTTP/1.1" 200 176684 "" "Prometheus/2.33.4" 2024-02-22T19:48:39.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:48:39.992 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:39.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:39.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:40.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:48:40.154 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:40.487 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 54 16 74157 74141 85.7172 56.5625 0.0250025 0.0116612 2024-02-22T19:48:41.050 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:48:41.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:41.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:48:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:41.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:41.228 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:39.824871+0000 mgr.y (mgr.14535) 749 : cluster [DBG] pgmap v1008: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.4 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 820 op/s; 7228/214461 objects misplaced (3.370%); 456 KiB/s, 7 objects/s recovering 2024-02-22T19:48:41.229 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:39.824871+0000 mgr.y (mgr.14535) 749 : cluster [DBG] pgmap v1008: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.4 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 820 op/s; 7228/214461 objects misplaced (3.370%); 456 KiB/s, 7 objects/s recovering 2024-02-22T19:48:41.234 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:39.824871+0000 mgr.y (mgr.14535) 749 : cluster [DBG] pgmap v1008: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.4 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 820 op/s; 7228/214461 objects misplaced (3.370%); 456 KiB/s, 7 objects/s recovering 2024-02-22T19:48:41.451 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:41.458 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:41.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:41.487 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 55 16 75049 75033 85.1725 55.75 0.0203039 0.011735 2024-02-22T19:48:41.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:41.619 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:41.762 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:41.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:41.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:41.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:41.915 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:41.937 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:41.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:41.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:42.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:42.393 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:42.401 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:42.401 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:42.402 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:48:42.249+0000 7fe2d794e700 -1 mon.a@0(leader).osd e492 definitely_dead 0 2024-02-22T19:48:42.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:42.403 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:42.490 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 56 16 75891 75875 84.5893 52.625 0.0217717 0.0118174 2024-02-22T19:48:42.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:42.561 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:42.962 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.1. 2024-02-22T19:48:43.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:41.825387+0000 mgr.y (mgr.14535) 750 : cluster [DBG] pgmap v1009: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.5 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 7112/220794 objects misplaced (3.221%); 919 KiB/s, 14 objects/s recovering 2024-02-22T19:48:43.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:42.250113+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.53:0/3182212604' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-02-22T19:48:43.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:42.250801+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-02-22T19:48:43.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:42.250908+0000 mon.a (mon.0) 1766 : cluster [INF] Client client.admin marked osd.1 out, while it was still marked up 2024-02-22T19:48:43.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:41.825387+0000 mgr.y (mgr.14535) 750 : cluster [DBG] pgmap v1009: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.5 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 7112/220794 objects misplaced (3.221%); 919 KiB/s, 14 objects/s recovering 2024-02-22T19:48:43.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:42.250113+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.53:0/3182212604' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-02-22T19:48:43.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:42.250801+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-02-22T19:48:43.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:42.250908+0000 mon.a (mon.0) 1766 : cluster [INF] Client client.admin marked osd.1 out, while it was still marked up 2024-02-22T19:48:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:41.825387+0000 mgr.y (mgr.14535) 750 : cluster [DBG] pgmap v1009: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.5 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 7112/220794 objects misplaced (3.221%); 919 KiB/s, 14 objects/s recovering 2024-02-22T19:48:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:42.250113+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.53:0/3182212604' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-02-22T19:48:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:42.250801+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-02-22T19:48:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:42.250908+0000 mon.a (mon.0) 1766 : cluster [INF] Client client.admin marked osd.1 out, while it was still marked up 2024-02-22T19:48:43.457 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:43.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:43.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:43.489 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 57 16 76273 76257 83.5251 23.875 0.0373544 0.0118492 2024-02-22T19:48:43.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:43.591 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:43.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:43.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:43.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:43.803 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:44.072 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:42.919480+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-02-22T19:48:44.072 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:42.919548+0000 mon.a (mon.0) 1768 : cluster [DBG] osdmap e493: 8 total, 8 up, 5 in 2024-02-22T19:48:44.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:42.919480+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-02-22T19:48:44.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:42.919548+0000 mon.a (mon.0) 1768 : cluster [DBG] osdmap e493: 8 total, 8 up, 5 in 2024-02-22T19:48:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:42.919480+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-02-22T19:48:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:42.919548+0000 mon.a (mon.0) 1768 : cluster [DBG] osdmap e493: 8 total, 8 up, 5 in 2024-02-22T19:48:44.489 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 58 16 76276 76260 82.0897 0.1875 1.14559 0.0118941 2024-02-22T19:48:44.968 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:48:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:48:45.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:43.825745+0000 mgr.y (mgr.14535) 751 : cluster [DBG] pgmap v1011: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 639 B/s rd, 33 MiB/s wr, 536 op/s; 7112/220794 objects misplaced (3.221%); 464 KiB/s, 7 objects/s recovering 2024-02-22T19:48:45.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:43.958475+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e494: 8 total, 8 up, 5 in 2024-02-22T19:48:45.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:43.825745+0000 mgr.y (mgr.14535) 751 : cluster [DBG] pgmap v1011: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 639 B/s rd, 33 MiB/s wr, 536 op/s; 7112/220794 objects misplaced (3.221%); 464 KiB/s, 7 objects/s recovering 2024-02-22T19:48:45.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:43.958475+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e494: 8 total, 8 up, 5 in 2024-02-22T19:48:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:43.825745+0000 mgr.y (mgr.14535) 751 : cluster [DBG] pgmap v1011: 138 pgs: 1 active+recovering+undersized+remapped, 137 active+clean; 4.5 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 639 B/s rd, 33 MiB/s wr, 536 op/s; 7112/220794 objects misplaced (3.221%); 464 KiB/s, 7 objects/s recovering 2024-02-22T19:48:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:43.958475+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e494: 8 total, 8 up, 5 in 2024-02-22T19:48:45.489 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 59 16 76475 76459 80.9102 12.4375 0.027187 0.012354 2024-02-22T19:48:45.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.519051+0000 osd.0 (osd.0) 177 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.519099+0000 osd.0 (osd.0) 178 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.957989+0000 osd.7 (osd.7) 321 : cluster [DBG] 5.9 starting backfill to osd.3 from (0'0,0'0] MAX to 484'21 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.958571+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e495: 8 total, 8 up, 5 in 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.960941+0000 osd.3 (osd.3) 343 : cluster [DBG] 11.0 starting backfill to osd.2 from (0'0,0'0] MAX to 492'4774 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.961098+0000 osd.7 (osd.7) 322 : cluster [DBG] 11.f starting backfill to osd.4 from (0'0,0'0] MAX to 492'4902 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.962561+0000 osd.4 (osd.4) 191 : cluster [DBG] 5.5 starting backfill to osd.3 from (0'0,0'0] MAX to 484'26 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.964522+0000 osd.5 (osd.5) 202 : cluster [DBG] 5.3 starting backfill to osd.7 from (0'0,0'0] MAX to 395'16 2024-02-22T19:48:46.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.965216+0000 osd.5 (osd.5) 203 : cluster [DBG] 5.7 starting backfill to osd.7 from (0'0,0'0] MAX to 473'28 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.965245+0000 osd.3 (osd.3) 344 : cluster [DBG] 11.b starting backfill to osd.5 from (0'0,0'0] MAX to 492'4719 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.965829+0000 osd.5 (osd.5) 204 : cluster [DBG] 11.7 starting backfill to osd.7 from (0'0,0'0] MAX to 492'4746 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.965895+0000 osd.2 (osd.2) 292 : cluster [DBG] 11.c starting backfill to osd.7 from (0'0,0'0] MAX to 492'4823 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.966021+0000 osd.2 (osd.2) 293 : cluster [DBG] 11.6 starting backfill to osd.5 from (0'0,0'0] MAX to 492'4889 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.968158+0000 osd.3 (osd.3) 345 : cluster [DBG] 11.a starting backfill to osd.2 from (0'0,0'0] MAX to 492'4684 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.969152+0000 osd.4 (osd.4) 192 : cluster [DBG] 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 492'4722 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.978102+0000 osd.3 (osd.3) 346 : cluster [DBG] 11.9 starting backfill to osd.5 from (0'0,0'0] MAX to 492'4803 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:44.988672+0000 osd.3 (osd.3) 347 : cluster [DBG] 5.0 starting backfill to osd.2 from (0'0,0'0] MAX to 473'16 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:45.459270+0000 mgr.y (mgr.14535) 752 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.519051+0000 osd.0 (osd.0) 177 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.519099+0000 osd.0 (osd.0) 178 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.957989+0000 osd.7 (osd.7) 321 : cluster [DBG] 5.9 starting backfill to osd.3 from (0'0,0'0] MAX to 484'21 2024-02-22T19:48:46.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.958571+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e495: 8 total, 8 up, 5 in 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.960941+0000 osd.3 (osd.3) 343 : cluster [DBG] 11.0 starting backfill to osd.2 from (0'0,0'0] MAX to 492'4774 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.961098+0000 osd.7 (osd.7) 322 : cluster [DBG] 11.f starting backfill to osd.4 from (0'0,0'0] MAX to 492'4902 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.962561+0000 osd.4 (osd.4) 191 : cluster [DBG] 5.5 starting backfill to osd.3 from (0'0,0'0] MAX to 484'26 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.964522+0000 osd.5 (osd.5) 202 : cluster [DBG] 5.3 starting backfill to osd.7 from (0'0,0'0] MAX to 395'16 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.965216+0000 osd.5 (osd.5) 203 : cluster [DBG] 5.7 starting backfill to osd.7 from (0'0,0'0] MAX to 473'28 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.965245+0000 osd.3 (osd.3) 344 : cluster [DBG] 11.b starting backfill to osd.5 from (0'0,0'0] MAX to 492'4719 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.965829+0000 osd.5 (osd.5) 204 : cluster [DBG] 11.7 starting backfill to osd.7 from (0'0,0'0] MAX to 492'4746 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.965895+0000 osd.2 (osd.2) 292 : cluster [DBG] 11.c starting backfill to osd.7 from (0'0,0'0] MAX to 492'4823 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.966021+0000 osd.2 (osd.2) 293 : cluster [DBG] 11.6 starting backfill to osd.5 from (0'0,0'0] MAX to 492'4889 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.968158+0000 osd.3 (osd.3) 345 : cluster [DBG] 11.a starting backfill to osd.2 from (0'0,0'0] MAX to 492'4684 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.969152+0000 osd.4 (osd.4) 192 : cluster [DBG] 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 492'4722 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.978102+0000 osd.3 (osd.3) 346 : cluster [DBG] 11.9 starting backfill to osd.5 from (0'0,0'0] MAX to 492'4803 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:44.988672+0000 osd.3 (osd.3) 347 : cluster [DBG] 5.0 starting backfill to osd.2 from (0'0,0'0] MAX to 473'16 2024-02-22T19:48:46.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:45.459270+0000 mgr.y (mgr.14535) 752 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:46.287 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:46.299 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:46.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:46.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.519051+0000 osd.0 (osd.0) 177 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.519099+0000 osd.0 (osd.0) 178 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.957989+0000 osd.7 (osd.7) 321 : cluster [DBG] 5.9 starting backfill to osd.3 from (0'0,0'0] MAX to 484'21 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.958571+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e495: 8 total, 8 up, 5 in 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.960941+0000 osd.3 (osd.3) 343 : cluster [DBG] 11.0 starting backfill to osd.2 from (0'0,0'0] MAX to 492'4774 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.961098+0000 osd.7 (osd.7) 322 : cluster [DBG] 11.f starting backfill to osd.4 from (0'0,0'0] MAX to 492'4902 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.962561+0000 osd.4 (osd.4) 191 : cluster [DBG] 5.5 starting backfill to osd.3 from (0'0,0'0] MAX to 484'26 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.964522+0000 osd.5 (osd.5) 202 : cluster [DBG] 5.3 starting backfill to osd.7 from (0'0,0'0] MAX to 395'16 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.965216+0000 osd.5 (osd.5) 203 : cluster [DBG] 5.7 starting backfill to osd.7 from (0'0,0'0] MAX to 473'28 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.965245+0000 osd.3 (osd.3) 344 : cluster [DBG] 11.b starting backfill to osd.5 from (0'0,0'0] MAX to 492'4719 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.965829+0000 osd.5 (osd.5) 204 : cluster [DBG] 11.7 starting backfill to osd.7 from (0'0,0'0] MAX to 492'4746 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.965895+0000 osd.2 (osd.2) 292 : cluster [DBG] 11.c starting backfill to osd.7 from (0'0,0'0] MAX to 492'4823 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.966021+0000 osd.2 (osd.2) 293 : cluster [DBG] 11.6 starting backfill to osd.5 from (0'0,0'0] MAX to 492'4889 2024-02-22T19:48:46.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.968158+0000 osd.3 (osd.3) 345 : cluster [DBG] 11.a starting backfill to osd.2 from (0'0,0'0] MAX to 492'4684 2024-02-22T19:48:46.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.969152+0000 osd.4 (osd.4) 192 : cluster [DBG] 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 492'4722 2024-02-22T19:48:46.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.978102+0000 osd.3 (osd.3) 346 : cluster [DBG] 11.9 starting backfill to osd.5 from (0'0,0'0] MAX to 492'4803 2024-02-22T19:48:46.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:44.988672+0000 osd.3 (osd.3) 347 : cluster [DBG] 5.0 starting backfill to osd.2 from (0'0,0'0] MAX to 473'16 2024-02-22T19:48:46.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:45.459270+0000 mgr.y (mgr.14535) 752 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:46.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:46.490 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:48:46.490458+0000 min lat: 0.00115108 max lat: 2.39342 avg lat: 0.0124375 2024-02-22T19:48:46.490 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:48:46.490 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 60 16 77250 77234 80.3695 48.4375 0.0189806 0.0124375 2024-02-22T19:48:46.492 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:45.826581+0000 mgr.y (mgr.14535) 753 : cluster [DBG] pgmap v1014: 138 pgs: 1 activating+degraded+remapped, 101 active+clean, 14 active+recovery_wait+degraded, 12 remapped+peering, 4 activating+remapped, 5 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 52 MiB/s wr, 828 op/s; 1044/229383 objects degraded (0.455%); 18998/229383 objects misplaced (8.282%); 1003 KiB/s, 15 objects/s recovering 2024-02-22T19:48:47.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:45.962542+0000 mon.a (mon.0) 1771 : cluster [WRN] Health check failed: Degraded data redundancy: 1044/229383 objects degraded (0.455%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T19:48:47.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:45.826581+0000 mgr.y (mgr.14535) 753 : cluster [DBG] pgmap v1014: 138 pgs: 1 activating+degraded+remapped, 101 active+clean, 14 active+recovery_wait+degraded, 12 remapped+peering, 4 activating+remapped, 5 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 52 MiB/s wr, 828 op/s; 1044/229383 objects degraded (0.455%); 18998/229383 objects misplaced (8.282%); 1003 KiB/s, 15 objects/s recovering 2024-02-22T19:48:47.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:45.962542+0000 mon.a (mon.0) 1771 : cluster [WRN] Health check failed: Degraded data redundancy: 1044/229383 objects degraded (0.455%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T19:48:47.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:45.826581+0000 mgr.y (mgr.14535) 753 : cluster [DBG] pgmap v1014: 138 pgs: 1 activating+degraded+remapped, 101 active+clean, 14 active+recovery_wait+degraded, 12 remapped+peering, 4 activating+remapped, 5 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 52 MiB/s wr, 828 op/s; 1044/229383 objects degraded (0.455%); 18998/229383 objects misplaced (8.282%); 1003 KiB/s, 15 objects/s recovering 2024-02-22T19:48:47.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:45.962542+0000 mon.a (mon.0) 1771 : cluster [WRN] Health check failed: Degraded data redundancy: 1044/229383 objects degraded (0.455%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T19:48:47.493 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 61 16 77930 77914 79.745 42.5 0.0321666 0.0125334 2024-02-22T19:48:47.707 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:47.708 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:47.711 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:47.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:47.015623+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e496: 8 total, 8 up, 5 in 2024-02-22T19:48:48.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:47.015623+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e496: 8 total, 8 up, 5 in 2024-02-22T19:48:48.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:47.015623+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e496: 8 total, 8 up, 5 in 2024-02-22T19:48:48.495 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 62 16 78522 78506 79.0533 37 0.0219703 0.0126426 2024-02-22T19:48:48.810 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:48:48.810 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:48:49.020 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:49.030 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:49.030 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:49.109 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:48:49.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:49.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:49.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:49.303 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:47.828464+0000 mgr.y (mgr.14535) 754 : cluster [DBG] pgmap v1016: 138 pgs: 1 activating+degraded+remapped, 101 active+clean, 14 active+recovery_wait+degraded, 12 remapped+peering, 4 activating+remapped, 5 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 36 MiB/s wr, 583 op/s; 1044/229383 objects degraded (0.455%); 18998/229383 objects misplaced (8.282%); 470 KiB/s, 7 objects/s recovering 2024-02-22T19:48:49.303 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:48.024869+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e497: 8 total, 8 up, 5 in 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:48.111127+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.53:0/1179827930' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:48.116932+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:49] "GET /metrics HTTP/1.1" 200 176684 "" "Prometheus/2.33.4" 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:47.828464+0000 mgr.y (mgr.14535) 754 : cluster [DBG] pgmap v1016: 138 pgs: 1 activating+degraded+remapped, 101 active+clean, 14 active+recovery_wait+degraded, 12 remapped+peering, 4 activating+remapped, 5 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 36 MiB/s wr, 583 op/s; 1044/229383 objects degraded (0.455%); 18998/229383 objects misplaced (8.282%); 470 KiB/s, 7 objects/s recovering 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:48.024869+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e497: 8 total, 8 up, 5 in 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:48.111127+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.53:0/1179827930' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:48:49.304 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:48.116932+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:48:49.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:48:49.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:47.828464+0000 mgr.y (mgr.14535) 754 : cluster [DBG] pgmap v1016: 138 pgs: 1 activating+degraded+remapped, 101 active+clean, 14 active+recovery_wait+degraded, 12 remapped+peering, 4 activating+remapped, 5 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 36 MiB/s wr, 583 op/s; 1044/229383 objects degraded (0.455%); 18998/229383 objects misplaced (8.282%); 470 KiB/s, 7 objects/s recovering 2024-02-22T19:48:49.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:48.024869+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e497: 8 total, 8 up, 5 in 2024-02-22T19:48:49.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:48.111127+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.53:0/1179827930' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:48:49.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:48.116932+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:48:49.345 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:48:49.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:49.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:48:49.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:49.495 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 63 16 79055 79039 78.3279 33.3125 0.025358 0.0127591 2024-02-22T19:48:49.528 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:49.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:49.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:49.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:48:49.708 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:49.735 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:48:49.738 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:49.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:48:49.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:49.951 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:50.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:49.045939+0000 mon.a (mon.0) 1775 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:50.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:49.066612+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:48:50.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:49.066696+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e498: 8 total, 8 up, 5 in 2024-02-22T19:48:50.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:49.045939+0000 mon.a (mon.0) 1775 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:50.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:49.066612+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:48:50.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:49.066696+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e498: 8 total, 8 up, 5 in 2024-02-22T19:48:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:49.045939+0000 mon.a (mon.0) 1775 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:48:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:49.066612+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:48:50.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:49.066696+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e498: 8 total, 8 up, 5 in 2024-02-22T19:48:50.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:48:50.501 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 64 16 79614 79598 77.6447 34.9375 0.0641381 0.0128724 2024-02-22T19:48:50.559 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:50.564 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:50.564 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:50.665 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:50.789 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:50.828 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:50.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:50.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:50.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:48:51.049 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:48:51.064 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:51.096 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:51.100 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:48:51.115 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:48:51.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:51.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:48:51.157 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:48:51.193 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:48:51.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:51.246 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:48:51.310 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:48:51.315 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:49.829061+0000 mgr.y (mgr.14535) 755 : cluster [DBG] pgmap v1019: 138 pgs: 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 peering, 1 activating+degraded+remapped, 117 active+clean, 2 active+recovery_wait+degraded, 3 remapped+peering, 3 active+remapped+backfilling, 4 activating+remapped, 3 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 840 B/s rd, 26 MiB/s wr, 422 op/s; 994/232608 objects degraded (0.427%); 41605/232608 objects misplaced (17.886%); 22 MiB/s, 364 objects/s recovering 2024-02-22T19:48:51.315 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:50.062718+0000 mon.a (mon.0) 1778 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:48:51.316 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:50.078682+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e499: 8 total, 8 up, 5 in 2024-02-22T19:48:51.316 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:50.994131+0000 mon.b (mon.1) 106 : audit [DBG] from='client.? 172.21.15.53:0/3224925231' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:51.316 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:51.316 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:49.829061+0000 mgr.y (mgr.14535) 755 : cluster [DBG] pgmap v1019: 138 pgs: 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 peering, 1 activating+degraded+remapped, 117 active+clean, 2 active+recovery_wait+degraded, 3 remapped+peering, 3 active+remapped+backfilling, 4 activating+remapped, 3 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 840 B/s rd, 26 MiB/s wr, 422 op/s; 994/232608 objects degraded (0.427%); 41605/232608 objects misplaced (17.886%); 22 MiB/s, 364 objects/s recovering 2024-02-22T19:48:51.316 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:50.062718+0000 mon.a (mon.0) 1778 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:48:51.316 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:50.078682+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e499: 8 total, 8 up, 5 in 2024-02-22T19:48:51.316 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:50.994131+0000 mon.b (mon.1) 106 : audit [DBG] from='client.? 172.21.15.53:0/3224925231' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:49.829061+0000 mgr.y (mgr.14535) 755 : cluster [DBG] pgmap v1019: 138 pgs: 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 peering, 1 activating+degraded+remapped, 117 active+clean, 2 active+recovery_wait+degraded, 3 remapped+peering, 3 active+remapped+backfilling, 4 activating+remapped, 3 active+recovering, 1 active+recovery_wait+undersized+degraded+remapped; 4.7 GiB data, 14 GiB used, 433 GiB / 447 GiB avail; 840 B/s rd, 26 MiB/s wr, 422 op/s; 994/232608 objects degraded (0.427%); 41605/232608 objects misplaced (17.886%); 22 MiB/s, 364 objects/s recovering 2024-02-22T19:48:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:50.062718+0000 mon.a (mon.0) 1778 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:48:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:50.078682+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e499: 8 total, 8 up, 5 in 2024-02-22T19:48:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:50.994131+0000 mon.b (mon.1) 106 : audit [DBG] from='client.? 172.21.15.53:0/3224925231' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:48:51.361 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:48:51.401 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.416 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:48:51.421 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:51.435 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:51.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:51.502 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 65 16 80087 80071 76.9038 29.5625 0.0307564 0.0129975 2024-02-22T19:48:51.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:51.586 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:51.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:51.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:51.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:51.769 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:51.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:51.806 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:51.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:51.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:48:51.969 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:52.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:48:52.504 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 66 16 80663 80647 76.2829 36 0.0223333 0.0131022 2024-02-22T19:48:52.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:52.065981+0000 mon.a (mon.0) 1780 : cluster [WRN] Health check update: Degraded data redundancy: 1/238203 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:52.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:52.065981+0000 mon.a (mon.0) 1780 : cluster [WRN] Health check update: Degraded data redundancy: 1/238203 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:52.506 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 5, 4] out_osds: [0, 6, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:48:52.506 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:48:52.506 INFO:tasks.thrashosds.thrasher:Shrinking pool default.rgw.meta 2024-02-22T19:48:52.506 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool default.rgw.meta by 10 min 8 2024-02-22T19:48:52.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:48:52.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:52.065981+0000 mon.a (mon.0) 1780 : cluster [WRN] Health check update: Degraded data redundancy: 1/238203 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:48:53.338 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:53.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:48:53.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:53.506 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 67 16 81179 81163 75.6263 32.25 0.0199117 0.0132162 2024-02-22T19:48:53.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:51.829531+0000 mgr.y (mgr.14535) 756 : cluster [DBG] pgmap v1021: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 1 peering, 122 active+clean, 5 active+remapped+backfilling; 4.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.0 KiB/s rd, 38 MiB/s wr, 611 op/s; 1/238203 objects degraded (0.000%); 58059/238203 objects misplaced (24.374%); 24 MiB/s, 405 objects/s recovering 2024-02-22T19:48:53.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:52.570580+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e500: 8 total, 8 up, 5 in 2024-02-22T19:48:53.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:52.862015+0000 mon.a (mon.0) 1782 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:48:53.512 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:51.829531+0000 mgr.y (mgr.14535) 756 : cluster [DBG] pgmap v1021: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 1 peering, 122 active+clean, 5 active+remapped+backfilling; 4.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.0 KiB/s rd, 38 MiB/s wr, 611 op/s; 1/238203 objects degraded (0.000%); 58059/238203 objects misplaced (24.374%); 24 MiB/s, 405 objects/s recovering 2024-02-22T19:48:53.512 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:52.570580+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e500: 8 total, 8 up, 5 in 2024-02-22T19:48:53.512 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:52.862015+0000 mon.a (mon.0) 1782 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:48:53.520 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:51.829531+0000 mgr.y (mgr.14535) 756 : cluster [DBG] pgmap v1021: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 1 peering, 122 active+clean, 5 active+remapped+backfilling; 4.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.0 KiB/s rd, 38 MiB/s wr, 611 op/s; 1/238203 objects degraded (0.000%); 58059/238203 objects misplaced (24.374%); 24 MiB/s, 405 objects/s recovering 2024-02-22T19:48:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:52.570580+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e500: 8 total, 8 up, 5 in 2024-02-22T19:48:53.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:52.862015+0000 mon.a (mon.0) 1782 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:48:54.506 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 68 16 81814 81798 75.0955 39.6875 0.0156526 0.0133108 2024-02-22T19:48:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:53.422773+0000 mon.a (mon.0) 1783 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:53.535142+0000 mon.a (mon.0) 1784 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:48:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:53.536144+0000 mon.a (mon.0) 1785 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:48:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:53.571470+0000 mon.a (mon.0) 1786 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:54.271425+0000 mon.a (mon.0) 1787 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:53.422773+0000 mon.a (mon.0) 1783 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:53.535142+0000 mon.a (mon.0) 1784 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:48:54.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:53.536144+0000 mon.a (mon.0) 1785 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:48:54.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:53.571470+0000 mon.a (mon.0) 1786 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:54.271425+0000 mon.a (mon.0) 1787 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:53.422773+0000 mon.a (mon.0) 1783 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:53.535142+0000 mon.a (mon.0) 1784 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:48:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:53.536144+0000 mon.a (mon.0) 1785 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:48:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:53.571470+0000 mon.a (mon.0) 1786 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:54.271425+0000 mon.a (mon.0) 1787 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:48:54.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:48:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:48:55.433 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:53.831394+0000 mgr.y (mgr.14535) 757 : cluster [DBG] pgmap v1023: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 1 peering, 122 active+clean, 5 active+remapped+backfilling; 4.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 881 B/s rd, 32 MiB/s wr, 507 op/s; 1/238203 objects degraded (0.000%); 58059/238203 objects misplaced (24.374%); 20 MiB/s, 335 objects/s recovering 2024-02-22T19:48:55.507 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 69 16 82428 82412 74.5632 38.375 0.0326617 0.0134056 2024-02-22T19:48:55.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:53.831394+0000 mgr.y (mgr.14535) 757 : cluster [DBG] pgmap v1023: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 1 peering, 122 active+clean, 5 active+remapped+backfilling; 4.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 881 B/s rd, 32 MiB/s wr, 507 op/s; 1/238203 objects degraded (0.000%); 58059/238203 objects misplaced (24.374%); 20 MiB/s, 335 objects/s recovering 2024-02-22T19:48:55.728 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:48:55.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:55.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:48:55.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:48:55.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:48:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:48:55.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:53.831394+0000 mgr.y (mgr.14535) 757 : cluster [DBG] pgmap v1023: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 1 peering, 122 active+clean, 5 active+remapped+backfilling; 4.8 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 881 B/s rd, 32 MiB/s wr, 507 op/s; 1/238203 objects degraded (0.000%); 58059/238203 objects misplaced (24.374%); 20 MiB/s, 335 objects/s recovering 2024-02-22T19:48:55.881 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:55.922 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:48:55.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:55.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:48:56.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:48:56.099 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:56.132 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:48:56.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:56.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:48:56.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:48:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:56.374 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:48:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:48:56.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:48:56.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:48:56.447 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:48:56.447 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-02-22T19:48:56.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-02-22T19:48:56.447 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-02-22T19:48:56.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.448 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209432 8.a 8:522e6d61:::benchmark_data_smithi053_130774_object97698:head [delete] snapc 0=[] ondisk+write+known_if_redirected e259)", 2024-02-22T19:48:56.448 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:06.927238+0000", 2024-02-22T19:48:56.448 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 589.51409996400002, 2024-02-22T19:48:56.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.34212656299999999, 2024-02-22T19:48:56.448 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.448 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209432 2024-02-22T19:48:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927238+0000", 2024-02-22T19:48:56.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927238+0000", 2024-02-22T19:48:56.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:48:56.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927236+0000", 2024-02-22T19:48:56.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8739999999999997e-06 2024-02-22T19:48:56.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927243+0000", 2024-02-22T19:48:56.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1793e-05 2024-02-22T19:48:56.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927255+0000", 2024-02-22T19:48:56.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6470000000000001e-06 2024-02-22T19:48:56.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927262+0000", 2024-02-22T19:48:56.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0491000000000002e-05 2024-02-22T19:48:56.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927312+0000", 2024-02-22T19:48:56.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013548299999999999 2024-02-22T19:48:56.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:56.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927448+0000", 2024-02-22T19:48:56.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015343600000000001 2024-02-22T19:48:56.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3,5", 2024-02-22T19:48:56.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.927601+0000", 2024-02-22T19:48:56.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004624815 2024-02-22T19:48:56.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:56.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.932226+0000", 2024-02-22T19:48:56.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017510169999999999 2024-02-22T19:48:56.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.933977+0000", 2024-02-22T19:48:56.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030616765000000001 2024-02-22T19:48:56.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.964594+0000", 2024-02-22T19:48:56.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9940000000000006e-06 2024-02-22T19:48:56.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.964603+0000", 2024-02-22T19:48:56.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.30467808000000002 2024-02-22T19:48:56.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.269281+0000", 2024-02-22T19:48:56.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3683999999999994e-05 2024-02-22T19:48:56.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.269364+0000", 2024-02-22T19:48:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.34212656299999999 2024-02-22T19:48:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.463 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209441 8.a 8:551a3331:::benchmark_data_smithi053_130774_object97707:head [delete] snapc 0=[] ondisk+write+known_if_redirected e259)", 2024-02-22T19:48:56.463 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:06.941518+0000", 2024-02-22T19:48:56.463 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 589.49981973299998, 2024-02-22T19:48:56.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.331210693, 2024-02-22T19:48:56.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.464 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:56.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:56.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209441 2024-02-22T19:48:56.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.465 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941518+0000", 2024-02-22T19:48:56.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941518+0000", 2024-02-22T19:48:56.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:48:56.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941517+0000", 2024-02-22T19:48:56.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0929999999999999e-06 2024-02-22T19:48:56.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941523+0000", 2024-02-22T19:48:56.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-02-22T19:48:56.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941530+0000", 2024-02-22T19:48:56.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.72e-06 2024-02-22T19:48:56.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941537+0000", 2024-02-22T19:48:56.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5241999999999996e-05 2024-02-22T19:48:56.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941602+0000", 2024-02-22T19:48:56.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144263 2024-02-22T19:48:56.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:56.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941746+0000", 2024-02-22T19:48:56.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015906699999999999 2024-02-22T19:48:56.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2,3,5", 2024-02-22T19:48:56.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.941905+0000", 2024-02-22T19:48:56.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022724146000000001 2024-02-22T19:48:56.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.964629+0000", 2024-02-22T19:48:56.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1650000000000003e-06 2024-02-22T19:48:56.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:06.964636+0000", 2024-02-22T19:48:56.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.30802108900000003 2024-02-22T19:48:56.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.272657+0000", 2024-02-22T19:48:56.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2099000000000002e-05 2024-02-22T19:48:56.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.272729+0000", 2024-02-22T19:48:56.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.331210693 2024-02-22T19:48:56.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.476 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.477 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209450 8.1 8:87718195:::benchmark_data_smithi053_130774_object97716:head [delete] snapc 0=[] ondisk+write+known_if_redirected e260)", 2024-02-22T19:48:56.477 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.003208+0000", 2024-02-22T19:48:56.477 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 589.43813000700004, 2024-02-22T19:48:56.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4444186480000001, 2024-02-22T19:48:56.478 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.478 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:56.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:56.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:56.479 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209450 2024-02-22T19:48:56.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.479 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003208+0000", 2024-02-22T19:48:56.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003208+0000", 2024-02-22T19:48:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003206+0000", 2024-02-22T19:48:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3529999999999995e-06 2024-02-22T19:48:56.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003214+0000", 2024-02-22T19:48:56.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1759e-05 2024-02-22T19:48:56.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003226+0000", 2024-02-22T19:48:56.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9700000000000002e-06 2024-02-22T19:48:56.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003233+0000", 2024-02-22T19:48:56.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000334999 2024-02-22T19:48:56.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003568+0000", 2024-02-22T19:48:56.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1613e-05 2024-02-22T19:48:56.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.003579+0000", 2024-02-22T19:48:56.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.21595956299999999 2024-02-22T19:48:56.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.219539+0000", 2024-02-22T19:48:56.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3499000000000001e-05 2024-02-22T19:48:56.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.219552+0000", 2024-02-22T19:48:56.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.203221345 2024-02-22T19:48:56.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.422774+0000", 2024-02-22T19:48:56.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010390700000000001 2024-02-22T19:48:56.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:56.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.422878+0000", 2024-02-22T19:48:56.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5383999999999994e-05 2024-02-22T19:48:56.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,2,5", 2024-02-22T19:48:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.422963+0000", 2024-02-22T19:48:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019337950000000001 2024-02-22T19:48:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.424897+0000", 2024-02-22T19:48:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042861030000000003 2024-02-22T19:48:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.429183+0000", 2024-02-22T19:48:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013967388000000001 2024-02-22T19:48:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.443150+0000", 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043055990000000002 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.447456+0000", 2024-02-22T19:48:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9536000000000001e-05 2024-02-22T19:48:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.447505+0000", 2024-02-22T19:48:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000120925 2024-02-22T19:48:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.447626+0000", 2024-02-22T19:48:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4444186480000001 2024-02-22T19:48:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.15054.0:209456 8.1 8:89799534:::benchmark_data_smithi053_130774_object97722:head [delete] snapc 0=[] ondisk+write+known_if_redirected e260)", 2024-02-22T19:48:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:39:07.013387+0000", 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 589.42795124600002, 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.434373015, 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.15054", 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2665541348", 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209456 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013387+0000", 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013387+0000", 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013385+0000", 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-02-22T19:48:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013393+0000", 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5058000000000001e-05 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013408+0000", 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4410000000000004e-06 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013415+0000", 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014715 2024-02-22T19:48:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013562+0000", 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5359999999999998e-06 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.013572+0000", 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.20600543199999999 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.219577+0000", 2024-02-22T19:48:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1200000000000002e-06 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:07.219585+0000", 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2053248329999999 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.424910+0000", 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2787999999999996e-05 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.424973+0000", 2024-02-22T19:48:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4627000000000002e-05 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0,2,5", 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.425048+0000", 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041273159999999998 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.429175+0000", 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5082999999999999e-05 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.429190+0000", 2024-02-22T19:48:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01399569 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.443186+0000", 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044648129999999998 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.447651+0000", 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9428e-05 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-02-22T19:48:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.447680+0000", 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9536999999999994e-05 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:39:08.447760+0000", 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.434373015 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60012 11.2 11.75d4a712 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.573996+0000", 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.867342192000002, 2024-02-22T19:48:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96044710300000002, 2024-02-22T19:48:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60012 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.573996+0000", 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.573996+0000", 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.573995+0000", 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5417000000000001e-05 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.574020+0000", 2024-02-22T19:48:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.574028+0000", 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.096e-06 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.574033+0000", 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010802399999999999 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.574141+0000", 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5220000000000001e-06 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.574148+0000", 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96028258899999996 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534431+0000", 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1776e-05 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534443+0000", 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.96044710300000002 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60025 11.2 11.6795a4d2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.589312+0000", 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.852025746000002, 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94516339000000005, 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60025 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589312+0000", 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589312+0000", 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:48:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589311+0000", 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6399e-05 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589327+0000", 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2590000000000003e-06 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589335+0000", 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5310000000000002e-06 2024-02-22T19:48:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589338+0000", 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3643999999999998e-05 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589372+0000", 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7759999999999997e-06 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.589379+0000", 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94508649700000003 2024-02-22T19:48:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534466+0000", 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4420000000000004e-06 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534475+0000", 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94516339000000005 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60028 11.2 11.734b4f82 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.591820+0000", 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.849517900000002, 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.942685683, 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60028 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591820+0000", 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591820+0000", 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591818+0000", 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5454000000000001e-05 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591844+0000", 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3719e-05 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591858+0000", 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9040000000000003e-06 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591864+0000", 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3884999999999997e-05 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591908+0000", 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8469999999999998e-06 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591915+0000", 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.942583528 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534499+0000", 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8539999999999999e-06 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534506+0000", 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.942685683 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60030 11.2 11.ee25162 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.591879+0000", 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.849458964, 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94264991600000003, 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60030 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591879+0000", 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591879+0000", 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591878+0000", 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5607999999999999e-05 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591903+0000", 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9529999999999999e-06 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591911+0000", 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7570000000000004e-06 2024-02-22T19:48:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591916+0000", 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9508000000000001e-05 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591966+0000", 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9330000000000007e-06 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.591975+0000", 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.942546263 2024-02-22T19:48:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534521+0000", 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9019999999999999e-06 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534529+0000", 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.94264991600000003 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60045 11.2 11.f674bff2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.605298+0000", 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.836039542999998, 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92925496399999996, 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60045 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605298+0000", 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605298+0000", 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605298+0000", 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9304e-05 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605317+0000", 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1349999999999997e-06 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605323+0000", 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0729999999999998e-06 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605327+0000", 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6055e-05 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605363+0000", 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3700000000000003e-06 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.605369+0000", 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92917891799999996 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534548+0000", 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6579999999999999e-06 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534553+0000", 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.92925496399999996 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60059 11.2 11.ef1727e2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.615641+0000", 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.825696299000001, 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91893085600000002, 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60059 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615641+0000", 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615641+0000", 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615640+0000", 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3932000000000001e-05 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615664+0000", 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1218e-05 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615675+0000", 2024-02-22T19:48:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4239999999999998e-06 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615681+0000", 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2024000000000001e-05 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615743+0000", 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4109999999999996e-06 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.615752+0000", 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91881345800000003 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534565+0000", 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0790000000000003e-06 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534572+0000", 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91893085600000002 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60063 11.2 11.f72b72e2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.619584+0000", 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.821754147999997, 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91502285400000005, 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60063 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619584+0000", 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619584+0000", 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619583+0000", 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8232e-05 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619611+0000", 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6759999999999996e-06 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619620+0000", 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.003e-06 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619626+0000", 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3459000000000003e-05 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619689+0000", 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9369999999999998e-06 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.619696+0000", 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91490108699999995 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534597+0000", 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1400000000000006e-06 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534606+0000", 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.91502285400000005 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60073 11.2 11.e2482cf2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.633152+0000", 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.808185520000002, 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90147560400000004, 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60073 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633152+0000", 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633152+0000", 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633151+0000", 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-05 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633176+0000", 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3149999999999999e-05 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633190+0000", 2024-02-22T19:48:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1529999999999996e-06 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633195+0000", 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4418999999999997e-05 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633229+0000", 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.633235+0000", 2024-02-22T19:48:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90138672600000003 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534622+0000", 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1580000000000002e-06 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534628+0000", 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.90147560400000004 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60089 11.2 11.3f9a0a02 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.649892+0000", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.791446219999997, 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.88475915699999996, 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60089 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.649892+0000", 2024-02-22T19:48:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.649892+0000", 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.649891+0000", 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9226000000000001e-05 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.649920+0000", 2024-02-22T19:48:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7130000000000005e-06 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.649928+0000", 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3959999999999997e-06 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.649933+0000", 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043860449999999999 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.654319+0000", 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3499999999999997e-06 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.654328+0000", 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.88031669800000001 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534644+0000", 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534651+0000", 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.88475915699999996 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60096 11.2 11.cae7b442 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.670213+0000", 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.771124479999997, 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86446349, 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60096 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670213+0000", 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670213+0000", 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670211+0000", 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.461e-05 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670236+0000", 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3055e-05 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670249+0000", 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5740000000000003e-06 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670255+0000", 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018394200000000001 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670439+0000", 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2499999999999997e-06 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.670444+0000", 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86422556399999995 2024-02-22T19:48:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534669+0000", 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534677+0000", 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.86446349 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60117 11.2 11.3d926f62 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.702600+0000", 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.738737522000001, 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.83210206499999995, 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60117 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702600+0000", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702600+0000", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702599+0000", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3918e-05 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702623+0000", 2024-02-22T19:48:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2352e-05 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702635+0000", 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4160000000000001e-06 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702641+0000", 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9608999999999999e-05 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702691+0000", 2024-02-22T19:48:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0629999999999996e-06 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.702696+0000", 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.83199970300000003 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534696+0000", 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6490000000000002e-06 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534702+0000", 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.83210206499999995 2024-02-22T19:48:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60180 11.2 11.7be53932 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.777986+0000", 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.663351280000001, 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.75674195899999996, 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60180 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.777986+0000", 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.777986+0000", 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.777985+0000", 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6658000000000001e-05 2024-02-22T19:48:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.778011+0000", 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5305e-05 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.778027+0000", 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1980000000000004e-06 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.778034+0000", 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6045999999999999e-05 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.778080+0000", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.34e-05 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.778093+0000", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.75662811600000002 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534721+0000", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0940000000000002e-06 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534728+0000", 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.75674195899999996 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60195 11.2 11.ec2efea2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.790895+0000", 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.650442505000001, 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.74385985300000002, 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60195 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790895+0000", 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790895+0000", 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790893+0000", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7560999999999999e-05 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790921+0000", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4029e-05 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790945+0000", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2123000000000001e-05 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790957+0000", 2024-02-22T19:48:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6625999999999998e-05 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790993+0000", 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1869999999999996e-06 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.790999+0000", 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.74374852700000005 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534747+0000", 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9929999999999995e-06 2024-02-22T19:48:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534755+0000", 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.74385985300000002 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60199 11.2 11.f0b261f2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.794973+0000", 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.646364888000001, 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73979918099999997, 2024-02-22T19:48:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60199 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.794973+0000", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.794973+0000", 2024-02-22T19:48:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.794971+0000", 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1352000000000001e-05 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.795003+0000", 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2143999999999999e-05 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.795015+0000", 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9649999999999996e-06 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.795021+0000", 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1952999999999997e-05 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.795053+0000", 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9459999999999997e-06 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.795058+0000", 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73970973600000001 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534767+0000", 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6859999999999997e-06 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534772+0000", 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.73979918099999997 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60254 11.2 11.1d7b4342 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.877196+0000", 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.564141652000004, 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65758903999999996, 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60254 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877196+0000", 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877196+0000", 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877195+0000", 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8144e-05 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877223+0000", 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.257e-05 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877236+0000", 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1709999999999996e-06 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877243+0000", 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5617999999999997e-05 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877278+0000", 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0209999999999999e-06 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.877283+0000", 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.657498158 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534781+0000", 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6509999999999999e-06 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534785+0000", 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.65758903999999996 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.25652.0:60257 11.2 11.c860af2 (undecoded) ondisk+write+known_if_redirected e488)", 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-02-22T19:48:22.885714+0000", 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.555624082999998, 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64908654499999996, 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-02-22T19:48:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.25652", 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 60257 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885714+0000", 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885714+0000", 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885712+0000", 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1157e-05 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885743+0000", 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0208e-05 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885753+0000", 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6550000000000003e-06 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885758+0000", 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5057e-05 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885793+0000", 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9899999999999997e-06 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:22.885798+0000", 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64899882499999995 2024-02-22T19:48:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534797+0000", 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5379999999999998e-06 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-02-22T19:48:23.534800+0000", 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64908654499999996 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:48:56.577 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:48:56.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:48:56.579 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 70 16 82978 82962 73.9896 34.375 0.0212117 0.0135101 2024-02-22T19:48:56.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:48:55.475762+0000 mgr.y (mgr.14535) 758 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:56.673 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:48:55.475762+0000 mgr.y (mgr.14535) 758 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:56.677 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:48:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:48:55.475762+0000 mgr.y (mgr.14535) 758 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:48:57.507 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 71 16 83590 83574 73.4867 38.25 0.00902252 0.0135983 2024-02-22T19:48:57.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:48:57.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:55.832106+0000 mgr.y (mgr.14535) 759 : cluster [DBG] pgmap v1024: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.0 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.5 KiB/s rd, 53 MiB/s wr, 846 op/s; 1/246534 objects degraded (0.000%); 56176/246534 objects misplaced (22.786%); 48 MiB/s, 783 objects/s recovering 2024-02-22T19:48:57.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:55.832106+0000 mgr.y (mgr.14535) 759 : cluster [DBG] pgmap v1024: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.0 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.5 KiB/s rd, 53 MiB/s wr, 846 op/s; 1/246534 objects degraded (0.000%); 56176/246534 objects misplaced (22.786%); 48 MiB/s, 783 objects/s recovering 2024-02-22T19:48:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:55.832106+0000 mgr.y (mgr.14535) 759 : cluster [DBG] pgmap v1024: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.0 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 1.5 KiB/s rd, 53 MiB/s wr, 846 op/s; 1/246534 objects degraded (0.000%); 56176/246534 objects misplaced (22.786%); 48 MiB/s, 783 objects/s recovering 2024-02-22T19:48:58.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:57.559296+0000 mon.a (mon.0) 1788 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:48:58.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:57.559333+0000 mon.a (mon.0) 1789 : cluster [WRN] Health check update: Degraded data redundancy: 1/246534 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:58.508 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 72 16 84146 84130 72.9482 34.75 0.0374651 0.0137014 2024-02-22T19:48:58.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:57.559296+0000 mon.a (mon.0) 1788 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:48:58.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:57.559333+0000 mon.a (mon.0) 1789 : cluster [WRN] Health check update: Degraded data redundancy: 1/246534 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:57.559296+0000 mon.a (mon.0) 1788 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:48:58.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:57.559333+0000 mon.a (mon.0) 1789 : cluster [WRN] Health check update: Degraded data redundancy: 1/246534 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:48:59.512 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 73 16 84738 84722 72.4544 37 0.0266885 0.0137965 2024-02-22T19:48:59.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:48:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:48:59] "GET /metrics HTTP/1.1" 200 176718 "" "Prometheus/2.33.4" 2024-02-22T19:48:59.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:48:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:57.832579+0000 mgr.y (mgr.14535) 760 : cluster [DBG] pgmap v1025: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.0 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 639 B/s rd, 36 MiB/s wr, 580 op/s; 1/246534 objects degraded (0.000%); 56176/246534 objects misplaced (22.786%); 27 MiB/s, 440 objects/s recovering 2024-02-22T19:48:59.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:48:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:57.832579+0000 mgr.y (mgr.14535) 760 : cluster [DBG] pgmap v1025: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.0 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 639 B/s rd, 36 MiB/s wr, 580 op/s; 1/246534 objects degraded (0.000%); 56176/246534 objects misplaced (22.786%); 27 MiB/s, 440 objects/s recovering 2024-02-22T19:48:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:48:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:57.832579+0000 mgr.y (mgr.14535) 760 : cluster [DBG] pgmap v1025: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.0 GiB data, 15 GiB used, 432 GiB / 447 GiB avail; 639 B/s rd, 36 MiB/s wr, 580 op/s; 1/246534 objects degraded (0.000%); 56176/246534 objects misplaced (22.786%); 27 MiB/s, 440 objects/s recovering 2024-02-22T19:49:00.511 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 74 16 85353 85337 71.9945 38.4375 0.0413705 0.013883 2024-02-22T19:49:01.348 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:01.358 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:01.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:01.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:01.511 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 75 16 85948 85932 71.5308 37.1875 0.01537 0.0139744 2024-02-22T19:49:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:01.546 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:01.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:01.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:01.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:01.717 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:01.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:48:59.833469+0000 mgr.y (mgr.14535) 761 : cluster [DBG] pgmap v1026: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.0 KiB/s rd, 25 MiB/s wr, 396 op/s; 1/249774 objects degraded (0.000%); 53320/249774 objects misplaced (21.347%); 42 MiB/s, 668 objects/s recovering 2024-02-22T19:49:01.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:48:59.833469+0000 mgr.y (mgr.14535) 761 : cluster [DBG] pgmap v1026: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.0 KiB/s rd, 25 MiB/s wr, 396 op/s; 1/249774 objects degraded (0.000%); 53320/249774 objects misplaced (21.347%); 42 MiB/s, 668 objects/s recovering 2024-02-22T19:49:01.759 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:48:59.833469+0000 mgr.y (mgr.14535) 761 : cluster [DBG] pgmap v1026: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.1 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1.0 KiB/s rd, 25 MiB/s wr, 396 op/s; 1/249774 objects degraded (0.000%); 53320/249774 objects misplaced (21.347%); 42 MiB/s, 668 objects/s recovering 2024-02-22T19:49:01.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:01.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:01.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:01.912 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:01.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:01.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:02.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:02.109 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:02.157 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:02.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:02.168 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:02.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:02.321 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:02.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:02.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:02.458 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:02.499 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:02.511 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 76 16 86558 86542 71.0916 38.125 0.0238418 0.0140611 2024-02-22T19:49:02.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:02.538 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:02.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:49:02.595 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:49:02.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:02.688 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:02.716 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:02.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:01.834156+0000 mgr.y (mgr.14535) 762 : cluster [DBG] pgmap v1027: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.2 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1023 B/s rd, 36 MiB/s wr, 569 op/s; 1/255270 objects degraded (0.000%); 54177/255270 objects misplaced (21.223%); 42 MiB/s, 675 objects/s recovering 2024-02-22T19:49:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:02.083829+0000 mgr.y (mgr.14535) 763 : audit [DBG] from='client.15834 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:49:02.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:02.562355+0000 mon.a (mon.0) 1790 : cluster [WRN] Health check update: Degraded data redundancy: 1/255270 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:02.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:01.834156+0000 mgr.y (mgr.14535) 762 : cluster [DBG] pgmap v1027: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.2 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1023 B/s rd, 36 MiB/s wr, 569 op/s; 1/255270 objects degraded (0.000%); 54177/255270 objects misplaced (21.223%); 42 MiB/s, 675 objects/s recovering 2024-02-22T19:49:02.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:02.083829+0000 mgr.y (mgr.14535) 763 : audit [DBG] from='client.15834 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:49:02.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:02.562355+0000 mon.a (mon.0) 1790 : cluster [WRN] Health check update: Degraded data redundancy: 1/255270 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:02.818 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:01.834156+0000 mgr.y (mgr.14535) 762 : cluster [DBG] pgmap v1027: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.2 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 1023 B/s rd, 36 MiB/s wr, 569 op/s; 1/255270 objects degraded (0.000%); 54177/255270 objects misplaced (21.223%); 42 MiB/s, 675 objects/s recovering 2024-02-22T19:49:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:02.083829+0000 mgr.y (mgr.14535) 763 : audit [DBG] from='client.15834 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:49:02.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:02.562355+0000 mon.a (mon.0) 1790 : cluster [WRN] Health check update: Degraded data redundancy: 1/255270 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:03.514 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 77 16 87081 87065 70.5914 32.6875 0.0174728 0.0141597 2024-02-22T19:49:03.898 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:04.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:04.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:04.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:04.156 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:04.187 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:04.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:04.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:04.350 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:04.385 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:04.391 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:04.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:04.492 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:04.513 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 78 16 87703 87687 70.185 38.875 0.0300558 0.0142425 2024-02-22T19:49:04.559 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:04.633 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:04.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:04.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:04.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:49:04.814 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:49:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:49:05.274 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":1027,"stamp":"2024-02-22T19:49:01.833647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5563881124,"num_objects":85090,"num_object_clones":0,"num_object_copies":255270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":54177,"num_objects_unfound":0,"num_objects_dirty":85090,"num_whiteouts":0,"num_read":2293,"num_read_kb":2045,"num_write":85612,"num_write_kb":5433513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9426,"num_bytes_recovered":578442781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58767,"ondisk_log_size":58767,"up":414,"acting":410,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":415,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":468725760,"kb_used":16748012,"kb_used_data":12855428,"kb_used_omap":0,"kb_used_meta":3892544,"kb_avail":451977748,"statfs":{"total":479975178240,"available":462825213952,"internally_reserved":0,"allocated":13163958272,"data_stored":13154306517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3985965056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,7,8,4,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":372834304,"num_objects":5689,"num_object_clones":0,"num_object_copies":17067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3882,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5689,"num_write_kb":364096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6755,"num_bytes_recovered":442695680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004423"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":54,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089860+0000","last_change":"2024-02-22T19:48:43.995271+0000","last_active":"2024-02-22T19:48:50.089860+0000","last_peered":"2024-02-22T19:48:50.089860+0000","last_clean":"2024-02-22T19:48:50.089860+0000","last_became_active":"2024-02-22T19:48:43.994473+0000","last_became_peered":"2024-02-22T19:48:43.994473+0000","last_unstale":"2024-02-22T19:48:50.089860+0000","last_undegraded":"2024-02-22T19:48:50.089860+0000","last_fullsized":"2024-02-22T19:48:50.089860+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:13.092057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.092057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":506,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085502+0000","last_change":"2024-02-22T19:47:40.523932+0000","last_active":"2024-02-22T19:48:50.085502+0000","last_peered":"2024-02-22T19:48:50.085502+0000","last_clean":"2024-02-22T19:48:50.085502+0000","last_became_active":"2024-02-22T19:47:40.523572+0000","last_became_peered":"2024-02-22T19:47:40.523572+0000","last_unstale":"2024-02-22T19:48:50.085502+0000","last_undegraded":"2024-02-22T19:48:50.085502+0000","last_fullsized":"2024-02-22T19:48:50.085502+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:20.457620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:20.457620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"263'15","reported_seq":69,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089393+0000","last_change":"2024-02-22T19:48:47.024508+0000","last_active":"2024-02-22T19:48:50.089393+0000","last_peered":"2024-02-22T19:48:50.089393+0000","last_clean":"2024-02-22T19:48:50.089393+0000","last_became_active":"2024-02-22T19:48:43.993795+0000","last_became_peered":"2024-02-22T19:48:43.993795+0000","last_unstale":"2024-02-22T19:48:50.089393+0000","last_undegraded":"2024-02-22T19:48:50.089393+0000","last_fullsized":"2024-02-22T19:48:50.089393+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:46:09.901351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.901351+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":486,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.113687+0000","last_change":"2024-02-22T19:48:43.998720+0000","last_active":"2024-02-22T19:48:50.113687+0000","last_peered":"2024-02-22T19:48:50.113687+0000","last_clean":"2024-02-22T19:48:50.113687+0000","last_became_active":"2024-02-22T19:48:43.997531+0000","last_became_peered":"2024-02-22T19:48:43.997531+0000","last_unstale":"2024-02-22T19:48:50.113687+0000","last_undegraded":"2024-02-22T19:48:50.113687+0000","last_fullsized":"2024-02-22T19:48:50.113687+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:14.159012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:23.788367+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.159012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":492,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114772+0000","last_change":"2024-02-22T19:48:43.996836+0000","last_active":"2024-02-22T19:48:50.114772+0000","last_peered":"2024-02-22T19:48:50.114772+0000","last_clean":"2024-02-22T19:48:50.114772+0000","last_became_active":"2024-02-22T19:48:43.995519+0000","last_became_peered":"2024-02-22T19:48:43.995519+0000","last_unstale":"2024-02-22T19:48:50.114772+0000","last_undegraded":"2024-02-22T19:48:50.114772+0000","last_fullsized":"2024-02-22T19:48:50.114772+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:15.211416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:48:15.211416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'27","reported_seq":590,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.104253+0000","last_change":"2024-02-22T19:48:02.098450+0000","last_active":"2024-02-22T19:48:50.104253+0000","last_peered":"2024-02-22T19:48:50.104253+0000","last_clean":"2024-02-22T19:48:50.104253+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:48:50.104253+0000","last_undegraded":"2024-02-22T19:48:50.104253+0000","last_fullsized":"2024-02-22T19:48:50.104253+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:48:02.098338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.098338+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":68,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079292+0000","last_change":"2024-02-22T19:48:43.978679+0000","last_active":"2024-02-22T19:48:50.079292+0000","last_peered":"2024-02-22T19:48:50.079292+0000","last_clean":"2024-02-22T19:48:50.079292+0000","last_became_active":"2024-02-22T19:48:43.977270+0000","last_became_peered":"2024-02-22T19:48:43.977270+0000","last_unstale":"2024-02-22T19:48:50.079292+0000","last_undegraded":"2024-02-22T19:48:50.079292+0000","last_fullsized":"2024-02-22T19:48:50.079292+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.905878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.905878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":338,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084952+0000","last_change":"2024-02-22T19:48:43.986479+0000","last_active":"2024-02-22T19:48:50.084952+0000","last_peered":"2024-02-22T19:48:50.084952+0000","last_clean":"2024-02-22T19:48:50.084952+0000","last_became_active":"2024-02-22T19:48:43.985993+0000","last_became_peered":"2024-02-22T19:48:43.985993+0000","last_unstale":"2024-02-22T19:48:50.084952+0000","last_undegraded":"2024-02-22T19:48:50.084952+0000","last_fullsized":"2024-02-22T19:48:50.084952+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:52.699342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:52.699342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"302'19","reported_seq":55,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089081+0000","last_change":"2024-02-22T19:48:02.141583+0000","last_active":"2024-02-22T19:48:50.089081+0000","last_peered":"2024-02-22T19:48:50.089081+0000","last_clean":"2024-02-22T19:48:50.089081+0000","last_became_active":"2024-02-22T19:47:40.534475+0000","last_became_peered":"2024-02-22T19:47:40.534475+0000","last_unstale":"2024-02-22T19:48:50.089081+0000","last_undegraded":"2024-02-22T19:48:50.089081+0000","last_fullsized":"2024-02-22T19:48:50.089081+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:48:02.141476+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.141476+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":311,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083808+0000","last_change":"2024-02-22T19:48:43.988376+0000","last_active":"2024-02-22T19:48:50.083808+0000","last_peered":"2024-02-22T19:48:50.083808+0000","last_clean":"2024-02-22T19:48:50.083808+0000","last_became_active":"2024-02-22T19:48:43.988033+0000","last_became_peered":"2024-02-22T19:48:43.988033+0000","last_unstale":"2024-02-22T19:48:50.083808+0000","last_undegraded":"2024-02-22T19:48:50.083808+0000","last_fullsized":"2024-02-22T19:48:50.083808+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.818051+0000","last_clean_scrub_stamp":"2024-02-22T19:45:03.818051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"484'2","reported_seq":364,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084601+0000","last_change":"2024-02-22T19:48:44.050963+0000","last_active":"2024-02-22T19:48:50.084601+0000","last_peered":"2024-02-22T19:48:50.084601+0000","last_clean":"2024-02-22T19:48:50.084601+0000","last_became_active":"2024-02-22T19:48:43.986263+0000","last_became_peered":"2024-02-22T19:48:43.986263+0000","last_unstale":"2024-02-22T19:48:50.084601+0000","last_undegraded":"2024-02-22T19:48:50.084601+0000","last_fullsized":"2024-02-22T19:48:50.084601+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"484'2","last_scrub_stamp":"2024-02-22T19:48:14.828252+0000","last_deep_scrub":"484'2","last_deep_scrub_stamp":"2024-02-22T19:48:14.828252+0000","last_clean_scrub_stamp":"2024-02-22T19:48:14.828252+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1790,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"264'7","reported_seq":66,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090037+0000","last_change":"2024-02-22T19:48:46.217575+0000","last_active":"2024-02-22T19:48:50.090037+0000","last_peered":"2024-02-22T19:48:50.090037+0000","last_clean":"2024-02-22T19:48:50.090037+0000","last_became_active":"2024-02-22T19:48:43.992302+0000","last_became_peered":"2024-02-22T19:48:43.992302+0000","last_unstale":"2024-02-22T19:48:50.090037+0000","last_undegraded":"2024-02-22T19:48:50.090037+0000","last_fullsized":"2024-02-22T19:48:50.090037+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:47:50.219857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:50.219857+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":40,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090073+0000","last_change":"2024-02-22T19:47:40.535662+0000","last_active":"2024-02-22T19:48:50.090073+0000","last_peered":"2024-02-22T19:48:50.090073+0000","last_clean":"2024-02-22T19:48:50.090073+0000","last_became_active":"2024-02-22T19:47:40.535327+0000","last_became_peered":"2024-02-22T19:47:40.535327+0000","last_unstale":"2024-02-22T19:48:50.090073+0000","last_undegraded":"2024-02-22T19:48:50.090073+0000","last_fullsized":"2024-02-22T19:48:50.090073+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:18.486267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.486267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":54,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090125+0000","last_change":"2024-02-22T19:48:43.992999+0000","last_active":"2024-02-22T19:48:50.090125+0000","last_peered":"2024-02-22T19:48:50.090125+0000","last_clean":"2024-02-22T19:48:50.090125+0000","last_became_active":"2024-02-22T19:48:43.992573+0000","last_became_peered":"2024-02-22T19:48:43.992573+0000","last_unstale":"2024-02-22T19:48:50.090125+0000","last_undegraded":"2024-02-22T19:48:50.090125+0000","last_fullsized":"2024-02-22T19:48:50.090125+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:10.057585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:48:10.057585+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"264'34","reported_seq":567,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083754+0000","last_change":"2024-02-22T19:48:45.370319+0000","last_active":"2024-02-22T19:48:50.083754+0000","last_peered":"2024-02-22T19:48:50.083754+0000","last_clean":"2024-02-22T19:48:50.083754+0000","last_became_active":"2024-02-22T19:48:43.993351+0000","last_became_peered":"2024-02-22T19:48:43.993351+0000","last_unstale":"2024-02-22T19:48:50.083754+0000","last_undegraded":"2024-02-22T19:48:50.083754+0000","last_fullsized":"2024-02-22T19:48:50.083754+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:47:48.162902+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:47:48.162902+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":504,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114669+0000","last_change":"2024-02-22T19:48:43.981936+0000","last_active":"2024-02-22T19:48:50.114669+0000","last_peered":"2024-02-22T19:48:50.114669+0000","last_clean":"2024-02-22T19:48:50.114669+0000","last_became_active":"2024-02-22T19:48:43.981593+0000","last_became_peered":"2024-02-22T19:48:43.981593+0000","last_unstale":"2024-02-22T19:48:50.114669+0000","last_undegraded":"2024-02-22T19:48:50.114669+0000","last_fullsized":"2024-02-22T19:48:50.114669+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:14.832482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:28.335750+0000","last_clean_scrub_stamp":"2024-02-22T19:45:14.832482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":54,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089959+0000","last_change":"2024-02-22T19:48:43.993194+0000","last_active":"2024-02-22T19:48:50.089959+0000","last_peered":"2024-02-22T19:48:50.089959+0000","last_clean":"2024-02-22T19:48:50.089959+0000","last_became_active":"2024-02-22T19:48:43.992908+0000","last_became_peered":"2024-02-22T19:48:43.992908+0000","last_unstale":"2024-02-22T19:48:50.089959+0000","last_undegraded":"2024-02-22T19:48:50.089959+0000","last_fullsized":"2024-02-22T19:48:50.089959+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:01.129956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:48:01.129956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"264'23","reported_seq":593,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114549+0000","last_change":"2024-02-22T19:48:44.439594+0000","last_active":"2024-02-22T19:48:50.114549+0000","last_peered":"2024-02-22T19:48:50.114549+0000","last_clean":"2024-02-22T19:48:50.114549+0000","last_became_active":"2024-02-22T19:48:43.985256+0000","last_became_peered":"2024-02-22T19:48:43.985256+0000","last_unstale":"2024-02-22T19:48:50.114549+0000","last_undegraded":"2024-02-22T19:48:50.114549+0000","last_fullsized":"2024-02-22T19:48:50.114549+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:48:06.055130+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:48:06.055130+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"137'2","reported_seq":516,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.081121+0000","last_change":"2024-02-22T19:47:05.294685+0000","last_active":"2024-02-22T19:48:50.081121+0000","last_peered":"2024-02-22T19:48:50.081121+0000","last_clean":"2024-02-22T19:48:50.081121+0000","last_became_active":"2024-02-22T19:37:57.184717+0000","last_became_peered":"2024-02-22T19:37:57.184717+0000","last_unstale":"2024-02-22T19:48:50.081121+0000","last_undegraded":"2024-02-22T19:48:50.081121+0000","last_fullsized":"2024-02-22T19:48:50.081121+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:47:05.294613+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-02-22T19:34:03.590356+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.294613+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":488,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.103411+0000","last_change":"2024-02-22T19:48:44.003822+0000","last_active":"2024-02-22T19:48:50.103411+0000","last_peered":"2024-02-22T19:48:50.103411+0000","last_clean":"2024-02-22T19:48:50.103411+0000","last_became_active":"2024-02-22T19:48:44.003326+0000","last_became_peered":"2024-02-22T19:48:44.003326+0000","last_unstale":"2024-02-22T19:48:50.103411+0000","last_undegraded":"2024-02-22T19:48:50.103411+0000","last_fullsized":"2024-02-22T19:48:50.103411+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:02.967304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.967304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'20","reported_seq":340,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.081134+0000","last_change":"2024-02-22T19:48:46.677297+0000","last_active":"2024-02-22T19:48:50.081134+0000","last_peered":"2024-02-22T19:48:50.081134+0000","last_clean":"2024-02-22T19:48:50.081134+0000","last_became_active":"2024-02-22T19:48:43.987091+0000","last_became_peered":"2024-02-22T19:48:43.987091+0000","last_unstale":"2024-02-22T19:48:50.081134+0000","last_undegraded":"2024-02-22T19:48:50.081134+0000","last_fullsized":"2024-02-22T19:48:50.081134+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:12.323915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.323915+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":488,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084767+0000","last_change":"2024-02-22T19:48:43.990483+0000","last_active":"2024-02-22T19:48:50.084767+0000","last_peered":"2024-02-22T19:48:50.084767+0000","last_clean":"2024-02-22T19:48:50.084767+0000","last_became_active":"2024-02-22T19:48:43.990252+0000","last_became_peered":"2024-02-22T19:48:43.990252+0000","last_unstale":"2024-02-22T19:48:50.084767+0000","last_undegraded":"2024-02-22T19:48:50.084767+0000","last_fullsized":"2024-02-22T19:48:50.084767+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:59.995375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:59.995375+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":512,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.102906+0000","last_change":"2024-02-22T19:48:43.995825+0000","last_active":"2024-02-22T19:48:50.102906+0000","last_peered":"2024-02-22T19:48:50.102906+0000","last_clean":"2024-02-22T19:48:50.102906+0000","last_became_active":"2024-02-22T19:48:43.995471+0000","last_became_peered":"2024-02-22T19:48:43.995471+0000","last_unstale":"2024-02-22T19:48:50.102906+0000","last_undegraded":"2024-02-22T19:48:50.102906+0000","last_fullsized":"2024-02-22T19:48:50.102906+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:56.165274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:47:56.165274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"264'19","reported_seq":561,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085601+0000","last_change":"2024-02-22T19:48:44.004005+0000","last_active":"2024-02-22T19:48:50.085601+0000","last_peered":"2024-02-22T19:48:50.085601+0000","last_clean":"2024-02-22T19:48:50.085601+0000","last_became_active":"2024-02-22T19:48:43.981861+0000","last_became_peered":"2024-02-22T19:48:43.981861+0000","last_unstale":"2024-02-22T19:48:50.085601+0000","last_undegraded":"2024-02-22T19:48:50.085601+0000","last_fullsized":"2024-02-22T19:48:50.085601+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:09.809806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.809806+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":494,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.080362+0000","last_change":"2024-02-22T19:47:40.527176+0000","last_active":"2024-02-22T19:48:50.080362+0000","last_peered":"2024-02-22T19:48:50.080362+0000","last_clean":"2024-02-22T19:48:50.080362+0000","last_became_active":"2024-02-22T19:47:40.526670+0000","last_became_peered":"2024-02-22T19:47:40.526670+0000","last_unstale":"2024-02-22T19:48:50.080362+0000","last_undegraded":"2024-02-22T19:48:50.080362+0000","last_fullsized":"2024-02-22T19:48:50.080362+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.262153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.262153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":506,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085452+0000","last_change":"2024-02-22T19:48:43.989900+0000","last_active":"2024-02-22T19:48:50.085452+0000","last_peered":"2024-02-22T19:48:50.085452+0000","last_clean":"2024-02-22T19:48:50.085452+0000","last_became_active":"2024-02-22T19:48:43.989628+0000","last_became_peered":"2024-02-22T19:48:43.989628+0000","last_unstale":"2024-02-22T19:48:50.085452+0000","last_undegraded":"2024-02-22T19:48:50.085452+0000","last_fullsized":"2024-02-22T19:48:50.085452+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:09.768846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:09.768846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"263'8","reported_seq":529,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.103011+0000","last_change":"2024-02-22T19:48:44.956639+0000","last_active":"2024-02-22T19:48:50.103011+0000","last_peered":"2024-02-22T19:48:50.103011+0000","last_clean":"2024-02-22T19:48:50.103011+0000","last_became_active":"2024-02-22T19:48:44.004946+0000","last_became_peered":"2024-02-22T19:48:44.004946+0000","last_unstale":"2024-02-22T19:48:50.103011+0000","last_undegraded":"2024-02-22T19:48:50.103011+0000","last_fullsized":"2024-02-22T19:48:50.103011+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:46:09.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.018318+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":515,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.113772+0000","last_change":"2024-02-22T19:47:40.535567+0000","last_active":"2024-02-22T19:48:50.113772+0000","last_peered":"2024-02-22T19:48:50.113772+0000","last_clean":"2024-02-22T19:48:50.113772+0000","last_became_active":"2024-02-22T19:47:40.535155+0000","last_became_peered":"2024-02-22T19:47:40.535155+0000","last_unstale":"2024-02-22T19:48:50.113772+0000","last_undegraded":"2024-02-22T19:48:50.113772+0000","last_fullsized":"2024-02-22T19:48:50.113772+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:12.135980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.135980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":311,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084830+0000","last_change":"2024-02-22T19:48:43.980972+0000","last_active":"2024-02-22T19:48:50.084830+0000","last_peered":"2024-02-22T19:48:50.084830+0000","last_clean":"2024-02-22T19:48:50.084830+0000","last_became_active":"2024-02-22T19:48:43.980325+0000","last_became_peered":"2024-02-22T19:48:43.980325+0000","last_unstale":"2024-02-22T19:48:50.084830+0000","last_undegraded":"2024-02-22T19:48:50.084830+0000","last_fullsized":"2024-02-22T19:48:50.084830+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:12.759362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:12.759362+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"264'19","reported_seq":524,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.104078+0000","last_change":"2024-02-22T19:47:57.137077+0000","last_active":"2024-02-22T19:48:50.104078+0000","last_peered":"2024-02-22T19:48:50.104078+0000","last_clean":"2024-02-22T19:48:50.104078+0000","last_became_active":"2024-02-22T19:41:52.117581+0000","last_became_peered":"2024-02-22T19:41:52.117581+0000","last_unstale":"2024-02-22T19:48:50.104078+0000","last_undegraded":"2024-02-22T19:48:50.104078+0000","last_fullsized":"2024-02-22T19:48:50.104078+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:57.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:57.136969+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":497,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083637+0000","last_change":"2024-02-22T19:48:43.997819+0000","last_active":"2024-02-22T19:48:50.083637+0000","last_peered":"2024-02-22T19:48:50.083637+0000","last_clean":"2024-02-22T19:48:50.083637+0000","last_became_active":"2024-02-22T19:48:43.997521+0000","last_became_peered":"2024-02-22T19:48:43.997521+0000","last_unstale":"2024-02-22T19:48:50.083637+0000","last_undegraded":"2024-02-22T19:48:50.083637+0000","last_fullsized":"2024-02-22T19:48:50.083637+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:54.743150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.743150+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":534,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.080783+0000","last_change":"2024-02-22T19:48:03.835497+0000","last_active":"2024-02-22T19:48:50.080783+0000","last_peered":"2024-02-22T19:48:50.080783+0000","last_clean":"2024-02-22T19:48:50.080783+0000","last_became_active":"2024-02-22T19:47:40.538414+0000","last_became_peered":"2024-02-22T19:47:40.538414+0000","last_unstale":"2024-02-22T19:48:50.080783+0000","last_undegraded":"2024-02-22T19:48:50.080783+0000","last_fullsized":"2024-02-22T19:48:50.080783+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:03.835423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:48:03.835423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"264'15","reported_seq":280,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084887+0000","last_change":"2024-02-22T19:48:44.353952+0000","last_active":"2024-02-22T19:48:50.084887+0000","last_peered":"2024-02-22T19:48:50.084887+0000","last_clean":"2024-02-22T19:48:50.084887+0000","last_became_active":"2024-02-22T19:48:43.978701+0000","last_became_peered":"2024-02-22T19:48:43.978701+0000","last_unstale":"2024-02-22T19:48:50.084887+0000","last_undegraded":"2024-02-22T19:48:50.084887+0000","last_fullsized":"2024-02-22T19:48:50.084887+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:48:18.011868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:18.011868+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":513,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.103335+0000","last_change":"2024-02-22T19:44:52.796416+0000","last_active":"2024-02-22T19:48:50.103335+0000","last_peered":"2024-02-22T19:48:50.103335+0000","last_clean":"2024-02-22T19:48:50.103335+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:48:50.103335+0000","last_undegraded":"2024-02-22T19:48:50.103335+0000","last_fullsized":"2024-02-22T19:48:50.103335+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:52.796310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.796310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089763+0000","last_change":"2024-02-22T19:48:00.092551+0000","last_active":"2024-02-22T19:48:50.089763+0000","last_peered":"2024-02-22T19:48:50.089763+0000","last_clean":"2024-02-22T19:48:50.089763+0000","last_became_active":"2024-02-22T19:47:40.530829+0000","last_became_peered":"2024-02-22T19:47:40.530829+0000","last_unstale":"2024-02-22T19:48:50.089763+0000","last_undegraded":"2024-02-22T19:48:50.089763+0000","last_fullsized":"2024-02-22T19:48:50.089763+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:00.092448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:00.092448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"264'19","reported_seq":78,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089944+0000","last_change":"2024-02-22T19:48:47.971956+0000","last_active":"2024-02-22T19:48:50.089944+0000","last_peered":"2024-02-22T19:48:50.089944+0000","last_clean":"2024-02-22T19:48:50.089944+0000","last_became_active":"2024-02-22T19:48:43.996274+0000","last_became_peered":"2024-02-22T19:48:43.996274+0000","last_unstale":"2024-02-22T19:48:50.089944+0000","last_undegraded":"2024-02-22T19:48:50.089944+0000","last_fullsized":"2024-02-22T19:48:50.089944+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":496,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:48:12.096428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:12.096428+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":314,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.081017+0000","last_change":"2024-02-22T19:47:40.541670+0000","last_active":"2024-02-22T19:48:50.081017+0000","last_peered":"2024-02-22T19:48:50.081017+0000","last_clean":"2024-02-22T19:48:50.081017+0000","last_became_active":"2024-02-22T19:47:40.541359+0000","last_became_peered":"2024-02-22T19:47:40.541359+0000","last_unstale":"2024-02-22T19:48:50.081017+0000","last_undegraded":"2024-02-22T19:48:50.081017+0000","last_fullsized":"2024-02-22T19:48:50.081017+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:24.509271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.509271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":507,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.087211+0000","last_change":"2024-02-22T19:47:58.764502+0000","last_active":"2024-02-22T19:48:50.087211+0000","last_peered":"2024-02-22T19:48:50.087211+0000","last_clean":"2024-02-22T19:48:50.087211+0000","last_became_active":"2024-02-22T19:47:40.526305+0000","last_became_peered":"2024-02-22T19:47:40.526305+0000","last_unstale":"2024-02-22T19:48:50.087211+0000","last_undegraded":"2024-02-22T19:48:50.087211+0000","last_fullsized":"2024-02-22T19:48:50.087211+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:58.764403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:47:58.764403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"264'25","reported_seq":528,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114971+0000","last_change":"2024-02-22T19:48:44.569603+0000","last_active":"2024-02-22T19:48:50.114971+0000","last_peered":"2024-02-22T19:48:50.114971+0000","last_clean":"2024-02-22T19:48:50.114971+0000","last_became_active":"2024-02-22T19:48:43.995441+0000","last_became_peered":"2024-02-22T19:48:43.995441+0000","last_unstale":"2024-02-22T19:48:50.114971+0000","last_undegraded":"2024-02-22T19:48:50.114971+0000","last_fullsized":"2024-02-22T19:48:50.114971+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:46:10.060783+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:46:10.060783+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":40,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089389+0000","last_change":"2024-02-22T19:47:40.532181+0000","last_active":"2024-02-22T19:48:50.089389+0000","last_peered":"2024-02-22T19:48:50.089389+0000","last_clean":"2024-02-22T19:48:50.089389+0000","last_became_active":"2024-02-22T19:47:40.531839+0000","last_became_peered":"2024-02-22T19:47:40.531839+0000","last_unstale":"2024-02-22T19:48:50.089389+0000","last_undegraded":"2024-02-22T19:48:50.089389+0000","last_fullsized":"2024-02-22T19:48:50.089389+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:08.073147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:26.761226+0000","last_clean_scrub_stamp":"2024-02-22T19:47:08.073147+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":305,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079422+0000","last_change":"2024-02-22T19:48:43.986179+0000","last_active":"2024-02-22T19:48:50.079422+0000","last_peered":"2024-02-22T19:48:50.079422+0000","last_clean":"2024-02-22T19:48:50.079422+0000","last_became_active":"2024-02-22T19:48:43.984998+0000","last_became_peered":"2024-02-22T19:48:43.984998+0000","last_unstale":"2024-02-22T19:48:50.079422+0000","last_undegraded":"2024-02-22T19:48:50.079422+0000","last_fullsized":"2024-02-22T19:48:50.079422+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:13.706799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.706799+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"264'19","reported_seq":321,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085188+0000","last_change":"2024-02-22T19:47:07.646871+0000","last_active":"2024-02-22T19:48:50.085188+0000","last_peered":"2024-02-22T19:48:50.085188+0000","last_clean":"2024-02-22T19:48:50.085188+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:48:50.085188+0000","last_undegraded":"2024-02-22T19:48:50.085188+0000","last_fullsized":"2024-02-22T19:48:50.085188+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:07.646805+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:47:07.646805+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":522,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083986+0000","last_change":"2024-02-22T19:47:40.542335+0000","last_active":"2024-02-22T19:48:50.083986+0000","last_peered":"2024-02-22T19:48:50.083986+0000","last_clean":"2024-02-22T19:48:50.083986+0000","last_became_active":"2024-02-22T19:47:40.542055+0000","last_became_peered":"2024-02-22T19:47:40.542055+0000","last_unstale":"2024-02-22T19:48:50.083986+0000","last_undegraded":"2024-02-22T19:48:50.083986+0000","last_fullsized":"2024-02-22T19:48:50.083986+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:05.985178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.985178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":534,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079792+0000","last_change":"2024-02-22T19:48:05.818844+0000","last_active":"2024-02-22T19:48:50.079792+0000","last_peered":"2024-02-22T19:48:50.079792+0000","last_clean":"2024-02-22T19:48:50.079792+0000","last_became_active":"2024-02-22T19:47:40.539063+0000","last_became_peered":"2024-02-22T19:47:40.539063+0000","last_unstale":"2024-02-22T19:48:50.079792+0000","last_undegraded":"2024-02-22T19:48:50.079792+0000","last_fullsized":"2024-02-22T19:48:50.079792+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:05.818758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:05.818758+0000","last_clean_scrub_stamp":"2024-02-22T19:48:05.818758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"263'15","reported_seq":298,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114911+0000","last_change":"2024-02-22T19:48:14.196056+0000","last_active":"2024-02-22T19:48:50.114911+0000","last_peered":"2024-02-22T19:48:50.114911+0000","last_clean":"2024-02-22T19:48:50.114911+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:48:50.114911+0000","last_undegraded":"2024-02-22T19:48:50.114911+0000","last_fullsized":"2024-02-22T19:48:50.114911+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:48:14.195977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:14.195977+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":529,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.080542+0000","last_change":"2024-02-22T19:47:40.519744+0000","last_active":"2024-02-22T19:48:50.080542+0000","last_peered":"2024-02-22T19:48:50.080542+0000","last_clean":"2024-02-22T19:48:50.080542+0000","last_became_active":"2024-02-22T19:47:40.519396+0000","last_became_peered":"2024-02-22T19:47:40.519396+0000","last_unstale":"2024-02-22T19:48:50.080542+0000","last_undegraded":"2024-02-22T19:48:50.080542+0000","last_fullsized":"2024-02-22T19:48:50.080542+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.314522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.314522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":474,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083539+0000","last_change":"2024-02-22T19:48:16.904341+0000","last_active":"2024-02-22T19:48:50.083539+0000","last_peered":"2024-02-22T19:48:50.083539+0000","last_clean":"2024-02-22T19:48:50.083539+0000","last_became_active":"2024-02-22T19:41:52.112348+0000","last_became_peered":"2024-02-22T19:41:52.112348+0000","last_unstale":"2024-02-22T19:48:50.083539+0000","last_undegraded":"2024-02-22T19:48:50.083539+0000","last_fullsized":"2024-02-22T19:48:50.083539+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:16.904276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:48:16.904276+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"264'12","reported_seq":507,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114759+0000","last_change":"2024-02-22T19:45:01.840336+0000","last_active":"2024-02-22T19:48:50.114759+0000","last_peered":"2024-02-22T19:48:50.114759+0000","last_clean":"2024-02-22T19:48:50.114759+0000","last_became_active":"2024-02-22T19:36:26.384446+0000","last_became_peered":"2024-02-22T19:36:26.384446+0000","last_unstale":"2024-02-22T19:48:50.114759+0000","last_undegraded":"2024-02-22T19:48:50.114759+0000","last_fullsized":"2024-02-22T19:48:50.114759+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:45:01.840227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.840227+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"500'5303","reported_seq":11404,"reported_epoch":500,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:48:59.510792+0000","last_change":"2024-02-22T19:48:46.246752+0000","last_active":"2024-02-22T19:48:59.510792+0000","last_peered":"2024-02-22T19:48:59.510792+0000","last_clean":"2024-02-22T19:48:42.923989+0000","last_became_active":"2024-02-22T19:48:45.115397+0000","last_became_peered":"2024-02-22T19:48:45.115397+0000","last_unstale":"2024-02-22T19:48:59.510792+0000","last_undegraded":"2024-02-22T19:48:59.510792+0000","last_fullsized":"2024-02-22T19:48:59.510792+0000","mapping_epoch":494,"log_start":"477'1717","ondisk_log_start":"477'1717","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3586,"ondisk_log_size":3586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":347537408,"num_objects":5303,"num_object_clones":0,"num_object_copies":15909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":339392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3025,"num_bytes_recovered":198246400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,1],"avail_no_missing":["5","1","4"],"object_location_counts":[{"shards":"1,4,5","objects":5303}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":347,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084243+0000","last_change":"2024-02-22T19:48:43.990311+0000","last_active":"2024-02-22T19:48:50.084243+0000","last_peered":"2024-02-22T19:48:50.084243+0000","last_clean":"2024-02-22T19:48:50.084243+0000","last_became_active":"2024-02-22T19:48:43.989441+0000","last_became_peered":"2024-02-22T19:48:43.989441+0000","last_unstale":"2024-02-22T19:48:50.084243+0000","last_undegraded":"2024-02-22T19:48:50.084243+0000","last_fullsized":"2024-02-22T19:48:50.084243+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.702473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":600,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.086650+0000","last_change":"2024-02-22T19:48:44.004372+0000","last_active":"2024-02-22T19:48:50.086650+0000","last_peered":"2024-02-22T19:48:50.086650+0000","last_clean":"2024-02-22T19:48:50.086650+0000","last_became_active":"2024-02-22T19:48:43.987849+0000","last_became_peered":"2024-02-22T19:48:43.987849+0000","last_unstale":"2024-02-22T19:48:50.086650+0000","last_undegraded":"2024-02-22T19:48:50.086650+0000","last_fullsized":"2024-02-22T19:48:50.086650+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.375370+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"264'27","reported_seq":386,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083437+0000","last_change":"2024-02-22T19:47:41.126913+0000","last_active":"2024-02-22T19:48:50.083437+0000","last_peered":"2024-02-22T19:48:50.083437+0000","last_clean":"2024-02-22T19:48:50.083437+0000","last_became_active":"2024-02-22T19:47:40.521680+0000","last_became_peered":"2024-02-22T19:47:40.521680+0000","last_unstale":"2024-02-22T19:48:50.083437+0000","last_undegraded":"2024-02-22T19:48:50.083437+0000","last_fullsized":"2024-02-22T19:48:50.083437+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:03.485065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:03.485065+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"495'23","reported_seq":840,"reported_epoch":500,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:48:54.445063+0000","last_change":"2024-02-22T19:48:45.041617+0000","last_active":"2024-02-22T19:48:54.445063+0000","last_peered":"2024-02-22T19:48:54.445063+0000","last_clean":"2024-02-22T19:48:39.438260+0000","last_became_active":"2024-02-22T19:48:45.013082+0000","last_became_peered":"2024-02-22T19:48:45.013082+0000","last_unstale":"2024-02-22T19:48:54.445063+0000","last_undegraded":"2024-02-22T19:48:54.445063+0000","last_fullsized":"2024-02-22T19:48:54.445063+0000","mapping_epoch":494,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'16","last_scrub_stamp":"2024-02-22T19:46:15.312651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.312651+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,1],"avail_no_missing":["7","1","4"],"object_location_counts":[{"shards":"1,4,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"500'5399","reported_seq":5450,"reported_epoch":500,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:48:59.883349+0000","last_change":"2024-02-22T19:48:45.604035+0000","last_active":"2024-02-22T19:48:59.883349+0000","last_peered":"2024-02-22T19:48:59.883349+0000","last_clean":"2024-02-22T19:48:42.896311+0000","last_became_active":"2024-02-22T19:48:45.105713+0000","last_became_peered":"2024-02-22T19:48:45.105713+0000","last_unstale":"2024-02-22T19:48:59.883349+0000","last_undegraded":"2024-02-22T19:48:59.883349+0000","last_fullsized":"2024-02-22T19:48:59.883349+0000","mapping_epoch":494,"log_start":"476'1439","ondisk_log_start":"476'1439","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3960,"ondisk_log_size":3960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":353828864,"num_objects":5399,"num_object_clones":0,"num_object_copies":16197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5399,"num_objects_unfound":0,"num_objects_dirty":5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5399,"num_write_kb":345536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,1],"avail_no_missing":["2","1","4"],"object_location_counts":[{"shards":"1,2,4","objects":5399}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":477,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.113738+0000","last_change":"2024-02-22T19:48:43.998310+0000","last_active":"2024-02-22T19:48:50.113738+0000","last_peered":"2024-02-22T19:48:50.113738+0000","last_clean":"2024-02-22T19:48:50.113738+0000","last_became_active":"2024-02-22T19:48:43.997123+0000","last_became_peered":"2024-02-22T19:48:43.997123+0000","last_unstale":"2024-02-22T19:48:50.113738+0000","last_undegraded":"2024-02-22T19:48:50.113738+0000","last_fullsized":"2024-02-22T19:48:50.113738+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:12.345713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:12.345713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":230,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085358+0000","last_change":"2024-02-22T19:48:44.008745+0000","last_active":"2024-02-22T19:48:50.085358+0000","last_peered":"2024-02-22T19:48:50.085358+0000","last_clean":"2024-02-22T19:48:50.085358+0000","last_became_active":"2024-02-22T19:48:44.008400+0000","last_became_peered":"2024-02-22T19:48:44.008400+0000","last_unstale":"2024-02-22T19:48:50.085358+0000","last_undegraded":"2024-02-22T19:48:50.085358+0000","last_fullsized":"2024-02-22T19:48:50.085358+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.069465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.069465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"263'18","reported_seq":78,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089673+0000","last_change":"2024-02-22T19:48:46.043660+0000","last_active":"2024-02-22T19:48:50.089673+0000","last_peered":"2024-02-22T19:48:50.089673+0000","last_clean":"2024-02-22T19:48:50.089673+0000","last_became_active":"2024-02-22T19:48:43.992577+0000","last_became_peered":"2024-02-22T19:48:43.992577+0000","last_unstale":"2024-02-22T19:48:50.089673+0000","last_undegraded":"2024-02-22T19:48:50.089673+0000","last_fullsized":"2024-02-22T19:48:50.089673+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:48:05.127961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:05.127961+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":537,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.086152+0000","last_change":"2024-02-22T19:48:44.008032+0000","last_active":"2024-02-22T19:48:50.086152+0000","last_peered":"2024-02-22T19:48:50.086152+0000","last_clean":"2024-02-22T19:48:50.086152+0000","last_became_active":"2024-02-22T19:48:44.007225+0000","last_became_peered":"2024-02-22T19:48:44.007225+0000","last_unstale":"2024-02-22T19:48:50.086152+0000","last_undegraded":"2024-02-22T19:48:50.086152+0000","last_fullsized":"2024-02-22T19:48:50.086152+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:09.025536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.025536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"500'5397","reported_seq":5436,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:59.510943+0000","last_change":"2024-02-22T19:47:40.538749+0000","last_active":"2024-02-22T19:48:59.510943+0000","last_peered":"2024-02-22T19:48:59.510943+0000","last_clean":"2024-02-22T19:48:59.510943+0000","last_became_active":"2024-02-22T19:47:40.538446+0000","last_became_peered":"2024-02-22T19:47:40.538446+0000","last_unstale":"2024-02-22T19:48:59.510943+0000","last_undegraded":"2024-02-22T19:48:59.510943+0000","last_fullsized":"2024-02-22T19:48:59.510943+0000","mapping_epoch":469,"log_start":"476'1699","ondisk_log_start":"476'1699","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3698,"ondisk_log_size":3698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":353697792,"num_objects":5397,"num_object_clones":0,"num_object_copies":16191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":345408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":474,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079378+0000","last_change":"2024-02-22T19:48:02.833667+0000","last_active":"2024-02-22T19:48:50.079378+0000","last_peered":"2024-02-22T19:48:50.079378+0000","last_clean":"2024-02-22T19:48:50.079378+0000","last_became_active":"2024-02-22T19:29:07.104298+0000","last_became_peered":"2024-02-22T19:29:07.104298+0000","last_unstale":"2024-02-22T19:48:50.079378+0000","last_undegraded":"2024-02-22T19:48:50.079378+0000","last_fullsized":"2024-02-22T19:48:50.079378+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:02.833563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.833563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":559,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089235+0000","last_change":"2024-02-22T19:48:18.180488+0000","last_active":"2024-02-22T19:48:50.089235+0000","last_peered":"2024-02-22T19:48:50.089235+0000","last_clean":"2024-02-22T19:48:50.089235+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:48:50.089235+0000","last_undegraded":"2024-02-22T19:48:50.089235+0000","last_fullsized":"2024-02-22T19:48:50.089235+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:18.180384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:48:18.180384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":512,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.081033+0000","last_change":"2024-02-22T19:48:14.743750+0000","last_active":"2024-02-22T19:48:50.081033+0000","last_peered":"2024-02-22T19:48:50.081033+0000","last_clean":"2024-02-22T19:48:50.081033+0000","last_became_active":"2024-02-22T19:41:52.109412+0000","last_became_peered":"2024-02-22T19:41:52.109412+0000","last_unstale":"2024-02-22T19:48:50.081033+0000","last_undegraded":"2024-02-22T19:48:50.081033+0000","last_fullsized":"2024-02-22T19:48:50.081033+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:14.743654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:14.743654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"264'35","reported_seq":322,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079937+0000","last_change":"2024-02-22T19:48:44.784250+0000","last_active":"2024-02-22T19:48:50.079937+0000","last_peered":"2024-02-22T19:48:50.079937+0000","last_clean":"2024-02-22T19:48:50.079937+0000","last_became_active":"2024-02-22T19:48:43.986917+0000","last_became_peered":"2024-02-22T19:48:43.986917+0000","last_unstale":"2024-02-22T19:48:50.079937+0000","last_undegraded":"2024-02-22T19:48:50.079937+0000","last_fullsized":"2024-02-22T19:48:50.079937+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:47:47.887865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.887865+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.4","version":"500'5214","reported_seq":5522,"reported_epoch":500,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:48:59.511067+0000","last_change":"2024-02-22T19:48:48.428891+0000","last_active":"2024-02-22T19:48:59.511067+0000","last_peered":"2024-02-22T19:48:59.511067+0000","last_clean":"2024-02-22T19:48:42.869299+0000","last_became_active":"2024-02-22T19:48:45.256504+0000","last_became_peered":"2024-02-22T19:48:45.256504+0000","last_unstale":"2024-02-22T19:48:59.511067+0000","last_undegraded":"2024-02-22T19:48:59.511067+0000","last_fullsized":"2024-02-22T19:48:44.958350+0000","mapping_epoch":494,"log_start":"476'1548","ondisk_log_start":"476'1548","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3666,"ondisk_log_size":3666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":341704704,"num_objects":5214,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4576,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5214,"num_write_kb":333696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":8519680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2],"avail_no_missing":["7","1","2"],"object_location_counts":[{"shards":"1,2,7","objects":4576},{"shards":"2,7","objects":638}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":296,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079934+0000","last_change":"2024-02-22T19:47:55.877350+0000","last_active":"2024-02-22T19:48:50.079934+0000","last_peered":"2024-02-22T19:48:50.079934+0000","last_clean":"2024-02-22T19:48:50.079934+0000","last_became_active":"2024-02-22T19:39:22.253703+0000","last_became_peered":"2024-02-22T19:39:22.253703+0000","last_unstale":"2024-02-22T19:48:50.079934+0000","last_undegraded":"2024-02-22T19:48:50.079934+0000","last_fullsized":"2024-02-22T19:48:50.079934+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.877246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.877246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":542,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084959+0000","last_change":"2024-02-22T19:47:40.540971+0000","last_active":"2024-02-22T19:48:50.084959+0000","last_peered":"2024-02-22T19:48:50.084959+0000","last_clean":"2024-02-22T19:48:50.084959+0000","last_became_active":"2024-02-22T19:47:40.540572+0000","last_became_peered":"2024-02-22T19:47:40.540572+0000","last_unstale":"2024-02-22T19:48:50.084959+0000","last_undegraded":"2024-02-22T19:48:50.084959+0000","last_fullsized":"2024-02-22T19:48:50.084959+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:00.864208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.864208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":324,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084182+0000","last_change":"2024-02-22T19:48:43.996679+0000","last_active":"2024-02-22T19:48:50.084182+0000","last_peered":"2024-02-22T19:48:50.084182+0000","last_clean":"2024-02-22T19:48:50.084182+0000","last_became_active":"2024-02-22T19:48:43.996177+0000","last_became_peered":"2024-02-22T19:48:43.996177+0000","last_unstale":"2024-02-22T19:48:50.084182+0000","last_undegraded":"2024-02-22T19:48:50.084182+0000","last_fullsized":"2024-02-22T19:48:50.084182+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.617526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.617526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"302'19","reported_seq":311,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090098+0000","last_change":"2024-02-22T19:48:48.401208+0000","last_active":"2024-02-22T19:48:50.090098+0000","last_peered":"2024-02-22T19:48:50.090098+0000","last_clean":"2024-02-22T19:48:50.090098+0000","last_became_active":"2024-02-22T19:48:43.998071+0000","last_became_peered":"2024-02-22T19:48:43.998071+0000","last_unstale":"2024-02-22T19:48:50.090098+0000","last_undegraded":"2024-02-22T19:48:50.090098+0000","last_fullsized":"2024-02-22T19:48:50.090098+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:47:23.007849+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:47:23.007849+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.b","version":"500'5287","reported_seq":5607,"reported_epoch":500,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:49:00.029240+0000","last_change":"2024-02-22T19:48:48.420750+0000","last_active":"2024-02-22T19:49:00.029240+0000","last_peered":"2024-02-22T19:49:00.029240+0000","last_clean":"2024-02-22T19:48:42.902791+0000","last_became_active":"2024-02-22T19:48:45.119349+0000","last_became_peered":"2024-02-22T19:48:45.119349+0000","last_unstale":"2024-02-22T19:49:00.029240+0000","last_undegraded":"2024-02-22T19:49:00.029240+0000","last_fullsized":"2024-02-22T19:49:00.029240+0000","mapping_epoch":494,"log_start":"477'1923","ondisk_log_start":"477'1923","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3364,"ondisk_log_size":3364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":346488832,"num_objects":5287,"num_object_clones":0,"num_object_copies":15861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5150,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":338368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":8847360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,1],"avail_no_missing":["3","1","4"],"object_location_counts":[{"shards":"1,3,4","objects":5287}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":349,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083728+0000","last_change":"2024-02-22T19:47:40.522595+0000","last_active":"2024-02-22T19:48:50.083728+0000","last_peered":"2024-02-22T19:48:50.083728+0000","last_clean":"2024-02-22T19:48:50.083728+0000","last_became_active":"2024-02-22T19:47:40.522250+0000","last_became_peered":"2024-02-22T19:47:40.522250+0000","last_unstale":"2024-02-22T19:48:50.083728+0000","last_undegraded":"2024-02-22T19:48:50.083728+0000","last_fullsized":"2024-02-22T19:48:50.083728+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.428898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.428898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":495,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.115000+0000","last_change":"2024-02-22T19:47:40.545351+0000","last_active":"2024-02-22T19:48:50.115000+0000","last_peered":"2024-02-22T19:48:50.115000+0000","last_clean":"2024-02-22T19:48:50.115000+0000","last_became_active":"2024-02-22T19:47:40.544519+0000","last_became_peered":"2024-02-22T19:47:40.544519+0000","last_unstale":"2024-02-22T19:48:50.115000+0000","last_undegraded":"2024-02-22T19:48:50.115000+0000","last_fullsized":"2024-02-22T19:48:50.115000+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.030866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.030866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":320,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084257+0000","last_change":"2024-02-22T19:48:43.981014+0000","last_active":"2024-02-22T19:48:50.084257+0000","last_peered":"2024-02-22T19:48:50.084257+0000","last_clean":"2024-02-22T19:48:50.084257+0000","last_became_active":"2024-02-22T19:48:43.980668+0000","last_became_peered":"2024-02-22T19:48:43.980668+0000","last_unstale":"2024-02-22T19:48:50.084257+0000","last_undegraded":"2024-02-22T19:48:50.084257+0000","last_fullsized":"2024-02-22T19:48:50.084257+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.553324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.553324+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090214+0000","last_change":"2024-02-22T19:48:43.996805+0000","last_active":"2024-02-22T19:48:50.090214+0000","last_peered":"2024-02-22T19:48:50.090214+0000","last_clean":"2024-02-22T19:48:50.090214+0000","last_became_active":"2024-02-22T19:48:43.996234+0000","last_became_peered":"2024-02-22T19:48:43.996234+0000","last_unstale":"2024-02-22T19:48:50.090214+0000","last_undegraded":"2024-02-22T19:48:50.090214+0000","last_fullsized":"2024-02-22T19:48:50.090214+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.718312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.718312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"494'41","reported_seq":467,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:54.442876+0000","last_change":"2024-02-22T19:48:45.542159+0000","last_active":"2024-02-22T19:48:54.442876+0000","last_peered":"2024-02-22T19:48:54.442876+0000","last_clean":"2024-02-22T19:48:54.442876+0000","last_became_active":"2024-02-22T19:48:43.981474+0000","last_became_peered":"2024-02-22T19:48:43.981474+0000","last_unstale":"2024-02-22T19:48:54.442876+0000","last_undegraded":"2024-02-22T19:48:54.442876+0000","last_fullsized":"2024-02-22T19:48:54.442876+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:46:14.875950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.875950+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"499'32","reported_seq":474,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:54.444360+0000","last_change":"2024-02-22T19:48:50.111168+0000","last_active":"2024-02-22T19:48:54.444360+0000","last_peered":"2024-02-22T19:48:54.444360+0000","last_clean":"2024-02-22T19:48:54.444360+0000","last_became_active":"2024-02-22T19:48:50.109653+0000","last_became_peered":"2024-02-22T19:48:50.109653+0000","last_unstale":"2024-02-22T19:48:54.444360+0000","last_undegraded":"2024-02-22T19:48:54.444360+0000","last_fullsized":"2024-02-22T19:48:54.444360+0000","mapping_epoch":498,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":499,"parent":"0.0","parent_split_bits":0,"last_scrub":"473'24","last_scrub_stamp":"2024-02-22T19:48:13.962118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.962118+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.8","version":"500'5210","reported_seq":5249,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:49:00.019216+0000","last_change":"2024-02-22T19:47:40.532408+0000","last_active":"2024-02-22T19:49:00.019216+0000","last_peered":"2024-02-22T19:49:00.019216+0000","last_clean":"2024-02-22T19:49:00.019216+0000","last_became_active":"2024-02-22T19:47:40.532111+0000","last_became_peered":"2024-02-22T19:47:40.532111+0000","last_unstale":"2024-02-22T19:49:00.019216+0000","last_undegraded":"2024-02-22T19:49:00.019216+0000","last_fullsized":"2024-02-22T19:49:00.019216+0000","mapping_epoch":469,"log_start":"477'1896","ondisk_log_start":"477'1896","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3314,"ondisk_log_size":3314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":341442560,"num_objects":5210,"num_object_clones":0,"num_object_copies":15630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5210,"num_write_kb":333440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":542,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.080346+0000","last_change":"2024-02-22T19:47:40.526995+0000","last_active":"2024-02-22T19:48:50.080346+0000","last_peered":"2024-02-22T19:48:50.080346+0000","last_clean":"2024-02-22T19:48:50.080346+0000","last_became_active":"2024-02-22T19:47:40.526255+0000","last_became_peered":"2024-02-22T19:47:40.526255+0000","last_unstale":"2024-02-22T19:48:50.080346+0000","last_undegraded":"2024-02-22T19:48:50.080346+0000","last_fullsized":"2024-02-22T19:48:50.080346+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.507020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.507020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":483,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.081354+0000","last_change":"2024-02-22T19:47:15.447527+0000","last_active":"2024-02-22T19:48:50.081354+0000","last_peered":"2024-02-22T19:48:50.081354+0000","last_clean":"2024-02-22T19:48:50.081354+0000","last_became_active":"2024-02-22T19:41:52.109279+0000","last_became_peered":"2024-02-22T19:41:52.109279+0000","last_unstale":"2024-02-22T19:48:50.081354+0000","last_undegraded":"2024-02-22T19:48:50.081354+0000","last_fullsized":"2024-02-22T19:48:50.081354+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.447427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.447427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":318,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085918+0000","last_change":"2024-02-22T19:48:44.008225+0000","last_active":"2024-02-22T19:48:50.085918+0000","last_peered":"2024-02-22T19:48:50.085918+0000","last_clean":"2024-02-22T19:48:50.085918+0000","last_became_active":"2024-02-22T19:48:44.007701+0000","last_became_peered":"2024-02-22T19:48:44.007701+0000","last_unstale":"2024-02-22T19:48:50.085918+0000","last_undegraded":"2024-02-22T19:48:50.085918+0000","last_fullsized":"2024-02-22T19:48:50.085918+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:58.646210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.646210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083335+0000","last_change":"2024-02-22T19:47:40.533498+0000","last_active":"2024-02-22T19:48:50.083335+0000","last_peered":"2024-02-22T19:48:50.083335+0000","last_clean":"2024-02-22T19:48:50.083335+0000","last_became_active":"2024-02-22T19:47:40.533130+0000","last_became_peered":"2024-02-22T19:47:40.533130+0000","last_unstale":"2024-02-22T19:48:50.083335+0000","last_undegraded":"2024-02-22T19:48:50.083335+0000","last_fullsized":"2024-02-22T19:48:50.083335+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.081880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.081880+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"264'17","reported_seq":335,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085362+0000","last_change":"2024-02-22T19:48:44.047286+0000","last_active":"2024-02-22T19:48:50.085362+0000","last_peered":"2024-02-22T19:48:50.085362+0000","last_clean":"2024-02-22T19:48:50.085362+0000","last_became_active":"2024-02-22T19:48:43.982177+0000","last_became_peered":"2024-02-22T19:48:43.982177+0000","last_unstale":"2024-02-22T19:48:50.085362+0000","last_undegraded":"2024-02-22T19:48:50.085362+0000","last_fullsized":"2024-02-22T19:48:50.085362+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:46:58.874350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.874350+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":23,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"415'30","reported_seq":594,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:54.442625+0000","last_change":"2024-02-22T19:47:10.006289+0000","last_active":"2024-02-22T19:48:54.442625+0000","last_peered":"2024-02-22T19:48:54.442625+0000","last_clean":"2024-02-22T19:48:54.442625+0000","last_became_active":"2024-02-22T19:45:01.732679+0000","last_became_peered":"2024-02-22T19:45:01.732679+0000","last_unstale":"2024-02-22T19:48:54.442625+0000","last_undegraded":"2024-02-22T19:48:54.442625+0000","last_fullsized":"2024-02-22T19:48:54.442625+0000","mapping_epoch":217,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":415,"parent":"0.0","parent_split_bits":0,"last_scrub":"415'30","last_scrub_stamp":"2024-02-22T19:47:10.006180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.006180+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"500'5225","reported_seq":5277,"reported_epoch":500,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:49:00.039917+0000","last_change":"2024-02-22T19:48:45.977427+0000","last_active":"2024-02-22T19:49:00.039917+0000","last_peered":"2024-02-22T19:49:00.039917+0000","last_clean":"2024-02-22T19:48:42.917141+0000","last_became_active":"2024-02-22T19:48:45.091169+0000","last_became_peered":"2024-02-22T19:48:45.091169+0000","last_unstale":"2024-02-22T19:49:00.039917+0000","last_undegraded":"2024-02-22T19:49:00.039917+0000","last_fullsized":"2024-02-22T19:49:00.039917+0000","mapping_epoch":494,"log_start":"477'1888","ondisk_log_start":"477'1888","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3337,"ondisk_log_size":3337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":342425600,"num_objects":5225,"num_object_clones":0,"num_object_copies":15675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5225,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":334400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,1],"avail_no_missing":["3","1","5"],"object_location_counts":[{"shards":"1,3,5","objects":5225}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":325,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085634+0000","last_change":"2024-02-22T19:48:08.833500+0000","last_active":"2024-02-22T19:48:50.085634+0000","last_peered":"2024-02-22T19:48:50.085634+0000","last_clean":"2024-02-22T19:48:50.085634+0000","last_became_active":"2024-02-22T19:47:40.531699+0000","last_became_peered":"2024-02-22T19:47:40.531699+0000","last_unstale":"2024-02-22T19:48:50.085634+0000","last_undegraded":"2024-02-22T19:48:50.085634+0000","last_fullsized":"2024-02-22T19:48:50.085634+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:08.833380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:48:08.833380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":542,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.079550+0000","last_change":"2024-02-22T19:48:43.988776+0000","last_active":"2024-02-22T19:48:50.079550+0000","last_peered":"2024-02-22T19:48:50.079550+0000","last_clean":"2024-02-22T19:48:50.079550+0000","last_became_active":"2024-02-22T19:48:43.987992+0000","last_became_peered":"2024-02-22T19:48:43.987992+0000","last_unstale":"2024-02-22T19:48:50.079550+0000","last_undegraded":"2024-02-22T19:48:50.079550+0000","last_fullsized":"2024-02-22T19:48:50.079550+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_clean_scrub_stamp":"2024-02-22T19:47:26.529483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":510,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.109807+0000","last_change":"2024-02-22T19:47:40.534850+0000","last_active":"2024-02-22T19:48:50.109807+0000","last_peered":"2024-02-22T19:48:50.109807+0000","last_clean":"2024-02-22T19:48:50.109807+0000","last_became_active":"2024-02-22T19:47:40.534548+0000","last_became_peered":"2024-02-22T19:47:40.534548+0000","last_unstale":"2024-02-22T19:48:50.109807+0000","last_undegraded":"2024-02-22T19:48:50.109807+0000","last_fullsized":"2024-02-22T19:48:50.109807+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:25.123659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:47:25.123659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"264'33","reported_seq":56,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089408+0000","last_change":"2024-02-22T19:47:43.296238+0000","last_active":"2024-02-22T19:48:50.089408+0000","last_peered":"2024-02-22T19:48:50.089408+0000","last_clean":"2024-02-22T19:48:50.089408+0000","last_became_active":"2024-02-22T19:47:40.534650+0000","last_became_peered":"2024-02-22T19:47:40.534650+0000","last_unstale":"2024-02-22T19:48:50.089408+0000","last_undegraded":"2024-02-22T19:48:50.089408+0000","last_fullsized":"2024-02-22T19:48:50.089408+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":472,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:46:12.320306+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.320306+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"423'13","reported_seq":585,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:54.449007+0000","last_change":"2024-02-22T19:47:54.799210+0000","last_active":"2024-02-22T19:48:54.449007+0000","last_peered":"2024-02-22T19:48:54.449007+0000","last_clean":"2024-02-22T19:48:54.449007+0000","last_became_active":"2024-02-22T19:45:09.547565+0000","last_became_peered":"2024-02-22T19:45:09.547565+0000","last_unstale":"2024-02-22T19:48:54.449007+0000","last_undegraded":"2024-02-22T19:48:54.449007+0000","last_fullsized":"2024-02-22T19:48:54.449007+0000","mapping_epoch":217,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"423'13","last_scrub_stamp":"2024-02-22T19:47:54.799098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.799098+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"500'5388","reported_seq":5441,"reported_epoch":500,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:48:59.977289+0000","last_change":"2024-02-22T19:48:45.612924+0000","last_active":"2024-02-22T19:48:59.977289+0000","last_peered":"2024-02-22T19:48:59.977289+0000","last_clean":"2024-02-22T19:48:42.912584+0000","last_became_active":"2024-02-22T19:48:45.117609+0000","last_became_peered":"2024-02-22T19:48:45.117609+0000","last_unstale":"2024-02-22T19:48:59.977289+0000","last_undegraded":"2024-02-22T19:48:59.977289+0000","last_fullsized":"2024-02-22T19:48:59.977289+0000","mapping_epoch":494,"log_start":"478'2007","ondisk_log_start":"478'2007","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3381,"ondisk_log_size":3381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":353107968,"num_objects":5388,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5388,"num_objects_unfound":0,"num_objects_dirty":5388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":344832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,1],"avail_no_missing":["3","1","7"],"object_location_counts":[{"shards":"1,3,7","objects":5388}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":370,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089602+0000","last_change":"2024-02-22T19:48:43.997139+0000","last_active":"2024-02-22T19:48:50.089602+0000","last_peered":"2024-02-22T19:48:50.089602+0000","last_clean":"2024-02-22T19:48:50.089602+0000","last_became_active":"2024-02-22T19:48:43.996694+0000","last_became_peered":"2024-02-22T19:48:43.996694+0000","last_unstale":"2024-02-22T19:48:50.089602+0000","last_undegraded":"2024-02-22T19:48:50.089602+0000","last_fullsized":"2024-02-22T19:48:50.089602+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.926641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.926641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":206,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084016+0000","last_change":"2024-02-22T19:48:43.987243+0000","last_active":"2024-02-22T19:48:50.084016+0000","last_peered":"2024-02-22T19:48:50.084016+0000","last_clean":"2024-02-22T19:48:50.084016+0000","last_became_active":"2024-02-22T19:48:43.986896+0000","last_became_peered":"2024-02-22T19:48:43.986896+0000","last_unstale":"2024-02-22T19:48:50.084016+0000","last_undegraded":"2024-02-22T19:48:50.084016+0000","last_fullsized":"2024-02-22T19:48:50.084016+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.792736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.792736+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089096+0000","last_change":"2024-02-22T19:47:40.536647+0000","last_active":"2024-02-22T19:48:50.089096+0000","last_peered":"2024-02-22T19:48:50.089096+0000","last_clean":"2024-02-22T19:48:50.089096+0000","last_became_active":"2024-02-22T19:47:40.535376+0000","last_became_peered":"2024-02-22T19:47:40.535376+0000","last_unstale":"2024-02-22T19:48:50.089096+0000","last_undegraded":"2024-02-22T19:48:50.089096+0000","last_fullsized":"2024-02-22T19:48:50.089096+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:16.015884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.015884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1730,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:58.034194+0000","last_change":"2024-02-22T19:48:46.765288+0000","last_active":"2024-02-22T19:48:58.034194+0000","last_peered":"2024-02-22T19:48:58.034194+0000","last_clean":"2024-02-22T19:48:58.034194+0000","last_became_active":"2024-02-22T19:48:43.987852+0000","last_became_peered":"2024-02-22T19:48:43.987852+0000","last_unstale":"2024-02-22T19:48:58.034194+0000","last_undegraded":"2024-02-22T19:48:58.034194+0000","last_fullsized":"2024-02-22T19:48:58.034194+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:47:19.438615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:19.438615+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1180,"num_read_kb":1175,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"264'24","reported_seq":75,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085493+0000","last_change":"2024-02-22T19:48:45.600114+0000","last_active":"2024-02-22T19:48:50.085493+0000","last_peered":"2024-02-22T19:48:50.085493+0000","last_clean":"2024-02-22T19:48:50.085493+0000","last_became_active":"2024-02-22T19:48:44.000675+0000","last_became_peered":"2024-02-22T19:48:44.000675+0000","last_unstale":"2024-02-22T19:48:50.085493+0000","last_undegraded":"2024-02-22T19:48:50.085493+0000","last_fullsized":"2024-02-22T19:48:50.085493+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:47:49.907139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:49.907139+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"498'31","reported_seq":477,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:54.443134+0000","last_change":"2024-02-22T19:48:48.044030+0000","last_active":"2024-02-22T19:48:54.443134+0000","last_peered":"2024-02-22T19:48:54.443134+0000","last_clean":"2024-02-22T19:48:54.443134+0000","last_became_active":"2024-02-22T19:48:48.043639+0000","last_became_peered":"2024-02-22T19:48:48.043639+0000","last_unstale":"2024-02-22T19:48:54.443134+0000","last_undegraded":"2024-02-22T19:48:54.443134+0000","last_fullsized":"2024-02-22T19:48:54.443134+0000","mapping_epoch":496,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'25","last_scrub_stamp":"2024-02-22T19:45:06.780605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:45:06.780605+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"500'5356","reported_seq":5724,"reported_epoch":500,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:48:59.851541+0000","last_change":"2024-02-22T19:48:48.412428+0000","last_active":"2024-02-22T19:48:59.851541+0000","last_peered":"2024-02-22T19:48:59.851541+0000","last_clean":"2024-02-22T19:48:42.896903+0000","last_became_active":"2024-02-22T19:48:45.153305+0000","last_became_peered":"2024-02-22T19:48:45.153305+0000","last_unstale":"2024-02-22T19:48:59.851541+0000","last_undegraded":"2024-02-22T19:48:59.851541+0000","last_fullsized":"2024-02-22T19:48:59.851541+0000","mapping_epoch":494,"log_start":"476'1373","ondisk_log_start":"476'1373","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3983,"ondisk_log_size":3983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":351010816,"num_objects":5356,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5192,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5356,"num_write_kb":342784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":10354688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,1],"avail_no_missing":["2","1","4"],"object_location_counts":[{"shards":"1,2,4","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":367,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114697+0000","last_change":"2024-02-22T19:47:40.545400+0000","last_active":"2024-02-22T19:48:50.114697+0000","last_peered":"2024-02-22T19:48:50.114697+0000","last_clean":"2024-02-22T19:48:50.114697+0000","last_became_active":"2024-02-22T19:47:40.544234+0000","last_became_peered":"2024-02-22T19:47:40.544234+0000","last_unstale":"2024-02-22T19:48:50.114697+0000","last_undegraded":"2024-02-22T19:48:50.114697+0000","last_fullsized":"2024-02-22T19:48:50.114697+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.024753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.024753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":307,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083901+0000","last_change":"2024-02-22T19:48:43.986718+0000","last_active":"2024-02-22T19:48:50.083901+0000","last_peered":"2024-02-22T19:48:50.083901+0000","last_clean":"2024-02-22T19:48:50.083901+0000","last_became_active":"2024-02-22T19:48:43.986337+0000","last_became_peered":"2024-02-22T19:48:43.986337+0000","last_unstale":"2024-02-22T19:48:50.083901+0000","last_undegraded":"2024-02-22T19:48:50.083901+0000","last_fullsized":"2024-02-22T19:48:50.083901+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:53.891414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:47:53.891414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":212,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083834+0000","last_change":"2024-02-22T19:48:43.993764+0000","last_active":"2024-02-22T19:48:50.083834+0000","last_peered":"2024-02-22T19:48:50.083834+0000","last_clean":"2024-02-22T19:48:50.083834+0000","last_became_active":"2024-02-22T19:48:43.993004+0000","last_became_peered":"2024-02-22T19:48:43.993004+0000","last_unstale":"2024-02-22T19:48:50.083834+0000","last_undegraded":"2024-02-22T19:48:50.083834+0000","last_fullsized":"2024-02-22T19:48:50.083834+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:46.844819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:47:46.844819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"463'37","reported_seq":356,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084197+0000","last_change":"2024-02-22T19:48:45.053337+0000","last_active":"2024-02-22T19:48:50.084197+0000","last_peered":"2024-02-22T19:48:50.084197+0000","last_clean":"2024-02-22T19:48:50.084197+0000","last_became_active":"2024-02-22T19:48:43.988945+0000","last_became_peered":"2024-02-22T19:48:43.988945+0000","last_unstale":"2024-02-22T19:48:50.084197+0000","last_undegraded":"2024-02-22T19:48:50.084197+0000","last_fullsized":"2024-02-22T19:48:50.084197+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"463'37","last_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_deep_scrub":"463'37","last_deep_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.474452+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":483,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083975+0000","last_change":"2024-02-22T19:44:54.804568+0000","last_active":"2024-02-22T19:48:50.083975+0000","last_peered":"2024-02-22T19:48:50.083975+0000","last_clean":"2024-02-22T19:48:50.083975+0000","last_became_active":"2024-02-22T19:44:45.398192+0000","last_became_peered":"2024-02-22T19:44:45.398192+0000","last_unstale":"2024-02-22T19:48:50.083975+0000","last_undegraded":"2024-02-22T19:48:50.083975+0000","last_fullsized":"2024-02-22T19:48:50.083975+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":399,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:54.804469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:54.804469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"500'5280","reported_seq":5659,"reported_epoch":500,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-02-22T19:48:59.856418+0000","last_change":"2024-02-22T19:48:45.613560+0000","last_active":"2024-02-22T19:48:59.856418+0000","last_peered":"2024-02-22T19:48:59.856418+0000","last_clean":"2024-02-22T19:48:42.880332+0000","last_became_active":"2024-02-22T19:48:45.204735+0000","last_became_peered":"2024-02-22T19:48:45.204735+0000","last_unstale":"2024-02-22T19:48:59.856418+0000","last_undegraded":"2024-02-22T19:48:45.007485+0000","last_fullsized":"2024-02-22T19:48:44.965703+0000","mapping_epoch":494,"log_start":"476'1312","ondisk_log_start":"476'1312","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3968,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":346030080,"num_objects":5280,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4595,"num_objects_unfound":0,"num_objects_dirty":5280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5280,"num_write_kb":337920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":10878976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4],"avail_no_missing":["2","1","4"],"object_location_counts":[{"shards":"1,2,4","objects":4595},{"shards":"2,4","objects":684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":215,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114170+0000","last_change":"2024-02-22T19:48:13.199988+0000","last_active":"2024-02-22T19:48:50.114170+0000","last_peered":"2024-02-22T19:48:50.114170+0000","last_clean":"2024-02-22T19:48:50.114170+0000","last_became_active":"2024-02-22T19:41:52.109694+0000","last_became_peered":"2024-02-22T19:41:52.109694+0000","last_unstale":"2024-02-22T19:48:50.114170+0000","last_undegraded":"2024-02-22T19:48:50.114170+0000","last_fullsized":"2024-02-22T19:48:50.114170+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:13.199920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:13.199920+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.199920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":296,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.102866+0000","last_change":"2024-02-22T19:48:04.068652+0000","last_active":"2024-02-22T19:48:50.102866+0000","last_peered":"2024-02-22T19:48:50.102866+0000","last_clean":"2024-02-22T19:48:50.102866+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:48:50.102866+0000","last_undegraded":"2024-02-22T19:48:50.102866+0000","last_fullsized":"2024-02-22T19:48:50.102866+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:04.068584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:48:04.068584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":302,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.085107+0000","last_change":"2024-02-22T19:46:19.538051+0000","last_active":"2024-02-22T19:48:50.085107+0000","last_peered":"2024-02-22T19:48:50.085107+0000","last_clean":"2024-02-22T19:48:50.085107+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:48:50.085107+0000","last_undegraded":"2024-02-22T19:48:50.085107+0000","last_fullsized":"2024-02-22T19:48:50.085107+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:19.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:46:19.537988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'20","reported_seq":371,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083871+0000","last_change":"2024-02-22T19:48:45.386827+0000","last_active":"2024-02-22T19:48:50.083871+0000","last_peered":"2024-02-22T19:48:50.083871+0000","last_clean":"2024-02-22T19:48:50.083871+0000","last_became_active":"2024-02-22T19:48:43.989903+0000","last_became_peered":"2024-02-22T19:48:43.989903+0000","last_unstale":"2024-02-22T19:48:50.083871+0000","last_undegraded":"2024-02-22T19:48:50.083871+0000","last_fullsized":"2024-02-22T19:48:50.083871+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:46:18.828884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.828884+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"498'19","reported_seq":638,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:48:54.443348+0000","last_change":"2024-02-22T19:48:48.053978+0000","last_active":"2024-02-22T19:48:54.443348+0000","last_peered":"2024-02-22T19:48:54.443348+0000","last_clean":"2024-02-22T19:48:54.443348+0000","last_became_active":"2024-02-22T19:48:48.053557+0000","last_became_peered":"2024-02-22T19:48:48.053557+0000","last_unstale":"2024-02-22T19:48:54.443348+0000","last_undegraded":"2024-02-22T19:48:54.443348+0000","last_fullsized":"2024-02-22T19:48:54.443348+0000","mapping_epoch":496,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'16","last_scrub_stamp":"2024-02-22T19:46:58.513473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.513473+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.e","version":"500'5253","reported_seq":7545,"reported_epoch":500,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:48:59.349648+0000","last_change":"2024-02-22T19:48:48.675530+0000","last_active":"2024-02-22T19:48:59.349648+0000","last_peered":"2024-02-22T19:48:59.349648+0000","last_clean":"2024-02-22T19:48:42.871723+0000","last_became_active":"2024-02-22T19:48:45.099086+0000","last_became_peered":"2024-02-22T19:48:45.099086+0000","last_unstale":"2024-02-22T19:48:59.349648+0000","last_undegraded":"2024-02-22T19:48:59.349648+0000","last_fullsized":"2024-02-22T19:48:59.349648+0000","mapping_epoch":494,"log_start":"476'1519","ondisk_log_start":"476'1519","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3734,"ondisk_log_size":3734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":344260608,"num_objects":5253,"num_object_clones":0,"num_object_copies":15759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4090,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5253,"num_write_kb":336192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":73465856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,1],"avail_no_missing":["4","1","7"],"object_location_counts":[{"shards":"1,4,7","objects":5253}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":387,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.113440+0000","last_change":"2024-02-22T19:47:18.018282+0000","last_active":"2024-02-22T19:48:50.113440+0000","last_peered":"2024-02-22T19:48:50.113440+0000","last_clean":"2024-02-22T19:48:50.113440+0000","last_became_active":"2024-02-22T19:41:52.103696+0000","last_became_peered":"2024-02-22T19:41:52.103696+0000","last_unstale":"2024-02-22T19:48:50.113440+0000","last_undegraded":"2024-02-22T19:48:50.113440+0000","last_fullsized":"2024-02-22T19:48:50.113440+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:18.018184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:18.018184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":495,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.087438+0000","last_change":"2024-02-22T19:48:43.987656+0000","last_active":"2024-02-22T19:48:50.087438+0000","last_peered":"2024-02-22T19:48:50.087438+0000","last_clean":"2024-02-22T19:48:50.087438+0000","last_became_active":"2024-02-22T19:48:43.987257+0000","last_became_peered":"2024-02-22T19:48:43.987257+0000","last_unstale":"2024-02-22T19:48:50.087438+0000","last_undegraded":"2024-02-22T19:48:50.087438+0000","last_fullsized":"2024-02-22T19:48:50.087438+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:47.829395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.829395+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":288,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090218+0000","last_change":"2024-02-22T19:48:43.995840+0000","last_active":"2024-02-22T19:48:50.090218+0000","last_peered":"2024-02-22T19:48:50.090218+0000","last_clean":"2024-02-22T19:48:50.090218+0000","last_became_active":"2024-02-22T19:48:43.995080+0000","last_became_peered":"2024-02-22T19:48:43.995080+0000","last_unstale":"2024-02-22T19:48:50.090218+0000","last_undegraded":"2024-02-22T19:48:50.090218+0000","last_fullsized":"2024-02-22T19:48:50.090218+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:54.924175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.924175+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"264'35","reported_seq":61,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089195+0000","last_change":"2024-02-22T19:47:51.225194+0000","last_active":"2024-02-22T19:48:50.089195+0000","last_peered":"2024-02-22T19:48:50.089195+0000","last_clean":"2024-02-22T19:48:50.089195+0000","last_became_active":"2024-02-22T19:47:40.521013+0000","last_became_peered":"2024-02-22T19:47:40.521013+0000","last_unstale":"2024-02-22T19:48:50.089195+0000","last_undegraded":"2024-02-22T19:48:50.089195+0000","last_fullsized":"2024-02-22T19:48:50.089195+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:47:51.225129+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:47:51.225129+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"495'18","reported_seq":813,"reported_epoch":500,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:48:54.444534+0000","last_change":"2024-02-22T19:48:45.105853+0000","last_active":"2024-02-22T19:48:54.444534+0000","last_peered":"2024-02-22T19:48:54.444534+0000","last_clean":"2024-02-22T19:48:39.438258+0000","last_became_active":"2024-02-22T19:48:45.074946+0000","last_became_peered":"2024-02-22T19:48:45.074946+0000","last_unstale":"2024-02-22T19:48:54.444534+0000","last_undegraded":"2024-02-22T19:48:54.444534+0000","last_fullsized":"2024-02-22T19:48:54.444534+0000","mapping_epoch":494,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:44:53.757594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.757594+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,1],"avail_no_missing":["3","1","4"],"object_location_counts":[{"shards":"1,3,4","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"500'5434","reported_seq":12603,"reported_epoch":500,"state":"active+remapped+backfilling","last_fresh":"2024-02-22T19:48:59.530437+0000","last_change":"2024-02-22T19:48:45.199564+0000","last_active":"2024-02-22T19:48:59.530437+0000","last_peered":"2024-02-22T19:48:59.530437+0000","last_clean":"2024-02-22T19:48:42.933736+0000","last_became_active":"2024-02-22T19:48:45.103451+0000","last_became_peered":"2024-02-22T19:48:45.103451+0000","last_unstale":"2024-02-22T19:48:59.530437+0000","last_undegraded":"2024-02-22T19:48:59.530437+0000","last_fullsized":"2024-02-22T19:48:59.530437+0000","mapping_epoch":494,"log_start":"476'1744","ondisk_log_start":"476'1744","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3690,"ondisk_log_size":3690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":356122624,"num_objects":5434,"num_object_clones":0,"num_object_copies":16302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":5434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5434,"num_write_kb":347776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3560,"num_bytes_recovered":233308160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,1],"avail_no_missing":["7","1","2"],"object_location_counts":[{"shards":"1,2,7","objects":5434}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089335+0000","last_change":"2024-02-22T19:47:40.532231+0000","last_active":"2024-02-22T19:48:50.089335+0000","last_peered":"2024-02-22T19:48:50.089335+0000","last_clean":"2024-02-22T19:48:50.089335+0000","last_became_active":"2024-02-22T19:47:40.531852+0000","last_became_peered":"2024-02-22T19:47:40.531852+0000","last_unstale":"2024-02-22T19:48:50.089335+0000","last_undegraded":"2024-02-22T19:48:50.089335+0000","last_fullsized":"2024-02-22T19:48:50.089335+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.070731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.070731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":45,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090040+0000","last_change":"2024-02-22T19:48:15.061095+0000","last_active":"2024-02-22T19:48:50.090040+0000","last_peered":"2024-02-22T19:48:50.090040+0000","last_clean":"2024-02-22T19:48:50.090040+0000","last_became_active":"2024-02-22T19:47:40.534711+0000","last_became_peered":"2024-02-22T19:47:40.534711+0000","last_unstale":"2024-02-22T19:48:50.090040+0000","last_undegraded":"2024-02-22T19:48:50.090040+0000","last_fullsized":"2024-02-22T19:48:50.090040+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:15.061001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:15.061001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":501,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114431+0000","last_change":"2024-02-22T19:45:58.995918+0000","last_active":"2024-02-22T19:48:50.114431+0000","last_peered":"2024-02-22T19:48:50.114431+0000","last_clean":"2024-02-22T19:48:50.114431+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:48:50.114431+0000","last_undegraded":"2024-02-22T19:48:50.114431+0000","last_fullsized":"2024-02-22T19:48:50.114431+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:58.995851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:45:58.995851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'30","reported_seq":632,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.105818+0000","last_change":"2024-02-22T19:48:46.245669+0000","last_active":"2024-02-22T19:48:50.105818+0000","last_peered":"2024-02-22T19:48:50.105818+0000","last_clean":"2024-02-22T19:48:50.105818+0000","last_became_active":"2024-02-22T19:48:44.004840+0000","last_became_peered":"2024-02-22T19:48:44.004840+0000","last_unstale":"2024-02-22T19:48:50.105818+0000","last_undegraded":"2024-02-22T19:48:50.105818+0000","last_fullsized":"2024-02-22T19:48:50.105818+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:45:09.692661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.692661+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":45,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089738+0000","last_change":"2024-02-22T19:47:55.133259+0000","last_active":"2024-02-22T19:48:50.089738+0000","last_peered":"2024-02-22T19:48:50.089738+0000","last_clean":"2024-02-22T19:48:50.089738+0000","last_became_active":"2024-02-22T19:47:40.535824+0000","last_became_peered":"2024-02-22T19:47:40.535824+0000","last_unstale":"2024-02-22T19:48:50.089738+0000","last_undegraded":"2024-02-22T19:48:50.089738+0000","last_fullsized":"2024-02-22T19:48:50.089738+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.133168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.133168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"500'5351","reported_seq":5412,"reported_epoch":500,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:49:00.017557+0000","last_change":"2024-02-22T19:48:46.248200+0000","last_active":"2024-02-22T19:49:00.017557+0000","last_peered":"2024-02-22T19:49:00.017557+0000","last_clean":"2024-02-22T19:48:42.906963+0000","last_became_active":"2024-02-22T19:48:45.076622+0000","last_became_peered":"2024-02-22T19:48:45.076622+0000","last_unstale":"2024-02-22T19:49:00.017557+0000","last_undegraded":"2024-02-22T19:49:00.017557+0000","last_fullsized":"2024-02-22T19:49:00.017557+0000","mapping_epoch":494,"log_start":"477'1978","ondisk_log_start":"477'1978","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3373,"ondisk_log_size":3373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":350683136,"num_objects":5351,"num_object_clones":0,"num_object_copies":16053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5350,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":342464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,1],"avail_no_missing":["3","1","7"],"object_location_counts":[{"shards":"1,3,7","objects":5351}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":323,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084435+0000","last_change":"2024-02-22T19:47:40.521785+0000","last_active":"2024-02-22T19:48:50.084435+0000","last_peered":"2024-02-22T19:48:50.084435+0000","last_clean":"2024-02-22T19:48:50.084435+0000","last_became_active":"2024-02-22T19:47:40.521465+0000","last_became_peered":"2024-02-22T19:47:40.521465+0000","last_unstale":"2024-02-22T19:48:50.084435+0000","last_undegraded":"2024-02-22T19:48:50.084435+0000","last_fullsized":"2024-02-22T19:48:50.084435+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.729907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.729907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":338,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084859+0000","last_change":"2024-02-22T19:48:43.989130+0000","last_active":"2024-02-22T19:48:50.084859+0000","last_peered":"2024-02-22T19:48:50.084859+0000","last_clean":"2024-02-22T19:48:50.084859+0000","last_became_active":"2024-02-22T19:48:43.988801+0000","last_became_peered":"2024-02-22T19:48:43.988801+0000","last_unstale":"2024-02-22T19:48:50.084859+0000","last_undegraded":"2024-02-22T19:48:50.084859+0000","last_fullsized":"2024-02-22T19:48:50.084859+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.513841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.513841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":488,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.105730+0000","last_change":"2024-02-22T19:48:43.995801+0000","last_active":"2024-02-22T19:48:50.105730+0000","last_peered":"2024-02-22T19:48:50.105730+0000","last_clean":"2024-02-22T19:48:50.105730+0000","last_became_active":"2024-02-22T19:48:43.995441+0000","last_became_peered":"2024-02-22T19:48:43.995441+0000","last_unstale":"2024-02-22T19:48:50.105730+0000","last_undegraded":"2024-02-22T19:48:50.105730+0000","last_fullsized":"2024-02-22T19:48:50.105730+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:49.123984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:49.123984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"264'27","reported_seq":196,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114523+0000","last_change":"2024-02-22T19:48:45.198931+0000","last_active":"2024-02-22T19:48:50.114523+0000","last_peered":"2024-02-22T19:48:50.114523+0000","last_clean":"2024-02-22T19:48:50.114523+0000","last_became_active":"2024-02-22T19:48:43.998654+0000","last_became_peered":"2024-02-22T19:48:43.998654+0000","last_unstale":"2024-02-22T19:48:50.114523+0000","last_undegraded":"2024-02-22T19:48:50.114523+0000","last_fullsized":"2024-02-22T19:48:50.114523+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:15.453047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.453047+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"500'5300","reported_seq":5339,"reported_epoch":500,"state":"active+clean","last_fresh":"2024-02-22T19:49:00.054473+0000","last_change":"2024-02-22T19:47:40.526886+0000","last_active":"2024-02-22T19:49:00.054473+0000","last_peered":"2024-02-22T19:49:00.054473+0000","last_clean":"2024-02-22T19:49:00.054473+0000","last_became_active":"2024-02-22T19:47:40.526501+0000","last_became_peered":"2024-02-22T19:47:40.526501+0000","last_unstale":"2024-02-22T19:49:00.054473+0000","last_undegraded":"2024-02-22T19:49:00.054473+0000","last_fullsized":"2024-02-22T19:49:00.054473+0000","mapping_epoch":469,"log_start":"477'1971","ondisk_log_start":"477'1971","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3329,"ondisk_log_size":3329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":347340800,"num_objects":5300,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5300,"num_write_kb":339200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":495,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.114880+0000","last_change":"2024-02-22T19:48:10.140949+0000","last_active":"2024-02-22T19:48:50.114880+0000","last_peered":"2024-02-22T19:48:50.114880+0000","last_clean":"2024-02-22T19:48:50.114880+0000","last_became_active":"2024-02-22T19:41:52.110021+0000","last_became_peered":"2024-02-22T19:41:52.110021+0000","last_unstale":"2024-02-22T19:48:50.114880+0000","last_undegraded":"2024-02-22T19:48:50.114880+0000","last_fullsized":"2024-02-22T19:48:50.114880+0000","mapping_epoch":322,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:10.140859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:10.140859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":45,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089818+0000","last_change":"2024-02-22T19:48:17.043511+0000","last_active":"2024-02-22T19:48:50.089818+0000","last_peered":"2024-02-22T19:48:50.089818+0000","last_clean":"2024-02-22T19:48:50.089818+0000","last_became_active":"2024-02-22T19:47:40.531155+0000","last_became_peered":"2024-02-22T19:47:40.531155+0000","last_unstale":"2024-02-22T19:48:50.089818+0000","last_undegraded":"2024-02-22T19:48:50.089818+0000","last_fullsized":"2024-02-22T19:48:50.089818+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:17.043439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:17.043439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"264'19","reported_seq":73,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090121+0000","last_change":"2024-02-22T19:48:48.410333+0000","last_active":"2024-02-22T19:48:50.090121+0000","last_peered":"2024-02-22T19:48:50.090121+0000","last_clean":"2024-02-22T19:48:50.090121+0000","last_became_active":"2024-02-22T19:48:43.998601+0000","last_became_peered":"2024-02-22T19:48:43.998601+0000","last_unstale":"2024-02-22T19:48:50.090121+0000","last_undegraded":"2024-02-22T19:48:50.090121+0000","last_fullsized":"2024-02-22T19:48:50.090121+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:10.692928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.692928+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"500'5259","reported_seq":6251,"reported_epoch":500,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:48:59.870291+0000","last_change":"2024-02-22T19:48:46.257521+0000","last_active":"2024-02-22T19:48:59.870291+0000","last_peered":"2024-02-22T19:48:59.870291+0000","last_clean":"2024-02-22T19:48:22.530905+0000","last_became_active":"2024-02-22T19:48:44.047570+0000","last_became_peered":"2024-02-22T19:48:44.047570+0000","last_unstale":"2024-02-22T19:48:59.870291+0000","last_undegraded":"2024-02-22T19:48:59.870291+0000","last_fullsized":"2024-02-22T19:48:43.962436+0000","mapping_epoch":493,"log_start":"476'1356","ondisk_log_start":"476'1356","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3903,"ondisk_log_size":3903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":344653824,"num_objects":5259,"num_object_clones":0,"num_object_copies":15777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":820,"num_objects_unfound":0,"num_objects_dirty":5259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5259,"num_write_kb":336576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":23986176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[{"shards":"1,2,5,7","objects":820},{"shards":"2,7","objects":4439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":302,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.083550+0000","last_change":"2024-02-22T19:44:48.769697+0000","last_active":"2024-02-22T19:48:50.083550+0000","last_peered":"2024-02-22T19:48:50.083550+0000","last_clean":"2024-02-22T19:48:50.083550+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:48:50.083550+0000","last_undegraded":"2024-02-22T19:48:50.083550+0000","last_fullsized":"2024-02-22T19:48:50.083550+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.769602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.769602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":513,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.084157+0000","last_change":"2024-02-22T19:47:47.396521+0000","last_active":"2024-02-22T19:48:50.084157+0000","last_peered":"2024-02-22T19:48:50.084157+0000","last_clean":"2024-02-22T19:48:50.084157+0000","last_became_active":"2024-02-22T19:47:40.532868+0000","last_became_peered":"2024-02-22T19:47:40.532868+0000","last_unstale":"2024-02-22T19:48:50.084157+0000","last_undegraded":"2024-02-22T19:48:50.084157+0000","last_fullsized":"2024-02-22T19:48:50.084157+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:47.396426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.396426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":49,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089161+0000","last_change":"2024-02-22T19:48:43.995464+0000","last_active":"2024-02-22T19:48:50.089161+0000","last_peered":"2024-02-22T19:48:50.089161+0000","last_clean":"2024-02-22T19:48:50.089161+0000","last_became_active":"2024-02-22T19:48:43.994933+0000","last_became_peered":"2024-02-22T19:48:43.994933+0000","last_unstale":"2024-02-22T19:48:50.089161+0000","last_undegraded":"2024-02-22T19:48:50.089161+0000","last_fullsized":"2024-02-22T19:48:50.089161+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.714507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.714507+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"264'31","reported_seq":85,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089452+0000","last_change":"2024-02-22T19:48:47.540788+0000","last_active":"2024-02-22T19:48:50.089452+0000","last_peered":"2024-02-22T19:48:50.089452+0000","last_clean":"2024-02-22T19:48:50.089452+0000","last_became_active":"2024-02-22T19:48:43.994403+0000","last_became_peered":"2024-02-22T19:48:43.994403+0000","last_unstale":"2024-02-22T19:48:50.089452+0000","last_undegraded":"2024-02-22T19:48:50.089452+0000","last_fullsized":"2024-02-22T19:48:50.089452+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":496,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:47:26.042105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:26.042105+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":38,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"500'5242","reported_seq":5614,"reported_epoch":500,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:48:59.459271+0000","last_change":"2024-02-22T19:48:45.615032+0000","last_active":"2024-02-22T19:48:59.459271+0000","last_peered":"2024-02-22T19:48:59.459271+0000","last_clean":"2024-02-22T19:48:42.910067+0000","last_became_active":"2024-02-22T19:48:45.176321+0000","last_became_peered":"2024-02-22T19:48:45.176321+0000","last_unstale":"2024-02-22T19:48:59.459271+0000","last_undegraded":"2024-02-22T19:48:59.459271+0000","last_fullsized":"2024-02-22T19:48:44.958051+0000","mapping_epoch":494,"log_start":"476'1559","ondisk_log_start":"476'1559","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":3683,"ondisk_log_size":3683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":343539712,"num_objects":5242,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4555,"num_objects_unfound":0,"num_objects_dirty":5242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5242,"num_write_kb":335488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10616832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5],"avail_no_missing":["7","1","5"],"object_location_counts":[{"shards":"1,5,7","objects":4555},{"shards":"5,7","objects":687}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":497,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.080027+0000","last_change":"2024-02-22T19:47:48.777053+0000","last_active":"2024-02-22T19:48:50.080027+0000","last_peered":"2024-02-22T19:48:50.080027+0000","last_clean":"2024-02-22T19:48:50.080027+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:48:50.080027+0000","last_undegraded":"2024-02-22T19:48:50.080027+0000","last_fullsized":"2024-02-22T19:48:50.080027+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:48.776953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:48.776953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":62,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.090172+0000","last_change":"2024-02-22T19:48:48.418310+0000","last_active":"2024-02-22T19:48:50.090172+0000","last_peered":"2024-02-22T19:48:50.090172+0000","last_clean":"2024-02-22T19:48:50.090172+0000","last_became_active":"2024-02-22T19:48:43.999054+0000","last_became_peered":"2024-02-22T19:48:43.999054+0000","last_unstale":"2024-02-22T19:48:50.090172+0000","last_undegraded":"2024-02-22T19:48:50.090172+0000","last_fullsized":"2024-02-22T19:48:50.090172+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:48:07.111947+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:48:07.111947+0000","last_clean_scrub_stamp":"2024-02-22T19:48:07.111947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":561,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.113653+0000","last_change":"2024-02-22T19:48:44.139400+0000","last_active":"2024-02-22T19:48:50.113653+0000","last_peered":"2024-02-22T19:48:50.113653+0000","last_clean":"2024-02-22T19:48:50.113653+0000","last_became_active":"2024-02-22T19:48:43.997941+0000","last_became_peered":"2024-02-22T19:48:43.997941+0000","last_unstale":"2024-02-22T19:48:50.113653+0000","last_undegraded":"2024-02-22T19:48:50.113653+0000","last_fullsized":"2024-02-22T19:48:50.113653+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:46:12.047696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.047696+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"264'20","reported_seq":78,"reported_epoch":499,"state":"active+clean","last_fresh":"2024-02-22T19:48:50.089857+0000","last_change":"2024-02-22T19:48:45.973378+0000","last_active":"2024-02-22T19:48:50.089857+0000","last_peered":"2024-02-22T19:48:50.089857+0000","last_clean":"2024-02-22T19:48:50.089857+0000","last_became_active":"2024-02-22T19:48:43.992574+0000","last_became_peered":"2024-02-22T19:48:43.992574+0000","last_unstale":"2024-02-22T19:48:50.089857+0000","last_undegraded":"2024-02-22T19:48:50.089857+0000","last_fullsized":"2024-02-22T19:48:50.089857+0000","mapping_epoch":493,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:53.131527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:53.131527+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5563875328,"num_objects":84898,"num_object_clones":0,"num_object_copies":254694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":54175,"num_objects_unfound":0,"num_objects_dirty":84898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84898,"num_write_kb":5433472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8826,"num_bytes_recovered":578420736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17359437824,"data_stored":17359437824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57969,"ondisk_log_size":57969,"up":48,"acting":44,"num_store_stats":6},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1180,"num_read_kb":1175,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":7672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1096,"num_read_kb":853,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":12040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":729,"ondisk_log_size":729,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":777,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266931,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3817332,"kb_used_data":2916332,"kb_used_omap":0,"kb_used_meta":900992,"kb_avail":89927820,"statfs":{"total":95995035648,"available":92086087680,"internally_reserved":0,"allocated":2986323968,"data_stored":2984376690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":922615808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.83599999999999997,"15min":1.28},"min":{"1min":0.253,"5min":0.219,"15min":0.19900000000000001},"max":{"1min":2.2759999999999998,"5min":5.516,"15min":7.9790000000000001},"last":2.3719999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.97199999999999998,"15min":1.361},"min":{"1min":0.316,"5min":0.255,"15min":0.216},"max":{"1min":4.5940000000000003,"5min":5.1349999999999998,"15min":9.8870000000000005},"last":2.048}]},{"osd":1,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9630000000000001,"5min":1.2569999999999999,"15min":1.7809999999999999},"min":{"1min":0.307,"5min":0.218,"15min":0.20000000000000001},"max":{"1min":20.588999999999999,"5min":20.588999999999999,"15min":54.951000000000001},"last":2.5139999999999998},{"interface":"front","average":{"1min":2.4140000000000001,"5min":1.6910000000000001,"15min":2.048},"min":{"1min":0.29899999999999999,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":20.923999999999999,"5min":20.923999999999999,"15min":57.012},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.97,"5min":2.3799999999999999,"15min":1.821},"min":{"1min":0.44600000000000001,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":18.606999999999999,"5min":42.377000000000002,"15min":42.377000000000002},"last":1.085},{"interface":"front","average":{"1min":1.859,"5min":2.222,"15min":2.0190000000000001},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":18.867000000000001,"5min":39.520000000000003,"15min":39.520000000000003},"last":2.4750000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.871,"15min":2.0409999999999999},"min":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":18.683,"5min":37.905000000000001,"15min":38.692999999999998},"last":8.3230000000000004},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.7549999999999999,"15min":2.097},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":18.896000000000001,"5min":38.25,"15min":38.25},"last":4.9939999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.61899999999999999,"15min":0.76500000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.5299999999999998,"5min":4.0910000000000002,"15min":6.4720000000000004},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.65500000000000003,"15min":0.82799999999999996},"min":{"1min":0.33500000000000002,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":1.9770000000000001,"5min":3.9119999999999999,"15min":7.6219999999999999},"last":1.135}]},{"osd":5,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57299999999999995,"15min":0.89600000000000002},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":6.9039999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72099999999999997,"15min":0.83799999999999997},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":4.8200000000000003,"5min":4.8200000000000003,"15min":7.3239999999999998},"last":0.436}]},{"osd":6,"last update":"Thu Feb 22 19:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73999999999999999,"15min":0.84199999999999997},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.4550000000000001,"5min":3.956,"15min":6.5},"last":2.9060000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.752,"15min":0.95099999999999996},"min":{"1min":0.33000000000000002,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":2.1459999999999999,"5min":6.6719999999999997,"15min":7.6390000000000002},"last":1.113}]}]},{"osd":6,"up_from":0,"seq":193273528568,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":0,"seq":55834575121,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":0,"seq":34359738646,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411595,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3643656,"kb_used_data":2752320,"kb_used_omap":0,"kb_used_meta":891328,"kb_avail":90101496,"statfs":{"total":95995035648,"available":92263931904,"internally_reserved":0,"allocated":2818375680,"data_stored":2816470945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912719872},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":1.498,"15min":1.8140000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.18099999999999999},"max":{"1min":2.355,"5min":8.7409999999999997,"15min":22.338999999999999},"last":7.1890000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.893,"15min":2.2690000000000001},"min":{"1min":0.249,"5min":0.216,"15min":0.17599999999999999},"max":{"1min":5.8029999999999999,"5min":9.3580000000000005,"15min":30.427},"last":0.872}]},{"osd":1,"last update":"Thu Feb 22 19:48:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8300000000000001,"5min":1.9790000000000001,"15min":2.0179999999999998},"min":{"1min":0.24399999999999999,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":6.2370000000000001,"5min":15.996,"15min":15.996},"last":5.2850000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":2.109,"15min":2.0419999999999998},"min":{"1min":0.26100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.645,"5min":11.986000000000001,"15min":13.782},"last":7.2460000000000004}]},{"osd":3,"last update":"Thu Feb 22 19:48:59 2024","interfaces":[{"interface":"back","average":{"1min":3.2149999999999999,"5min":2.3479999999999999,"15min":2.2690000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":7.5469999999999997,"5min":8.9469999999999992,"15min":22.302},"last":7.3769999999999998},{"interface":"front","average":{"1min":3.4510000000000001,"5min":2.2170000000000001,"15min":2.4020000000000001},"min":{"1min":0.25900000000000001,"5min":0.183,"15min":0.183},"max":{"1min":11.978999999999999,"5min":11.978999999999999,"15min":30.282},"last":4.7169999999999996}]},{"osd":4,"last update":"Thu Feb 22 19:48:15 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.4450000000000001,"15min":1.5429999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.26400000000000001},"max":{"1min":5.6219999999999999,"5min":8.2289999999999992,"15min":12.43},"last":1.052},{"interface":"front","average":{"1min":1.605,"5min":1.4350000000000001,"15min":1.5069999999999999},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.253},"max":{"1min":8.8209999999999997,"5min":8.8209999999999997,"15min":10.34},"last":7.274}]},{"osd":5,"last update":"Thu Feb 22 19:48:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":1.577,"15min":1.746},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.28000000000000003},"max":{"1min":2.3199999999999998,"5min":14.045,"15min":40.633000000000003},"last":7.3209999999999997},{"interface":"front","average":{"1min":1.875,"5min":1.6830000000000001,"15min":1.9199999999999999},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.26600000000000001},"max":{"1min":5.5259999999999998,"5min":14.513999999999999,"15min":40.997999999999998},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:48:15 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.371,"15min":1.6699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.311},"max":{"1min":5.7089999999999996,"5min":8.2530000000000001,"15min":21.640000000000001},"last":5.3170000000000002},{"interface":"front","average":{"1min":1.71,"5min":1.585,"15min":1.595},"min":{"1min":0.41399999999999998,"5min":0.33900000000000002,"15min":0.30299999999999999},"max":{"1min":8.9220000000000006,"5min":8.9220000000000006,"15min":10.369999999999999},"last":7.4020000000000001}]},{"osd":7,"last update":"Thu Feb 22 19:48:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":5.444,"5min":14.199,"15min":14.199},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.8140000000000001,"15min":1.766},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.29899999999999999},"max":{"1min":2.4380000000000002,"5min":18.021999999999998,"15min":18.021999999999998},"last":7.298}]}]},{"osd":3,"up_from":26,"seq":111669149957,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3664180,"kb_used_data":2768428,"kb_used_omap":0,"kb_used_meta":895744,"kb_avail":90080972,"statfs":{"total":95995035648,"available":92242915328,"internally_reserved":0,"allocated":2834870272,"data_stored":2832922615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":917241856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.6299999999999999,"15min":2.1000000000000001},"min":{"1min":0.24399999999999999,"5min":0.187,"15min":0.187},"max":{"1min":6.944,"5min":9.3599999999999994,"15min":23.48},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.7629999999999999,"15min":2.4079999999999999},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.20799999999999999},"max":{"1min":4.8499999999999996,"5min":10.297000000000001,"15min":23.411999999999999},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":2.8740000000000001,"5min":3.778,"15min":3.3559999999999999},"min":{"1min":0.28599999999999998,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":17.734000000000002,"5min":94.454999999999998,"15min":94.454999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":2.706,"5min":3.5510000000000002,"15min":2.8700000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.214},"max":{"1min":17.696999999999999,"5min":95.197999999999993,"15min":95.197999999999993},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":2.036,"5min":2.3450000000000002,"15min":2.4510000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":17.736000000000001,"5min":17.736000000000001,"15min":20.105},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.5680000000000001,"5min":2.4039999999999999,"15min":2.46},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":7.1760000000000002,"5min":17.986000000000001,"15min":35.939999999999998},"last":1.329}]},{"osd":4,"last update":"Thu Feb 22 19:48:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.357,"15min":1.8520000000000001},"min":{"1min":0.38600000000000001,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":6.2199999999999998,"5min":6.5659999999999998,"15min":13.680999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":1.456,"15min":1.7090000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":2.9180000000000001,"5min":7.8719999999999999,"15min":13.548},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5640000000000001,"15min":1.726},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.3609999999999998,"5min":9.3160000000000007,"15min":12.879},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4219999999999999,"15min":1.8340000000000001},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":6.1539999999999999,"5min":6.6029999999999998,"15min":13.707000000000001},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:48:05 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.5600000000000001,"15min":1.754},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":6.9059999999999997,"5min":9.423,"15min":12.898999999999999},"last":1.8120000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.518,"15min":1.95},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.26600000000000001},"max":{"1min":4.3289999999999997,"5min":7.3520000000000003,"15min":13.803000000000001},"last":11.635}]},{"osd":7,"last update":"Thu Feb 22 19:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":1.5129999999999999,"15min":1.74},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":3.0089999999999999,"5min":8.0879999999999992,"15min":13.483000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":1.5920000000000001,"15min":1.8109999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.238,"5min":9.3930000000000007,"15min":12.846},"last":0.42299999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953731,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2918416,"kb_used_data":2608904,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":90826736,"statfs":{"total":95995035648,"available":93006577664,"internally_reserved":0,"allocated":2671517696,"data_stored":2669605739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:48:29 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":0.97799999999999998,"15min":1.5369999999999999},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.20499999999999999},"max":{"1min":8.3469999999999995,"5min":8.3469999999999995,"15min":69.251999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.2669999999999999,"15min":1.6200000000000001},"min":{"1min":0.33200000000000002,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":9.3780000000000001,"5min":9.4900000000000002,"15min":69.373000000000005},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.1619999999999999,"15min":1.573},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.215},"max":{"1min":3.9359999999999999,"5min":4.9930000000000003,"15min":27.234999999999999},"last":1.482},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.081,"15min":1.341},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.248},"max":{"1min":3.274,"5min":5.8170000000000002,"15min":28.879000000000001},"last":1.589}]},{"osd":2,"last update":"Thu Feb 22 19:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.3129999999999999,"15min":1.4099999999999999},"min":{"1min":0.318,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.9430000000000001,"5min":7.0739999999999998,"15min":12.359},"last":6.9320000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.262,"15min":1.486},"min":{"1min":0.41299999999999998,"5min":0.26100000000000001,"15min":0.24399999999999999},"max":{"1min":2.3290000000000002,"5min":4.9080000000000004,"15min":13.244999999999999},"last":2.569}]},{"osd":3,"last update":"Thu Feb 22 19:48:29 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.373,"15min":1.752},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.23100000000000001},"max":{"1min":5.7999999999999998,"5min":5.7999999999999998,"15min":25.547000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.3009999999999999,"15min":1.7509999999999999},"min":{"1min":0.377,"5min":0.30399999999999999,"15min":0.25900000000000001},"max":{"1min":3.0830000000000002,"5min":9.4529999999999994,"15min":25.623000000000001},"last":1.5089999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.72399999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":2.3359999999999999,"5min":2.8610000000000002,"15min":5.4850000000000003},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.67700000000000005,"15min":0.82399999999999995},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.249},"max":{"1min":3.3450000000000002,"5min":4.9800000000000004,"15min":13.211},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61299999999999999,"15min":0.74399999999999999},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.25900000000000001},"max":{"1min":2.0150000000000001,"5min":2.4910000000000001,"15min":7.2670000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58599999999999997,"15min":0.78900000000000003},"min":{"1min":0.33200000000000002,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":1.0660000000000001,"5min":2.1179999999999999,"15min":13.353},"last":0.434}]},{"osd":7,"last update":"Thu Feb 22 19:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.629,"15min":0.78800000000000003},"min":{"1min":0.33500000000000002,"5min":0.28799999999999998,"15min":0.27900000000000003},"max":{"1min":1.903,"5min":2.8180000000000001,"15min":11.111000000000001},"last":1.754},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.64000000000000001,"15min":0.74299999999999999},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.307},"max":{"1min":4.4210000000000003,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":0.65100000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724797,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2704428,"kb_used_data":1809444,"kb_used_omap":0,"kb_used_meta":894976,"kb_avail":91040724,"statfs":{"total":95995035648,"available":93225701376,"internally_reserved":0,"allocated":1852870656,"data_stored":1850930528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":916455424},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,3,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:48:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.95599999999999996,"15min":1.321},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23300000000000001},"max":{"1min":3.169,"5min":7.0960000000000001,"15min":17.684000000000001},"last":2.8929999999999998},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.081,"15min":1.5409999999999999},"min":{"1min":0.26600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.7530000000000001,"5min":5.2789999999999999,"15min":17.402999999999999},"last":1.7030000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:48:52 2024","interfaces":[{"interface":"back","average":{"1min":3.907,"5min":1.7070000000000001,"15min":1.6279999999999999},"min":{"1min":0.28000000000000003,"5min":0.22700000000000001,"15min":0.19},"max":{"1min":37.009999999999998,"5min":37.009999999999998,"15min":37.009999999999998},"last":2.9249999999999998},{"interface":"front","average":{"1min":4.1589999999999998,"5min":1.9890000000000001,"15min":1.855},"min":{"1min":0.35099999999999998,"5min":0.23699999999999999,"15min":0.218},"max":{"1min":39.131999999999998,"5min":39.131999999999998,"15min":39.131999999999998},"last":1.825}]},{"osd":2,"last update":"Thu Feb 22 19:48:52 2024","interfaces":[{"interface":"back","average":{"1min":2.9100000000000001,"5min":2.1080000000000001,"15min":1.962},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":37.204000000000001,"5min":47.859000000000002,"15min":47.859000000000002},"last":6.2889999999999997},{"interface":"front","average":{"1min":2.6989999999999998,"5min":1.8859999999999999,"15min":1.829},"min":{"1min":0.30499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":38.709000000000003,"5min":45.183,"15min":45.183},"last":7.7270000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:48:52 2024","interfaces":[{"interface":"back","average":{"1min":3.2850000000000001,"5min":1.9530000000000001,"15min":2.0209999999999999},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.254},"max":{"1min":37.707999999999998,"5min":43.616999999999997,"15min":43.616999999999997},"last":3.0179999999999998},{"interface":"front","average":{"1min":3.6469999999999998,"5min":2.3149999999999999,"15min":1.9259999999999999},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":47.814,"5min":47.814,"15min":47.814},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Feb 22 19:48:52 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.60999999999999999,"15min":0.78700000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.223},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":7.5419999999999998},"last":6.266},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.56200000000000006,"15min":0.878},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.3519999999999999,"5min":3.3519999999999999,"15min":10.407},"last":8.0109999999999992}]},{"osd":6,"last update":"Thu Feb 22 19:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.57299999999999995,"15min":0.65600000000000003},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.247},"max":{"1min":0.83299999999999996,"5min":3.2599999999999998,"15min":5.1150000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.77200000000000002,"15min":0.86299999999999999},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.26400000000000001},"max":{"1min":1.8440000000000001,"5min":4.2160000000000002,"15min":10.436},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.77200000000000002},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28299999999999997},"max":{"1min":1.405,"5min":2.9689999999999999,"15min":6.7119999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.747,"15min":0.93600000000000005},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29199999999999998},"max":{"1min":2.6139999999999999,"5min":5.6120000000000001,"15min":7.6379999999999999},"last":4.1790000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":2932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4144496640,"data_stored":4144496640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2829975552,"data_stored":2829975552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2854486016,"data_stored":2854486016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2670592000,"data_stored":2670592000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1856438272,"data_stored":1856438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3003449344,"data_stored":3003449344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:49:05.280 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-02-22T19:49:05.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set default.rgw.meta pg_num 22 2024-02-22T19:49:05.304 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:49:05.314 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:05.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:49:05.358 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:49:05.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:05.466 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:05.510 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:05.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:05.515 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:05.519 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 79 16 88246 88230 69.726 33.9375 0.0151446 0.0143363 2024-02-22T19:49:05.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:49:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:03.835160+0000 mgr.y (mgr.14535) 764 : cluster [DBG] pgmap v1028: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.2 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 908 B/s rd, 32 MiB/s wr, 505 op/s; 1/255270 objects degraded (0.000%); 54177/255270 objects misplaced (21.223%); 37 MiB/s, 599 objects/s recovering 2024-02-22T19:49:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:04.300540+0000 mon.a (mon.0) 1791 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:04.916127+0000 osd.3 (osd.3) 348 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:04.916182+0000 osd.3 (osd.3) 349 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:05.151007+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.53:0/1623185977' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:05.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:05.151680+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:05.616 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:03.835160+0000 mgr.y (mgr.14535) 764 : cluster [DBG] pgmap v1028: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.2 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 908 B/s rd, 32 MiB/s wr, 505 op/s; 1/255270 objects degraded (0.000%); 54177/255270 objects misplaced (21.223%); 37 MiB/s, 599 objects/s recovering 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:04.300540+0000 mon.a (mon.0) 1791 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:04.916127+0000 osd.3 (osd.3) 348 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:04.916182+0000 osd.3 (osd.3) 349 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:05.151007+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.53:0/1623185977' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:05.151680+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:05.735 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:03.835160+0000 mgr.y (mgr.14535) 764 : cluster [DBG] pgmap v1028: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.2 GiB data, 16 GiB used, 431 GiB / 447 GiB avail; 908 B/s rd, 32 MiB/s wr, 505 op/s; 1/255270 objects degraded (0.000%); 54177/255270 objects misplaced (21.223%); 37 MiB/s, 599 objects/s recovering 2024-02-22T19:49:05.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:04.300540+0000 mon.a (mon.0) 1791 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:05.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:04.916127+0000 osd.3 (osd.3) 348 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:05.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:04.916182+0000 osd.3 (osd.3) 349 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:05.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:05.151007+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.53:0/1623185977' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:05.736 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:05.151680+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:05.739 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:06.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:49:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:06.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:06.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:06.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:06.294 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:06.318 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:06.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:06.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:06.426 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:05.298365+0000 mon.a (mon.0) 1793 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:05.321244+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:05.321308+0000 mon.a (mon.0) 1795 : cluster [DBG] osdmap e501: 8 total, 8 up, 5 in 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:05.486810+0000 mgr.y (mgr.14535) 765 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:05.298365+0000 mon.a (mon.0) 1793 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:05.321244+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:05.321308+0000 mon.a (mon.0) 1795 : cluster [DBG] osdmap e501: 8 total, 8 up, 5 in 2024-02-22T19:49:06.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:05.486810+0000 mgr.y (mgr.14535) 765 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:06.518 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:2024-02-22T19:49:06.519099+0000 min lat: 0.00115108 max lat: 2.39342 avg lat: 0.014429 2024-02-22T19:49:06.518 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-02-22T19:49:06.518 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 80 16 88784 88768 69.2718 33.625 0.0301886 0.014429 2024-02-22T19:49:06.542 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:05.298365+0000 mon.a (mon.0) 1793 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:49:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:05.321244+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:49:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:05.321308+0000 mon.a (mon.0) 1795 : cluster [DBG] osdmap e501: 8 total, 8 up, 5 in 2024-02-22T19:49:06.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:05.486810+0000 mgr.y (mgr.14535) 765 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:07.041 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:07.046 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:07.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:07.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:07.294 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:07.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:07.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:07.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:07.491 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:07.520 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 81 16 89366 89350 68.8644 36.375 0.0181063 0.0145108 2024-02-22T19:49:07.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:05.838560+0000 mgr.y (mgr.14535) 766 : cluster [DBG] pgmap v1030: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1023 B/s rd, 37 MiB/s wr, 588 op/s; 1/264165 objects degraded (0.000%); 52258/264165 objects misplaced (19.782%); 41 MiB/s, 663 objects/s recovering 2024-02-22T19:49:07.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:05.838560+0000 mgr.y (mgr.14535) 766 : cluster [DBG] pgmap v1030: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1023 B/s rd, 37 MiB/s wr, 588 op/s; 1/264165 objects degraded (0.000%); 52258/264165 objects misplaced (19.782%); 41 MiB/s, 663 objects/s recovering 2024-02-22T19:49:07.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:05.838560+0000 mgr.y (mgr.14535) 766 : cluster [DBG] pgmap v1030: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1023 B/s rd, 37 MiB/s wr, 588 op/s; 1/264165 objects degraded (0.000%); 52258/264165 objects misplaced (19.782%); 41 MiB/s, 663 objects/s recovering 2024-02-22T19:49:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:08.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:08.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:08.520 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 82 15 89510 89495 68.1359 9.0625 0.0169621 0.0146686 2024-02-22T19:49:08.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:08.608 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:08.634 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:08.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:07.355338+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e502: 8 total, 8 up, 5 in 2024-02-22T19:49:08.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:07.574165+0000 mon.a (mon.0) 1797 : cluster [WRN] Health check update: Degraded data redundancy: 1/264165 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:08.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:08.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:08.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:08.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:07.355338+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e502: 8 total, 8 up, 5 in 2024-02-22T19:49:08.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:07.574165+0000 mon.a (mon.0) 1797 : cluster [WRN] Health check update: Degraded data redundancy: 1/264165 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:08.763 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:07.355338+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e502: 8 total, 8 up, 5 in 2024-02-22T19:49:08.763 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:07.574165+0000 mon.a (mon.0) 1797 : cluster [WRN] Health check update: Degraded data redundancy: 1/264165 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:08.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:49:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:09.194 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:09.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:09.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:09.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:09.337 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:09.394 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:09.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:09.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:09.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:09.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:09] "GET /metrics HTTP/1.1" 200 176721 "" "Prometheus/2.33.4" 2024-02-22T19:49:09.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:07.839053+0000 mgr.y (mgr.14535) 767 : cluster [DBG] pgmap v1032: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 639 B/s rd, 37 MiB/s wr, 599 op/s; 1/264165 objects degraded (0.000%); 52258/264165 objects misplaced (19.782%); 27 MiB/s, 433 objects/s recovering 2024-02-22T19:49:09.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:08.381574+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e503: 8 total, 8 up, 5 in 2024-02-22T19:49:09.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:08.394975+0000 mon.a (mon.0) 1799 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:09.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:07.839053+0000 mgr.y (mgr.14535) 767 : cluster [DBG] pgmap v1032: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 639 B/s rd, 37 MiB/s wr, 599 op/s; 1/264165 objects degraded (0.000%); 52258/264165 objects misplaced (19.782%); 27 MiB/s, 433 objects/s recovering 2024-02-22T19:49:09.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:08.381574+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e503: 8 total, 8 up, 5 in 2024-02-22T19:49:09.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:08.394975+0000 mon.a (mon.0) 1799 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:09.520 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 83 16 89803 89787 67.5354 18.25 0.0559192 0.0148006 2024-02-22T19:49:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:09.571 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:09.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:09.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:09.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:07.839053+0000 mgr.y (mgr.14535) 767 : cluster [DBG] pgmap v1032: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 123 active+clean, 5 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 639 B/s rd, 37 MiB/s wr, 599 op/s; 1/264165 objects degraded (0.000%); 52258/264165 objects misplaced (19.782%); 27 MiB/s, 433 objects/s recovering 2024-02-22T19:49:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:08.381574+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e503: 8 total, 8 up, 5 in 2024-02-22T19:49:09.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:08.394975+0000 mon.a (mon.0) 1799 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:10.520 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 84 16 90432 90416 67.1998 39.3125 0.0246831 0.0148764 2024-02-22T19:49:10.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:09.383195+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e504: 8 total, 8 up, 5 in 2024-02-22T19:49:10.687 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:09.383195+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e504: 8 total, 8 up, 5 in 2024-02-22T19:49:10.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:09.383195+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e504: 8 total, 8 up, 5 in 2024-02-22T19:49:10.997 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:11.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:11.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:11.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:11.148 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:11.180 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:11.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:11.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:11.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:49:11.342 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:49:11.371 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:11.400 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:49:11.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:09.791606+0000 osd.5 (osd.5) 205 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:11.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:09.791645+0000 osd.5 (osd.5) 206 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:11.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:09.839659+0000 mgr.y (mgr.14535) 768 : cluster [DBG] pgmap v1035: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 186 op/s; 1/266511 objects degraded (0.000%); 49766/266511 objects misplaced (18.673%); 36 MiB/s, 570 objects/s recovering 2024-02-22T19:49:11.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:10.422968+0000 mon.a (mon.0) 1801 : cluster [DBG] osdmap e505: 8 total, 8 up, 5 in 2024-02-22T19:49:11.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:11.277879+0000 mon.c (mon.2) 87 : audit [DBG] from='client.? 172.21.15.53:0/2045671406' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:11.456 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:49:11.522 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 85 16 91012 90996 66.835 36.25 0.012009 0.0149555 2024-02-22T19:49:11.522 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:49:11.582 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:49:11.646 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:49:11.702 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:09.791606+0000 osd.5 (osd.5) 205 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:09.791645+0000 osd.5 (osd.5) 206 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:09.839659+0000 mgr.y (mgr.14535) 768 : cluster [DBG] pgmap v1035: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 186 op/s; 1/266511 objects degraded (0.000%); 49766/266511 objects misplaced (18.673%); 36 MiB/s, 570 objects/s recovering 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:10.422968+0000 mon.a (mon.0) 1801 : cluster [DBG] osdmap e505: 8 total, 8 up, 5 in 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:11.277879+0000 mon.c (mon.2) 87 : audit [DBG] from='client.? 172.21.15.53:0/2045671406' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:09.791606+0000 osd.5 (osd.5) 205 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:09.791645+0000 osd.5 (osd.5) 206 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:11.727 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:09.839659+0000 mgr.y (mgr.14535) 768 : cluster [DBG] pgmap v1035: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 186 op/s; 1/266511 objects degraded (0.000%); 49766/266511 objects misplaced (18.673%); 36 MiB/s, 570 objects/s recovering 2024-02-22T19:49:11.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:10.422968+0000 mon.a (mon.0) 1801 : cluster [DBG] osdmap e505: 8 total, 8 up, 5 in 2024-02-22T19:49:11.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:11.277879+0000 mon.c (mon.2) 87 : audit [DBG] from='client.? 172.21.15.53:0/2045671406' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:11.757 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:49:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:49:12.524 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 86 16 91661 91645 66.5279 40.5625 0.0328196 0.0150245 2024-02-22T19:49:13.524 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 87 16 92169 92153 66.1285 31.75 0.0164341 0.0151139 2024-02-22T19:49:13.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:11.847654+0000 mgr.y (mgr.14535) 769 : cluster [DBG] pgmap v1037: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 16 KiB/s rd, 30 MiB/s wr, 500 op/s; 1/270546 objects degraded (0.000%); 50316/270546 objects misplaced (18.598%); 39 MiB/s, 626 objects/s recovering 2024-02-22T19:49:13.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:12.579608+0000 mon.a (mon.0) 1802 : cluster [WRN] Health check update: Degraded data redundancy: 1/270546 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:13.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:12.600868+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e506: 8 total, 8 up, 5 in 2024-02-22T19:49:13.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:11.847654+0000 mgr.y (mgr.14535) 769 : cluster [DBG] pgmap v1037: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 16 KiB/s rd, 30 MiB/s wr, 500 op/s; 1/270546 objects degraded (0.000%); 50316/270546 objects misplaced (18.598%); 39 MiB/s, 626 objects/s recovering 2024-02-22T19:49:13.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:12.579608+0000 mon.a (mon.0) 1802 : cluster [WRN] Health check update: Degraded data redundancy: 1/270546 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:13.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:12.600868+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e506: 8 total, 8 up, 5 in 2024-02-22T19:49:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:11.847654+0000 mgr.y (mgr.14535) 769 : cluster [DBG] pgmap v1037: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 16 KiB/s rd, 30 MiB/s wr, 500 op/s; 1/270546 objects degraded (0.000%); 50316/270546 objects misplaced (18.598%); 39 MiB/s, 626 objects/s recovering 2024-02-22T19:49:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:12.579608+0000 mon.a (mon.0) 1802 : cluster [WRN] Health check update: Degraded data redundancy: 1/270546 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:12.600868+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e506: 8 total, 8 up, 5 in 2024-02-22T19:49:14.532 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 88 16 92816 92800 65.8317 40.4375 0.0170039 0.0151849 2024-02-22T19:49:14.555 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:49:14.555 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:14.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:14.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:49:14.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:14.745 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:15.103 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:49:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:49:15.508 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 6 pg_num to 22 2024-02-22T19:49:15.532 INFO:tasks.radosbench.radosbench.0.smithi053.stdout: 89 16 93379 93363 65.4873 35.1875 0.0329951 0.0152634 2024-02-22T19:49:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:13.848148+0000 mgr.y (mgr.14535) 770 : cluster [DBG] pgmap v1039: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 14 KiB/s rd, 24 MiB/s wr, 411 op/s; 1/270546 objects degraded (0.000%); 50316/270546 objects misplaced (18.598%); 32 MiB/s, 515 objects/s recovering 2024-02-22T19:49:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:15.047636+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.53:0/1383454595' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:49:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:15.048306+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:49:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:13.848148+0000 mgr.y (mgr.14535) 770 : cluster [DBG] pgmap v1039: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 14 KiB/s rd, 24 MiB/s wr, 411 op/s; 1/270546 objects degraded (0.000%); 50316/270546 objects misplaced (18.598%); 32 MiB/s, 515 objects/s recovering 2024-02-22T19:49:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:15.047636+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.53:0/1383454595' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:49:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:15.048306+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:49:15.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:49:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:49:15.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:13.848148+0000 mgr.y (mgr.14535) 770 : cluster [DBG] pgmap v1039: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.5 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 14 KiB/s rd, 24 MiB/s wr, 411 op/s; 1/270546 objects degraded (0.000%); 50316/270546 objects misplaced (18.598%); 32 MiB/s, 515 objects/s recovering 2024-02-22T19:49:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:15.047636+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.53:0/1383454595' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:49:15.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:15.048306+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-22T19:49:16.234 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:16.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:16.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:16.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:49:16.433 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total time run: 90.0394 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Total writes made: 93930 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Write size: 65536 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Object size: 65536 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Bandwidth (MB/sec): 65.2007 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Bandwidth: 39.1412 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max bandwidth (MB/sec): 185.375 2024-02-22T19:49:16.542 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min bandwidth (MB/sec): 0 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average IOPS: 1043 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev IOPS: 626.259 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max IOPS: 2966 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min IOPS: 0 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Average Latency(s): 0.0153296 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Stddev Latency(s): 0.0447669 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Max latency(s): 2.39342 2024-02-22T19:49:16.543 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Min latency(s): 0.00115108 2024-02-22T19:49:16.575 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Cleaning up (deleting benchmark objects) 2024-02-22T19:49:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:49:16.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:16.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:49:16.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:15.454451+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-22T19:49:16.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:15.454527+0000 mon.a (mon.0) 1806 : cluster [DBG] osdmap e507: 8 total, 8 up, 5 in 2024-02-22T19:49:16.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:15.498990+0000 mgr.y (mgr.14535) 771 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:16.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:15.851957+0000 mon.a (mon.0) 1807 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-22T19:49:16.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:15.454451+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-22T19:49:16.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:15.454527+0000 mon.a (mon.0) 1806 : cluster [DBG] osdmap e507: 8 total, 8 up, 5 in 2024-02-22T19:49:16.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:15.498990+0000 mgr.y (mgr.14535) 771 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:16.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:15.851957+0000 mon.a (mon.0) 1807 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-22T19:49:16.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:16.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:15.454451+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-22T19:49:16.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:15.454527+0000 mon.a (mon.0) 1806 : cluster [DBG] osdmap e507: 8 total, 8 up, 5 in 2024-02-22T19:49:16.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:15.498990+0000 mgr.y (mgr.14535) 771 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:16.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:15.851957+0000 mon.a (mon.0) 1807 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-22T19:49:16.840 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:16.864 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:16.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:16.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:16.932 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:49:16.932 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:49:16.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:17.028 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:17.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:17.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:17.175 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:17.205 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:17.213 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:17.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:17.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:17.384 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:17.535 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:15.850493+0000 mgr.y (mgr.14535) 772 : cluster [DBG] pgmap v1041: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 67 KiB/s rd, 45 MiB/s wr, 828 op/s; 1/279459 objects degraded (0.000%); 50573/279459 objects misplaced (18.097%); 12 MiB/s, 193 objects/s recovering 2024-02-22T19:49:17.535 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:16.505432+0000 mon.a (mon.0) 1808 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-22T19:49:17.535 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:16.505627+0000 mon.a (mon.0) 1809 : cluster [DBG] osdmap e508: 8 total, 8 up, 5 in 2024-02-22T19:49:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:15.850493+0000 mgr.y (mgr.14535) 772 : cluster [DBG] pgmap v1041: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 67 KiB/s rd, 45 MiB/s wr, 828 op/s; 1/279459 objects degraded (0.000%); 50573/279459 objects misplaced (18.097%); 12 MiB/s, 193 objects/s recovering 2024-02-22T19:49:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:16.505432+0000 mon.a (mon.0) 1808 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-22T19:49:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:16.505627+0000 mon.a (mon.0) 1809 : cluster [DBG] osdmap e508: 8 total, 8 up, 5 in 2024-02-22T19:49:17.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:15.850493+0000 mgr.y (mgr.14535) 772 : cluster [DBG] pgmap v1041: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 67 KiB/s rd, 45 MiB/s wr, 828 op/s; 1/279459 objects degraded (0.000%); 50573/279459 objects misplaced (18.097%); 12 MiB/s, 193 objects/s recovering 2024-02-22T19:49:17.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:16.505432+0000 mon.a (mon.0) 1808 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-22T19:49:17.760 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:16.505627+0000 mon.a (mon.0) 1809 : cluster [DBG] osdmap e508: 8 total, 8 up, 5 in 2024-02-22T19:49:18.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:49:18.564 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:17.510492+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e509: 8 total, 8 up, 5 in 2024-02-22T19:49:18.564 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:17.595148+0000 mon.a (mon.0) 1811 : cluster [WRN] Health check update: Degraded data redundancy: 1/279459 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:18.564 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:17.601749+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e510: 8 total, 8 up, 5 in 2024-02-22T19:49:18.568 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:17.510492+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e509: 8 total, 8 up, 5 in 2024-02-22T19:49:18.568 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:17.595148+0000 mon.a (mon.0) 1811 : cluster [WRN] Health check update: Degraded data redundancy: 1/279459 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:18.568 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:17.601749+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e510: 8 total, 8 up, 5 in 2024-02-22T19:49:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:17.510492+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e509: 8 total, 8 up, 5 in 2024-02-22T19:49:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:17.595148+0000 mon.a (mon.0) 1811 : cluster [WRN] Health check update: Degraded data redundancy: 1/279459 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:18.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:17.601749+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e510: 8 total, 8 up, 5 in 2024-02-22T19:49:19.509 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:49:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:19] "GET /metrics HTTP/1.1" 200 176721 "" "Prometheus/2.33.4" 2024-02-22T19:49:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:17.850933+0000 mgr.y (mgr.14535) 773 : cluster [DBG] pgmap v1045: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 91 KiB/s rd, 46 MiB/s wr, 893 op/s; 1/279459 objects degraded (0.000%); 50573/279459 objects misplaced (18.097%); 15 MiB/s, 232 objects/s recovering 2024-02-22T19:49:19.917 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:17.850933+0000 mgr.y (mgr.14535) 773 : cluster [DBG] pgmap v1045: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 91 KiB/s rd, 46 MiB/s wr, 893 op/s; 1/279459 objects degraded (0.000%); 50573/279459 objects misplaced (18.097%); 15 MiB/s, 232 objects/s recovering 2024-02-22T19:49:19.917 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:17.850933+0000 mgr.y (mgr.14535) 773 : cluster [DBG] pgmap v1045: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 5 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 91 KiB/s rd, 46 MiB/s wr, 893 op/s; 1/279459 objects degraded (0.000%); 50573/279459 objects misplaced (18.097%); 15 MiB/s, 232 objects/s recovering 2024-02-22T19:49:20.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:49:20.631 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:20.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:20.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:20.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:20.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:19.851680+0000 mgr.y (mgr.14535) 774 : cluster [DBG] pgmap v1046: 138 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.1 KiB/s rd, 6.1 MiB/s wr, 240 op/s; 1/278892 objects degraded (0.000%); 49908/278892 objects misplaced (17.895%); 9.4 MiB/s, 149 objects/s recovering 2024-02-22T19:49:20.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:19.851680+0000 mgr.y (mgr.14535) 774 : cluster [DBG] pgmap v1046: 138 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.1 KiB/s rd, 6.1 MiB/s wr, 240 op/s; 1/278892 objects degraded (0.000%); 49908/278892 objects misplaced (17.895%); 9.4 MiB/s, 149 objects/s recovering 2024-02-22T19:49:20.770 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:20.803 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:20.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:20.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:20.931 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:19.851680+0000 mgr.y (mgr.14535) 774 : cluster [DBG] pgmap v1046: 138 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 125 active+clean, 3 active+remapped+backfilling; 5.7 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1.1 KiB/s rd, 6.1 MiB/s wr, 240 op/s; 1/278892 objects degraded (0.000%); 49908/278892 objects misplaced (17.895%); 9.4 MiB/s, 149 objects/s recovering 2024-02-22T19:49:20.934 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:20.944 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:20.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:20.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:21.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:21.132 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:21.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:21.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:21.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:21.370 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:21.493 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:21.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:21.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:21.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:21.643 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:21.679 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:21.680 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:21.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:21.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:21.813 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:21.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:21.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:21.929 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:21.853995+0000 mon.a (mon.0) 1813 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-22T19:49:21.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:21.853995+0000 mon.a (mon.0) 1813 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-22T19:49:21.934 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:21.853995+0000 mon.a (mon.0) 1813 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-22T19:49:21.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:21.979 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:22.017 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:22.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:22.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:22.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:22.154 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:22.182 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:22.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:22.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:22.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:22.363 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:22.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:22.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:22.542 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:22.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:22.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:22.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:22.764 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:49:23.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:21.852532+0000 mgr.y (mgr.14535) 775 : cluster [DBG] pgmap v1047: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 126 active+clean, 4 active+remapped+backfilling; 5.6 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1023 B/s rd, 10 MiB/s wr, 474 op/s; 1/276756 objects degraded (0.000%); 49194/276756 objects misplaced (17.775%); 9.7 MiB/s, 155 objects/s recovering 2024-02-22T19:49:23.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:21.904614+0000 mon.a (mon.0) 1814 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:23.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:21.929001+0000 mon.a (mon.0) 1815 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:21.929395+0000 mon.a (mon.0) 1816 : cluster [DBG] osdmap e511: 8 total, 8 up, 5 in 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:22.269155+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.53:0/1144668519' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:22.269706+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:22.599003+0000 mon.a (mon.0) 1818 : cluster [WRN] Health check update: Degraded data redundancy: 1/276756 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:22.599600+0000 mon.a (mon.0) 1819 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:22.612707+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:49:23.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:22.612771+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e512: 8 total, 8 up, 5 in 2024-02-22T19:49:23.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:21.852532+0000 mgr.y (mgr.14535) 775 : cluster [DBG] pgmap v1047: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 126 active+clean, 4 active+remapped+backfilling; 5.6 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1023 B/s rd, 10 MiB/s wr, 474 op/s; 1/276756 objects degraded (0.000%); 49194/276756 objects misplaced (17.775%); 9.7 MiB/s, 155 objects/s recovering 2024-02-22T19:49:23.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:21.904614+0000 mon.a (mon.0) 1814 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:23.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:21.929001+0000 mon.a (mon.0) 1815 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-22T19:49:23.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:21.929395+0000 mon.a (mon.0) 1816 : cluster [DBG] osdmap e511: 8 total, 8 up, 5 in 2024-02-22T19:49:23.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:22.269155+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.53:0/1144668519' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:49:23.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:22.269706+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:49:23.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:22.599003+0000 mon.a (mon.0) 1818 : cluster [WRN] Health check update: Degraded data redundancy: 1/276756 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:23.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:22.599600+0000 mon.a (mon.0) 1819 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:23.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:22.612707+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:49:23.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:22.612771+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e512: 8 total, 8 up, 5 in 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:21.852532+0000 mgr.y (mgr.14535) 775 : cluster [DBG] pgmap v1047: 138 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 126 active+clean, 4 active+remapped+backfilling; 5.6 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1023 B/s rd, 10 MiB/s wr, 474 op/s; 1/276756 objects degraded (0.000%); 49194/276756 objects misplaced (17.775%); 9.7 MiB/s, 155 objects/s recovering 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:21.904614+0000 mon.a (mon.0) 1814 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:21.929001+0000 mon.a (mon.0) 1815 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:21.929395+0000 mon.a (mon.0) 1816 : cluster [DBG] osdmap e511: 8 total, 8 up, 5 in 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:22.269155+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.53:0/1144668519' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:22.269706+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:22.599003+0000 mon.a (mon.0) 1818 : cluster [WRN] Health check update: Degraded data redundancy: 1/276756 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:23.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:22.599600+0000 mon.a (mon.0) 1819 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:23.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:22.612707+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:49:23.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:22.612771+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e512: 8 total, 8 up, 5 in 2024-02-22T19:49:23.442 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:23.450 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:23.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:23.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:23.758 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:23.775 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:23.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:23.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:24.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:24.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:24.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:23.238384+0000 osd.7 (osd.7) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:23.238429+0000 osd.7 (osd.7) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:23.403130+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:23.650654+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e513: 8 total, 8 up, 5 in 2024-02-22T19:49:24.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:23.855524+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-22T19:49:24.760 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:23.238384+0000 osd.7 (osd.7) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:24.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:23.238429+0000 osd.7 (osd.7) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:24.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:23.403130+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:24.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:23.650654+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e513: 8 total, 8 up, 5 in 2024-02-22T19:49:24.761 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:23.855524+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-22T19:49:24.782 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:24.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:49:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:49:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:23.238384+0000 osd.7 (osd.7) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:23.238429+0000 osd.7 (osd.7) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:23.403130+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:23.650654+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e513: 8 total, 8 up, 5 in 2024-02-22T19:49:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:23.855524+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-22T19:49:25.034 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:25.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:25.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:25.044 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 5, 4] out_osds: [0, 6, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:49:25.044 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:49:25.045 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-02-22T19:49:25.045 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 6 2024-02-22T19:49:25.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:25.177 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:25.204 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:25.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:25.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:25.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:25.451 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:25.564 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:49:25.645 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:49:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:23.852954+0000 mgr.y (mgr.14535) 776 : cluster [DBG] pgmap v1051: 137 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 125 active+clean, 4 active+remapped+backfilling; 5.6 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1023 B/s rd, 10 MiB/s wr, 474 op/s; 1/276756 objects degraded (0.000%); 49194/276756 objects misplaced (17.775%); 9.7 MiB/s, 155 objects/s recovering 2024-02-22T19:49:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:24.694301+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-22T19:49:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:24.694358+0000 mon.a (mon.0) 1826 : cluster [DBG] osdmap e514: 8 total, 8 up, 5 in 2024-02-22T19:49:25.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:23.852954+0000 mgr.y (mgr.14535) 776 : cluster [DBG] pgmap v1051: 137 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 125 active+clean, 4 active+remapped+backfilling; 5.6 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1023 B/s rd, 10 MiB/s wr, 474 op/s; 1/276756 objects degraded (0.000%); 49194/276756 objects misplaced (17.775%); 9.7 MiB/s, 155 objects/s recovering 2024-02-22T19:49:25.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:24.694301+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-22T19:49:25.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:24.694358+0000 mon.a (mon.0) 1826 : cluster [DBG] osdmap e514: 8 total, 8 up, 5 in 2024-02-22T19:49:25.808 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:49:25.850 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:49:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:23.852954+0000 mgr.y (mgr.14535) 776 : cluster [DBG] pgmap v1051: 137 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 125 active+clean, 4 active+remapped+backfilling; 5.6 GiB data, 18 GiB used, 429 GiB / 447 GiB avail; 1023 B/s rd, 10 MiB/s wr, 474 op/s; 1/276756 objects degraded (0.000%); 49194/276756 objects misplaced (17.775%); 9.7 MiB/s, 155 objects/s recovering 2024-02-22T19:49:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:24.694301+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-22T19:49:25.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:24.694358+0000 mon.a (mon.0) 1826 : cluster [DBG] osdmap e514: 8 total, 8 up, 5 in 2024-02-22T19:49:25.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:49:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:49:25.889 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:49:25.931 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:49:25.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:49:26.488 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:49:26.763 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:25.520369+0000 mgr.y (mgr.14535) 777 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:26.763 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:25.524756+0000 mon.c (mon.2) 90 : audit [DBG] from='client.? 172.21.15.53:0/4059232161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:26.763 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:25.704884+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e515: 8 total, 8 up, 5 in 2024-02-22T19:49:26.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:25.520369+0000 mgr.y (mgr.14535) 777 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:26.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:25.524756+0000 mon.c (mon.2) 90 : audit [DBG] from='client.? 172.21.15.53:0/4059232161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:26.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:25.704884+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e515: 8 total, 8 up, 5 in 2024-02-22T19:49:26.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:25.520369+0000 mgr.y (mgr.14535) 777 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:26.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:25.524756+0000 mon.c (mon.2) 90 : audit [DBG] from='client.? 172.21.15.53:0/4059232161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:26.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:25.704884+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e515: 8 total, 8 up, 5 in 2024-02-22T19:49:26.938 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:26.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:26.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:27.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:27.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:27.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:27.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:49:27.349 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:27.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:25.860669+0000 mgr.y (mgr.14535) 778 : cluster [DBG] pgmap v1054: 137 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 125 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 740 op/s; 1/268032 objects degraded (0.000%); 47149/268032 objects misplaced (17.591%); 15 MiB/s, 238 objects/s recovering 2024-02-22T19:49:27.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:26.733379+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e516: 8 total, 8 up, 5 in 2024-02-22T19:49:27.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:25.860669+0000 mgr.y (mgr.14535) 778 : cluster [DBG] pgmap v1054: 137 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 125 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 740 op/s; 1/268032 objects degraded (0.000%); 47149/268032 objects misplaced (17.591%); 15 MiB/s, 238 objects/s recovering 2024-02-22T19:49:27.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:26.733379+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e516: 8 total, 8 up, 5 in 2024-02-22T19:49:27.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:25.860669+0000 mgr.y (mgr.14535) 778 : cluster [DBG] pgmap v1054: 137 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 125 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 740 op/s; 1/268032 objects degraded (0.000%); 47149/268032 objects misplaced (17.591%); 15 MiB/s, 238 objects/s recovering 2024-02-22T19:49:27.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:26.733379+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e516: 8 total, 8 up, 5 in 2024-02-22T19:49:28.546 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:49:28.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:28.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:27.428509+0000 osd.1 (osd.1) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:27.428554+0000 osd.1 (osd.1) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:27.606438+0000 mon.a (mon.0) 1829 : cluster [WRN] Health check update: Degraded data redundancy: 1/268032 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:27.748304+0000 mon.a (mon.0) 1830 : cluster [DBG] osdmap e517: 8 total, 8 up, 5 in 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:27.862678+0000 mon.a (mon.0) 1831 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:27.428509+0000 osd.1 (osd.1) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:27.428554+0000 osd.1 (osd.1) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:27.606438+0000 mon.a (mon.0) 1829 : cluster [WRN] Health check update: Degraded data redundancy: 1/268032 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:27.748304+0000 mon.a (mon.0) 1830 : cluster [DBG] osdmap e517: 8 total, 8 up, 5 in 2024-02-22T19:49:28.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:27.862678+0000 mon.a (mon.0) 1831 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-22T19:49:28.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:28.756 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:28.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:27.428509+0000 osd.1 (osd.1) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:28.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:27.428554+0000 osd.1 (osd.1) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:28.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:27.606438+0000 mon.a (mon.0) 1829 : cluster [WRN] Health check update: Degraded data redundancy: 1/268032 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:28.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:27.748304+0000 mon.a (mon.0) 1830 : cluster [DBG] osdmap e517: 8 total, 8 up, 5 in 2024-02-22T19:49:28.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:27.862678+0000 mon.a (mon.0) 1831 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-22T19:49:28.888 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:28.893 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:28.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:28.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:29.049 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:29.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:29.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:29.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:29.244 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:29.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:29.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:29.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:29.486 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:29.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:29.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:27.861174+0000 mgr.y (mgr.14535) 779 : cluster [DBG] pgmap v1057: 136 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 124 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 726 op/s; 1/268032 objects degraded (0.000%); 47149/268032 objects misplaced (17.591%); 15 MiB/s, 234 objects/s recovering 2024-02-22T19:49:29.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:28.770350+0000 mon.a (mon.0) 1832 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-22T19:49:29.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:28.770409+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e518: 8 total, 8 up, 5 in 2024-02-22T19:49:29.508 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:29] "GET /metrics HTTP/1.1" 200 176729 "" "Prometheus/2.33.4" 2024-02-22T19:49:29.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:27.861174+0000 mgr.y (mgr.14535) 779 : cluster [DBG] pgmap v1057: 136 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 124 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 726 op/s; 1/268032 objects degraded (0.000%); 47149/268032 objects misplaced (17.591%); 15 MiB/s, 234 objects/s recovering 2024-02-22T19:49:29.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:28.770350+0000 mon.a (mon.0) 1832 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-22T19:49:29.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:28.770409+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e518: 8 total, 8 up, 5 in 2024-02-22T19:49:29.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:29.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:29.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:29.787 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:27.861174+0000 mgr.y (mgr.14535) 779 : cluster [DBG] pgmap v1057: 136 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 124 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 726 op/s; 1/268032 objects degraded (0.000%); 47149/268032 objects misplaced (17.591%); 15 MiB/s, 234 objects/s recovering 2024-02-22T19:49:29.787 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:28.770350+0000 mon.a (mon.0) 1832 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-22T19:49:29.787 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:28.770409+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e518: 8 total, 8 up, 5 in 2024-02-22T19:49:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:29.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:29.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:49:29.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:29.957 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:29.995 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:29.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:29.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:30.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:30.188 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:30.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:49:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:29.780263+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e519: 8 total, 8 up, 5 in 2024-02-22T19:49:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:29.866791+0000 mgr.y (mgr.14535) 780 : cluster [DBG] pgmap v1060: 136 pgs: 2 peering, 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 251 op/s; 1/265026 objects degraded (0.000%); 46489/265026 objects misplaced (17.541%); 9.9 MiB/s, 158 objects/s recovering 2024-02-22T19:49:31.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:29.780263+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e519: 8 total, 8 up, 5 in 2024-02-22T19:49:31.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:29.866791+0000 mgr.y (mgr.14535) 780 : cluster [DBG] pgmap v1060: 136 pgs: 2 peering, 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 251 op/s; 1/265026 objects degraded (0.000%); 46489/265026 objects misplaced (17.541%); 9.9 MiB/s, 158 objects/s recovering 2024-02-22T19:49:31.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:29.780263+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e519: 8 total, 8 up, 5 in 2024-02-22T19:49:31.261 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:29.866791+0000 mgr.y (mgr.14535) 780 : cluster [DBG] pgmap v1060: 136 pgs: 2 peering, 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.4 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 251 op/s; 1/265026 objects degraded (0.000%); 46489/265026 objects misplaced (17.541%); 9.9 MiB/s, 158 objects/s recovering 2024-02-22T19:49:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:30.804281+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e520: 8 total, 8 up, 5 in 2024-02-22T19:49:32.260 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:30.804281+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e520: 8 total, 8 up, 5 in 2024-02-22T19:49:32.260 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:30.804281+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e520: 8 total, 8 up, 5 in 2024-02-22T19:49:32.428 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:32.436 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:32.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:32.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:32.592 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:32.621 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:32.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:32.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:32.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:32.772 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:32.802 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:32.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:32.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:32.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:32.948 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:32.982 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:32.986 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:32.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:33.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:33.133 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:31.850743+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e521: 8 total, 8 up, 5 in 2024-02-22T19:49:33.133 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:31.868127+0000 mgr.y (mgr.14535) 781 : cluster [DBG] pgmap v1063: 135 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 697 op/s; 1/259662 objects degraded (0.000%); 45184/259662 objects misplaced (17.401%); 14 MiB/s, 230 objects/s recovering 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:31.869362+0000 mon.a (mon.0) 1837 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:32.607945+0000 mon.a (mon.0) 1838 : cluster [WRN] Health check update: Degraded data redundancy: 1/265026 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:32.612446+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:32.612486+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e522: 8 total, 8 up, 5 in 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:31.850743+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e521: 8 total, 8 up, 5 in 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:31.868127+0000 mgr.y (mgr.14535) 781 : cluster [DBG] pgmap v1063: 135 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 697 op/s; 1/259662 objects degraded (0.000%); 45184/259662 objects misplaced (17.401%); 14 MiB/s, 230 objects/s recovering 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:31.869362+0000 mon.a (mon.0) 1837 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:32.607945+0000 mon.a (mon.0) 1838 : cluster [WRN] Health check update: Degraded data redundancy: 1/265026 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:32.612446+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-22T19:49:33.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:32.612486+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e522: 8 total, 8 up, 5 in 2024-02-22T19:49:33.135 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:33.245 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:33.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:33.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:31.850743+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e521: 8 total, 8 up, 5 in 2024-02-22T19:49:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:31.868127+0000 mgr.y (mgr.14535) 781 : cluster [DBG] pgmap v1063: 135 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 697 op/s; 1/259662 objects degraded (0.000%); 45184/259662 objects misplaced (17.401%); 14 MiB/s, 230 objects/s recovering 2024-02-22T19:49:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:31.869362+0000 mon.a (mon.0) 1837 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-22T19:49:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:32.607945+0000 mon.a (mon.0) 1838 : cluster [WRN] Health check update: Degraded data redundancy: 1/265026 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:49:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:32.612446+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-22T19:49:33.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:32.612486+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e522: 8 total, 8 up, 5 in 2024-02-22T19:49:33.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:33.481 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:34.573 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:34.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:34.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:34.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:49:34.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:34.729 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:34.751 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:34.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:34.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:34.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:33.626352+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e523: 8 total, 8 up, 5 in 2024-02-22T19:49:34.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:33.868556+0000 mgr.y (mgr.14535) 782 : cluster [DBG] pgmap v1066: 135 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 0 B/s wr, 446 op/s; 1/259662 objects degraded (0.000%); 45184/259662 objects misplaced (17.401%); 4.5 MiB/s, 72 objects/s recovering 2024-02-22T19:49:34.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:33.626352+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e523: 8 total, 8 up, 5 in 2024-02-22T19:49:34.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:33.868556+0000 mgr.y (mgr.14535) 782 : cluster [DBG] pgmap v1066: 135 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 0 B/s wr, 446 op/s; 1/259662 objects degraded (0.000%); 45184/259662 objects misplaced (17.401%); 4.5 MiB/s, 72 objects/s recovering 2024-02-22T19:49:34.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:34.914 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:34.916 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:33.626352+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e523: 8 total, 8 up, 5 in 2024-02-22T19:49:34.916 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:33.868556+0000 mgr.y (mgr.14535) 782 : cluster [DBG] pgmap v1066: 135 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 122 active+clean, 4 active+remapped+backfilling; 5.3 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 0 B/s wr, 446 op/s; 1/259662 objects degraded (0.000%); 45184/259662 objects misplaced (17.401%); 4.5 MiB/s, 72 objects/s recovering 2024-02-22T19:49:34.916 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:49:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:49:34.917 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:34.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:35.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:35.056 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:35.081 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:35.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:35.085 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:35.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:35.224 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:35.256 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:35.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:35.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:35.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:35.416 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:35.419 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:35.422 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:35.423 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:35.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:35.601 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:35.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:35.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:35.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:35.763 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:49:35.795 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:34.675988+0000 mon.a (mon.0) 1842 : cluster [DBG] osdmap e524: 8 total, 8 up, 5 in 2024-02-22T19:49:35.795 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:35.299882+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? 172.21.15.53:0/3323127402' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:35.795 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:35.535260+0000 mgr.y (mgr.14535) 783 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:35.795 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:49:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:49:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:35.852 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:35.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:35.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:35.957 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:36.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:34.675988+0000 mon.a (mon.0) 1842 : cluster [DBG] osdmap e524: 8 total, 8 up, 5 in 2024-02-22T19:49:36.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:35.299882+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? 172.21.15.53:0/3323127402' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:36.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:35.535260+0000 mgr.y (mgr.14535) 783 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:36.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:34.675988+0000 mon.a (mon.0) 1842 : cluster [DBG] osdmap e524: 8 total, 8 up, 5 in 2024-02-22T19:49:36.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:35.299882+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? 172.21.15.53:0/3323127402' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:49:36.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:35.535260+0000 mgr.y (mgr.14535) 783 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:36.065 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:36.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:36.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:36.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:36.245 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:36.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:36.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:36.382 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:36.411 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:36.416 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:36.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:36.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:36.576 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:36.581 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:36.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:36.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:36.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:35.671953+0000 mon.a (mon.0) 1844 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:36.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:35.715505+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? 172.21.15.53:0/3323127402' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:49:36.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:35.715612+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e525: 8 total, 8 up, 5 in 2024-02-22T19:49:36.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:35.869064+0000 mgr.y (mgr.14535) 784 : cluster [DBG] pgmap v1069: 134 pgs: 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 121 active+clean, 4 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 717 op/s; 199/251064 objects degraded (0.079%); 42975/251064 objects misplaced (17.117%); 15 MiB/s, 239 objects/s recovering 2024-02-22T19:49:36.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:35.871966+0000 mon.a (mon.0) 1847 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-22T19:49:36.689 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:35.671953+0000 mon.a (mon.0) 1844 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:36.690 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:35.715505+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? 172.21.15.53:0/3323127402' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:49:36.690 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:35.715612+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e525: 8 total, 8 up, 5 in 2024-02-22T19:49:36.690 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:35.869064+0000 mgr.y (mgr.14535) 784 : cluster [DBG] pgmap v1069: 134 pgs: 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 121 active+clean, 4 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 717 op/s; 199/251064 objects degraded (0.079%); 42975/251064 objects misplaced (17.117%); 15 MiB/s, 239 objects/s recovering 2024-02-22T19:49:36.690 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:35.871966+0000 mon.a (mon.0) 1847 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-22T19:49:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:36.986 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:35.671953+0000 mon.a (mon.0) 1844 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:36.986 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:35.715505+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? 172.21.15.53:0/3323127402' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:49:36.986 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:35.715612+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e525: 8 total, 8 up, 5 in 2024-02-22T19:49:36.987 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:35.869064+0000 mgr.y (mgr.14535) 784 : cluster [DBG] pgmap v1069: 134 pgs: 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 121 active+clean, 4 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 717 op/s; 199/251064 objects degraded (0.079%); 42975/251064 objects misplaced (17.117%); 15 MiB/s, 239 objects/s recovering 2024-02-22T19:49:36.987 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:35.871966+0000 mon.a (mon.0) 1847 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-22T19:49:36.987 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:49:36.975+0000 7fe2d794e700 -1 mon.a@0(leader).osd e526 definitely_dead 0 2024-02-22T19:49:37.782 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.6. 2024-02-22T19:49:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:36.751453+0000 mon.a (mon.0) 1848 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-22T19:49:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:36.751529+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e526: 8 total, 8 up, 5 in 2024-02-22T19:49:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:36.976422+0000 mon.a (mon.0) 1850 : audit [DBG] from='client.? 172.21.15.53:0/2702970143' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-02-22T19:49:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:37.609777+0000 mon.a (mon.0) 1851 : cluster [WRN] Health check update: Degraded data redundancy: 199/251064 objects degraded (0.079%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:36.751453+0000 mon.a (mon.0) 1848 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-22T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:36.751529+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e526: 8 total, 8 up, 5 in 2024-02-22T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:36.976422+0000 mon.a (mon.0) 1850 : audit [DBG] from='client.? 172.21.15.53:0/2702970143' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-02-22T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:37.609777+0000 mon.a (mon.0) 1851 : cluster [WRN] Health check update: Degraded data redundancy: 199/251064 objects degraded (0.079%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T19:49:38.157 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:36.751453+0000 mon.a (mon.0) 1848 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-22T19:49:38.157 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:36.751529+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e526: 8 total, 8 up, 5 in 2024-02-22T19:49:38.157 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:36.976422+0000 mon.a (mon.0) 1850 : audit [DBG] from='client.? 172.21.15.53:0/2702970143' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-02-22T19:49:38.157 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:37.609777+0000 mon.a (mon.0) 1851 : cluster [WRN] Health check update: Degraded data redundancy: 199/251064 objects degraded (0.079%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T19:49:38.254 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:38.268 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:38.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:38.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:38.401 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:38.747 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-02-22T19:49:38.757 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:38.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:38.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:38.861 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:39.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:37.762742+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? 172.21.15.53:0/2702970143' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-02-22T19:49:39.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:37.762796+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e527: 8 total, 8 up, 6 in 2024-02-22T19:49:39.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:37.870229+0000 mgr.y (mgr.14535) 785 : cluster [DBG] pgmap v1072: 134 pgs: 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 121 active+clean, 4 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 717 op/s; 199/251064 objects degraded (0.079%); 42975/251064 objects misplaced (17.117%); 15 MiB/s, 239 objects/s recovering 2024-02-22T19:49:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:37.762742+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? 172.21.15.53:0/2702970143' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-02-22T19:49:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:37.762796+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e527: 8 total, 8 up, 6 in 2024-02-22T19:49:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:37.870229+0000 mgr.y (mgr.14535) 785 : cluster [DBG] pgmap v1072: 134 pgs: 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 121 active+clean, 4 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 717 op/s; 199/251064 objects degraded (0.079%); 42975/251064 objects misplaced (17.117%); 15 MiB/s, 239 objects/s recovering 2024-02-22T19:49:39.021 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:39.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:37.762742+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? 172.21.15.53:0/2702970143' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-02-22T19:49:39.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:37.762796+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e527: 8 total, 8 up, 6 in 2024-02-22T19:49:39.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:37.870229+0000 mgr.y (mgr.14535) 785 : cluster [DBG] pgmap v1072: 134 pgs: 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 121 active+clean, 4 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 430 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 717 op/s; 199/251064 objects degraded (0.079%); 42975/251064 objects misplaced (17.117%); 15 MiB/s, 239 objects/s recovering 2024-02-22T19:49:39.032 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:39.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:39.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:39.167 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:39.270 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:39.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:39.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:39.400 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:39.422 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:39.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:39.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:39.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:39] "GET /metrics HTTP/1.1" 200 176731 "" "Prometheus/2.33.4" 2024-02-22T19:49:39.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:39.582 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:39.585 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:39.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:39.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:39.712 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:39.736 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:39.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:39.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:39.875 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:39.914 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:39.917 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:39.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:39.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:38.767546+0000 mon.a (mon.0) 1854 : cluster [DBG] osdmap e528: 8 total, 8 up, 6 in 2024-02-22T19:49:39.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:38.770212+0000 osd.3 (osd.3) 350 : cluster [DBG] 11.9 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6681 2024-02-22T19:49:39.919 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:38.770244+0000 osd.3 (osd.3) 351 : cluster [DBG] 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6601 2024-02-22T19:49:39.919 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:38.795323+0000 osd.3 (osd.3) 352 : cluster [DBG] 11.a starting backfill to osd.6 from (0'0,0'0] MAX to 526'6509 2024-02-22T19:49:39.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:38.767546+0000 mon.a (mon.0) 1854 : cluster [DBG] osdmap e528: 8 total, 8 up, 6 in 2024-02-22T19:49:39.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:38.770212+0000 osd.3 (osd.3) 350 : cluster [DBG] 11.9 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6681 2024-02-22T19:49:39.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:38.770244+0000 osd.3 (osd.3) 351 : cluster [DBG] 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6601 2024-02-22T19:49:39.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:38.795323+0000 osd.3 (osd.3) 352 : cluster [DBG] 11.a starting backfill to osd.6 from (0'0,0'0] MAX to 526'6509 2024-02-22T19:49:40.019 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:49:40.078 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:40.156 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:38.767546+0000 mon.a (mon.0) 1854 : cluster [DBG] osdmap e528: 8 total, 8 up, 6 in 2024-02-22T19:49:40.156 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:38.770212+0000 osd.3 (osd.3) 350 : cluster [DBG] 11.9 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6681 2024-02-22T19:49:40.156 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:38.770244+0000 osd.3 (osd.3) 351 : cluster [DBG] 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6601 2024-02-22T19:49:40.156 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:38.795323+0000 osd.3 (osd.3) 352 : cluster [DBG] 11.a starting backfill to osd.6 from (0'0,0'0] MAX to 526'6509 2024-02-22T19:49:40.177 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:49:40.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:40.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:49:40.283 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:40.332 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:40.399 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:40.400 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:49:40.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:40.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:40.432 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:49:40.462 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:49:40.488 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:49:40.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:40.523 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:49:40.550 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:49:40.589 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:49:40.651 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.666 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:49:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:40.871 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:40.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:40.881 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:40.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.711265+0000 osd.4 (osd.4) 193 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.711311+0000 osd.4 (osd.4) 194 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.771908+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e529: 8 total, 8 up, 6 in 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.772774+0000 osd.3 (osd.3) 353 : cluster [DBG] 11.8 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6484 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.773606+0000 osd.7 (osd.7) 325 : cluster [DBG] 11.5 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6705 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.773615+0000 osd.7 (osd.7) 326 : cluster [DBG] 11.3 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6522 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.773749+0000 osd.3 (osd.3) 354 : cluster [DBG] 6.c starting backfill to osd.6 from (0'0,0'0] MAX to 137'2 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.774325+0000 osd.3 (osd.3) 355 : cluster [DBG] 5.3 starting backfill to osd.6 from (0'0,0'0] MAX to 498'19 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.774597+0000 osd.7 (osd.7) 327 : cluster [DBG] 11.4 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6467 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.775395+0000 osd.2 (osd.2) 294 : cluster [DBG] 11.d starting backfill to osd.6 from (0'0,0'0] MAX to 526'6594 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.775913+0000 osd.4 (osd.4) 195 : cluster [DBG] 5.5 starting backfill to osd.6 from (0'0,0'0] MAX to 499'32 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.777594+0000 osd.2 (osd.2) 295 : cluster [DBG] 11.1 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6603 2024-02-22T19:49:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.779753+0000 osd.7 (osd.7) 328 : cluster [DBG] 5.9 starting backfill to osd.6 from (0'0,0'0] MAX to 495'23 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:39.870801+0000 mgr.y (mgr.14535) 786 : cluster [DBG] pgmap v1075: 133 pgs: 5 remapped+peering, 6 active+recovery_wait+degraded, 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering, 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 110 active+clean, 3 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 140 op/s; 102/249387 objects degraded (0.041%); 34654/249387 objects misplaced (13.896%); 3.3 MiB/s, 60 objects/s recovering 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:39.872674+0000 mon.a (mon.0) 1856 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:40.363624+0000 mon.c (mon.2) 91 : audit [DBG] from='client.? 172.21.15.53:0/2614639372' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.711265+0000 osd.4 (osd.4) 193 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.711311+0000 osd.4 (osd.4) 194 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.771908+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e529: 8 total, 8 up, 6 in 2024-02-22T19:49:41.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.772774+0000 osd.3 (osd.3) 353 : cluster [DBG] 11.8 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6484 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.773606+0000 osd.7 (osd.7) 325 : cluster [DBG] 11.5 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6705 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.773615+0000 osd.7 (osd.7) 326 : cluster [DBG] 11.3 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6522 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.773749+0000 osd.3 (osd.3) 354 : cluster [DBG] 6.c starting backfill to osd.6 from (0'0,0'0] MAX to 137'2 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.774325+0000 osd.3 (osd.3) 355 : cluster [DBG] 5.3 starting backfill to osd.6 from (0'0,0'0] MAX to 498'19 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.774597+0000 osd.7 (osd.7) 327 : cluster [DBG] 11.4 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6467 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.775395+0000 osd.2 (osd.2) 294 : cluster [DBG] 11.d starting backfill to osd.6 from (0'0,0'0] MAX to 526'6594 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.775913+0000 osd.4 (osd.4) 195 : cluster [DBG] 5.5 starting backfill to osd.6 from (0'0,0'0] MAX to 499'32 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.777594+0000 osd.2 (osd.2) 295 : cluster [DBG] 11.1 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6603 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.779753+0000 osd.7 (osd.7) 328 : cluster [DBG] 5.9 starting backfill to osd.6 from (0'0,0'0] MAX to 495'23 2024-02-22T19:49:41.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:39.870801+0000 mgr.y (mgr.14535) 786 : cluster [DBG] pgmap v1075: 133 pgs: 5 remapped+peering, 6 active+recovery_wait+degraded, 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering, 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 110 active+clean, 3 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 140 op/s; 102/249387 objects degraded (0.041%); 34654/249387 objects misplaced (13.896%); 3.3 MiB/s, 60 objects/s recovering 2024-02-22T19:49:41.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:39.872674+0000 mon.a (mon.0) 1856 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-22T19:49:41.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:40.363624+0000 mon.c (mon.2) 91 : audit [DBG] from='client.? 172.21.15.53:0/2614639372' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:41.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.711265+0000 osd.4 (osd.4) 193 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:41.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.711311+0000 osd.4 (osd.4) 194 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:41.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.771908+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e529: 8 total, 8 up, 6 in 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.772774+0000 osd.3 (osd.3) 353 : cluster [DBG] 11.8 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6484 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.773606+0000 osd.7 (osd.7) 325 : cluster [DBG] 11.5 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6705 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.773615+0000 osd.7 (osd.7) 326 : cluster [DBG] 11.3 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6522 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.773749+0000 osd.3 (osd.3) 354 : cluster [DBG] 6.c starting backfill to osd.6 from (0'0,0'0] MAX to 137'2 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.774325+0000 osd.3 (osd.3) 355 : cluster [DBG] 5.3 starting backfill to osd.6 from (0'0,0'0] MAX to 498'19 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.774597+0000 osd.7 (osd.7) 327 : cluster [DBG] 11.4 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6467 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.775395+0000 osd.2 (osd.2) 294 : cluster [DBG] 11.d starting backfill to osd.6 from (0'0,0'0] MAX to 526'6594 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.775913+0000 osd.4 (osd.4) 195 : cluster [DBG] 5.5 starting backfill to osd.6 from (0'0,0'0] MAX to 499'32 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.777594+0000 osd.2 (osd.2) 295 : cluster [DBG] 11.1 starting backfill to osd.6 from (0'0,0'0] MAX to 526'6603 2024-02-22T19:49:41.052 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.779753+0000 osd.7 (osd.7) 328 : cluster [DBG] 5.9 starting backfill to osd.6 from (0'0,0'0] MAX to 495'23 2024-02-22T19:49:41.053 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:39.870801+0000 mgr.y (mgr.14535) 786 : cluster [DBG] pgmap v1075: 133 pgs: 5 remapped+peering, 6 active+recovery_wait+degraded, 1 active+recovering+undersized+remapped, 1 peering, 3 active+recovering, 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 2 active+remapped+backfill_toofull, 110 active+clean, 3 active+remapped+backfilling; 5.1 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 140 op/s; 102/249387 objects degraded (0.041%); 34654/249387 objects misplaced (13.896%); 3.3 MiB/s, 60 objects/s recovering 2024-02-22T19:49:41.053 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:39.872674+0000 mon.a (mon.0) 1856 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-22T19:49:41.053 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:40.363624+0000 mon.c (mon.2) 91 : audit [DBG] from='client.? 172.21.15.53:0/2614639372' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:41.060 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:41.094 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:41.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:41.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:41.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:41.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:41.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:41.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:41.444 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:41.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:41.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:41.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:41.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:49:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:41.839 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:41.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:41.941 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:41.985 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:40.769237+0000 mon.a (mon.0) 1857 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:40.789847+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:40.789924+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e530: 8 total, 8 up, 6 in 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:41.473712+0000 osd.2 (osd.2) 296 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:41.473758+0000 osd.2 (osd.2) 297 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:40.769237+0000 mon.a (mon.0) 1857 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:40.789847+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:40.789924+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e530: 8 total, 8 up, 6 in 2024-02-22T19:49:42.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:41.473712+0000 osd.2 (osd.2) 296 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:42.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:41.473758+0000 osd.2 (osd.2) 297 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:42.015 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:42.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:42.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:40.769237+0000 mon.a (mon.0) 1857 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:40.789847+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-22T19:49:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:40.789924+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e530: 8 total, 8 up, 6 in 2024-02-22T19:49:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:41.473712+0000 osd.2 (osd.2) 296 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:41.473758+0000 osd.2 (osd.2) 297 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:42.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:42.167 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:42.201 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:42.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:42.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:42.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:42.384 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:42.475 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:42.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:42.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:42.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:42.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:42.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:42.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:42.871 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:49:42.918 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:42.926 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:42.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:42.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:43.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:43.088 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:41.820465+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e531: 8 total, 8 up, 6 in 2024-02-22T19:49:43.088 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:41.871435+0000 mgr.y (mgr.14535) 787 : cluster [DBG] pgmap v1078: 133 pgs: 1 active+recovering+undersized+remapped, 5 remapped+peering, 6 active+recovery_wait+degraded, 1 peering, 4 active+remapped+backfill_wait, 4 active+recovering, 6 active+remapped+backfill_toofull, 1 active+recovery_wait+degraded+remapped, 105 active+clean; 5.0 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 336 op/s; 4186/247032 objects degraded (1.695%); 41382/247032 objects misplaced (16.752%); 6.6 MiB/s, 117 objects/s recovering 2024-02-22T19:49:43.088 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:42.612238+0000 mon.a (mon.0) 1861 : cluster [WRN] Health check update: Degraded data redundancy: 102/249387 objects degraded (0.041%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T19:49:43.088 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:42.625063+0000 mon.a (mon.0) 1862 : cluster [DBG] osdmap e532: 8 total, 8 up, 6 in 2024-02-22T19:49:43.089 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:41.820465+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e531: 8 total, 8 up, 6 in 2024-02-22T19:49:43.089 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:41.871435+0000 mgr.y (mgr.14535) 787 : cluster [DBG] pgmap v1078: 133 pgs: 1 active+recovering+undersized+remapped, 5 remapped+peering, 6 active+recovery_wait+degraded, 1 peering, 4 active+remapped+backfill_wait, 4 active+recovering, 6 active+remapped+backfill_toofull, 1 active+recovery_wait+degraded+remapped, 105 active+clean; 5.0 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 336 op/s; 4186/247032 objects degraded (1.695%); 41382/247032 objects misplaced (16.752%); 6.6 MiB/s, 117 objects/s recovering 2024-02-22T19:49:43.089 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:42.612238+0000 mon.a (mon.0) 1861 : cluster [WRN] Health check update: Degraded data redundancy: 102/249387 objects degraded (0.041%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T19:49:43.089 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:42.625063+0000 mon.a (mon.0) 1862 : cluster [DBG] osdmap e532: 8 total, 8 up, 6 in 2024-02-22T19:49:43.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:41.820465+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e531: 8 total, 8 up, 6 in 2024-02-22T19:49:43.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:41.871435+0000 mgr.y (mgr.14535) 787 : cluster [DBG] pgmap v1078: 133 pgs: 1 active+recovering+undersized+remapped, 5 remapped+peering, 6 active+recovery_wait+degraded, 1 peering, 4 active+remapped+backfill_wait, 4 active+recovering, 6 active+remapped+backfill_toofull, 1 active+recovery_wait+degraded+remapped, 105 active+clean; 5.0 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 336 op/s; 4186/247032 objects degraded (1.695%); 41382/247032 objects misplaced (16.752%); 6.6 MiB/s, 117 objects/s recovering 2024-02-22T19:49:43.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:42.612238+0000 mon.a (mon.0) 1861 : cluster [WRN] Health check update: Degraded data redundancy: 102/249387 objects degraded (0.041%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T19:49:43.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:42.625063+0000 mon.a (mon.0) 1862 : cluster [DBG] osdmap e532: 8 total, 8 up, 6 in 2024-02-22T19:49:43.190 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:43.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:43.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:43.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:43.360 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:43.389 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:43.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:43.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:43.595 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:43.748 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 5, 4, 6] out_osds: [0, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:49:43.748 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:49:43.748 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-02-22T19:49:43.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 1 2024-02-22T19:49:44.090 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:44.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:44.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:44.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:44.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:49:44.278 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:44.547 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:44.558 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:44.559 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:44.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:49:44.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:44.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:43.634423+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e533: 8 total, 8 up, 6 in 2024-02-22T19:49:44.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:43.872640+0000 mgr.y (mgr.14535) 788 : cluster [DBG] pgmap v1081: 132 pgs: 1 active+recovering+undersized+remapped, 5 remapped+peering, 6 active+recovery_wait+degraded, 1 peering, 4 active+remapped+backfill_wait, 4 active+recovering, 6 active+remapped+backfill_toofull, 1 active+recovery_wait+degraded+remapped, 104 active+clean; 5.0 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 0 B/s wr, 196 op/s; 4186/247032 objects degraded (1.695%); 41382/247032 objects misplaced (16.752%); 3.3 MiB/s, 56 objects/s recovering 2024-02-22T19:49:44.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:43.875209+0000 mon.a (mon.0) 1864 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-22T19:49:44.812 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:44.067905+0000 osd.6 (osd.6) 164 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:44.812 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:44.067956+0000 osd.6 (osd.6) 165 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:44.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:43.634423+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e533: 8 total, 8 up, 6 in 2024-02-22T19:49:44.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:43.872640+0000 mgr.y (mgr.14535) 788 : cluster [DBG] pgmap v1081: 132 pgs: 1 active+recovering+undersized+remapped, 5 remapped+peering, 6 active+recovery_wait+degraded, 1 peering, 4 active+remapped+backfill_wait, 4 active+recovering, 6 active+remapped+backfill_toofull, 1 active+recovery_wait+degraded+remapped, 104 active+clean; 5.0 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 0 B/s wr, 196 op/s; 4186/247032 objects degraded (1.695%); 41382/247032 objects misplaced (16.752%); 3.3 MiB/s, 56 objects/s recovering 2024-02-22T19:49:44.813 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:43.875209+0000 mon.a (mon.0) 1864 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-22T19:49:44.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:44.067905+0000 osd.6 (osd.6) 164 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:44.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:44.067956+0000 osd.6 (osd.6) 165 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:45.046 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:45.061 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:45.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:45.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:45.265 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:45.328 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:49:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:49:45.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:43.634423+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e533: 8 total, 8 up, 6 in 2024-02-22T19:49:45.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:43.872640+0000 mgr.y (mgr.14535) 788 : cluster [DBG] pgmap v1081: 132 pgs: 1 active+recovering+undersized+remapped, 5 remapped+peering, 6 active+recovery_wait+degraded, 1 peering, 4 active+remapped+backfill_wait, 4 active+recovering, 6 active+remapped+backfill_toofull, 1 active+recovery_wait+degraded+remapped, 104 active+clean; 5.0 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 0 B/s wr, 196 op/s; 4186/247032 objects degraded (1.695%); 41382/247032 objects misplaced (16.752%); 3.3 MiB/s, 56 objects/s recovering 2024-02-22T19:49:45.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:43.875209+0000 mon.a (mon.0) 1864 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-22T19:49:45.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:44.067905+0000 osd.6 (osd.6) 164 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:49:45.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:44.067956+0000 osd.6 (osd.6) 165 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:49:45.460 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:45.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:45.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:45.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:44.677617+0000 mon.a (mon.0) 1865 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-22T19:49:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:44.677675+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e534: 8 total, 8 up, 6 in 2024-02-22T19:49:45.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:45.545250+0000 mgr.y (mgr.14535) 789 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:45.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:44.677617+0000 mon.a (mon.0) 1865 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-22T19:49:45.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:44.677675+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e534: 8 total, 8 up, 6 in 2024-02-22T19:49:45.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:45.545250+0000 mgr.y (mgr.14535) 789 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:45.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:49:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:49:45.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:44.677617+0000 mon.a (mon.0) 1865 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-22T19:49:45.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:44.677675+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e534: 8 total, 8 up, 6 in 2024-02-22T19:49:45.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:45.545250+0000 mgr.y (mgr.14535) 789 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:46.116 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:46.147 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:46.152 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:46.152 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:46.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:46.337 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:46.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:46.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:46.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:46.534 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:46.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:46.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:46.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:46.705 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:46.786 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:49:46.795 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:46.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:46.802 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:46.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:46.803 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:46.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:45.673763+0000 mon.a (mon.0) 1867 : cluster [DBG] osdmap e535: 8 total, 8 up, 6 in 2024-02-22T19:49:46.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:45.877514+0000 mgr.y (mgr.14535) 790 : cluster [DBG] pgmap v1084: 132 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+recovering+remapped, 1 active+recovery_wait+degraded, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 114 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 730 op/s; 5/238263 objects degraded (0.002%); 55349/238263 objects misplaced (23.230%); 9.9 MiB/s, 169 objects/s recovering 2024-02-22T19:49:46.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:46.202040+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.53:0/3908129995' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:49:46.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:46.202575+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:49:46.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:46.973 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:45.673763+0000 mon.a (mon.0) 1867 : cluster [DBG] osdmap e535: 8 total, 8 up, 6 in 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:45.877514+0000 mgr.y (mgr.14535) 790 : cluster [DBG] pgmap v1084: 132 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+recovering+remapped, 1 active+recovery_wait+degraded, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 114 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 730 op/s; 5/238263 objects degraded (0.002%); 55349/238263 objects misplaced (23.230%); 9.9 MiB/s, 169 objects/s recovering 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:46.202040+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.53:0/3908129995' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:46.202575+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:45.673763+0000 mon.a (mon.0) 1867 : cluster [DBG] osdmap e535: 8 total, 8 up, 6 in 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:45.877514+0000 mgr.y (mgr.14535) 790 : cluster [DBG] pgmap v1084: 132 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+recovering+remapped, 1 active+recovery_wait+degraded, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 114 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 730 op/s; 5/238263 objects degraded (0.002%); 55349/238263 objects misplaced (23.230%); 9.9 MiB/s, 169 objects/s recovering 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:46.202040+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.53:0/3908129995' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:49:46.974 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:46.202575+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:49:46.975 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:48.009 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:46.685543+0000 mon.a (mon.0) 1869 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:46.690452+0000 mon.a (mon.0) 1870 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:46.737101+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:46.737161+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e536: 8 total, 8 up, 6 in 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:47.614814+0000 mon.a (mon.0) 1873 : cluster [WRN] Health check update: Degraded data redundancy: 5/238263 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:46.685543+0000 mon.a (mon.0) 1869 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:46.690452+0000 mon.a (mon.0) 1870 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:48.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:46.737101+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:49:48.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:46.737161+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e536: 8 total, 8 up, 6 in 2024-02-22T19:49:48.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:47.614814+0000 mon.a (mon.0) 1873 : cluster [WRN] Health check update: Degraded data redundancy: 5/238263 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:46.685543+0000 mon.a (mon.0) 1869 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:49:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:46.690452+0000 mon.a (mon.0) 1870 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:49:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:46.737101+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:49:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:46.737161+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e536: 8 total, 8 up, 6 in 2024-02-22T19:49:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:47.614814+0000 mon.a (mon.0) 1873 : cluster [WRN] Health check update: Degraded data redundancy: 5/238263 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:49.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:47.755125+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e537: 8 total, 8 up, 6 in 2024-02-22T19:49:49.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:47.878028+0000 mgr.y (mgr.14535) 791 : cluster [DBG] pgmap v1087: 131 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+recovering+remapped, 1 active+recovery_wait+degraded, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 113 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 731 op/s; 5/238263 objects degraded (0.002%); 55349/238263 objects misplaced (23.230%); 9.9 MiB/s, 169 objects/s recovering 2024-02-22T19:49:49.070 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:47.879682+0000 mon.a (mon.0) 1875 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-22T19:49:49.080 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:47.755125+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e537: 8 total, 8 up, 6 in 2024-02-22T19:49:49.080 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:47.878028+0000 mgr.y (mgr.14535) 791 : cluster [DBG] pgmap v1087: 131 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+recovering+remapped, 1 active+recovery_wait+degraded, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 113 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 731 op/s; 5/238263 objects degraded (0.002%); 55349/238263 objects misplaced (23.230%); 9.9 MiB/s, 169 objects/s recovering 2024-02-22T19:49:49.080 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:47.879682+0000 mon.a (mon.0) 1875 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-22T19:49:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:49.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:49.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:47.755125+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e537: 8 total, 8 up, 6 in 2024-02-22T19:49:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:47.878028+0000 mgr.y (mgr.14535) 791 : cluster [DBG] pgmap v1087: 131 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+recovering+remapped, 1 active+recovery_wait+degraded, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 113 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 731 op/s; 5/238263 objects degraded (0.002%); 55349/238263 objects misplaced (23.230%); 9.9 MiB/s, 169 objects/s recovering 2024-02-22T19:49:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:47.879682+0000 mon.a (mon.0) 1875 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-22T19:49:49.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:49.265 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:49.310 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:49.317 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:49.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:49.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:49.464 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:49:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:49] "GET /metrics HTTP/1.1" 200 176731 "" "Prometheus/2.33.4" 2024-02-22T19:49:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:50.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:48.766050+0000 mon.a (mon.0) 1876 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-22T19:49:50.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:48.766125+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e538: 8 total, 8 up, 6 in 2024-02-22T19:49:50.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:48.766050+0000 mon.a (mon.0) 1876 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-22T19:49:50.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:48.766125+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e538: 8 total, 8 up, 6 in 2024-02-22T19:49:50.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:48.766050+0000 mon.a (mon.0) 1876 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-22T19:49:50.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:48.766125+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e538: 8 total, 8 up, 6 in 2024-02-22T19:49:51.056 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:49.769650+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e539: 8 total, 8 up, 6 in 2024-02-22T19:49:51.062 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:49.878613+0000 mgr.y (mgr.14535) 792 : cluster [DBG] pgmap v1090: 131 pgs: 2 peering, 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 296 op/s; 5/234723 objects degraded (0.002%); 54724/234723 objects misplaced (23.314%); 6.2 MiB/s, 98 objects/s recovering 2024-02-22T19:49:51.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:49.769650+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e539: 8 total, 8 up, 6 in 2024-02-22T19:49:51.063 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:49.878613+0000 mgr.y (mgr.14535) 792 : cluster [DBG] pgmap v1090: 131 pgs: 2 peering, 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 296 op/s; 5/234723 objects degraded (0.002%); 54724/234723 objects misplaced (23.314%); 6.2 MiB/s, 98 objects/s recovering 2024-02-22T19:49:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:51.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:51.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:49.769650+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e539: 8 total, 8 up, 6 in 2024-02-22T19:49:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:49.878613+0000 mgr.y (mgr.14535) 792 : cluster [DBG] pgmap v1090: 131 pgs: 2 peering, 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.8 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 296 op/s; 5/234723 objects degraded (0.002%); 54724/234723 objects misplaced (23.314%); 6.2 MiB/s, 98 objects/s recovering 2024-02-22T19:49:51.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:51.254 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:51.817 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:50.801345+0000 mon.a (mon.0) 1879 : cluster [DBG] osdmap e540: 8 total, 8 up, 6 in 2024-02-22T19:49:51.833 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:51.839 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:51.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:51.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:52.010 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:50.801345+0000 mon.a (mon.0) 1879 : cluster [DBG] osdmap e540: 8 total, 8 up, 6 in 2024-02-22T19:49:52.232 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:52.234 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:50.801345+0000 mon.a (mon.0) 1879 : cluster [DBG] osdmap e540: 8 total, 8 up, 6 in 2024-02-22T19:49:52.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:52.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:52.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:52.409 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:52.470 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:52.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:52.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:52.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:52.636 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:52.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:52.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:52.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:53.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:51.823553+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e541: 8 total, 8 up, 6 in 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:51.879142+0000 mgr.y (mgr.14535) 793 : cluster [DBG] pgmap v1093: 130 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 788 op/s; 5/228813 objects degraded (0.002%); 52884/228813 objects misplaced (23.112%); 11 MiB/s, 172 objects/s recovering 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:51.880721+0000 mon.a (mon.0) 1881 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:52.619153+0000 mon.a (mon.0) 1882 : cluster [WRN] Health check update: Degraded data redundancy: 5/234723 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:52.628244+0000 mon.a (mon.0) 1883 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:52.628308+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e542: 8 total, 8 up, 6 in 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:51.823553+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e541: 8 total, 8 up, 6 in 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:51.879142+0000 mgr.y (mgr.14535) 793 : cluster [DBG] pgmap v1093: 130 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 788 op/s; 5/228813 objects degraded (0.002%); 52884/228813 objects misplaced (23.112%); 11 MiB/s, 172 objects/s recovering 2024-02-22T19:49:53.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:51.880721+0000 mon.a (mon.0) 1881 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-22T19:49:53.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:52.619153+0000 mon.a (mon.0) 1882 : cluster [WRN] Health check update: Degraded data redundancy: 5/234723 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:53.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:52.628244+0000 mon.a (mon.0) 1883 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-22T19:49:53.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:52.628308+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e542: 8 total, 8 up, 6 in 2024-02-22T19:49:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:51.823553+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e541: 8 total, 8 up, 6 in 2024-02-22T19:49:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:51.879142+0000 mgr.y (mgr.14535) 793 : cluster [DBG] pgmap v1093: 130 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 788 op/s; 5/228813 objects degraded (0.002%); 52884/228813 objects misplaced (23.112%); 11 MiB/s, 172 objects/s recovering 2024-02-22T19:49:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:51.880721+0000 mon.a (mon.0) 1881 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-22T19:49:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:52.619153+0000 mon.a (mon.0) 1882 : cluster [WRN] Health check update: Degraded data redundancy: 5/234723 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:52.628244+0000 mon.a (mon.0) 1883 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-22T19:49:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:52.628308+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e542: 8 total, 8 up, 6 in 2024-02-22T19:49:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:53.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:53.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:53.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:53.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:53.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:53.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:49:53.825 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:53.584668+0000 mon.a (mon.0) 1885 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:49:53.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:53.648122+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e543: 8 total, 8 up, 6 in 2024-02-22T19:49:53.984 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:49:53.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:53.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:49:54.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:54.135 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:53.584668+0000 mon.a (mon.0) 1885 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:49:54.135 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:53.648122+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e543: 8 total, 8 up, 6 in 2024-02-22T19:49:54.139 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:53.584668+0000 mon.a (mon.0) 1885 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:49:54.139 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:53.648122+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e543: 8 total, 8 up, 6 in 2024-02-22T19:49:54.156 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:54.650 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:54.660 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:54.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:54.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:49:54.810 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:49:54.841 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:54.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:49:54.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:54.944 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:53.884774+0000 mgr.y (mgr.14535) 794 : cluster [DBG] pgmap v1096: 130 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 0 B/s wr, 491 op/s; 5/228813 objects degraded (0.002%); 52884/228813 objects misplaced (23.112%); 4.6 MiB/s, 74 objects/s recovering 2024-02-22T19:49:54.944 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:54.654804+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e544: 8 total, 8 up, 6 in 2024-02-22T19:49:54.945 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:53.884774+0000 mgr.y (mgr.14535) 794 : cluster [DBG] pgmap v1096: 130 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 0 B/s wr, 491 op/s; 5/228813 objects degraded (0.002%); 52884/228813 objects misplaced (23.112%); 4.6 MiB/s, 74 objects/s recovering 2024-02-22T19:49:54.945 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:54.654804+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e544: 8 total, 8 up, 6 in 2024-02-22T19:49:55.032 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:53.884774+0000 mgr.y (mgr.14535) 794 : cluster [DBG] pgmap v1096: 130 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 111 active+clean; 4.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 0 B/s wr, 491 op/s; 5/228813 objects degraded (0.002%); 52884/228813 objects misplaced (23.112%); 4.6 MiB/s, 74 objects/s recovering 2024-02-22T19:49:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:54.654804+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e544: 8 total, 8 up, 6 in 2024-02-22T19:49:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:49:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:49:55.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:49:55.195+0000 7fe2d794e700 -1 mon.a@0(leader).osd e544 definitely_dead 0 2024-02-22T19:49:55.699 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.1. 2024-02-22T19:49:55.847 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:49:55.848 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:49:55.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:49:56.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.196171+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? 172.21.15.53:0/1557276852' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-02-22T19:49:56.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.196864+0000 mon.a (mon.0) 1888 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-02-22T19:49:56.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.198945+0000 mon.a (mon.0) 1889 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:49:56.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.199824+0000 mon.a (mon.0) 1890 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:49:56.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.207189+0000 mon.a (mon.0) 1891 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:56.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.555208+0000 mgr.y (mgr.14535) 795 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.653686+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:55.653753+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e545: 8 total, 8 up, 7 in 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.196171+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? 172.21.15.53:0/1557276852' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.196864+0000 mon.a (mon.0) 1888 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.198945+0000 mon.a (mon.0) 1889 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.199824+0000 mon.a (mon.0) 1890 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.207189+0000 mon.a (mon.0) 1891 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.555208+0000 mgr.y (mgr.14535) 795 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.653686+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-02-22T19:49:56.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:55.653753+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e545: 8 total, 8 up, 7 in 2024-02-22T19:49:56.162 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.196171+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? 172.21.15.53:0/1557276852' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-02-22T19:49:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.196864+0000 mon.a (mon.0) 1888 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-02-22T19:49:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.198945+0000 mon.a (mon.0) 1889 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:49:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.199824+0000 mon.a (mon.0) 1890 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:49:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.207189+0000 mon.a (mon.0) 1891 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:49:56.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.555208+0000 mgr.y (mgr.14535) 795 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:49:56.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.653686+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-02-22T19:49:56.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:55.653753+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e545: 8 total, 8 up, 7 in 2024-02-22T19:49:56.714 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:56.718 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:56.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:56.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:49:56.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:56.848 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:56.874 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:56.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:56.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:49:57.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:55.885226+0000 mgr.y (mgr.14535) 796 : cluster [DBG] pgmap v1099: 129 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 1 clean+premerge+peered, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 110 active+clean; 4.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 776 op/s; 5/219492 objects degraded (0.002%); 50486/219492 objects misplaced (23.001%); 11 MiB/s, 174 objects/s recovering 2024-02-22T19:49:57.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:55.886010+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-22T19:49:57.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:56.654840+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-22T19:49:57.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:56.654905+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e546: 8 total, 8 up, 7 in 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:56.655033+0000 osd.4 (osd.4) 196 : cluster [DBG] 5.5 starting backfill to osd.1 from (0'0,0'0] MAX to 529'36 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:56.655197+0000 osd.4 (osd.4) 197 : cluster [INF] 5.5 continuing backfill to osd.6 from (387'17,529'36] MIN to 529'36 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:55.885226+0000 mgr.y (mgr.14535) 796 : cluster [DBG] pgmap v1099: 129 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 1 clean+premerge+peered, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 110 active+clean; 4.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 776 op/s; 5/219492 objects degraded (0.002%); 50486/219492 objects misplaced (23.001%); 11 MiB/s, 174 objects/s recovering 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:55.886010+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:56.654840+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:56.654905+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e546: 8 total, 8 up, 7 in 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:56.655033+0000 osd.4 (osd.4) 196 : cluster [DBG] 5.5 starting backfill to osd.1 from (0'0,0'0] MAX to 529'36 2024-02-22T19:49:57.002 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:56.655197+0000 osd.4 (osd.4) 197 : cluster [INF] 5.5 continuing backfill to osd.6 from (387'17,529'36] MIN to 529'36 2024-02-22T19:49:57.004 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:55.885226+0000 mgr.y (mgr.14535) 796 : cluster [DBG] pgmap v1099: 129 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 1 clean+premerge+peered, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 110 active+clean; 4.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 776 op/s; 5/219492 objects degraded (0.002%); 50486/219492 objects misplaced (23.001%); 11 MiB/s, 174 objects/s recovering 2024-02-22T19:49:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:55.886010+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-22T19:49:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:56.654840+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-22T19:49:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:56.654905+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e546: 8 total, 8 up, 7 in 2024-02-22T19:49:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:56.655033+0000 osd.4 (osd.4) 196 : cluster [DBG] 5.5 starting backfill to osd.1 from (0'0,0'0] MAX to 529'36 2024-02-22T19:49:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:56.655197+0000 osd.4 (osd.4) 197 : cluster [INF] 5.5 continuing backfill to osd.6 from (387'17,529'36] MIN to 529'36 2024-02-22T19:49:57.709 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:49:57.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:57.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:49:57.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:57.877 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:57.908 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:57.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:57.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.621223+0000 mon.a (mon.0) 1897 : cluster [WRN] Health check update: Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.655894+0000 mon.a (mon.0) 1898 : cluster [DBG] osdmap e547: 8 total, 8 up, 7 in 2024-02-22T19:49:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.659273+0000 osd.7 (osd.7) 329 : cluster [DBG] 11.f starting backfill to osd.1 from (477'2044,501'5746] MIN to 544'7484 2024-02-22T19:49:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.660440+0000 osd.4 (osd.4) 198 : cluster [DBG] 6.e starting backfill to osd.1 from (0'0,0'0] MAX to 85'2 2024-02-22T19:49:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.660594+0000 osd.4 (osd.4) 199 : cluster [DBG] 11.e starting backfill to osd.1 from (480'2558,525'6473] MIN to 544'7040 2024-02-22T19:49:58.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.670836+0000 osd.7 (osd.7) 330 : cluster [INF] 11.3 continuing backfill to osd.6 from (480'2759,544'7135] MIN to 544'7135 2024-02-22T19:49:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.621223+0000 mon.a (mon.0) 1897 : cluster [WRN] Health check update: Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.655894+0000 mon.a (mon.0) 1898 : cluster [DBG] osdmap e547: 8 total, 8 up, 7 in 2024-02-22T19:49:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.659273+0000 osd.7 (osd.7) 329 : cluster [DBG] 11.f starting backfill to osd.1 from (477'2044,501'5746] MIN to 544'7484 2024-02-22T19:49:58.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.660440+0000 osd.4 (osd.4) 198 : cluster [DBG] 6.e starting backfill to osd.1 from (0'0,0'0] MAX to 85'2 2024-02-22T19:49:58.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.660594+0000 osd.4 (osd.4) 199 : cluster [DBG] 11.e starting backfill to osd.1 from (480'2558,525'6473] MIN to 544'7040 2024-02-22T19:49:58.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.670836+0000 osd.7 (osd.7) 330 : cluster [INF] 11.3 continuing backfill to osd.6 from (480'2759,544'7135] MIN to 544'7135 2024-02-22T19:49:58.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:49:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:49:58.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:58.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:49:58.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:58.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.621223+0000 mon.a (mon.0) 1897 : cluster [WRN] Health check update: Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:49:58.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.655894+0000 mon.a (mon.0) 1898 : cluster [DBG] osdmap e547: 8 total, 8 up, 7 in 2024-02-22T19:49:58.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.659273+0000 osd.7 (osd.7) 329 : cluster [DBG] 11.f starting backfill to osd.1 from (477'2044,501'5746] MIN to 544'7484 2024-02-22T19:49:58.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.660440+0000 osd.4 (osd.4) 198 : cluster [DBG] 6.e starting backfill to osd.1 from (0'0,0'0] MAX to 85'2 2024-02-22T19:49:58.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.660594+0000 osd.4 (osd.4) 199 : cluster [DBG] 11.e starting backfill to osd.1 from (480'2558,525'6473] MIN to 544'7040 2024-02-22T19:49:58.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.670836+0000 osd.7 (osd.7) 330 : cluster [INF] 11.3 continuing backfill to osd.6 from (480'2759,544'7135] MIN to 544'7135 2024-02-22T19:49:58.235 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:58.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:58.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:58.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:49:58.395 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-02-22T19:49:58.410 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:49:58.452 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:58.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:49:58.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:58.601 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:58.634 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:58.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:58.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:49:58.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:49:58.792 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:49:58.839 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:58.850 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:49:58.881 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:49:58.922 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:49:58.931 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:49:58.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:58.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:49:58.963 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:49:59.010 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:49:59.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:49:59.044 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:49:59.112 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:49:59.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.657771+0000 osd.3 (osd.3) 356 : cluster [DBG] 5.0 starting backfill to osd.1 from (0'0,0'0] MAX to 508'19 2024-02-22T19:49:59.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.661308+0000 osd.3 (osd.3) 357 : cluster [INF] 11.9 continuing backfill to osd.6 from (485'3348,544'7286] MIN to 544'7286 2024-02-22T19:49:59.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.671900+0000 osd.5 (osd.5) 207 : cluster [DBG] 11.7 starting backfill to osd.1 from (478'2017,502'5603] MIN to 544'7215 2024-02-22T19:49:59.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.671918+0000 osd.5 (osd.5) 208 : cluster [DBG] 5.7 starting backfill to osd.1 from (0'0,0'0] MAX to 498'31 2024-02-22T19:49:59.147 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:57.885655+0000 mgr.y (mgr.14535) 797 : cluster [DBG] pgmap v1102: 129 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 1 clean+premerge+peered, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 110 active+clean; 4.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 5/219492 objects degraded (0.002%); 50486/219492 objects misplaced (23.001%); 11 MiB/s, 174 objects/s recovering 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:58.658180+0000 mon.a (mon.0) 1899 : cluster [DBG] osdmap e548: 8 total, 8 up, 7 in 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:49:58.735888+0000 mon.a (mon.0) 1900 : audit [DBG] from='client.? 172.21.15.53:0/4179647112' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.657771+0000 osd.3 (osd.3) 356 : cluster [DBG] 5.0 starting backfill to osd.1 from (0'0,0'0] MAX to 508'19 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.661308+0000 osd.3 (osd.3) 357 : cluster [INF] 11.9 continuing backfill to osd.6 from (485'3348,544'7286] MIN to 544'7286 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.671900+0000 osd.5 (osd.5) 207 : cluster [DBG] 11.7 starting backfill to osd.1 from (478'2017,502'5603] MIN to 544'7215 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.671918+0000 osd.5 (osd.5) 208 : cluster [DBG] 5.7 starting backfill to osd.1 from (0'0,0'0] MAX to 498'31 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:57.885655+0000 mgr.y (mgr.14535) 797 : cluster [DBG] pgmap v1102: 129 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 1 clean+premerge+peered, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 110 active+clean; 4.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 5/219492 objects degraded (0.002%); 50486/219492 objects misplaced (23.001%); 11 MiB/s, 174 objects/s recovering 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:58.658180+0000 mon.a (mon.0) 1899 : cluster [DBG] osdmap e548: 8 total, 8 up, 7 in 2024-02-22T19:49:59.148 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:49:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:49:58.735888+0000 mon.a (mon.0) 1900 : audit [DBG] from='client.? 172.21.15.53:0/4179647112' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:49:59.151 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:59.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.657771+0000 osd.3 (osd.3) 356 : cluster [DBG] 5.0 starting backfill to osd.1 from (0'0,0'0] MAX to 508'19 2024-02-22T19:49:59.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.661308+0000 osd.3 (osd.3) 357 : cluster [INF] 11.9 continuing backfill to osd.6 from (485'3348,544'7286] MIN to 544'7286 2024-02-22T19:49:59.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.671900+0000 osd.5 (osd.5) 207 : cluster [DBG] 11.7 starting backfill to osd.1 from (478'2017,502'5603] MIN to 544'7215 2024-02-22T19:49:59.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.671918+0000 osd.5 (osd.5) 208 : cluster [DBG] 5.7 starting backfill to osd.1 from (0'0,0'0] MAX to 498'31 2024-02-22T19:49:59.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:57.885655+0000 mgr.y (mgr.14535) 797 : cluster [DBG] pgmap v1102: 129 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+degraded, 2 active+recovering+remapped, 1 clean+premerge+peered, 3 active+remapped+backfill_wait, 9 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 110 active+clean; 4.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 5/219492 objects degraded (0.002%); 50486/219492 objects misplaced (23.001%); 11 MiB/s, 174 objects/s recovering 2024-02-22T19:49:59.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:58.658180+0000 mon.a (mon.0) 1899 : cluster [DBG] osdmap e548: 8 total, 8 up, 7 in 2024-02-22T19:49:59.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:49:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:49:58.735888+0000 mon.a (mon.0) 1900 : audit [DBG] from='client.? 172.21.15.53:0/4179647112' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:49:59.505 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:49:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:49:59] "GET /metrics HTTP/1.1" 200 176727 "" "Prometheus/2.33.4" 2024-02-22T19:49:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:49:59.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:49:59.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:49:59.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:49:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:49:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:49:59.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:49:59.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:00.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:00.105 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:00.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:00.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:00.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:00.653 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:00.690 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:00.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:00.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:00.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:59.662335+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e549: 8 total, 8 up, 7 in 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:49:59.887991+0000 mgr.y (mgr.14535) 798 : cluster [DBG] pgmap v1105: 128 pgs: 1 active+recovering+remapped, 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 9 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 108 active+clean; 4.4 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 194 op/s; 1/217158 objects degraded (0.000%); 54049/217158 objects misplaced (24.889%); 2.6 MiB/s, 66 objects/s recovering 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000139+0000 mon.a (mon.0) 1902 : cluster [WRN] Health detail: HEALTH_WARN nodeep-scrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull; Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000184+0000 mon.a (mon.0) 1903 : cluster [WRN] [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000198+0000 mon.a (mon.0) 1904 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000211+0000 mon.a (mon.0) 1905 : cluster [WRN] pg 5.3 is active+remapped+backfill_toofull, acting [3,5,7] 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000224+0000 mon.a (mon.0) 1906 : cluster [WRN] pg 5.5 is active+remapped+backfill_toofull, acting [4,7,3] 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000247+0000 mon.a (mon.0) 1907 : cluster [WRN] pg 5.9 is active+remapped+backfill_toofull, acting [7,4,1] 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000260+0000 mon.a (mon.0) 1908 : cluster [WRN] pg 11.0 is active+remapped+backfill_toofull, acting [3,7,1] 2024-02-22T19:50:01.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000271+0000 mon.a (mon.0) 1909 : cluster [WRN] pg 11.1 is active+remapped+backfill_toofull, acting [2,3,4] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000281+0000 mon.a (mon.0) 1910 : cluster [WRN] pg 11.5 is active+remapped+backfill_toofull, acting [7,5,3] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000291+0000 mon.a (mon.0) 1911 : cluster [WRN] pg 11.8 is active+remapped+backfill_toofull, acting [3,5,2] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000301+0000 mon.a (mon.0) 1912 : cluster [WRN] pg 11.b is active+remapped+backfill_toofull, acting [3,4,1] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000312+0000 mon.a (mon.0) 1913 : cluster [WRN] pg 11.c is active+remapped+backfill_toofull, acting [2,4,1] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000322+0000 mon.a (mon.0) 1914 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000336+0000 mon.a (mon.0) 1915 : cluster [WRN] pg 4.18 is active+recovery_wait+degraded, acting [4,6,3] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000347+0000 mon.a (mon.0) 1916 : cluster [WRN] pg 11.2 is stuck undersized for 73s, current state active+recovering+undersized+remapped, last acting [2,7] 2024-02-22T19:50:01.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.000357+0000 mon.a (mon.0) 1917 : cluster [WRN] pg 11.d is active+recovering+degraded+remapped, acting [2,4,5] 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:59.662335+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e549: 8 total, 8 up, 7 in 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:49:59.887991+0000 mgr.y (mgr.14535) 798 : cluster [DBG] pgmap v1105: 128 pgs: 1 active+recovering+remapped, 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 9 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 108 active+clean; 4.4 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 194 op/s; 1/217158 objects degraded (0.000%); 54049/217158 objects misplaced (24.889%); 2.6 MiB/s, 66 objects/s recovering 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000139+0000 mon.a (mon.0) 1902 : cluster [WRN] Health detail: HEALTH_WARN nodeep-scrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull; Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000184+0000 mon.a (mon.0) 1903 : cluster [WRN] [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000198+0000 mon.a (mon.0) 1904 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000211+0000 mon.a (mon.0) 1905 : cluster [WRN] pg 5.3 is active+remapped+backfill_toofull, acting [3,5,7] 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000224+0000 mon.a (mon.0) 1906 : cluster [WRN] pg 5.5 is active+remapped+backfill_toofull, acting [4,7,3] 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000247+0000 mon.a (mon.0) 1907 : cluster [WRN] pg 5.9 is active+remapped+backfill_toofull, acting [7,4,1] 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000260+0000 mon.a (mon.0) 1908 : cluster [WRN] pg 11.0 is active+remapped+backfill_toofull, acting [3,7,1] 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000271+0000 mon.a (mon.0) 1909 : cluster [WRN] pg 11.1 is active+remapped+backfill_toofull, acting [2,3,4] 2024-02-22T19:50:01.010 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000281+0000 mon.a (mon.0) 1910 : cluster [WRN] pg 11.5 is active+remapped+backfill_toofull, acting [7,5,3] 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000291+0000 mon.a (mon.0) 1911 : cluster [WRN] pg 11.8 is active+remapped+backfill_toofull, acting [3,5,2] 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000301+0000 mon.a (mon.0) 1912 : cluster [WRN] pg 11.b is active+remapped+backfill_toofull, acting [3,4,1] 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000312+0000 mon.a (mon.0) 1913 : cluster [WRN] pg 11.c is active+remapped+backfill_toofull, acting [2,4,1] 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000322+0000 mon.a (mon.0) 1914 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000336+0000 mon.a (mon.0) 1915 : cluster [WRN] pg 4.18 is active+recovery_wait+degraded, acting [4,6,3] 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000347+0000 mon.a (mon.0) 1916 : cluster [WRN] pg 11.2 is stuck undersized for 73s, current state active+recovering+undersized+remapped, last acting [2,7] 2024-02-22T19:50:01.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.000357+0000 mon.a (mon.0) 1917 : cluster [WRN] pg 11.d is active+recovering+degraded+remapped, acting [2,4,5] 2024-02-22T19:50:01.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:59.662335+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e549: 8 total, 8 up, 7 in 2024-02-22T19:50:01.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:49:59.887991+0000 mgr.y (mgr.14535) 798 : cluster [DBG] pgmap v1105: 128 pgs: 1 active+recovering+remapped, 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 9 active+remapped+backfill_wait, 6 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 108 active+clean; 4.4 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 194 op/s; 1/217158 objects degraded (0.000%); 54049/217158 objects misplaced (24.889%); 2.6 MiB/s, 66 objects/s recovering 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000139+0000 mon.a (mon.0) 1902 : cluster [WRN] Health detail: HEALTH_WARN nodeep-scrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull; Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000184+0000 mon.a (mon.0) 1903 : cluster [WRN] [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000198+0000 mon.a (mon.0) 1904 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000211+0000 mon.a (mon.0) 1905 : cluster [WRN] pg 5.3 is active+remapped+backfill_toofull, acting [3,5,7] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000224+0000 mon.a (mon.0) 1906 : cluster [WRN] pg 5.5 is active+remapped+backfill_toofull, acting [4,7,3] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000247+0000 mon.a (mon.0) 1907 : cluster [WRN] pg 5.9 is active+remapped+backfill_toofull, acting [7,4,1] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000260+0000 mon.a (mon.0) 1908 : cluster [WRN] pg 11.0 is active+remapped+backfill_toofull, acting [3,7,1] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000271+0000 mon.a (mon.0) 1909 : cluster [WRN] pg 11.1 is active+remapped+backfill_toofull, acting [2,3,4] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000281+0000 mon.a (mon.0) 1910 : cluster [WRN] pg 11.5 is active+remapped+backfill_toofull, acting [7,5,3] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000291+0000 mon.a (mon.0) 1911 : cluster [WRN] pg 11.8 is active+remapped+backfill_toofull, acting [3,5,2] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000301+0000 mon.a (mon.0) 1912 : cluster [WRN] pg 11.b is active+remapped+backfill_toofull, acting [3,4,1] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000312+0000 mon.a (mon.0) 1913 : cluster [WRN] pg 11.c is active+remapped+backfill_toofull, acting [2,4,1] 2024-02-22T19:50:01.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000322+0000 mon.a (mon.0) 1914 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 5/219492 objects degraded (0.002%), 2 pgs degraded, 1 pg undersized 2024-02-22T19:50:01.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000336+0000 mon.a (mon.0) 1915 : cluster [WRN] pg 4.18 is active+recovery_wait+degraded, acting [4,6,3] 2024-02-22T19:50:01.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000347+0000 mon.a (mon.0) 1916 : cluster [WRN] pg 11.2 is stuck undersized for 73s, current state active+recovering+undersized+remapped, last acting [2,7] 2024-02-22T19:50:01.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.000357+0000 mon.a (mon.0) 1917 : cluster [WRN] pg 11.d is active+recovering+degraded+remapped, acting [2,4,5] 2024-02-22T19:50:01.160 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:01.166 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:01.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:01.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:01.338 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:02.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:00.659563+0000 mon.a (mon.0) 1918 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:50:02.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:00.659563+0000 mon.a (mon.0) 1918 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:50:02.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:00.659563+0000 mon.a (mon.0) 1918 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:50:02.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:50:02.732 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:01.888642+0000 mgr.y (mgr.14535) 799 : cluster [DBG] pgmap v1106: 128 pgs: 3 active+recovery_wait+degraded, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 317 op/s; 58/214506 objects degraded (0.027%); 51666/214506 objects misplaced (24.086%); 3.7 MiB/s, 83 objects/s recovering 2024-02-22T19:50:02.732 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:02.622155+0000 mon.a (mon.0) 1919 : cluster [WRN] Health check update: Degraded data redundancy: 1/217158 objects degraded (0.000%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:50:02.733 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:01.888642+0000 mgr.y (mgr.14535) 799 : cluster [DBG] pgmap v1106: 128 pgs: 3 active+recovery_wait+degraded, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 317 op/s; 58/214506 objects degraded (0.027%); 51666/214506 objects misplaced (24.086%); 3.7 MiB/s, 83 objects/s recovering 2024-02-22T19:50:02.733 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:02.622155+0000 mon.a (mon.0) 1919 : cluster [WRN] Health check update: Degraded data redundancy: 1/217158 objects degraded (0.000%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:50:02.733 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:02.633363+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e550: 8 total, 8 up, 7 in 2024-02-22T19:50:03.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:02.633363+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e550: 8 total, 8 up, 7 in 2024-02-22T19:50:03.031 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:03.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:03.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:03.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:01.888642+0000 mgr.y (mgr.14535) 799 : cluster [DBG] pgmap v1106: 128 pgs: 3 active+recovery_wait+degraded, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 317 op/s; 58/214506 objects degraded (0.027%); 51666/214506 objects misplaced (24.086%); 3.7 MiB/s, 83 objects/s recovering 2024-02-22T19:50:03.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:02.622155+0000 mon.a (mon.0) 1919 : cluster [WRN] Health check update: Degraded data redundancy: 1/217158 objects degraded (0.000%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:50:03.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:02.633363+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e550: 8 total, 8 up, 7 in 2024-02-22T19:50:03.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:03.235 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:03.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:03.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:03.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:03.396 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 5, 4, 6, 1] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:50:03.396 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:50:03.397 INFO:tasks.thrashosds.thrasher:fixing pg num pool .rgw.root 2024-02-22T19:50:03.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:50:03.400 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:03.450 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:03.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:03.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:03.611 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:03.638 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:03.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:03.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:03.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:03.790 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:03.823 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:03.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:03.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:03.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:04.057 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:04.058 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:04.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:04.436 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:04.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:04.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:04.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:50:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:50:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:05.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:05.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:05.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:05.404 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:05.410 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:05.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:05.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:03.889159+0000 mgr.y (mgr.14535) 800 : cluster [DBG] pgmap v1108: 128 pgs: 3 active+recovery_wait+degraded, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 276 op/s; 58/214506 objects degraded (0.027%); 51666/214506 objects misplaced (24.086%); 3.3 MiB/s, 72 objects/s recovering 2024-02-22T19:50:05.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:04.391795+0000 mon.a (mon.0) 1921 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:05.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:04.940349+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? 172.21.15.53:0/2330847217' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:05.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:03.889159+0000 mgr.y (mgr.14535) 800 : cluster [DBG] pgmap v1108: 128 pgs: 3 active+recovery_wait+degraded, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 276 op/s; 58/214506 objects degraded (0.027%); 51666/214506 objects misplaced (24.086%); 3.3 MiB/s, 72 objects/s recovering 2024-02-22T19:50:05.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:04.391795+0000 mon.a (mon.0) 1921 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:05.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:04.940349+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? 172.21.15.53:0/2330847217' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:05.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:05.574 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:50:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:05.830 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:05.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:05.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:50:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:50:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:03.889159+0000 mgr.y (mgr.14535) 800 : cluster [DBG] pgmap v1108: 128 pgs: 3 active+recovery_wait+degraded, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 276 op/s; 58/214506 objects degraded (0.027%); 51666/214506 objects misplaced (24.086%); 3.3 MiB/s, 72 objects/s recovering 2024-02-22T19:50:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:04.391795+0000 mon.a (mon.0) 1921 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:05.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:04.940349+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? 172.21.15.53:0/2330847217' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:05.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:05.981 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:06.226 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:06.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:06.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:06.290 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:06.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:06.411 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:06.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:05.390071+0000 mon.a (mon.0) 1923 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:50:06.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:05.418285+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.53:0/2330847217' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:50:06.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:05.418351+0000 mon.a (mon.0) 1925 : cluster [DBG] osdmap e551: 8 total, 8 up, 7 in 2024-02-22T19:50:06.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:05.566984+0000 mgr.y (mgr.14535) 801 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:06.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:05.390071+0000 mon.a (mon.0) 1923 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:50:06.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:05.418285+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.53:0/2330847217' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:50:06.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:05.418351+0000 mon.a (mon.0) 1925 : cluster [DBG] osdmap e551: 8 total, 8 up, 7 in 2024-02-22T19:50:06.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:05.566984+0000 mgr.y (mgr.14535) 801 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:06.516 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:06.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:06.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:06.553 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:06.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:06.682 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:05.390071+0000 mon.a (mon.0) 1923 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:50:06.682 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:05.418285+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.53:0/2330847217' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:50:06.682 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:05.418351+0000 mon.a (mon.0) 1925 : cluster [DBG] osdmap e551: 8 total, 8 up, 7 in 2024-02-22T19:50:06.682 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:05.566984+0000 mgr.y (mgr.14535) 801 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:06.688 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:06.724 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:06.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:06.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:07.002 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:50:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:50:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:50:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:50:07.003 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:50:07.003 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:07.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:07.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:07.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:07.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:05.889635+0000 mgr.y (mgr.14535) 802 : cluster [DBG] pgmap v1110: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 822 B/s rd, 0 B/s wr, 662 op/s; 209/204792 objects degraded (0.102%); 49269/204792 objects misplaced (24.058%); 6.6 MiB/s, 110 objects/s recovering 2024-02-22T19:50:07.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:05.889635+0000 mgr.y (mgr.14535) 802 : cluster [DBG] pgmap v1110: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 822 B/s rd, 0 B/s wr, 662 op/s; 209/204792 objects degraded (0.102%); 49269/204792 objects misplaced (24.058%); 6.6 MiB/s, 110 objects/s recovering 2024-02-22T19:50:07.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:05.889635+0000 mgr.y (mgr.14535) 802 : cluster [DBG] pgmap v1110: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 822 B/s rd, 0 B/s wr, 662 op/s; 209/204792 objects degraded (0.102%); 49269/204792 objects misplaced (24.058%); 6.6 MiB/s, 110 objects/s recovering 2024-02-22T19:50:08.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:50:08.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:07.626863+0000 mon.a (mon.0) 1926 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:50:08.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:07.626888+0000 mon.a (mon.0) 1927 : cluster [WRN] Health check update: Degraded data redundancy: 209/204792 objects degraded (0.102%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:08.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:07.638271+0000 mon.a (mon.0) 1928 : cluster [DBG] osdmap e552: 8 total, 8 up, 7 in 2024-02-22T19:50:08.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:08.419384+0000 mon.a (mon.0) 1929 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:07.626863+0000 mon.a (mon.0) 1926 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:50:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:07.626888+0000 mon.a (mon.0) 1927 : cluster [WRN] Health check update: Degraded data redundancy: 209/204792 objects degraded (0.102%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:07.638271+0000 mon.a (mon.0) 1928 : cluster [DBG] osdmap e552: 8 total, 8 up, 7 in 2024-02-22T19:50:08.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:08.419384+0000 mon.a (mon.0) 1929 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:08.765 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:08.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:08.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:07.626863+0000 mon.a (mon.0) 1926 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:50:08.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:07.626888+0000 mon.a (mon.0) 1927 : cluster [WRN] Health check update: Degraded data redundancy: 209/204792 objects degraded (0.102%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:08.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:07.638271+0000 mon.a (mon.0) 1928 : cluster [DBG] osdmap e552: 8 total, 8 up, 7 in 2024-02-22T19:50:08.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:08.419384+0000 mon.a (mon.0) 1929 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:08.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:08.946 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:09.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:07.890527+0000 mgr.y (mgr.14535) 803 : cluster [DBG] pgmap v1112: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 540 op/s; 209/204792 objects degraded (0.102%); 49269/204792 objects misplaced (24.058%); 5.4 MiB/s, 85 objects/s recovering 2024-02-22T19:50:09.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:09.405527+0000 mon.a (mon.0) 1930 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:09.757 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:50:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:09] "GET /metrics HTTP/1.1" 200 176734 "" "Prometheus/2.33.4" 2024-02-22T19:50:09.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:07.890527+0000 mgr.y (mgr.14535) 803 : cluster [DBG] pgmap v1112: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 540 op/s; 209/204792 objects degraded (0.102%); 49269/204792 objects misplaced (24.058%); 5.4 MiB/s, 85 objects/s recovering 2024-02-22T19:50:09.758 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:09.405527+0000 mon.a (mon.0) 1930 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:09.847 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:09.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:09.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:09.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:07.890527+0000 mgr.y (mgr.14535) 803 : cluster [DBG] pgmap v1112: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 540 op/s; 209/204792 objects degraded (0.102%); 49269/204792 objects misplaced (24.058%); 5.4 MiB/s, 85 objects/s recovering 2024-02-22T19:50:09.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:09.405527+0000 mon.a (mon.0) 1930 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:09.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:10.049 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:10.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:10.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:10.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:10.320 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:10.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:10.348 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:50:10.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:10.582 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:10.753 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:09.398818+0000 osd.0 (osd.0) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:10.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:09.398901+0000 osd.0 (osd.0) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:10.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:10.421235+0000 mon.a (mon.0) 1931 : cluster [DBG] osdmap e553: 8 total, 8 up, 7 in 2024-02-22T19:50:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:09.398818+0000 osd.0 (osd.0) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:09.398901+0000 osd.0 (osd.0) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:10.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:10.421235+0000 mon.a (mon.0) 1931 : cluster [DBG] osdmap e553: 8 total, 8 up, 7 in 2024-02-22T19:50:10.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:09.398818+0000 osd.0 (osd.0) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:10.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:09.398901+0000 osd.0 (osd.0) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:10.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:10.421235+0000 mon.a (mon.0) 1931 : cluster [DBG] osdmap e553: 8 total, 8 up, 7 in 2024-02-22T19:50:11.565 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:09.892028+0000 mgr.y (mgr.14535) 804 : cluster [DBG] pgmap v1113: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.1 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 663 op/s; 209/200097 objects degraded (0.104%); 48195/200097 objects misplaced (24.086%); 6.6 MiB/s, 105 objects/s recovering 2024-02-22T19:50:11.565 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:09.892028+0000 mgr.y (mgr.14535) 804 : cluster [DBG] pgmap v1113: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.1 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 663 op/s; 209/200097 objects degraded (0.104%); 48195/200097 objects misplaced (24.086%); 6.6 MiB/s, 105 objects/s recovering 2024-02-22T19:50:11.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:09.892028+0000 mgr.y (mgr.14535) 804 : cluster [DBG] pgmap v1113: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 5 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.1 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 663 op/s; 209/200097 objects degraded (0.104%); 48195/200097 objects misplaced (24.086%); 6.6 MiB/s, 105 objects/s recovering 2024-02-22T19:50:12.911 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:12.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:12.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:12.992 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:50:13.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:13.022 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:50:13.064 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:50:13.114 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:50:13.131 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:13.173 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:50:13.221 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:50:13.246 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:50:13.284 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:50:13.310 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.310 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:50:13.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:11.892442+0000 mgr.y (mgr.14535) 805 : cluster [DBG] pgmap v1115: 128 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 790 B/s rd, 0 B/s wr, 486 op/s; 349/195366 objects degraded (0.179%); 46937/195366 objects misplaced (24.025%); 4.9 MiB/s, 78 objects/s recovering 2024-02-22T19:50:13.653 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:12.629031+0000 mon.a (mon.0) 1932 : cluster [WRN] Health check update: Degraded data redundancy: 349/195366 objects degraded (0.179%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:13.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:12.648659+0000 mon.a (mon.0) 1933 : cluster [DBG] osdmap e554: 8 total, 8 up, 7 in 2024-02-22T19:50:13.654 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:12.972109+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.53:0/3360639887' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:13.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:11.892442+0000 mgr.y (mgr.14535) 805 : cluster [DBG] pgmap v1115: 128 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 790 B/s rd, 0 B/s wr, 486 op/s; 349/195366 objects degraded (0.179%); 46937/195366 objects misplaced (24.025%); 4.9 MiB/s, 78 objects/s recovering 2024-02-22T19:50:13.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:12.629031+0000 mon.a (mon.0) 1932 : cluster [WRN] Health check update: Degraded data redundancy: 349/195366 objects degraded (0.179%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:13.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:12.648659+0000 mon.a (mon.0) 1933 : cluster [DBG] osdmap e554: 8 total, 8 up, 7 in 2024-02-22T19:50:13.655 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:12.972109+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.53:0/3360639887' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:11.892442+0000 mgr.y (mgr.14535) 805 : cluster [DBG] pgmap v1115: 128 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 790 B/s rd, 0 B/s wr, 486 op/s; 349/195366 objects degraded (0.179%); 46937/195366 objects misplaced (24.025%); 4.9 MiB/s, 78 objects/s recovering 2024-02-22T19:50:13.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:12.629031+0000 mon.a (mon.0) 1932 : cluster [WRN] Health check update: Degraded data redundancy: 349/195366 objects degraded (0.179%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:13.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:12.648659+0000 mon.a (mon.0) 1933 : cluster [DBG] osdmap e554: 8 total, 8 up, 7 in 2024-02-22T19:50:13.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:12.972109+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.53:0/3360639887' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:13.909 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:13.918 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:13.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:14.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:14.091 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:14.146 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:14.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:14.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:14.158 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:14.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:14.414 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:14.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:14.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:14.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:14.575 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:14.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:14.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:14.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:14.740 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:14.786 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:14.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:14.854 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:50:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:50:14.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:14.972 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:14.978 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:14.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:15.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:15.141 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:15.192 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:15.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:15.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:15.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:15.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:15.438 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:50:15.452 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:50:15.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:15.513 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:13.894087+0000 mgr.y (mgr.14535) 806 : cluster [DBG] pgmap v1117: 128 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 818 B/s rd, 0 B/s wr, 502 op/s; 349/195366 objects degraded (0.179%); 46937/195366 objects misplaced (24.025%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:13.894087+0000 mgr.y (mgr.14535) 806 : cluster [DBG] pgmap v1117: 128 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 818 B/s rd, 0 B/s wr, 502 op/s; 349/195366 objects degraded (0.179%); 46937/195366 objects misplaced (24.025%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:13.894087+0000 mgr.y (mgr.14535) 806 : cluster [DBG] pgmap v1117: 128 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 1 active+recovering+degraded+remapped, 107 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 818 B/s rd, 0 B/s wr, 502 op/s; 349/195366 objects degraded (0.179%); 46937/195366 objects misplaced (24.025%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:15.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:50:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:50:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:15.874 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:15.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:15.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:16.082 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:16.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:16.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:16.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:16.269 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:14.880276+0000 mgr.y (mgr.14535) 807 : audit [DBG] from='client.25750 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:50:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:15.581291+0000 mgr.y (mgr.14535) 808 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:14.880276+0000 mgr.y (mgr.14535) 807 : audit [DBG] from='client.25750 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:50:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:15.581291+0000 mgr.y (mgr.14535) 808 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:16.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:14.880276+0000 mgr.y (mgr.14535) 807 : audit [DBG] from='client.25750 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:50:16.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:15.581291+0000 mgr.y (mgr.14535) 808 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:15.898103+0000 mgr.y (mgr.14535) 809 : cluster [DBG] pgmap v1118: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 513/186147 objects degraded (0.276%); 44632/186147 objects misplaced (23.977%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:17.886 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:15.898103+0000 mgr.y (mgr.14535) 809 : cluster [DBG] pgmap v1118: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 513/186147 objects degraded (0.276%); 44632/186147 objects misplaced (23.977%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:17.886 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:15.898103+0000 mgr.y (mgr.14535) 809 : cluster [DBG] pgmap v1118: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 513/186147 objects degraded (0.276%); 44632/186147 objects misplaced (23.977%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:17.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:50:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:18.503 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:18.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:18.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:18.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:18.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:18.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:17.635261+0000 mon.a (mon.0) 1934 : cluster [WRN] Health check update: Degraded data redundancy: 513/186147 objects degraded (0.276%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:18.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:17.635261+0000 mon.a (mon.0) 1934 : cluster [WRN] Health check update: Degraded data redundancy: 513/186147 objects degraded (0.276%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:18.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:18.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:17.635261+0000 mon.a (mon.0) 1934 : cluster [WRN] Health check update: Degraded data redundancy: 513/186147 objects degraded (0.276%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:18.874 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:18.935 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:18.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:18.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:19.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:19.119 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:19.507 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:50:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:19] "GET /metrics HTTP/1.1" 200 176734 "" "Prometheus/2.33.4" 2024-02-22T19:50:19.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:17.899260+0000 mgr.y (mgr.14535) 810 : cluster [DBG] pgmap v1119: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 581 op/s; 513/186147 objects degraded (0.276%); 44632/186147 objects misplaced (23.977%); 5.8 MiB/s, 93 objects/s recovering 2024-02-22T19:50:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:17.899260+0000 mgr.y (mgr.14535) 810 : cluster [DBG] pgmap v1119: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 581 op/s; 513/186147 objects degraded (0.276%); 44632/186147 objects misplaced (23.977%); 5.8 MiB/s, 93 objects/s recovering 2024-02-22T19:50:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:17.899260+0000 mgr.y (mgr.14535) 810 : cluster [DBG] pgmap v1119: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 581 op/s; 513/186147 objects degraded (0.276%); 44632/186147 objects misplaced (23.977%); 5.8 MiB/s, 93 objects/s recovering 2024-02-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:20.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:20.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:20.206 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:20.270 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:20.305 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:20.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:20.307 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:20.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:20.478 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":1117,"stamp":"2024-02-22T19:50:13.893718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4255192764,"num_objects":65122,"num_object_clones":0,"num_object_copies":195366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":349,"num_objects_misplaced":46937,"num_objects_unfound":0,"num_objects_dirty":65122,"num_whiteouts":0,"num_read":2785,"num_read_kb":2537,"num_write":123932,"num_write_kb":6011562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24171,"num_bytes_recovered":1530752573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75741,"ondisk_log_size":75741,"up":384,"acting":382,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":19038816,"kb_used_data":13940712,"kb_used_omap":0,"kb_used_meta":5097536,"kb_avail":637177248,"statfs":{"total":671965249536,"available":652469501952,"internally_reserved":0,"allocated":14275289088,"data_stored":14260183513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5219876864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,8,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-205914112,"num_objects":-3142,"num_object_clones":0,"num_object_copies":-9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":140,"num_objects_misplaced":-2332,"num_objects_unfound":0,"num_objects_dirty":-3142,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3142,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":508,"num_bytes_recovered":33292288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.256476"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":567,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444082+0000","last_change":"2024-02-22T19:49:38.793457+0000","last_active":"2024-02-22T19:50:05.444082+0000","last_peered":"2024-02-22T19:50:05.444082+0000","last_clean":"2024-02-22T19:50:05.444082+0000","last_became_active":"2024-02-22T19:49:38.793084+0000","last_became_peered":"2024-02-22T19:49:38.793084+0000","last_unstale":"2024-02-22T19:50:05.444082+0000","last_undegraded":"2024-02-22T19:50:05.444082+0000","last_fullsized":"2024-02-22T19:50:05.444082+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:20.457620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:20.457620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"504'21","reported_seq":161,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428227+0000","last_change":"2024-02-22T19:49:57.037858+0000","last_active":"2024-02-22T19:50:05.428227+0000","last_peered":"2024-02-22T19:50:05.428227+0000","last_clean":"2024-02-22T19:50:05.428227+0000","last_became_active":"2024-02-22T19:49:56.672847+0000","last_became_peered":"2024-02-22T19:49:56.672847+0000","last_unstale":"2024-02-22T19:50:05.428227+0000","last_undegraded":"2024-02-22T19:50:05.428227+0000","last_fullsized":"2024-02-22T19:50:05.428227+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:46:09.901351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.901351+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":553,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444347+0000","last_change":"2024-02-22T19:49:38.809030+0000","last_active":"2024-02-22T19:50:05.444347+0000","last_peered":"2024-02-22T19:50:05.444347+0000","last_clean":"2024-02-22T19:50:05.444347+0000","last_became_active":"2024-02-22T19:49:38.802266+0000","last_became_peered":"2024-02-22T19:49:38.802266+0000","last_unstale":"2024-02-22T19:50:05.444347+0000","last_undegraded":"2024-02-22T19:50:05.444347+0000","last_fullsized":"2024-02-22T19:50:05.444347+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:15.211416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:48:15.211416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"506'39","reported_seq":660,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.448591+0000","last_change":"2024-02-22T19:48:02.098450+0000","last_active":"2024-02-22T19:50:05.448591+0000","last_peered":"2024-02-22T19:50:05.448591+0000","last_clean":"2024-02-22T19:50:05.448591+0000","last_became_active":"2024-02-22T19:39:22.269808+0000","last_became_peered":"2024-02-22T19:39:22.269808+0000","last_unstale":"2024-02-22T19:50:05.448591+0000","last_undegraded":"2024-02-22T19:50:05.448591+0000","last_fullsized":"2024-02-22T19:50:05.448591+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:48:02.098338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.098338+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":399,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444424+0000","last_change":"2024-02-22T19:49:38.797993+0000","last_active":"2024-02-22T19:50:05.444424+0000","last_peered":"2024-02-22T19:50:05.444424+0000","last_clean":"2024-02-22T19:50:05.444424+0000","last_became_active":"2024-02-22T19:49:38.797444+0000","last_became_peered":"2024-02-22T19:49:38.797444+0000","last_unstale":"2024-02-22T19:50:05.444424+0000","last_undegraded":"2024-02-22T19:50:05.444424+0000","last_fullsized":"2024-02-22T19:50:05.444424+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:52.699342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:52.699342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"505'25","reported_seq":116,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428694+0000","last_change":"2024-02-22T19:48:02.141583+0000","last_active":"2024-02-22T19:50:05.428694+0000","last_peered":"2024-02-22T19:50:05.428694+0000","last_clean":"2024-02-22T19:50:05.428694+0000","last_became_active":"2024-02-22T19:47:40.534475+0000","last_became_peered":"2024-02-22T19:47:40.534475+0000","last_unstale":"2024-02-22T19:50:05.428694+0000","last_undegraded":"2024-02-22T19:50:05.428694+0000","last_fullsized":"2024-02-22T19:50:05.428694+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:48:02.141476+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.141476+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"484'2","reported_seq":442,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443883+0000","last_change":"2024-02-22T19:49:57.387995+0000","last_active":"2024-02-22T19:50:05.443883+0000","last_peered":"2024-02-22T19:50:05.443883+0000","last_clean":"2024-02-22T19:50:05.443883+0000","last_became_active":"2024-02-22T19:49:56.700262+0000","last_became_peered":"2024-02-22T19:49:56.700262+0000","last_unstale":"2024-02-22T19:50:05.443883+0000","last_undegraded":"2024-02-22T19:50:05.443883+0000","last_fullsized":"2024-02-22T19:50:05.443883+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"484'2","last_scrub_stamp":"2024-02-22T19:48:14.828252+0000","last_deep_scrub":"484'2","last_deep_scrub_stamp":"2024-02-22T19:48:14.828252+0000","last_clean_scrub_stamp":"2024-02-22T19:48:14.828252+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"505'9","reported_seq":136,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430530+0000","last_change":"2024-02-22T19:49:59.048672+0000","last_active":"2024-02-22T19:50:05.430530+0000","last_peered":"2024-02-22T19:50:05.430530+0000","last_clean":"2024-02-22T19:50:05.430530+0000","last_became_active":"2024-02-22T19:49:56.694783+0000","last_became_peered":"2024-02-22T19:49:56.694783+0000","last_unstale":"2024-02-22T19:50:05.430530+0000","last_undegraded":"2024-02-22T19:50:05.430530+0000","last_fullsized":"2024-02-22T19:50:05.430530+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":548,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-02-22T19:47:50.219857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:50.219857+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":115,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430576+0000","last_change":"2024-02-22T19:49:56.673734+0000","last_active":"2024-02-22T19:50:05.430576+0000","last_peered":"2024-02-22T19:50:05.430576+0000","last_clean":"2024-02-22T19:50:05.430576+0000","last_became_active":"2024-02-22T19:49:56.673235+0000","last_became_peered":"2024-02-22T19:49:56.673235+0000","last_unstale":"2024-02-22T19:50:05.430576+0000","last_undegraded":"2024-02-22T19:50:05.430576+0000","last_fullsized":"2024-02-22T19:50:05.430576+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:10.057585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:48:10.057585+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"505'46","reported_seq":662,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.435234+0000","last_change":"2024-02-22T19:49:58.594941+0000","last_active":"2024-02-22T19:50:05.435234+0000","last_peered":"2024-02-22T19:50:05.435234+0000","last_clean":"2024-02-22T19:50:05.435234+0000","last_became_active":"2024-02-22T19:49:56.720148+0000","last_became_peered":"2024-02-22T19:49:56.720148+0000","last_unstale":"2024-02-22T19:50:05.435234+0000","last_undegraded":"2024-02-22T19:50:05.435234+0000","last_fullsized":"2024-02-22T19:50:05.435234+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-02-22T19:47:48.162902+0000","last_deep_scrub":"172'22","last_deep_scrub_stamp":"2024-02-22T19:36:24.021111+0000","last_clean_scrub_stamp":"2024-02-22T19:47:48.162902+0000","log_size":46,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":115,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428827+0000","last_change":"2024-02-22T19:49:56.857891+0000","last_active":"2024-02-22T19:50:05.428827+0000","last_peered":"2024-02-22T19:50:05.428827+0000","last_clean":"2024-02-22T19:50:05.428827+0000","last_became_active":"2024-02-22T19:49:56.857508+0000","last_became_peered":"2024-02-22T19:49:56.857508+0000","last_unstale":"2024-02-22T19:50:05.428827+0000","last_undegraded":"2024-02-22T19:50:05.428827+0000","last_fullsized":"2024-02-22T19:50:05.428827+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:01.129956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:48:01.129956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"506'29","reported_seq":696,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.445904+0000","last_change":"2024-02-22T19:49:57.990330+0000","last_active":"2024-02-22T19:50:05.445904+0000","last_peered":"2024-02-22T19:50:05.445904+0000","last_clean":"2024-02-22T19:50:05.445904+0000","last_became_active":"2024-02-22T19:49:56.726217+0000","last_became_peered":"2024-02-22T19:49:56.726217+0000","last_unstale":"2024-02-22T19:50:05.445904+0000","last_undegraded":"2024-02-22T19:50:05.445904+0000","last_fullsized":"2024-02-22T19:50:05.445904+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'23","last_scrub_stamp":"2024-02-22T19:48:06.055130+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:48:06.055130+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430607+0000","last_change":"2024-02-22T19:49:56.713968+0000","last_active":"2024-02-22T19:50:05.430607+0000","last_peered":"2024-02-22T19:50:05.430607+0000","last_clean":"2024-02-22T19:50:05.430607+0000","last_became_active":"2024-02-22T19:49:56.713674+0000","last_became_peered":"2024-02-22T19:49:56.713674+0000","last_unstale":"2024-02-22T19:50:05.430607+0000","last_undegraded":"2024-02-22T19:50:05.430607+0000","last_fullsized":"2024-02-22T19:50:05.430607+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:02.967304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.967304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"506'30","reported_seq":428,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.439718+0000","last_change":"2024-02-22T19:49:58.864228+0000","last_active":"2024-02-22T19:50:05.439718+0000","last_peered":"2024-02-22T19:50:05.439718+0000","last_clean":"2024-02-22T19:50:05.439718+0000","last_became_active":"2024-02-22T19:49:56.695996+0000","last_became_peered":"2024-02-22T19:49:56.695996+0000","last_unstale":"2024-02-22T19:50:05.439718+0000","last_undegraded":"2024-02-22T19:50:05.439718+0000","last_fullsized":"2024-02-22T19:50:05.439718+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":548,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:12.323915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.323915+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":573,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.449326+0000","last_change":"2024-02-22T19:49:56.727748+0000","last_active":"2024-02-22T19:50:05.449326+0000","last_peered":"2024-02-22T19:50:05.449326+0000","last_clean":"2024-02-22T19:50:05.449326+0000","last_became_active":"2024-02-22T19:49:56.727363+0000","last_became_peered":"2024-02-22T19:49:56.727363+0000","last_unstale":"2024-02-22T19:50:05.449326+0000","last_undegraded":"2024-02-22T19:50:05.449326+0000","last_fullsized":"2024-02-22T19:50:05.449326+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:56.165274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:47:56.165274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"506'27","reported_seq":67,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437069+0000","last_change":"2024-02-22T19:49:56.737115+0000","last_active":"2024-02-22T19:50:05.437069+0000","last_peered":"2024-02-22T19:50:05.437069+0000","last_clean":"2024-02-22T19:50:05.437069+0000","last_became_active":"2024-02-22T19:49:56.676012+0000","last_became_peered":"2024-02-22T19:49:56.676012+0000","last_unstale":"2024-02-22T19:50:05.437069+0000","last_undegraded":"2024-02-22T19:50:05.437069+0000","last_fullsized":"2024-02-22T19:50:05.437069+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:45:09.809806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.809806+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":546,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437571+0000","last_change":"2024-02-22T19:47:40.527176+0000","last_active":"2024-02-22T19:50:05.437571+0000","last_peered":"2024-02-22T19:50:05.437571+0000","last_clean":"2024-02-22T19:50:05.437571+0000","last_became_active":"2024-02-22T19:47:40.526670+0000","last_became_peered":"2024-02-22T19:47:40.526670+0000","last_unstale":"2024-02-22T19:50:05.437571+0000","last_undegraded":"2024-02-22T19:50:05.437571+0000","last_fullsized":"2024-02-22T19:50:05.437571+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:06.262153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:06.262153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":567,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444075+0000","last_change":"2024-02-22T19:49:38.790649+0000","last_active":"2024-02-22T19:50:05.444075+0000","last_peered":"2024-02-22T19:50:05.444075+0000","last_clean":"2024-02-22T19:50:05.444075+0000","last_became_active":"2024-02-22T19:49:38.790206+0000","last_became_peered":"2024-02-22T19:49:38.790206+0000","last_unstale":"2024-02-22T19:50:05.444075+0000","last_undegraded":"2024-02-22T19:50:05.444075+0000","last_fullsized":"2024-02-22T19:50:05.444075+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:09.768846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:09.768846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"505'12","reported_seq":609,"reported_epoch":551,"state":"active+recovery_wait+degraded","last_fresh":"2024-02-22T19:50:05.449663+0000","last_change":"2024-02-22T19:49:56.729080+0000","last_active":"2024-02-22T19:50:05.449663+0000","last_peered":"2024-02-22T19:50:05.449663+0000","last_clean":"2024-02-22T19:49:54.663642+0000","last_became_active":"2024-02-22T19:49:56.728926+0000","last_became_peered":"2024-02-22T19:49:56.728926+0000","last_unstale":"2024-02-22T19:50:05.449663+0000","last_undegraded":"2024-02-22T19:49:56.672189+0000","last_fullsized":"2024-02-22T19:50:05.449663+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'8","last_scrub_stamp":"2024-02-22T19:46:09.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:09.018318+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":["3","4","6"],"object_location_counts":[{"shards":"3,4,6","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":576,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444319+0000","last_change":"2024-02-22T19:49:38.806492+0000","last_active":"2024-02-22T19:50:05.444319+0000","last_peered":"2024-02-22T19:50:05.444319+0000","last_clean":"2024-02-22T19:50:05.444319+0000","last_became_active":"2024-02-22T19:49:38.802055+0000","last_became_peered":"2024-02-22T19:49:38.802055+0000","last_unstale":"2024-02-22T19:50:05.444319+0000","last_undegraded":"2024-02-22T19:50:05.444319+0000","last_fullsized":"2024-02-22T19:50:05.444319+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:12.135980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:12.135980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":372,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444986+0000","last_change":"2024-02-22T19:49:56.703436+0000","last_active":"2024-02-22T19:50:05.444986+0000","last_peered":"2024-02-22T19:50:05.444986+0000","last_clean":"2024-02-22T19:50:05.444986+0000","last_became_active":"2024-02-22T19:49:56.703113+0000","last_became_peered":"2024-02-22T19:49:56.703113+0000","last_unstale":"2024-02-22T19:50:05.444986+0000","last_undegraded":"2024-02-22T19:50:05.444986+0000","last_fullsized":"2024-02-22T19:50:05.444986+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:12.759362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:12.759362+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"506'27","reported_seq":609,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.448476+0000","last_change":"2024-02-22T19:49:40.020725+0000","last_active":"2024-02-22T19:50:05.448476+0000","last_peered":"2024-02-22T19:50:05.448476+0000","last_clean":"2024-02-22T19:50:05.448476+0000","last_became_active":"2024-02-22T19:49:38.881889+0000","last_became_peered":"2024-02-22T19:49:38.881889+0000","last_unstale":"2024-02-22T19:50:05.448476+0000","last_undegraded":"2024-02-22T19:50:05.448476+0000","last_fullsized":"2024-02-22T19:50:05.448476+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":529,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:57.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:57.136969+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":567,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443114+0000","last_change":"2024-02-22T19:49:56.700708+0000","last_active":"2024-02-22T19:50:05.443114+0000","last_peered":"2024-02-22T19:50:05.443114+0000","last_clean":"2024-02-22T19:50:05.443114+0000","last_became_active":"2024-02-22T19:49:56.700417+0000","last_became_peered":"2024-02-22T19:49:56.700417+0000","last_unstale":"2024-02-22T19:50:05.443114+0000","last_undegraded":"2024-02-22T19:50:05.443114+0000","last_fullsized":"2024-02-22T19:50:05.443114+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:54.743150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.743150+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":586,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.439634+0000","last_change":"2024-02-22T19:48:03.835497+0000","last_active":"2024-02-22T19:50:05.439634+0000","last_peered":"2024-02-22T19:50:05.439634+0000","last_clean":"2024-02-22T19:50:05.439634+0000","last_became_active":"2024-02-22T19:47:40.538414+0000","last_became_peered":"2024-02-22T19:47:40.538414+0000","last_unstale":"2024-02-22T19:50:05.439634+0000","last_undegraded":"2024-02-22T19:50:05.439634+0000","last_fullsized":"2024-02-22T19:50:05.439634+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:03.835423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:48:03.835423+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"505'21","reported_seq":26,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430902+0000","last_change":"2024-02-22T19:49:57.041793+0000","last_active":"2024-02-22T19:50:05.430902+0000","last_peered":"2024-02-22T19:50:05.430902+0000","last_clean":"2024-02-22T19:50:05.430902+0000","last_became_active":"2024-02-22T19:49:56.866114+0000","last_became_peered":"2024-02-22T19:49:56.866114+0000","last_unstale":"2024-02-22T19:50:05.430902+0000","last_undegraded":"2024-02-22T19:50:05.430902+0000","last_fullsized":"2024-02-22T19:50:05.430902+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'15","last_scrub_stamp":"2024-02-22T19:48:18.011868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:18.011868+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":565,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.448113+0000","last_change":"2024-02-22T19:44:52.796416+0000","last_active":"2024-02-22T19:50:05.448113+0000","last_peered":"2024-02-22T19:50:05.448113+0000","last_clean":"2024-02-22T19:50:05.448113+0000","last_became_active":"2024-02-22T19:39:22.269365+0000","last_became_peered":"2024-02-22T19:39:22.269365+0000","last_unstale":"2024-02-22T19:50:05.448113+0000","last_undegraded":"2024-02-22T19:50:05.448113+0000","last_fullsized":"2024-02-22T19:50:05.448113+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:52.796310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:44:52.796310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":106,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428733+0000","last_change":"2024-02-22T19:49:38.806393+0000","last_active":"2024-02-22T19:50:05.428733+0000","last_peered":"2024-02-22T19:50:05.428733+0000","last_clean":"2024-02-22T19:50:05.428733+0000","last_became_active":"2024-02-22T19:49:38.806030+0000","last_became_peered":"2024-02-22T19:49:38.806030+0000","last_unstale":"2024-02-22T19:50:05.428733+0000","last_undegraded":"2024-02-22T19:50:05.428733+0000","last_fullsized":"2024-02-22T19:50:05.428733+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:00.092448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:00.092448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"506'27","reported_seq":184,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430456+0000","last_change":"2024-02-22T19:49:57.558752+0000","last_active":"2024-02-22T19:50:05.430456+0000","last_peered":"2024-02-22T19:50:05.430456+0000","last_clean":"2024-02-22T19:50:05.430456+0000","last_became_active":"2024-02-22T19:49:56.671777+0000","last_became_peered":"2024-02-22T19:49:56.671777+0000","last_unstale":"2024-02-22T19:50:05.430456+0000","last_undegraded":"2024-02-22T19:50:05.430456+0000","last_fullsized":"2024-02-22T19:50:05.430456+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:48:12.096428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:12.096428+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":559,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.440166+0000","last_change":"2024-02-22T19:47:58.764502+0000","last_active":"2024-02-22T19:50:05.440166+0000","last_peered":"2024-02-22T19:50:05.440166+0000","last_clean":"2024-02-22T19:50:05.440166+0000","last_became_active":"2024-02-22T19:47:40.526305+0000","last_became_peered":"2024-02-22T19:47:40.526305+0000","last_unstale":"2024-02-22T19:50:05.440166+0000","last_undegraded":"2024-02-22T19:50:05.440166+0000","last_fullsized":"2024-02-22T19:50:05.440166+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:58.764403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:39.596165+0000","last_clean_scrub_stamp":"2024-02-22T19:47:58.764403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"506'35","reported_seq":602,"reported_epoch":551,"state":"active+recovery_wait+degraded","last_fresh":"2024-02-22T19:50:05.443926+0000","last_change":"2024-02-22T19:49:56.730164+0000","last_active":"2024-02-22T19:50:05.443926+0000","last_peered":"2024-02-22T19:50:05.443926+0000","last_clean":"2024-02-22T19:49:54.664706+0000","last_became_active":"2024-02-22T19:49:56.729526+0000","last_became_peered":"2024-02-22T19:49:56.729526+0000","last_unstale":"2024-02-22T19:50:05.443926+0000","last_undegraded":"2024-02-22T19:49:56.677638+0000","last_fullsized":"2024-02-22T19:50:05.443926+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-02-22T19:46:10.060783+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:46:10.060783+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":["3","2","7"],"object_location_counts":[{"shards":"2,3,7","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":366,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437327+0000","last_change":"2024-02-22T19:49:56.685770+0000","last_active":"2024-02-22T19:50:05.437327+0000","last_peered":"2024-02-22T19:50:05.437327+0000","last_clean":"2024-02-22T19:50:05.437327+0000","last_became_active":"2024-02-22T19:49:56.685437+0000","last_became_peered":"2024-02-22T19:49:56.685437+0000","last_unstale":"2024-02-22T19:50:05.437327+0000","last_undegraded":"2024-02-22T19:50:05.437327+0000","last_fullsized":"2024-02-22T19:50:05.437327+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:13.706799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.706799+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"506'25","reported_seq":382,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444656+0000","last_change":"2024-02-22T19:47:07.646871+0000","last_active":"2024-02-22T19:50:05.444656+0000","last_peered":"2024-02-22T19:50:05.444656+0000","last_clean":"2024-02-22T19:50:05.444656+0000","last_became_active":"2024-02-22T19:37:57.178041+0000","last_became_peered":"2024-02-22T19:37:57.178041+0000","last_unstale":"2024-02-22T19:50:05.444656+0000","last_undegraded":"2024-02-22T19:50:05.444656+0000","last_fullsized":"2024-02-22T19:50:05.444656+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:07.646805+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-02-22T19:41:31.388143+0000","last_clean_scrub_stamp":"2024-02-22T19:47:07.646805+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":574,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.435128+0000","last_change":"2024-02-22T19:47:40.542335+0000","last_active":"2024-02-22T19:50:05.435128+0000","last_peered":"2024-02-22T19:50:05.435128+0000","last_clean":"2024-02-22T19:50:05.435128+0000","last_became_active":"2024-02-22T19:47:40.542055+0000","last_became_peered":"2024-02-22T19:47:40.542055+0000","last_unstale":"2024-02-22T19:50:05.435128+0000","last_undegraded":"2024-02-22T19:50:05.435128+0000","last_fullsized":"2024-02-22T19:50:05.435128+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:05.985178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:47:05.985178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":595,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437798+0000","last_change":"2024-02-22T19:49:38.805810+0000","last_active":"2024-02-22T19:50:05.437798+0000","last_peered":"2024-02-22T19:50:05.437798+0000","last_clean":"2024-02-22T19:50:05.437798+0000","last_became_active":"2024-02-22T19:49:38.805139+0000","last_became_peered":"2024-02-22T19:49:38.805139+0000","last_unstale":"2024-02-22T19:50:05.437798+0000","last_undegraded":"2024-02-22T19:50:05.437798+0000","last_fullsized":"2024-02-22T19:50:05.437798+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:05.818758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:05.818758+0000","last_clean_scrub_stamp":"2024-02-22T19:48:05.818758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"505'21","reported_seq":359,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444488+0000","last_change":"2024-02-22T19:48:14.196056+0000","last_active":"2024-02-22T19:50:05.444488+0000","last_peered":"2024-02-22T19:50:05.444488+0000","last_clean":"2024-02-22T19:50:05.444488+0000","last_became_active":"2024-02-22T19:39:22.270283+0000","last_became_peered":"2024-02-22T19:39:22.270283+0000","last_unstale":"2024-02-22T19:50:05.444488+0000","last_undegraded":"2024-02-22T19:50:05.444488+0000","last_fullsized":"2024-02-22T19:50:05.444488+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'15","last_scrub_stamp":"2024-02-22T19:48:14.195977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:14.195977+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":590,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.439558+0000","last_change":"2024-02-22T19:49:38.798612+0000","last_active":"2024-02-22T19:50:05.439558+0000","last_peered":"2024-02-22T19:50:05.439558+0000","last_clean":"2024-02-22T19:50:05.439558+0000","last_became_active":"2024-02-22T19:49:38.797702+0000","last_became_peered":"2024-02-22T19:49:38.797702+0000","last_unstale":"2024-02-22T19:50:05.439558+0000","last_undegraded":"2024-02-22T19:50:05.439558+0000","last_fullsized":"2024-02-22T19:50:05.439558+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.314522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.314522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437063+0000","last_change":"2024-02-22T19:49:38.844516+0000","last_active":"2024-02-22T19:50:05.437063+0000","last_peered":"2024-02-22T19:50:05.437063+0000","last_clean":"2024-02-22T19:50:05.437063+0000","last_became_active":"2024-02-22T19:49:38.844187+0000","last_became_peered":"2024-02-22T19:49:38.844187+0000","last_unstale":"2024-02-22T19:50:05.437063+0000","last_undegraded":"2024-02-22T19:50:05.437063+0000","last_fullsized":"2024-02-22T19:50:05.437063+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:16.904276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:05.811946+0000","last_clean_scrub_stamp":"2024-02-22T19:48:16.904276+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"506'18","reported_seq":585,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443921+0000","last_change":"2024-02-22T19:49:58.350469+0000","last_active":"2024-02-22T19:50:05.443921+0000","last_peered":"2024-02-22T19:50:05.443921+0000","last_clean":"2024-02-22T19:50:05.443921+0000","last_became_active":"2024-02-22T19:49:56.728115+0000","last_became_peered":"2024-02-22T19:49:56.728115+0000","last_unstale":"2024-02-22T19:50:05.443921+0000","last_undegraded":"2024-02-22T19:50:05.443921+0000","last_fullsized":"2024-02-22T19:50:05.443921+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-02-22T19:45:01.840227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:01.840227+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"552'7711","reported_seq":18300,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:09.528506+0000","last_change":"2024-02-22T19:49:57.832513+0000","last_active":"2024-02-22T19:50:09.528506+0000","last_peered":"2024-02-22T19:50:09.528506+0000","last_clean":"2024-02-22T19:49:55.643336+0000","last_became_active":"2024-02-22T19:49:57.819790+0000","last_became_peered":"2024-02-22T19:49:57.819790+0000","last_unstale":"2024-02-22T19:50:09.528506+0000","last_undegraded":"2024-02-22T19:50:09.528506+0000","last_fullsized":"2024-02-22T19:50:09.528506+0000","mapping_epoch":546,"log_start":"481'2817","ondisk_log_start":"481'2817","created":469,"last_epoch_clean":504,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4894,"ondisk_log_size":4894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":267583488,"num_objects":4083,"num_object_clones":0,"num_object_copies":12249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4083,"num_objects_unfound":0,"num_objects_dirty":4083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7711,"num_write_kb":377408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5231,"num_bytes_recovered":342818816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,4,7],"avail_no_missing":["5","4","7"],"object_location_counts":[{"shards":"4,5,7","objects":4083}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":426,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443561+0000","last_change":"2024-02-22T19:49:43.651280+0000","last_active":"2024-02-22T19:50:05.443561+0000","last_peered":"2024-02-22T19:50:05.443561+0000","last_clean":"2024-02-22T19:50:05.443561+0000","last_became_active":"2024-02-22T19:49:43.651033+0000","last_became_peered":"2024-02-22T19:49:43.651033+0000","last_unstale":"2024-02-22T19:50:05.443561+0000","last_undegraded":"2024-02-22T19:50:05.443561+0000","last_fullsized":"2024-02-22T19:50:05.443561+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":533,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:52.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:45:52.702473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":667,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.439199+0000","last_change":"2024-02-22T19:49:57.127928+0000","last_active":"2024-02-22T19:50:05.439199+0000","last_peered":"2024-02-22T19:50:05.439199+0000","last_clean":"2024-02-22T19:50:05.439199+0000","last_became_active":"2024-02-22T19:49:56.693371+0000","last_became_peered":"2024-02-22T19:49:56.693371+0000","last_unstale":"2024-02-22T19:50:05.439199+0000","last_undegraded":"2024-02-22T19:50:05.439199+0000","last_fullsized":"2024-02-22T19:50:05.439199+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.375370+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"505'39","reported_seq":481,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.442902+0000","last_change":"2024-02-22T19:49:39.401352+0000","last_active":"2024-02-22T19:50:05.442902+0000","last_peered":"2024-02-22T19:50:05.442902+0000","last_clean":"2024-02-22T19:50:05.442902+0000","last_became_active":"2024-02-22T19:49:38.790485+0000","last_became_peered":"2024-02-22T19:49:38.790485+0000","last_unstale":"2024-02-22T19:50:05.442902+0000","last_undegraded":"2024-02-22T19:50:05.442902+0000","last_fullsized":"2024-02-22T19:50:05.442902+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:03.485065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:03.485065+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"529'26","reported_seq":929,"reported_epoch":552,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:50:07.660866+0000","last_change":"2024-02-22T19:49:39.862560+0000","last_active":"2024-02-22T19:50:07.660866+0000","last_peered":"2024-02-22T19:50:07.660866+0000","last_clean":"2024-02-22T19:49:35.929015+0000","last_became_active":"2024-02-22T19:49:39.840512+0000","last_became_peered":"2024-02-22T19:49:39.840512+0000","last_unstale":"2024-02-22T19:50:07.660866+0000","last_undegraded":"2024-02-22T19:50:07.660866+0000","last_fullsized":"2024-02-22T19:50:07.660866+0000","mapping_epoch":528,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":525,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'16","last_scrub_stamp":"2024-02-22T19:46:15.312651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.312651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,4,1],"avail_no_missing":["7","1","3","4"],"object_location_counts":[{"shards":"1,4,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"552'7811","reported_seq":8292,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:10.079247+0000","last_change":"2024-02-22T19:49:57.695426+0000","last_active":"2024-02-22T19:50:10.079247+0000","last_peered":"2024-02-22T19:50:10.079247+0000","last_clean":"2024-02-22T19:50:10.079247+0000","last_became_active":"2024-02-22T19:49:57.672073+0000","last_became_peered":"2024-02-22T19:49:57.672073+0000","last_unstale":"2024-02-22T19:50:10.079247+0000","last_undegraded":"2024-02-22T19:50:10.079247+0000","last_fullsized":"2024-02-22T19:50:10.079247+0000","mapping_epoch":546,"log_start":"479'2598","ondisk_log_start":"479'2598","created":469,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":5213,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":269549568,"num_objects":4113,"num_object_clones":0,"num_object_copies":12339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7811,"num_write_kb":381568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":11862016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":557,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444272+0000","last_change":"2024-02-22T19:49:56.728695+0000","last_active":"2024-02-22T19:50:05.444272+0000","last_peered":"2024-02-22T19:50:05.444272+0000","last_clean":"2024-02-22T19:50:05.444272+0000","last_became_active":"2024-02-22T19:49:56.728419+0000","last_became_peered":"2024-02-22T19:49:56.728419+0000","last_unstale":"2024-02-22T19:50:05.444272+0000","last_undegraded":"2024-02-22T19:50:05.444272+0000","last_fullsized":"2024-02-22T19:50:05.444272+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:12.345713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:12.345713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":291,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437593+0000","last_change":"2024-02-22T19:49:56.717667+0000","last_active":"2024-02-22T19:50:05.437593+0000","last_peered":"2024-02-22T19:50:05.437593+0000","last_clean":"2024-02-22T19:50:05.437593+0000","last_became_active":"2024-02-22T19:49:56.717355+0000","last_became_peered":"2024-02-22T19:49:56.717355+0000","last_unstale":"2024-02-22T19:50:05.437593+0000","last_undegraded":"2024-02-22T19:50:05.437593+0000","last_fullsized":"2024-02-22T19:50:05.437593+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.069465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.069465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"505'24","reported_seq":181,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430753+0000","last_change":"2024-02-22T19:49:59.032052+0000","last_active":"2024-02-22T19:50:05.430753+0000","last_peered":"2024-02-22T19:50:05.430753+0000","last_clean":"2024-02-22T19:50:05.430753+0000","last_became_active":"2024-02-22T19:49:56.685005+0000","last_became_peered":"2024-02-22T19:49:56.685005+0000","last_unstale":"2024-02-22T19:50:05.430753+0000","last_undegraded":"2024-02-22T19:50:05.430753+0000","last_fullsized":"2024-02-22T19:50:05.430753+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":548,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'18","last_scrub_stamp":"2024-02-22T19:48:05.127961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:48:05.127961+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":598,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430277+0000","last_change":"2024-02-22T19:49:56.710660+0000","last_active":"2024-02-22T19:50:05.430277+0000","last_peered":"2024-02-22T19:50:05.430277+0000","last_clean":"2024-02-22T19:50:05.430277+0000","last_became_active":"2024-02-22T19:49:56.710320+0000","last_became_peered":"2024-02-22T19:49:56.710320+0000","last_unstale":"2024-02-22T19:50:05.430277+0000","last_undegraded":"2024-02-22T19:50:05.430277+0000","last_fullsized":"2024-02-22T19:50:05.430277+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:09.025536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:09.025536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"552'7790","reported_seq":7893,"reported_epoch":552,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-22T19:50:09.549852+0000","last_change":"2024-02-22T19:49:39.943887+0000","last_active":"2024-02-22T19:50:09.549852+0000","last_peered":"2024-02-22T19:50:09.549852+0000","last_clean":"2024-02-22T19:49:37.681321+0000","last_became_active":"2024-02-22T19:49:39.920578+0000","last_became_peered":"2024-02-22T19:49:39.920578+0000","last_unstale":"2024-02-22T19:50:09.549852+0000","last_undegraded":"2024-02-22T19:50:09.549852+0000","last_fullsized":"2024-02-22T19:50:09.549852+0000","mapping_epoch":528,"log_start":"482'2999","ondisk_log_start":"482'2999","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4791,"ondisk_log_size":4791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":271843328,"num_objects":4148,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4148,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7790,"num_write_kb":382016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,3],"avail_no_missing":["7","3","5"],"object_location_counts":[{"shards":"3,5,7","objects":4148}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":544,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.436707+0000","last_change":"2024-02-22T19:49:51.842814+0000","last_active":"2024-02-22T19:50:05.436707+0000","last_peered":"2024-02-22T19:50:05.436707+0000","last_clean":"2024-02-22T19:50:05.436707+0000","last_became_active":"2024-02-22T19:49:51.842580+0000","last_became_peered":"2024-02-22T19:49:51.842580+0000","last_unstale":"2024-02-22T19:50:05.436707+0000","last_undegraded":"2024-02-22T19:50:05.436707+0000","last_fullsized":"2024-02-22T19:50:05.436707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":541,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:02.833563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:48:02.833563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":611,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438146+0000","last_change":"2024-02-22T19:48:18.180488+0000","last_active":"2024-02-22T19:50:05.438146+0000","last_peered":"2024-02-22T19:50:05.438146+0000","last_clean":"2024-02-22T19:50:05.438146+0000","last_became_active":"2024-02-22T19:39:22.258692+0000","last_became_peered":"2024-02-22T19:39:22.258692+0000","last_unstale":"2024-02-22T19:50:05.438146+0000","last_undegraded":"2024-02-22T19:50:05.438146+0000","last_fullsized":"2024-02-22T19:50:05.438146+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:18.180384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:48:18.180384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":573,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.439957+0000","last_change":"2024-02-22T19:49:38.798629+0000","last_active":"2024-02-22T19:50:05.439957+0000","last_peered":"2024-02-22T19:50:05.439957+0000","last_clean":"2024-02-22T19:50:05.439957+0000","last_became_active":"2024-02-22T19:49:38.798274+0000","last_became_peered":"2024-02-22T19:49:38.798274+0000","last_unstale":"2024-02-22T19:50:05.439957+0000","last_undegraded":"2024-02-22T19:50:05.439957+0000","last_fullsized":"2024-02-22T19:50:05.439957+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:14.743654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:14.743654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"506'51","reported_seq":83,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438013+0000","last_change":"2024-02-22T19:49:57.513348+0000","last_active":"2024-02-22T19:50:05.438013+0000","last_peered":"2024-02-22T19:50:05.438013+0000","last_clean":"2024-02-22T19:50:05.438013+0000","last_became_active":"2024-02-22T19:49:56.676335+0000","last_became_peered":"2024-02-22T19:49:56.676335+0000","last_unstale":"2024-02-22T19:50:05.438013+0000","last_undegraded":"2024-02-22T19:50:05.438013+0000","last_fullsized":"2024-02-22T19:50:05.438013+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:47:47.887865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.887865+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.4","version":"552'7530","reported_seq":10903,"reported_epoch":552,"state":"active+recovering+remapped","last_fresh":"2024-02-22T19:50:09.598165+0000","last_change":"2024-02-22T19:49:40.476727+0000","last_active":"2024-02-22T19:50:09.598165+0000","last_peered":"2024-02-22T19:50:09.598165+0000","last_clean":"2024-02-22T19:48:42.869299+0000","last_became_active":"2024-02-22T19:49:40.030281+0000","last_became_peered":"2024-02-22T19:49:40.030281+0000","last_unstale":"2024-02-22T19:50:09.598165+0000","last_undegraded":"2024-02-22T19:50:09.598165+0000","last_fullsized":"2024-02-22T19:50:09.598165+0000","mapping_epoch":528,"log_start":"480'2748","ondisk_log_start":"480'2748","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4782,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":260177920,"num_objects":3970,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7530,"num_write_kb":368000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":106889216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,2,3],"avail_no_missing":["7","1","2"],"object_location_counts":[{"shards":"1,2,7","objects":2925},{"shards":"2,3,7","objects":1045}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":366,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438196+0000","last_change":"2024-02-22T19:49:47.790398+0000","last_active":"2024-02-22T19:50:05.438196+0000","last_peered":"2024-02-22T19:50:05.438196+0000","last_clean":"2024-02-22T19:50:05.438196+0000","last_became_active":"2024-02-22T19:49:47.790052+0000","last_became_peered":"2024-02-22T19:49:47.790052+0000","last_unstale":"2024-02-22T19:50:05.438196+0000","last_undegraded":"2024-02-22T19:50:05.438196+0000","last_fullsized":"2024-02-22T19:50:05.438196+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":537,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.877246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.877246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":594,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.435646+0000","last_change":"2024-02-22T19:47:40.540971+0000","last_active":"2024-02-22T19:50:05.435646+0000","last_peered":"2024-02-22T19:50:05.435646+0000","last_clean":"2024-02-22T19:50:05.435646+0000","last_became_active":"2024-02-22T19:47:40.540572+0000","last_became_peered":"2024-02-22T19:47:40.540572+0000","last_unstale":"2024-02-22T19:50:05.435646+0000","last_undegraded":"2024-02-22T19:50:05.435646+0000","last_fullsized":"2024-02-22T19:50:05.435646+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:00.864208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:00.864208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":385,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443792+0000","last_change":"2024-02-22T19:49:38.796813+0000","last_active":"2024-02-22T19:50:05.443792+0000","last_peered":"2024-02-22T19:50:05.443792+0000","last_clean":"2024-02-22T19:50:05.443792+0000","last_became_active":"2024-02-22T19:49:38.796284+0000","last_became_peered":"2024-02-22T19:49:38.796284+0000","last_unstale":"2024-02-22T19:50:05.443792+0000","last_undegraded":"2024-02-22T19:50:05.443792+0000","last_fullsized":"2024-02-22T19:50:05.443792+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:03.617526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:03.617526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"506'25","reported_seq":26,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.431168+0000","last_change":"2024-02-22T19:49:56.748909+0000","last_active":"2024-02-22T19:50:05.431168+0000","last_peered":"2024-02-22T19:50:05.431168+0000","last_clean":"2024-02-22T19:50:05.431168+0000","last_became_active":"2024-02-22T19:49:56.720861+0000","last_became_peered":"2024-02-22T19:49:56.720861+0000","last_unstale":"2024-02-22T19:50:05.431168+0000","last_undegraded":"2024-02-22T19:50:05.431168+0000","last_fullsized":"2024-02-22T19:50:05.431168+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'19","last_scrub_stamp":"2024-02-22T19:47:23.007849+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:47:23.007849+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"552'7674","reported_seq":8470,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:10.119655+0000","last_change":"2024-02-22T19:49:57.693266+0000","last_active":"2024-02-22T19:50:10.119655+0000","last_peered":"2024-02-22T19:50:10.119655+0000","last_clean":"2024-02-22T19:50:10.119655+0000","last_became_active":"2024-02-22T19:49:57.675986+0000","last_became_peered":"2024-02-22T19:49:57.675986+0000","last_unstale":"2024-02-22T19:50:10.119655+0000","last_undegraded":"2024-02-22T19:50:10.119655+0000","last_fullsized":"2024-02-22T19:50:10.119655+0000","mapping_epoch":546,"log_start":"487'3492","ondisk_log_start":"487'3492","created":469,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4182,"ondisk_log_size":4182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":264241152,"num_objects":4032,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7674,"num_write_kb":374592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":22151168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":401,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443978+0000","last_change":"2024-02-22T19:47:40.522595+0000","last_active":"2024-02-22T19:50:05.443978+0000","last_peered":"2024-02-22T19:50:05.443978+0000","last_clean":"2024-02-22T19:50:05.443978+0000","last_became_active":"2024-02-22T19:47:40.522250+0000","last_became_peered":"2024-02-22T19:47:40.522250+0000","last_unstale":"2024-02-22T19:50:05.443978+0000","last_undegraded":"2024-02-22T19:50:05.443978+0000","last_fullsized":"2024-02-22T19:50:05.443978+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:11.428898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:46:11.428898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":565,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444564+0000","last_change":"2024-02-22T19:49:59.691946+0000","last_active":"2024-02-22T19:50:05.444564+0000","last_peered":"2024-02-22T19:50:05.444564+0000","last_clean":"2024-02-22T19:50:05.444564+0000","last_became_active":"2024-02-22T19:49:59.691559+0000","last_became_peered":"2024-02-22T19:49:59.691559+0000","last_unstale":"2024-02-22T19:50:05.444564+0000","last_undegraded":"2024-02-22T19:50:05.444564+0000","last_fullsized":"2024-02-22T19:50:05.444564+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":549,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:13.030866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:13.030866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":390,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443483+0000","last_change":"2024-02-22T19:49:56.701978+0000","last_active":"2024-02-22T19:50:05.443483+0000","last_peered":"2024-02-22T19:50:05.443483+0000","last_clean":"2024-02-22T19:50:05.443483+0000","last_became_active":"2024-02-22T19:49:56.701645+0000","last_became_peered":"2024-02-22T19:49:56.701645+0000","last_unstale":"2024-02-22T19:50:05.443483+0000","last_undegraded":"2024-02-22T19:50:05.443483+0000","last_fullsized":"2024-02-22T19:50:05.443483+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.553324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.553324+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":119,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430654+0000","last_change":"2024-02-22T19:49:56.673859+0000","last_active":"2024-02-22T19:50:05.430654+0000","last_peered":"2024-02-22T19:50:05.430654+0000","last_clean":"2024-02-22T19:50:05.430654+0000","last_became_active":"2024-02-22T19:49:56.673250+0000","last_became_peered":"2024-02-22T19:49:56.673250+0000","last_unstale":"2024-02-22T19:50:05.430654+0000","last_undegraded":"2024-02-22T19:50:05.430654+0000","last_fullsized":"2024-02-22T19:50:05.430654+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.718312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.718312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"548'55","reported_seq":40,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.431681+0000","last_change":"2024-02-22T19:49:56.780381+0000","last_active":"2024-02-22T19:50:05.431681+0000","last_peered":"2024-02-22T19:50:05.431681+0000","last_clean":"2024-02-22T19:50:05.431681+0000","last_became_active":"2024-02-22T19:49:56.725545+0000","last_became_peered":"2024-02-22T19:49:56.725545+0000","last_unstale":"2024-02-22T19:50:05.431681+0000","last_undegraded":"2024-02-22T19:50:05.431681+0000","last_fullsized":"2024-02-22T19:50:05.431681+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'40","last_scrub_stamp":"2024-02-22T19:46:14.875950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:14.875950+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"548'38","reported_seq":578,"reported_epoch":551,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:05.429216+0000","last_change":"2024-02-22T19:49:56.695411+0000","last_active":"2024-02-22T19:50:05.429216+0000","last_peered":"2024-02-22T19:50:05.429216+0000","last_clean":"2024-02-22T19:49:36.751346+0000","last_became_active":"2024-02-22T19:49:56.694845+0000","last_became_peered":"2024-02-22T19:49:56.694845+0000","last_unstale":"2024-02-22T19:50:05.429216+0000","last_undegraded":"2024-02-22T19:50:05.429216+0000","last_fullsized":"2024-02-22T19:50:05.429216+0000","mapping_epoch":545,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":499,"parent":"0.0","parent_split_bits":0,"last_scrub":"473'24","last_scrub_stamp":"2024-02-22T19:48:13.962118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.962118+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[4,7,3],"avail_no_missing":["4","3","7"],"object_location_counts":[{"shards":"3,4,7","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.8","version":"552'7580","reported_seq":7684,"reported_epoch":552,"state":"active+remapped+backfill_wait+backfill_toofull","last_fresh":"2024-02-22T19:50:10.153467+0000","last_change":"2024-02-22T19:50:10.013974+0000","last_active":"2024-02-22T19:50:10.153467+0000","last_peered":"2024-02-22T19:50:10.153467+0000","last_clean":"2024-02-22T19:49:37.510212+0000","last_became_active":"2024-02-22T19:49:39.896795+0000","last_became_peered":"2024-02-22T19:49:39.896795+0000","last_unstale":"2024-02-22T19:50:10.153467+0000","last_undegraded":"2024-02-22T19:50:10.153467+0000","last_fullsized":"2024-02-22T19:50:10.153467+0000","mapping_epoch":528,"log_start":"487'3396","ondisk_log_start":"487'3396","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4184,"ondisk_log_size":4184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":255459328,"num_objects":3898,"num_object_clones":0,"num_object_copies":11694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3898,"num_objects_unfound":0,"num_objects_dirty":3898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7580,"num_write_kb":367296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,5,2],"avail_no_missing":["3","2","5"],"object_location_counts":[{"shards":"2,3,5","objects":3898}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":594,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437480+0000","last_change":"2024-02-22T19:47:40.526995+0000","last_active":"2024-02-22T19:50:05.437480+0000","last_peered":"2024-02-22T19:50:05.437480+0000","last_clean":"2024-02-22T19:50:05.437480+0000","last_became_active":"2024-02-22T19:47:40.526255+0000","last_became_peered":"2024-02-22T19:47:40.526255+0000","last_unstale":"2024-02-22T19:50:05.437480+0000","last_undegraded":"2024-02-22T19:50:05.437480+0000","last_fullsized":"2024-02-22T19:50:05.437480+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.507020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.507020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":544,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.440147+0000","last_change":"2024-02-22T19:49:38.805158+0000","last_active":"2024-02-22T19:50:05.440147+0000","last_peered":"2024-02-22T19:50:05.440147+0000","last_clean":"2024-02-22T19:50:05.440147+0000","last_became_active":"2024-02-22T19:49:38.804804+0000","last_became_peered":"2024-02-22T19:49:38.804804+0000","last_unstale":"2024-02-22T19:50:05.440147+0000","last_undegraded":"2024-02-22T19:50:05.440147+0000","last_fullsized":"2024-02-22T19:50:05.440147+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:15.447427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:47:15.447427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.432731+0000","last_change":"2024-02-22T19:49:56.724662+0000","last_active":"2024-02-22T19:50:05.432731+0000","last_peered":"2024-02-22T19:50:05.432731+0000","last_clean":"2024-02-22T19:50:05.432731+0000","last_became_active":"2024-02-22T19:49:56.724355+0000","last_became_peered":"2024-02-22T19:49:56.724355+0000","last_unstale":"2024-02-22T19:50:05.432731+0000","last_undegraded":"2024-02-22T19:50:05.432731+0000","last_fullsized":"2024-02-22T19:50:05.432731+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:58.646210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.646210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":92,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.442942+0000","last_change":"2024-02-22T19:47:40.533498+0000","last_active":"2024-02-22T19:50:05.442942+0000","last_peered":"2024-02-22T19:50:05.442942+0000","last_clean":"2024-02-22T19:50:05.442942+0000","last_became_active":"2024-02-22T19:47:40.533130+0000","last_became_peered":"2024-02-22T19:47:40.533130+0000","last_unstale":"2024-02-22T19:50:05.442942+0000","last_undegraded":"2024-02-22T19:50:05.442942+0000","last_fullsized":"2024-02-22T19:50:05.442942+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:17.081880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:52.175144+0000","last_clean_scrub_stamp":"2024-02-22T19:47:17.081880+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"505'21","reported_seq":28,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.432301+0000","last_change":"2024-02-22T19:49:56.754556+0000","last_active":"2024-02-22T19:50:05.432301+0000","last_peered":"2024-02-22T19:50:05.432301+0000","last_clean":"2024-02-22T19:50:05.432301+0000","last_became_active":"2024-02-22T19:49:56.719422+0000","last_became_peered":"2024-02-22T19:49:56.719422+0000","last_unstale":"2024-02-22T19:50:05.432301+0000","last_undegraded":"2024-02-22T19:50:05.432301+0000","last_fullsized":"2024-02-22T19:50:05.432301+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'17","last_scrub_stamp":"2024-02-22T19:46:58.874350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.874350+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":28,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":2310,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"415'30","reported_seq":660,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:07.662861+0000","last_change":"2024-02-22T19:47:10.006289+0000","last_active":"2024-02-22T19:50:07.662861+0000","last_peered":"2024-02-22T19:50:07.662861+0000","last_clean":"2024-02-22T19:50:07.662861+0000","last_became_active":"2024-02-22T19:45:01.732679+0000","last_became_peered":"2024-02-22T19:45:01.732679+0000","last_unstale":"2024-02-22T19:50:07.662861+0000","last_undegraded":"2024-02-22T19:50:07.662861+0000","last_fullsized":"2024-02-22T19:50:07.662861+0000","mapping_epoch":217,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":415,"parent":"0.0","parent_split_bits":0,"last_scrub":"415'30","last_scrub_stamp":"2024-02-22T19:47:10.006180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.006180+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"552'7566","reported_seq":8218,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:10.178010+0000","last_change":"2024-02-22T19:49:57.128923+0000","last_active":"2024-02-22T19:50:10.178010+0000","last_peered":"2024-02-22T19:50:10.178010+0000","last_clean":"2024-02-22T19:48:42.917141+0000","last_became_active":"2024-02-22T19:49:38.871535+0000","last_became_peered":"2024-02-22T19:49:38.871535+0000","last_unstale":"2024-02-22T19:50:10.178010+0000","last_undegraded":"2024-02-22T19:50:10.178010+0000","last_fullsized":"2024-02-22T19:50:10.178010+0000","mapping_epoch":527,"log_start":"487'3429","ondisk_log_start":"487'3429","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4137,"ondisk_log_size":4137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":261881856,"num_objects":3996,"num_object_clones":0,"num_object_copies":11988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3988,"num_objects_unfound":0,"num_objects_dirty":3996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7566,"num_write_kb":369984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17235968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,5,1],"avail_no_missing":["3","1","2","5"],"object_location_counts":[{"shards":"1,3,5","objects":3996}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":395,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444166+0000","last_change":"2024-02-22T19:49:55.691529+0000","last_active":"2024-02-22T19:50:05.444166+0000","last_peered":"2024-02-22T19:50:05.444166+0000","last_clean":"2024-02-22T19:50:05.444166+0000","last_became_active":"2024-02-22T19:49:55.691326+0000","last_became_peered":"2024-02-22T19:49:55.691326+0000","last_unstale":"2024-02-22T19:50:05.444166+0000","last_undegraded":"2024-02-22T19:50:05.444166+0000","last_fullsized":"2024-02-22T19:50:05.444166+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:08.833380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:17.656518+0000","last_clean_scrub_stamp":"2024-02-22T19:48:08.833380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":612,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.436889+0000","last_change":"2024-02-22T19:49:56.674730+0000","last_active":"2024-02-22T19:50:05.436889+0000","last_peered":"2024-02-22T19:50:05.436889+0000","last_clean":"2024-02-22T19:50:05.436889+0000","last_became_active":"2024-02-22T19:49:56.674265+0000","last_became_peered":"2024-02-22T19:49:56.674265+0000","last_unstale":"2024-02-22T19:50:05.436889+0000","last_undegraded":"2024-02-22T19:50:05.436889+0000","last_fullsized":"2024-02-22T19:50:05.436889+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_clean_scrub_stamp":"2024-02-22T19:47:26.529483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":571,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443488+0000","last_change":"2024-02-22T19:49:38.871775+0000","last_active":"2024-02-22T19:50:05.443488+0000","last_peered":"2024-02-22T19:50:05.443488+0000","last_clean":"2024-02-22T19:50:05.443488+0000","last_became_active":"2024-02-22T19:49:38.871417+0000","last_became_peered":"2024-02-22T19:49:38.871417+0000","last_unstale":"2024-02-22T19:50:05.443488+0000","last_undegraded":"2024-02-22T19:50:05.443488+0000","last_fullsized":"2024-02-22T19:50:05.443488+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:25.123659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:47:25.123659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"506'45","reported_seq":50,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438088+0000","last_change":"2024-02-22T19:49:38.906599+0000","last_active":"2024-02-22T19:50:05.438088+0000","last_peered":"2024-02-22T19:50:05.438088+0000","last_clean":"2024-02-22T19:50:05.438088+0000","last_became_active":"2024-02-22T19:49:38.800372+0000","last_became_peered":"2024-02-22T19:49:38.800372+0000","last_unstale":"2024-02-22T19:50:05.438088+0000","last_undegraded":"2024-02-22T19:50:05.438088+0000","last_fullsized":"2024-02-22T19:50:05.438088+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-02-22T19:46:12.320306+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.320306+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":50,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"423'13","reported_seq":651,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:07.662088+0000","last_change":"2024-02-22T19:47:54.799210+0000","last_active":"2024-02-22T19:50:07.662088+0000","last_peered":"2024-02-22T19:50:07.662088+0000","last_clean":"2024-02-22T19:50:07.662088+0000","last_became_active":"2024-02-22T19:45:09.547565+0000","last_became_peered":"2024-02-22T19:45:09.547565+0000","last_unstale":"2024-02-22T19:50:07.662088+0000","last_undegraded":"2024-02-22T19:50:07.662088+0000","last_fullsized":"2024-02-22T19:50:07.662088+0000","mapping_epoch":217,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"423'13","last_scrub_stamp":"2024-02-22T19:47:54.799098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.799098+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"552'7777","reported_seq":8366,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:10.130944+0000","last_change":"2024-02-22T19:49:57.704668+0000","last_active":"2024-02-22T19:50:10.130944+0000","last_peered":"2024-02-22T19:50:10.130944+0000","last_clean":"2024-02-22T19:48:42.912584+0000","last_became_active":"2024-02-22T19:49:57.672647+0000","last_became_peered":"2024-02-22T19:49:57.672647+0000","last_unstale":"2024-02-22T19:50:10.130944+0000","last_undegraded":"2024-02-22T19:50:10.130944+0000","last_fullsized":"2024-02-22T19:50:10.130944+0000","mapping_epoch":546,"log_start":"487'3648","ondisk_log_start":"487'3648","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4129,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":272695296,"num_objects":4161,"num_object_clones":0,"num_object_copies":12483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4161,"num_objects_unfound":0,"num_objects_dirty":4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7777,"num_write_kb":382016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":15204352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,1,7],"avail_no_missing":["3","1","7"],"object_location_counts":[{"shards":"1,3,7","objects":4161}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.431385+0000","last_change":"2024-02-22T19:49:56.700043+0000","last_active":"2024-02-22T19:50:05.431385+0000","last_peered":"2024-02-22T19:50:05.431385+0000","last_clean":"2024-02-22T19:50:05.431385+0000","last_became_active":"2024-02-22T19:49:56.693319+0000","last_became_peered":"2024-02-22T19:49:56.693319+0000","last_unstale":"2024-02-22T19:50:05.431385+0000","last_undegraded":"2024-02-22T19:50:05.431385+0000","last_fullsized":"2024-02-22T19:50:05.431385+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.926641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.926641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.431758+0000","last_change":"2024-02-22T19:49:56.719017+0000","last_active":"2024-02-22T19:50:05.431758+0000","last_peered":"2024-02-22T19:50:05.431758+0000","last_clean":"2024-02-22T19:50:05.431758+0000","last_became_active":"2024-02-22T19:49:56.718724+0000","last_became_peered":"2024-02-22T19:49:56.718724+0000","last_unstale":"2024-02-22T19:50:05.431758+0000","last_undegraded":"2024-02-22T19:50:05.431758+0000","last_fullsized":"2024-02-22T19:50:05.431758+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.792736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.792736+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":92,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430592+0000","last_change":"2024-02-22T19:47:40.536647+0000","last_active":"2024-02-22T19:50:05.430592+0000","last_peered":"2024-02-22T19:50:05.430592+0000","last_clean":"2024-02-22T19:50:05.430592+0000","last_became_active":"2024-02-22T19:47:40.535376+0000","last_became_peered":"2024-02-22T19:47:40.535376+0000","last_unstale":"2024-02-22T19:50:05.430592+0000","last_undegraded":"2024-02-22T19:50:05.430592+0000","last_fullsized":"2024-02-22T19:50:05.430592+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:16.015884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:47:16.015884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"64'5","reported_seq":1864,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:07.733237+0000","last_change":"2024-02-22T19:49:57.124754+0000","last_active":"2024-02-22T19:50:07.733237+0000","last_peered":"2024-02-22T19:50:07.733237+0000","last_clean":"2024-02-22T19:50:07.733237+0000","last_became_active":"2024-02-22T19:49:56.686710+0000","last_became_peered":"2024-02-22T19:49:56.686710+0000","last_unstale":"2024-02-22T19:50:07.733237+0000","last_undegraded":"2024-02-22T19:50:07.733237+0000","last_fullsized":"2024-02-22T19:50:07.733237+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-02-22T19:47:19.438615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:19.438615+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1248,"num_read_kb":1243,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"505'36","reported_seq":30,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.434270+0000","last_change":"2024-02-22T19:49:56.765382+0000","last_active":"2024-02-22T19:50:05.434270+0000","last_peered":"2024-02-22T19:50:05.434270+0000","last_clean":"2024-02-22T19:50:05.434270+0000","last_became_active":"2024-02-22T19:49:56.724392+0000","last_became_peered":"2024-02-22T19:49:56.724392+0000","last_unstale":"2024-02-22T19:50:05.434270+0000","last_undegraded":"2024-02-22T19:50:05.434270+0000","last_fullsized":"2024-02-22T19:50:05.434270+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'24","last_scrub_stamp":"2024-02-22T19:47:49.907139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:49.907139+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"548'33","reported_seq":554,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:07.655044+0000","last_change":"2024-02-22T19:49:57.697794+0000","last_active":"2024-02-22T19:50:07.655044+0000","last_peered":"2024-02-22T19:50:07.655044+0000","last_clean":"2024-02-22T19:49:54.659182+0000","last_became_active":"2024-02-22T19:49:57.697465+0000","last_became_peered":"2024-02-22T19:49:57.697465+0000","last_unstale":"2024-02-22T19:50:07.655044+0000","last_undegraded":"2024-02-22T19:50:07.655044+0000","last_fullsized":"2024-02-22T19:50:07.655044+0000","mapping_epoch":546,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'25","last_scrub_stamp":"2024-02-22T19:45:06.780605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:45:06.780605+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,4,7],"avail_no_missing":["5","4","7"],"object_location_counts":[{"shards":"4,5,7","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"552'7820","reported_seq":9164,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:09.441338+0000","last_change":"2024-02-22T19:49:57.744613+0000","last_active":"2024-02-22T19:50:09.441338+0000","last_peered":"2024-02-22T19:50:09.441338+0000","last_clean":"2024-02-22T19:50:09.441338+0000","last_became_active":"2024-02-22T19:49:57.727263+0000","last_became_peered":"2024-02-22T19:49:57.727263+0000","last_unstale":"2024-02-22T19:50:09.441338+0000","last_undegraded":"2024-02-22T19:50:09.441338+0000","last_fullsized":"2024-02-22T19:50:09.441338+0000","mapping_epoch":546,"log_start":"485'3326","ondisk_log_start":"485'3326","created":469,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4494,"ondisk_log_size":4494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":263192576,"num_objects":4016,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7820,"num_write_kb":378752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":40042496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438501+0000","last_change":"2024-02-22T19:49:38.808909+0000","last_active":"2024-02-22T19:50:05.438501+0000","last_peered":"2024-02-22T19:50:05.438501+0000","last_clean":"2024-02-22T19:50:05.438501+0000","last_became_active":"2024-02-22T19:49:38.808320+0000","last_became_peered":"2024-02-22T19:49:38.808320+0000","last_unstale":"2024-02-22T19:50:05.438501+0000","last_undegraded":"2024-02-22T19:50:05.438501+0000","last_fullsized":"2024-02-22T19:50:05.438501+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:55.024753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:45:55.024753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430525+0000","last_change":"2024-02-22T19:49:56.728261+0000","last_active":"2024-02-22T19:50:05.430525+0000","last_peered":"2024-02-22T19:50:05.430525+0000","last_clean":"2024-02-22T19:50:05.430525+0000","last_became_active":"2024-02-22T19:49:56.723819+0000","last_became_peered":"2024-02-22T19:49:56.723819+0000","last_unstale":"2024-02-22T19:50:05.430525+0000","last_undegraded":"2024-02-22T19:50:05.430525+0000","last_fullsized":"2024-02-22T19:50:05.430525+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:53.891414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:47:53.891414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.431930+0000","last_change":"2024-02-22T19:49:56.727574+0000","last_active":"2024-02-22T19:50:05.431930+0000","last_peered":"2024-02-22T19:50:05.431930+0000","last_clean":"2024-02-22T19:50:05.431930+0000","last_became_active":"2024-02-22T19:49:56.726055+0000","last_became_peered":"2024-02-22T19:49:56.726055+0000","last_unstale":"2024-02-22T19:50:05.431930+0000","last_undegraded":"2024-02-22T19:50:05.431930+0000","last_fullsized":"2024-02-22T19:50:05.431930+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:46.844819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:39:48.361427+0000","last_clean_scrub_stamp":"2024-02-22T19:47:46.844819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"506'47","reported_seq":450,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443511+0000","last_change":"2024-02-22T19:49:58.077926+0000","last_active":"2024-02-22T19:50:05.443511+0000","last_peered":"2024-02-22T19:50:05.443511+0000","last_clean":"2024-02-22T19:50:05.443511+0000","last_became_active":"2024-02-22T19:49:56.714017+0000","last_became_peered":"2024-02-22T19:49:56.714017+0000","last_unstale":"2024-02-22T19:50:05.443511+0000","last_undegraded":"2024-02-22T19:50:05.443511+0000","last_fullsized":"2024-02-22T19:50:05.443511+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"463'37","last_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_deep_scrub":"463'37","last_deep_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_clean_scrub_stamp":"2024-02-22T19:47:24.474452+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1650,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":34,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438088+0000","last_change":"2024-02-22T19:49:38.802810+0000","last_active":"2024-02-22T19:50:05.438088+0000","last_peered":"2024-02-22T19:50:05.438088+0000","last_clean":"2024-02-22T19:50:05.438088+0000","last_became_active":"2024-02-22T19:49:38.802359+0000","last_became_peered":"2024-02-22T19:49:38.802359+0000","last_unstale":"2024-02-22T19:50:05.438088+0000","last_undegraded":"2024-02-22T19:50:05.438088+0000","last_fullsized":"2024-02-22T19:50:05.438088+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:54.804469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:54.804469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"552'7671","reported_seq":11152,"reported_epoch":552,"state":"active+recovering+degraded+remapped","last_fresh":"2024-02-22T19:50:10.138275+0000","last_change":"2024-02-22T19:49:41.228044+0000","last_active":"2024-02-22T19:50:10.138275+0000","last_peered":"2024-02-22T19:50:10.138275+0000","last_clean":"2024-02-22T19:48:42.880332+0000","last_became_active":"2024-02-22T19:49:39.970692+0000","last_became_peered":"2024-02-22T19:49:39.970692+0000","last_unstale":"2024-02-22T19:50:10.138275+0000","last_undegraded":"2024-02-22T19:49:39.942855+0000","last_fullsized":"2024-02-22T19:50:10.138275+0000","mapping_epoch":528,"log_start":"479'2412","ondisk_log_start":"479'2412","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":5259,"ondisk_log_size":5259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":265158680,"num_objects":4047,"num_object_clones":0,"num_object_copies":12141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":2948,"num_objects_unfound":0,"num_objects_dirty":4047,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7671,"num_write_kb":374913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1685,"num_bytes_recovered":110428160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,5],"avail_no_missing":["2","1","4"],"object_location_counts":[{"shards":"1,2,4","objects":2948},{"shards":"2,4","objects":1},{"shards":"2,4,5","objects":1098}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":34,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437150+0000","last_change":"2024-02-22T19:49:38.810103+0000","last_active":"2024-02-22T19:50:05.437150+0000","last_peered":"2024-02-22T19:50:05.437150+0000","last_clean":"2024-02-22T19:50:05.437150+0000","last_became_active":"2024-02-22T19:49:38.809765+0000","last_became_peered":"2024-02-22T19:49:38.809765+0000","last_unstale":"2024-02-22T19:50:05.437150+0000","last_undegraded":"2024-02-22T19:50:05.437150+0000","last_fullsized":"2024-02-22T19:50:05.437150+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:13.199920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:13.199920+0000","last_clean_scrub_stamp":"2024-02-22T19:48:13.199920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":348,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.449220+0000","last_change":"2024-02-22T19:48:04.068652+0000","last_active":"2024-02-22T19:50:05.449220+0000","last_peered":"2024-02-22T19:50:05.449220+0000","last_clean":"2024-02-22T19:50:05.449220+0000","last_became_active":"2024-02-22T19:39:22.270548+0000","last_became_peered":"2024-02-22T19:39:22.270548+0000","last_unstale":"2024-02-22T19:50:05.449220+0000","last_undegraded":"2024-02-22T19:50:05.449220+0000","last_fullsized":"2024-02-22T19:50:05.449220+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:04.068584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:48:04.068584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":354,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444568+0000","last_change":"2024-02-22T19:46:19.538051+0000","last_active":"2024-02-22T19:50:05.444568+0000","last_peered":"2024-02-22T19:50:05.444568+0000","last_clean":"2024-02-22T19:50:05.444568+0000","last_became_active":"2024-02-22T19:37:57.190156+0000","last_became_peered":"2024-02-22T19:37:57.190156+0000","last_unstale":"2024-02-22T19:50:05.444568+0000","last_undegraded":"2024-02-22T19:50:05.444568+0000","last_fullsized":"2024-02-22T19:50:05.444568+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:19.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:46:19.537988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"506'30","reported_seq":28,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430824+0000","last_change":"2024-02-22T19:49:56.867047+0000","last_active":"2024-02-22T19:50:05.430824+0000","last_peered":"2024-02-22T19:50:05.430824+0000","last_clean":"2024-02-22T19:50:05.430824+0000","last_became_active":"2024-02-22T19:49:56.774151+0000","last_became_peered":"2024-02-22T19:49:56.774151+0000","last_unstale":"2024-02-22T19:50:05.430824+0000","last_undegraded":"2024-02-22T19:50:05.430824+0000","last_fullsized":"2024-02-22T19:50:05.430824+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:46:18.828884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:18.828884+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"529'21","reported_seq":120,"reported_epoch":552,"state":"active+remapped+backfill_wait+backfill_toofull","last_fresh":"2024-02-22T19:50:09.487992+0000","last_change":"2024-02-22T19:50:10.020161+0000","last_active":"2024-02-22T19:50:09.487992+0000","last_peered":"2024-02-22T19:50:09.487992+0000","last_clean":"2024-02-22T19:39:06.068713+0000","last_became_active":"2024-02-22T19:49:39.948487+0000","last_became_peered":"2024-02-22T19:49:39.948487+0000","last_unstale":"2024-02-22T19:50:09.487992+0000","last_undegraded":"2024-02-22T19:50:09.487992+0000","last_fullsized":"2024-02-22T19:50:09.487992+0000","mapping_epoch":528,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":497,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'16","last_scrub_stamp":"2024-02-22T19:46:58.513473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:46:58.513473+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[3,5,7],"avail_no_missing":["3","5","7"],"object_location_counts":[{"shards":"3,5,7","objects":1}],"blocked_by":[],"up_primary":6,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"552'7499","reported_seq":18401,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:09.322057+0000","last_change":"2024-02-22T19:49:57.849472+0000","last_active":"2024-02-22T19:50:09.322057+0000","last_peered":"2024-02-22T19:50:09.322057+0000","last_clean":"2024-02-22T19:49:55.649006+0000","last_became_active":"2024-02-22T19:49:57.832951+0000","last_became_peered":"2024-02-22T19:49:57.832951+0000","last_unstale":"2024-02-22T19:50:09.322057+0000","last_undegraded":"2024-02-22T19:50:09.322057+0000","last_fullsized":"2024-02-22T19:50:09.322057+0000","mapping_epoch":546,"log_start":"480'2558","ondisk_log_start":"480'2558","created":469,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4941,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":267059200,"num_objects":4075,"num_object_clones":0,"num_object_copies":12225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4075,"num_objects_unfound":0,"num_objects_dirty":4075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7499,"num_write_kb":370368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5389,"num_bytes_recovered":353173504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,7,3],"avail_no_missing":["4","3","7"],"object_location_counts":[{"shards":"3,4,7","objects":4075}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":448,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443707+0000","last_change":"2024-02-22T19:49:38.817530+0000","last_active":"2024-02-22T19:50:05.443707+0000","last_peered":"2024-02-22T19:50:05.443707+0000","last_clean":"2024-02-22T19:50:05.443707+0000","last_became_active":"2024-02-22T19:49:38.806979+0000","last_became_peered":"2024-02-22T19:49:38.806979+0000","last_unstale":"2024-02-22T19:50:05.443707+0000","last_undegraded":"2024-02-22T19:50:05.443707+0000","last_fullsized":"2024-02-22T19:50:05.443707+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:18.018184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:18.018184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":556,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.440401+0000","last_change":"2024-02-22T19:49:38.795888+0000","last_active":"2024-02-22T19:50:05.440401+0000","last_peered":"2024-02-22T19:50:05.440401+0000","last_clean":"2024-02-22T19:50:05.440401+0000","last_became_active":"2024-02-22T19:49:38.795530+0000","last_became_peered":"2024-02-22T19:49:38.795530+0000","last_unstale":"2024-02-22T19:50:05.440401+0000","last_undegraded":"2024-02-22T19:50:05.440401+0000","last_fullsized":"2024-02-22T19:50:05.440401+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:47.829395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.829395+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.434445+0000","last_change":"2024-02-22T19:49:56.681985+0000","last_active":"2024-02-22T19:50:05.434445+0000","last_peered":"2024-02-22T19:50:05.434445+0000","last_clean":"2024-02-22T19:50:05.434445+0000","last_became_active":"2024-02-22T19:49:56.681469+0000","last_became_peered":"2024-02-22T19:49:56.681469+0000","last_unstale":"2024-02-22T19:50:05.434445+0000","last_undegraded":"2024-02-22T19:50:05.434445+0000","last_fullsized":"2024-02-22T19:50:05.434445+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:54.924175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:54.924175+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"506'49","reported_seq":159,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430675+0000","last_change":"2024-02-22T19:49:39.483576+0000","last_active":"2024-02-22T19:50:05.430675+0000","last_peered":"2024-02-22T19:50:05.430675+0000","last_clean":"2024-02-22T19:50:05.430675+0000","last_became_active":"2024-02-22T19:49:38.795181+0000","last_became_peered":"2024-02-22T19:49:38.795181+0000","last_unstale":"2024-02-22T19:50:05.430675+0000","last_undegraded":"2024-02-22T19:50:05.430675+0000","last_fullsized":"2024-02-22T19:50:05.430675+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-22T19:47:51.225129+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:54.022453+0000","last_clean_scrub_stamp":"2024-02-22T19:47:51.225129+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"548'21","reported_seq":907,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:07.665425+0000","last_change":"2024-02-22T19:49:57.672897+0000","last_active":"2024-02-22T19:50:07.665425+0000","last_peered":"2024-02-22T19:50:07.665425+0000","last_clean":"2024-02-22T19:49:54.664346+0000","last_became_active":"2024-02-22T19:49:57.672569+0000","last_became_peered":"2024-02-22T19:49:57.672569+0000","last_unstale":"2024-02-22T19:50:07.665425+0000","last_undegraded":"2024-02-22T19:50:07.665425+0000","last_fullsized":"2024-02-22T19:50:07.665425+0000","mapping_epoch":546,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":508,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'11","last_scrub_stamp":"2024-02-22T19:44:53.757594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:44:53.757594+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,4,2],"avail_no_missing":["3","2","4"],"object_location_counts":[{"shards":"2,3,4","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"552'7946","reported_seq":18676,"reported_epoch":552,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:09.507378+0000","last_change":"2024-02-22T19:49:57.815455+0000","last_active":"2024-02-22T19:50:09.507378+0000","last_peered":"2024-02-22T19:50:09.507378+0000","last_clean":"2024-02-22T19:49:55.652439+0000","last_became_active":"2024-02-22T19:49:57.803865+0000","last_became_peered":"2024-02-22T19:49:57.803865+0000","last_unstale":"2024-02-22T19:50:09.507378+0000","last_undegraded":"2024-02-22T19:50:09.507378+0000","last_fullsized":"2024-02-22T19:50:09.507378+0000","mapping_epoch":546,"log_start":"482'3054","ondisk_log_start":"482'3054","created":469,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4892,"ondisk_log_size":4892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":274595840,"num_objects":4190,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7946,"num_write_kb":388352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5304,"num_bytes_recovered":347602944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,2,4],"avail_no_missing":["7","2","4"],"object_location_counts":[{"shards":"2,4,7","objects":4190}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.429425+0000","last_change":"2024-02-22T19:49:38.807262+0000","last_active":"2024-02-22T19:50:05.429425+0000","last_peered":"2024-02-22T19:50:05.429425+0000","last_clean":"2024-02-22T19:50:05.429425+0000","last_became_active":"2024-02-22T19:49:38.806549+0000","last_became_peered":"2024-02-22T19:49:38.806549+0000","last_unstale":"2024-02-22T19:50:05.429425+0000","last_undegraded":"2024-02-22T19:50:05.429425+0000","last_fullsized":"2024-02-22T19:50:05.429425+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:21.070731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:21.070731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":97,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428917+0000","last_change":"2024-02-22T19:48:15.061095+0000","last_active":"2024-02-22T19:50:05.428917+0000","last_peered":"2024-02-22T19:50:05.428917+0000","last_clean":"2024-02-22T19:50:05.428917+0000","last_became_active":"2024-02-22T19:47:40.534711+0000","last_became_peered":"2024-02-22T19:47:40.534711+0000","last_unstale":"2024-02-22T19:50:05.428917+0000","last_undegraded":"2024-02-22T19:50:05.428917+0000","last_fullsized":"2024-02-22T19:50:05.428917+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:15.061001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:15.061001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":553,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443695+0000","last_change":"2024-02-22T19:45:58.995918+0000","last_active":"2024-02-22T19:50:05.443695+0000","last_peered":"2024-02-22T19:50:05.443695+0000","last_clean":"2024-02-22T19:50:05.443695+0000","last_became_active":"2024-02-22T19:39:22.242789+0000","last_became_peered":"2024-02-22T19:39:22.242789+0000","last_unstale":"2024-02-22T19:50:05.443695+0000","last_undegraded":"2024-02-22T19:50:05.443695+0000","last_fullsized":"2024-02-22T19:50:05.443695+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:58.995851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:45:58.995851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"506'42","reported_seq":729,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.449910+0000","last_change":"2024-02-22T19:49:58.375881+0000","last_active":"2024-02-22T19:50:05.449910+0000","last_peered":"2024-02-22T19:50:05.449910+0000","last_clean":"2024-02-22T19:50:05.449910+0000","last_became_active":"2024-02-22T19:49:56.715296+0000","last_became_peered":"2024-02-22T19:49:56.715296+0000","last_unstale":"2024-02-22T19:50:05.449910+0000","last_undegraded":"2024-02-22T19:50:05.449910+0000","last_fullsized":"2024-02-22T19:50:05.449910+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-02-22T19:45:09.692661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:45:09.692661+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":97,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430822+0000","last_change":"2024-02-22T19:47:55.133259+0000","last_active":"2024-02-22T19:50:05.430822+0000","last_peered":"2024-02-22T19:50:05.430822+0000","last_clean":"2024-02-22T19:50:05.430822+0000","last_became_active":"2024-02-22T19:47:40.535824+0000","last_became_peered":"2024-02-22T19:47:40.535824+0000","last_unstale":"2024-02-22T19:50:05.430822+0000","last_undegraded":"2024-02-22T19:50:05.430822+0000","last_fullsized":"2024-02-22T19:50:05.430822+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:55.133168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:47:55.133168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"552'7698","reported_seq":7833,"reported_epoch":552,"state":"active+clean","last_fresh":"2024-02-22T19:50:09.440956+0000","last_change":"2024-02-22T19:49:57.704557+0000","last_active":"2024-02-22T19:50:09.440956+0000","last_peered":"2024-02-22T19:50:09.440956+0000","last_clean":"2024-02-22T19:50:09.440956+0000","last_became_active":"2024-02-22T19:49:57.681972+0000","last_became_peered":"2024-02-22T19:49:57.681972+0000","last_unstale":"2024-02-22T19:50:09.440956+0000","last_undegraded":"2024-02-22T19:50:09.440956+0000","last_fullsized":"2024-02-22T19:50:09.440956+0000","mapping_epoch":546,"log_start":"485'3321","ondisk_log_start":"485'3321","created":469,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4377,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":265682944,"num_objects":4054,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7698,"num_write_kb":376064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":384,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443634+0000","last_change":"2024-02-22T19:49:38.802134+0000","last_active":"2024-02-22T19:50:05.443634+0000","last_peered":"2024-02-22T19:50:05.443634+0000","last_clean":"2024-02-22T19:50:05.443634+0000","last_became_active":"2024-02-22T19:49:38.801396+0000","last_became_peered":"2024-02-22T19:49:38.801396+0000","last_unstale":"2024-02-22T19:50:05.443634+0000","last_undegraded":"2024-02-22T19:50:05.443634+0000","last_fullsized":"2024-02-22T19:50:05.443634+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:45:00.729907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:45:00.729907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":426,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.444347+0000","last_change":"2024-02-22T19:49:56.702055+0000","last_active":"2024-02-22T19:50:05.444347+0000","last_peered":"2024-02-22T19:50:05.444347+0000","last_clean":"2024-02-22T19:50:05.444347+0000","last_became_active":"2024-02-22T19:49:56.701710+0000","last_became_peered":"2024-02-22T19:49:56.701710+0000","last_unstale":"2024-02-22T19:50:05.444347+0000","last_undegraded":"2024-02-22T19:50:05.444347+0000","last_fullsized":"2024-02-22T19:50:05.444347+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:46:17.513841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:46:17.513841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":549,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.449863+0000","last_change":"2024-02-22T19:49:56.727701+0000","last_active":"2024-02-22T19:50:05.449863+0000","last_peered":"2024-02-22T19:50:05.449863+0000","last_clean":"2024-02-22T19:50:05.449863+0000","last_became_active":"2024-02-22T19:49:56.726891+0000","last_became_peered":"2024-02-22T19:49:56.726891+0000","last_unstale":"2024-02-22T19:50:05.449863+0000","last_undegraded":"2024-02-22T19:50:05.449863+0000","last_fullsized":"2024-02-22T19:50:05.449863+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:49.123984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:49.123984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"506'39","reported_seq":32,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.432218+0000","last_change":"2024-02-22T19:49:56.754350+0000","last_active":"2024-02-22T19:50:05.432218+0000","last_peered":"2024-02-22T19:50:05.432218+0000","last_clean":"2024-02-22T19:50:05.432218+0000","last_became_active":"2024-02-22T19:49:56.714365+0000","last_became_peered":"2024-02-22T19:49:56.714365+0000","last_unstale":"2024-02-22T19:50:05.432218+0000","last_undegraded":"2024-02-22T19:50:05.432218+0000","last_fullsized":"2024-02-22T19:50:05.432218+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-02-22T19:46:15.453047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:46:15.453047+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"552'7686","reported_seq":7787,"reported_epoch":552,"state":"active+remapped+backfill_wait+backfill_toofull","last_fresh":"2024-02-22T19:50:10.075970+0000","last_change":"2024-02-22T19:50:09.964200+0000","last_active":"2024-02-22T19:50:10.075970+0000","last_peered":"2024-02-22T19:50:10.075970+0000","last_clean":"2024-02-22T19:49:37.541632+0000","last_became_active":"2024-02-22T19:49:39.932413+0000","last_became_peered":"2024-02-22T19:49:39.932413+0000","last_unstale":"2024-02-22T19:50:10.075970+0000","last_undegraded":"2024-02-22T19:50:10.075970+0000","last_fullsized":"2024-02-22T19:50:10.075970+0000","mapping_epoch":528,"log_start":"485'3328","ondisk_log_start":"485'3328","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4358,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":261488640,"num_objects":3990,"num_object_clones":0,"num_object_copies":11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3990,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7686,"num_write_kb":373632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[2,3,4],"avail_no_missing":["2","3","4"],"object_location_counts":[{"shards":"2,3,4","objects":3990}],"blocked_by":[],"up_primary":6,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"137'2","reported_seq":590,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443846+0000","last_change":"2024-02-22T19:49:57.691930+0000","last_active":"2024-02-22T19:50:05.443846+0000","last_peered":"2024-02-22T19:50:05.443846+0000","last_clean":"2024-02-22T19:50:05.443846+0000","last_became_active":"2024-02-22T19:49:57.691592+0000","last_became_peered":"2024-02-22T19:49:57.691592+0000","last_unstale":"2024-02-22T19:50:05.443846+0000","last_undegraded":"2024-02-22T19:50:05.443846+0000","last_fullsized":"2024-02-22T19:50:05.443846+0000","mapping_epoch":546,"log_start":"137'2","ondisk_log_start":"137'2","created":62,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:10.140859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:48:10.140859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":97,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.430961+0000","last_change":"2024-02-22T19:48:17.043511+0000","last_active":"2024-02-22T19:50:05.430961+0000","last_peered":"2024-02-22T19:50:05.430961+0000","last_clean":"2024-02-22T19:50:05.430961+0000","last_became_active":"2024-02-22T19:47:40.531155+0000","last_became_peered":"2024-02-22T19:47:40.531155+0000","last_unstale":"2024-02-22T19:50:05.430961+0000","last_undegraded":"2024-02-22T19:50:05.430961+0000","last_fullsized":"2024-02-22T19:50:05.430961+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:48:17.043439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:48:17.043439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"506'27","reported_seq":179,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428942+0000","last_change":"2024-02-22T19:49:58.431139+0000","last_active":"2024-02-22T19:50:05.428942+0000","last_peered":"2024-02-22T19:50:05.428942+0000","last_clean":"2024-02-22T19:50:05.428942+0000","last_became_active":"2024-02-22T19:49:56.675817+0000","last_became_peered":"2024-02-22T19:49:56.675817+0000","last_unstale":"2024-02-22T19:50:05.428942+0000","last_undegraded":"2024-02-22T19:50:05.428942+0000","last_fullsized":"2024-02-22T19:50:05.428942+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-02-22T19:47:10.692928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:10.692928+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"552'7584","reported_seq":10351,"reported_epoch":552,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-02-22T19:50:10.126557+0000","last_change":"2024-02-22T19:49:58.393476+0000","last_active":"2024-02-22T19:50:10.126557+0000","last_peered":"2024-02-22T19:50:10.126557+0000","last_clean":"2024-02-22T19:48:22.530905+0000","last_became_active":"2024-02-22T19:49:56.858895+0000","last_became_peered":"2024-02-22T19:49:56.858895+0000","last_unstale":"2024-02-22T19:50:10.126557+0000","last_undegraded":"2024-02-22T19:49:56.848345+0000","last_fullsized":"2024-02-22T19:49:56.665768+0000","mapping_epoch":545,"log_start":"479'2404","ondisk_log_start":"479'2404","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":5180,"ondisk_log_size":5180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":267124736,"num_objects":4076,"num_object_clones":0,"num_object_copies":12228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":340,"num_objects_misplaced":6532,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7584,"num_write_kb":373120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1153,"num_bytes_recovered":75431936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[{"shards":"2,3,7","objects":3096},{"shards":"2,7","objects":640}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":354,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.443036+0000","last_change":"2024-02-22T19:44:48.769697+0000","last_active":"2024-02-22T19:50:05.443036+0000","last_peered":"2024-02-22T19:50:05.443036+0000","last_clean":"2024-02-22T19:50:05.443036+0000","last_became_active":"2024-02-22T19:37:57.189498+0000","last_became_peered":"2024-02-22T19:37:57.189498+0000","last_unstale":"2024-02-22T19:50:05.443036+0000","last_undegraded":"2024-02-22T19:50:05.443036+0000","last_fullsized":"2024-02-22T19:50:05.443036+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:44:48.769602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:44:48.769602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":583,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.435102+0000","last_change":"2024-02-22T19:49:24.740210+0000","last_active":"2024-02-22T19:50:05.435102+0000","last_peered":"2024-02-22T19:50:05.435102+0000","last_clean":"2024-02-22T19:50:05.435102+0000","last_became_active":"2024-02-22T19:49:24.739876+0000","last_became_peered":"2024-02-22T19:49:24.739876+0000","last_unstale":"2024-02-22T19:50:05.435102+0000","last_undegraded":"2024-02-22T19:50:05.435102+0000","last_fullsized":"2024-02-22T19:50:05.435102+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":514,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:47.396426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:47:47.396426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":43,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.437931+0000","last_change":"2024-02-22T19:49:56.676678+0000","last_active":"2024-02-22T19:50:05.437931+0000","last_peered":"2024-02-22T19:50:05.437931+0000","last_clean":"2024-02-22T19:50:05.437931+0000","last_became_active":"2024-02-22T19:49:56.675940+0000","last_became_peered":"2024-02-22T19:49:56.675940+0000","last_unstale":"2024-02-22T19:50:05.437931+0000","last_undegraded":"2024-02-22T19:50:05.437931+0000","last_fullsized":"2024-02-22T19:50:05.437931+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:14.714507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:47:14.714507+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"506'45","reported_seq":212,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.429496+0000","last_change":"2024-02-22T19:49:57.386002+0000","last_active":"2024-02-22T19:50:05.429496+0000","last_peered":"2024-02-22T19:50:05.429496+0000","last_clean":"2024-02-22T19:50:05.429496+0000","last_became_active":"2024-02-22T19:49:56.673755+0000","last_became_peered":"2024-02-22T19:49:56.673755+0000","last_unstale":"2024-02-22T19:50:05.429496+0000","last_undegraded":"2024-02-22T19:50:05.429496+0000","last_fullsized":"2024-02-22T19:50:05.429496+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'31","last_scrub_stamp":"2024-02-22T19:47:26.042105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:26.042105+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":59,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"552'7589","reported_seq":10389,"reported_epoch":552,"state":"active+recovering+undersized+remapped","last_fresh":"2024-02-22T19:50:09.563111+0000","last_change":"2024-02-22T19:49:59.074758+0000","last_active":"2024-02-22T19:50:09.563111+0000","last_peered":"2024-02-22T19:50:09.563111+0000","last_clean":"2024-02-22T19:48:42.910067+0000","last_became_active":"2024-02-22T19:49:57.832730+0000","last_became_peered":"2024-02-22T19:49:57.832730+0000","last_unstale":"2024-02-22T19:50:09.563111+0000","last_undegraded":"2024-02-22T19:50:09.563111+0000","last_fullsized":"2024-02-22T19:49:57.656911+0000","mapping_epoch":546,"log_start":"480'2759","ondisk_log_start":"480'2759","created":469,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:39.489272+0000","last_clean_scrub_stamp":"2024-02-22T19:47:39.489272+0000","log_size":4830,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":267452416,"num_objects":4081,"num_object_clones":0,"num_object_copies":12243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7589,"num_write_kb":373440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1339,"num_bytes_recovered":87752704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[{"shards":"2,5,7","objects":995},{"shards":"5,7","objects":3086}],"blocked_by":[],"up_primary":6,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":549,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.438343+0000","last_change":"2024-02-22T19:47:48.777053+0000","last_active":"2024-02-22T19:50:05.438343+0000","last_peered":"2024-02-22T19:50:05.438343+0000","last_clean":"2024-02-22T19:50:05.438343+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:50:05.438343+0000","last_undegraded":"2024-02-22T19:50:05.438343+0000","last_fullsized":"2024-02-22T19:50:05.438343+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:47:48.776953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:47:48.776953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":142,"reported_epoch":551,"state":"active+remapped+backfill_wait","last_fresh":"2024-02-22T19:50:05.429052+0000","last_change":"2024-02-22T19:49:57.674904+0000","last_active":"2024-02-22T19:50:05.429052+0000","last_peered":"2024-02-22T19:50:05.429052+0000","last_clean":"2024-02-22T19:49:54.657804+0000","last_became_active":"2024-02-22T19:49:57.674557+0000","last_became_peered":"2024-02-22T19:49:57.674557+0000","last_unstale":"2024-02-22T19:50:05.429052+0000","last_undegraded":"2024-02-22T19:50:05.429052+0000","last_fullsized":"2024-02-22T19:50:05.429052+0000","mapping_epoch":546,"log_start":"85'2","ondisk_log_start":"85'2","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:48:07.111947+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:48:07.111947+0000","last_clean_scrub_stamp":"2024-02-22T19:48:07.111947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,2,5],"avail_no_missing":["4","2","5"],"object_location_counts":[{"shards":"2,4,5","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":620,"reported_epoch":551,"state":"active+recovery_wait+degraded","last_fresh":"2024-02-22T19:50:05.444174+0000","last_change":"2024-02-22T19:49:56.735646+0000","last_active":"2024-02-22T19:50:05.444174+0000","last_peered":"2024-02-22T19:50:05.444174+0000","last_clean":"2024-02-22T19:49:54.665034+0000","last_became_active":"2024-02-22T19:49:56.735475+0000","last_became_peered":"2024-02-22T19:49:56.735475+0000","last_unstale":"2024-02-22T19:50:05.444174+0000","last_undegraded":"2024-02-22T19:49:56.675376+0000","last_fullsized":"2024-02-22T19:50:05.444174+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":494,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:46:12.047696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:46:12.047696+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":["3","4","7"],"object_location_counts":[{"shards":"3,4,7","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"505'30","reported_seq":166,"reported_epoch":551,"state":"active+clean","last_fresh":"2024-02-22T19:50:05.428821+0000","last_change":"2024-02-22T19:49:40.444424+0000","last_active":"2024-02-22T19:50:05.428821+0000","last_peered":"2024-02-22T19:50:05.428821+0000","last_clean":"2024-02-22T19:50:05.428821+0000","last_became_active":"2024-02-22T19:49:38.805896+0000","last_became_peered":"2024-02-22T19:49:38.805896+0000","last_unstale":"2024-02-22T19:50:05.428821+0000","last_undegraded":"2024-02-22T19:50:05.428821+0000","last_fullsized":"2024-02-22T19:50:05.428821+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":529,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'20","last_scrub_stamp":"2024-02-22T19:47:53.131527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:47:53.131527+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":4255186968,"num_objects":64930,"num_object_clones":0,"num_object_copies":194790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":341,"num_objects_misplaced":46928,"num_objects_unfound":0,"num_objects_dirty":64930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":122932,"num_write_kb":6011521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23360,"num_bytes_recovered":1530724352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14182068224,"data_stored":14182056008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74643,"ondisk_log_size":74643,"up":48,"acting":46,"num_store_stats":7},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1248,"num_read_kb":1243,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":8480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1519,"num_read_kb":1276,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":741,"num_bytes_recovered":16650,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1017,"ondisk_log_size":1017,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":66,"acting":66,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266945,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3292856,"kb_used_data":2387696,"kb_used_omap":0,"kb_used_meta":905088,"kb_avail":90452296,"statfs":{"total":95995035648,"available":92623151104,"internally_reserved":0,"allocated":2445000704,"data_stored":2442846625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":926810112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3599999999999999,"5min":1.1299999999999999,"15min":1.3340000000000001},"min":{"1min":0.25600000000000001,"5min":0.219,"15min":0.219},"max":{"1min":9.9220000000000006,"5min":9.9220000000000006,"15min":9.9220000000000006},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.077,"15min":1.343},"min":{"1min":0.27900000000000003,"5min":0.255,"15min":0.216},"max":{"1min":4.4359999999999999,"5min":5.1349999999999998,"15min":9.8870000000000005},"last":2.3090000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":3.004,"5min":1.7070000000000001,"15min":1.8720000000000001},"min":{"1min":0.28799999999999998,"5min":0.218,"15min":0.20000000000000001},"max":{"1min":12.574999999999999,"5min":20.588999999999999,"15min":54.951000000000001},"last":6.2930000000000001},{"interface":"front","average":{"1min":3.8679999999999999,"5min":2.298,"15min":2.1749999999999998},"min":{"1min":0.33400000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":12.484999999999999,"5min":20.923999999999999,"15min":57.012},"last":2.3380000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":2.464,"5min":2.6739999999999999,"15min":1.9139999999999999},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.23999999999999999},"max":{"1min":13.175000000000001,"5min":42.377000000000002,"15min":42.377000000000002},"last":9.4640000000000004},{"interface":"front","average":{"1min":4.5179999999999998,"5min":2.867,"15min":2.2170000000000001},"min":{"1min":0.33600000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":14.436,"5min":39.520000000000003,"15min":39.520000000000003},"last":1.538}]},{"osd":3,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":2.6970000000000001,"5min":2.194,"15min":2.0859999999999999},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":8.3230000000000004,"5min":37.905000000000001,"15min":38.692999999999998},"last":9.6259999999999994},{"interface":"front","average":{"1min":3.2090000000000001,"5min":2.194,"15min":2.194},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.361000000000001,"5min":38.25,"15min":38.25},"last":2.3679999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":0.76700000000000002,"15min":0.81799999999999995},"min":{"1min":0.251,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.5499999999999998,"5min":5.5499999999999998,"15min":6.4720000000000004},"last":2.3570000000000002},{"interface":"front","average":{"1min":1.2470000000000001,"5min":0.77900000000000003,"15min":0.87},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26900000000000002},"max":{"1min":5.5940000000000003,"5min":5.5940000000000003,"15min":7.6219999999999999},"last":0.29799999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":0.85299999999999998,"15min":0.98499999999999999},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.321999999999999,"5min":12.321999999999999,"15min":12.321999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":0.80600000000000005,"15min":0.872},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":3.9529999999999998,"5min":4.8200000000000003,"15min":7.3239999999999998},"last":2.8300000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:49:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.84699999999999998,"15min":0.89200000000000002},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.8739999999999997,"5min":4.8739999999999997,"15min":6.5},"last":2.673},{"interface":"front","average":{"1min":1.276,"5min":0.80200000000000005,"15min":0.995},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":7.6150000000000002,"5min":7.6150000000000002,"15min":7.6390000000000002},"last":9.5969999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528582,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314904,"kb_used_data":5392,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430248,"statfs":{"total":95995035648,"available":95672573952,"internally_reserved":0,"allocated":5521408,"data_stored":3398562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":0.89100000000000001,"15min":1.1890000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.17499999999999999},"max":{"1min":3.8370000000000002,"5min":4.8360000000000003,"15min":8.2089999999999996},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.1339999999999999,"15min":1.359},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.21199999999999999},"max":{"1min":9.9779999999999998,"5min":9.9779999999999998,"15min":9.9779999999999998},"last":1.175}]},{"osd":1,"last update":"Thu Feb 22 19:49:09 2024","interfaces":[{"interface":"back","average":{"1min":3.6379999999999999,"5min":1.772,"15min":1.744},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":44.101999999999997,"5min":44.101999999999997,"15min":44.101999999999997},"last":7.5030000000000001},{"interface":"front","average":{"1min":3.6059999999999999,"5min":1.5700000000000001,"15min":1.605},"min":{"1min":0.309,"5min":0.28699999999999998,"15min":0.222},"max":{"1min":44.018000000000001,"5min":44.018000000000001,"15min":44.018000000000001},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Feb 22 19:49:09 2024","interfaces":[{"interface":"back","average":{"1min":4.3390000000000004,"5min":2.5390000000000001,"15min":2.052},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.249},"max":{"1min":44.006999999999998,"5min":63.305,"15min":63.305},"last":2.403},{"interface":"front","average":{"1min":3.9569999999999999,"5min":2.8740000000000001,"15min":1.998},"min":{"1min":0.35899999999999999,"5min":0.24299999999999999,"15min":0.24099999999999999},"max":{"1min":44.031999999999996,"5min":62.859999999999999,"15min":62.859999999999999},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.214,"15min":1.456},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":4.0339999999999998,"5min":4.5819999999999999,"15min":18.661999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.1399999999999999,"15min":1.671},"min":{"1min":0.437,"5min":0.28499999999999998,"15min":0.27000000000000002},"max":{"1min":6.375,"5min":11.73,"15min":19.154},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.51800000000000002,"15min":0.65100000000000002},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.23200000000000001},"max":{"1min":2.738,"5min":2.738,"15min":8.1549999999999994},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.56599999999999995,"15min":0.78000000000000003},"min":{"1min":0.29599999999999999,"5min":0.27400000000000002,"15min":0.23300000000000001},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":9.3949999999999996},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Feb 22 19:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59199999999999997,"15min":0.93999999999999995},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.255},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":21.675000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.63,"15min":0.77700000000000002},"min":{"1min":0.31900000000000001,"5min":0.28199999999999997,"15min":0.27600000000000002},"max":{"1min":2.754,"5min":2.754,"15min":21.634},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:49:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.65300000000000002,"15min":0.84299999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.26300000000000001},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":22.138999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.625,"15min":0.91100000000000003},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.315},"max":{"1min":1.7070000000000001,"5min":1.974,"15min":22.678000000000001},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575135,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4271596,"kb_used_data":3379876,"kb_used_omap":0,"kb_used_meta":891392,"kb_avail":89473556,"statfs":{"total":95995035648,"available":91620921344,"internally_reserved":0,"allocated":3460993024,"data_stored":3458786486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912785408},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:49:13 2024","interfaces":[{"interface":"back","average":{"1min":3.0699999999999998,"5min":1.8080000000000001,"15min":2.036},"min":{"1min":0.28799999999999998,"5min":0.19900000000000001,"15min":0.185},"max":{"1min":9.4410000000000007,"5min":9.4410000000000007,"15min":51.091000000000001},"last":0.316},{"interface":"front","average":{"1min":2.96,"5min":1.8919999999999999,"15min":2.1070000000000002},"min":{"1min":0.30299999999999999,"5min":0.25800000000000001,"15min":0.17499999999999999},"max":{"1min":15.390000000000001,"5min":15.390000000000001,"15min":51.289999999999999},"last":4.0810000000000004}]},{"osd":2,"last update":"Thu Feb 22 19:49:28 2024","interfaces":[{"interface":"back","average":{"1min":6.5380000000000003,"5min":3.3250000000000002,"15min":2.4359999999999999},"min":{"1min":0.29899999999999999,"5min":0.21299999999999999,"15min":0.191},"max":{"1min":36.279000000000003,"5min":36.279000000000003,"15min":51.027000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":7.2119999999999997,"5min":3.391,"15min":2.7149999999999999},"min":{"1min":0.39200000000000002,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":39.929000000000002,"5min":39.929000000000002,"15min":51.369999999999997},"last":7.6459999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:50:06 2024","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":2.649,"15min":2.3500000000000001},"min":{"1min":0.28899999999999998,"5min":0.26100000000000001,"15min":0.23100000000000001},"max":{"1min":8.8810000000000002,"5min":82.034000000000006,"15min":82.034000000000006},"last":0.28899999999999998},{"interface":"front","average":{"1min":3.0579999999999998,"5min":3.3540000000000001,"15min":2.6680000000000001},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.20200000000000001},"max":{"1min":8.5619999999999994,"5min":81.765000000000001,"15min":81.765000000000001},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:49:25 2024","interfaces":[{"interface":"back","average":{"1min":2.5529999999999999,"5min":1.79,"15min":1.885},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.24399999999999999},"max":{"1min":8.4209999999999994,"5min":11.706,"15min":51.158999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.921,"5min":1.234,"15min":1.466},"min":{"1min":0.32900000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":8.1210000000000004,"5min":11.763999999999999,"15min":51.276000000000003},"last":0.32700000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:49:46 2024","interfaces":[{"interface":"back","average":{"1min":3.6230000000000002,"5min":2.0640000000000001,"15min":2.0539999999999998},"min":{"1min":0.29399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":8.532,"5min":11.733000000000001,"15min":51.183999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":2.0699999999999998,"5min":1.6479999999999999,"15min":1.7769999999999999},"min":{"1min":0.37,"5min":0.31,"15min":0.28799999999999998},"max":{"1min":6.2199999999999998,"5min":11.781000000000001,"15min":51.212000000000003},"last":4.1139999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.3340000000000001,"15min":1.617},"min":{"1min":0.35799999999999998,"5min":0.32200000000000001,"15min":0.30499999999999999},"max":{"1min":7.9400000000000004,"5min":8.1690000000000005,"15min":51.079999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":3.0070000000000001,"5min":1.9339999999999999,"15min":2.032},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":10.218,"5min":10.218,"15min":51.119999999999997},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:49:28 2024","interfaces":[{"interface":"back","average":{"1min":2.3610000000000002,"5min":1.696,"15min":1.9710000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":8.8089999999999993,"5min":11.826000000000001,"15min":51.244999999999997},"last":4.1349999999999998},{"interface":"front","average":{"1min":2.7890000000000001,"5min":1.5309999999999999,"15min":1.6639999999999999},"min":{"1min":0.53200000000000003,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":8.1489999999999991,"5min":11.603,"15min":51.317999999999998},"last":3.806}]}]},{"osd":0,"up_from":0,"seq":34359738660,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":18,"seq":77309411609,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3054432,"kb_used_data":2163032,"kb_used_omap":0,"kb_used_meta":891328,"kb_avail":90690720,"statfs":{"total":95995035648,"available":92867297280,"internally_reserved":0,"allocated":2214944768,"data_stored":2212813382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912719872},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:49:24 2024","interfaces":[{"interface":"back","average":{"1min":3.3010000000000002,"5min":1.9359999999999999,"15min":1.901},"min":{"1min":0.223,"5min":0.216,"15min":0.18099999999999999},"max":{"1min":13.494999999999999,"5min":13.494999999999999,"15min":22.338999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":4.9169999999999998,"5min":2.6379999999999999,"15min":2.4489999999999998},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.17599999999999999},"max":{"1min":19.16,"5min":19.16,"15min":30.427},"last":1.254}]},{"osd":1,"last update":"Thu Feb 22 19:49:24 2024","interfaces":[{"interface":"back","average":{"1min":7.5419999999999998,"5min":3.2400000000000002,"15min":2.327},"min":{"1min":0.217,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":48.305,"5min":48.305,"15min":48.305},"last":0.40600000000000003},{"interface":"front","average":{"1min":7.5190000000000001,"5min":3.2709999999999999,"15min":2.3860000000000001},"min":{"1min":0.32300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":50.448999999999998,"5min":50.448999999999998,"15min":50.448999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:50:03 2024","interfaces":[{"interface":"back","average":{"1min":4.8079999999999998,"5min":2.8279999999999998,"15min":2.4180000000000001},"min":{"1min":0.441,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":18.280000000000001,"5min":18.280000000000001,"15min":22.302},"last":3.3279999999999998},{"interface":"front","average":{"1min":6.298,"5min":3.1459999999999999,"15min":2.665},"min":{"1min":0.48499999999999999,"5min":0.183,"15min":0.183},"max":{"1min":13.925000000000001,"5min":13.925000000000001,"15min":30.282},"last":3.4430000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:49:16 2024","interfaces":[{"interface":"back","average":{"1min":3.609,"5min":1.97,"15min":1.669},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26400000000000001},"max":{"1min":11.861000000000001,"5min":11.861000000000001,"15min":12.43},"last":0.441},{"interface":"front","average":{"1min":3.1080000000000001,"5min":1.887,"15min":1.639},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.253},"max":{"1min":13.478999999999999,"5min":13.478999999999999,"15min":13.478999999999999},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:49:40 2024","interfaces":[{"interface":"back","average":{"1min":5.0309999999999997,"5min":2.3210000000000002,"15min":1.9930000000000001},"min":{"1min":0.44700000000000001,"5min":0.29799999999999999,"15min":0.28000000000000003},"max":{"1min":13.429,"5min":14.045,"15min":40.633000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":4.8899999999999997,"5min":2.4089999999999998,"15min":2.089},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.26600000000000001},"max":{"1min":13.741,"5min":14.513999999999999,"15min":40.997999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:49:16 2024","interfaces":[{"interface":"back","average":{"1min":3.6749999999999998,"5min":1.8879999999999999,"15min":1.7749999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.311},"max":{"1min":11.904,"5min":11.904,"15min":21.640000000000001},"last":3.8420000000000001},{"interface":"front","average":{"1min":3.395,"5min":2.0449999999999999,"15min":1.746},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.30299999999999999},"max":{"1min":13.51,"5min":13.51,"15min":13.51},"last":3.2389999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:49:24 2024","interfaces":[{"interface":"back","average":{"1min":4.1219999999999999,"5min":2.3300000000000001,"15min":1.919},"min":{"1min":0.378,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":11.955,"5min":14.199,"15min":14.199},"last":3.8100000000000001},{"interface":"front","average":{"1min":3.6429999999999998,"5min":2.2229999999999999,"15min":1.9179999999999999},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.29899999999999999},"max":{"1min":13.458,"5min":18.021999999999998,"15min":18.021999999999998},"last":0.56699999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149971,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3323936,"kb_used_data":2432216,"kb_used_omap":0,"kb_used_meta":891648,"kb_avail":90421216,"statfs":{"total":95995035648,"available":92591325184,"internally_reserved":0,"allocated":2490589184,"data_stored":2488395506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":913047552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.306,"5min":1.992,"15min":1.9399999999999999},"min":{"1min":0.20399999999999999,"5min":0.187,"15min":0.187},"max":{"1min":9.1950000000000003,"5min":9.3599999999999994,"15min":23.48},"last":0.22700000000000001},{"interface":"front","average":{"1min":2.012,"5min":1.9950000000000001,"15min":2.2069999999999999},"min":{"1min":0.28499999999999998,"5min":0.222,"15min":0.20799999999999999},"max":{"1min":7.7279999999999998,"5min":9.9600000000000009,"15min":23.411999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.601,"5min":4.2000000000000002,"15min":3.198},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":13.576000000000001,"5min":94.454999999999998,"15min":94.454999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":3.944,"5min":4.1879999999999997,"15min":2.9380000000000002},"min":{"1min":0.32900000000000001,"5min":0.22,"15min":0.214},"max":{"1min":10.887,"5min":95.197999999999993,"15min":95.197999999999993},"last":8.516}]},{"osd":2,"last update":"Thu Feb 22 19:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.5099999999999998,"5min":2.573,"15min":2.2930000000000001},"min":{"1min":0.26600000000000001,"5min":0.224,"15min":0.224},"max":{"1min":8.7560000000000002,"5min":17.736000000000001,"15min":20.105},"last":1.1499999999999999},{"interface":"front","average":{"1min":3.5819999999999999,"5min":2.8399999999999999,"15min":2.371},"min":{"1min":0.375,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":17.02,"5min":17.986000000000001,"15min":35.939999999999998},"last":5.234}]},{"osd":4,"last update":"Thu Feb 22 19:49:17 2024","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.514,"15min":1.835},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":13.384,"5min":13.384,"15min":13.680999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":2.4249999999999998,"5min":1.585,"15min":1.708},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":6.7050000000000001,"5min":7.8719999999999999,"15min":13.548},"last":3.8780000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:49:49 2024","interfaces":[{"interface":"back","average":{"1min":2.6819999999999999,"5min":1.919,"15min":1.746},"min":{"1min":0.40999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.7059999999999995,"5min":9.3160000000000007,"15min":11.161},"last":0.34999999999999998},{"interface":"front","average":{"1min":3.3460000000000001,"5min":1.8400000000000001,"15min":1.9079999999999999},"min":{"1min":0.36099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":13.353,"5min":13.353,"15min":13.353},"last":0.41699999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.98,"5min":1.8129999999999999,"15min":1.669},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":6.1879999999999997,"5min":9.423,"15min":11.263},"last":0.495},{"interface":"front","average":{"1min":3.0419999999999998,"5min":2.218,"15min":2.0150000000000001},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.26600000000000001},"max":{"1min":12.058,"5min":13.407999999999999,"15min":13.407999999999999},"last":3.778}]},{"osd":7,"last update":"Thu Feb 22 19:49:22 2024","interfaces":[{"interface":"back","average":{"1min":2.7040000000000002,"5min":1.7130000000000001,"15min":1.794},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":6.992,"5min":7.907,"15min":13.483000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":2.165,"5min":1.7370000000000001,"15min":1.7709999999999999},"min":{"1min":0.42299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.2240000000000002,"5min":9.3930000000000007,"15min":11.539},"last":0.437}]}]},{"osd":4,"up_from":32,"seq":137438953745,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2414716,"kb_used_data":2105204,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":91330436,"statfs":{"total":95995035648,"available":93522366464,"internally_reserved":0,"allocated":2155728896,"data_stored":2153570617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:49:32 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.032,"15min":1.4399999999999999},"min":{"1min":0.26200000000000001,"5min":0.24199999999999999,"15min":0.20499999999999999},"max":{"1min":2.2879999999999998,"5min":8.3469999999999995,"15min":69.251999999999995},"last":2.4609999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.266,"15min":1.5940000000000001},"min":{"1min":0.318,"5min":0.26700000000000002,"15min":0.20999999999999999},"max":{"1min":5.5090000000000003,"5min":9.3780000000000001,"15min":69.373000000000005},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Feb 22 19:49:32 2024","interfaces":[{"interface":"back","average":{"1min":2.4980000000000002,"5min":1.5,"15min":1.607},"min":{"1min":0.27000000000000002,"5min":0.23400000000000001,"15min":0.215},"max":{"1min":9.0530000000000008,"5min":9.0530000000000008,"15min":27.234999999999999},"last":8.1449999999999996},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.2050000000000001,"15min":1.3600000000000001},"min":{"1min":0.28499999999999998,"5min":0.254,"15min":0.248},"max":{"1min":8.8510000000000009,"5min":8.8510000000000009,"15min":28.879000000000001},"last":2.379}]},{"osd":2,"last update":"Thu Feb 22 19:49:32 2024","interfaces":[{"interface":"back","average":{"1min":2.911,"5min":1.605,"15min":1.4850000000000001},"min":{"1min":0.318,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":8.9819999999999993,"5min":8.9819999999999993,"15min":12.359},"last":2.4220000000000002},{"interface":"front","average":{"1min":2.4510000000000001,"5min":1.5549999999999999,"15min":1.4930000000000001},"min":{"1min":0.31,"5min":0.26100000000000001,"15min":0.246},"max":{"1min":6.8239999999999998,"5min":6.8239999999999998,"15min":13.244999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Feb 22 19:49:32 2024","interfaces":[{"interface":"back","average":{"1min":7.1870000000000003,"5min":2.5859999999999999,"15min":2.036},"min":{"1min":0.40400000000000003,"5min":0.25900000000000001,"15min":0.23100000000000001},"max":{"1min":94.703999999999994,"5min":94.703999999999994,"15min":94.703999999999994},"last":0.33800000000000002},{"interface":"front","average":{"1min":7.343,"5min":2.4950000000000001,"15min":2.0649999999999999},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.25900000000000001},"max":{"1min":95.034000000000006,"5min":95.034000000000006,"15min":95.034000000000006},"last":5.7389999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.61299999999999999,"15min":0.73599999999999999},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":2.2999999999999998,"5min":2.8610000000000002,"15min":5.4850000000000003},"last":3.6760000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.70099999999999996,"15min":0.83399999999999996},"min":{"1min":0.30499999999999999,"5min":0.254,"15min":0.249},"max":{"1min":2.5009999999999999,"5min":4.9800000000000004,"15min":13.211},"last":0.36499999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.745,"15min":0.80900000000000005},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":7.2670000000000003},"last":2.496},{"interface":"front","average":{"1min":1.8360000000000001,"5min":0.89700000000000002,"15min":0.91100000000000003},"min":{"1min":0.378,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":6.4560000000000004,"5min":6.4560000000000004,"15min":13.353},"last":5.5019999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.69899999999999995,"15min":0.81100000000000005},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.27900000000000003},"max":{"1min":4.8529999999999998,"5min":4.8529999999999998,"15min":11.111000000000001},"last":5.7130000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.66200000000000003,"15min":0.75800000000000001},"min":{"1min":0.34499999999999997,"5min":0.313,"15min":0.307},"max":{"1min":3.7829999999999999,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":2.5249999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724811,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2366376,"kb_used_data":1467296,"kb_used_omap":0,"kb_used_meta":899072,"kb_avail":91378776,"statfs":{"total":95995035648,"available":93571866624,"internally_reserved":0,"allocated":1502511104,"data_stored":1500372335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":920649728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:49:54 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.0349999999999999,"15min":1.3160000000000001},"min":{"1min":0.27100000000000002,"5min":0.23999999999999999,"15min":0.23899999999999999},"max":{"1min":7.5010000000000003,"5min":7.5010000000000003,"15min":17.684000000000001},"last":0.504},{"interface":"front","average":{"1min":2.4940000000000002,"5min":1.3899999999999999,"15min":1.5629999999999999},"min":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":10.308999999999999,"5min":10.308999999999999,"15min":17.402999999999999},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:49:54 2024","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.8819999999999999,"15min":1.6200000000000001},"min":{"1min":0.251,"5min":0.22700000000000001,"15min":0.19},"max":{"1min":7.5369999999999999,"5min":37.009999999999998,"15min":37.009999999999998},"last":5.9740000000000002},{"interface":"front","average":{"1min":2.5289999999999999,"5min":2.298,"15min":1.8580000000000001},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.372,"5min":39.131999999999998,"15min":39.131999999999998},"last":4.3259999999999996}]},{"osd":2,"last update":"Thu Feb 22 19:49:54 2024","interfaces":[{"interface":"back","average":{"1min":3.121,"5min":2.488,"15min":1.974},"min":{"1min":0.28899999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.683999999999999,"5min":47.859000000000002,"15min":47.859000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":3.0289999999999999,"5min":2.2469999999999999,"15min":1.8440000000000001},"min":{"1min":0.30499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":12.332000000000001,"5min":45.183,"15min":45.183},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Feb 22 19:49:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":2.0529999999999999,"15min":1.901},"min":{"1min":0.33200000000000002,"5min":0.26100000000000001,"15min":0.254},"max":{"1min":3.4700000000000002,"5min":43.616999999999997,"15min":43.616999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":3.3069999999999999,"5min":2.7080000000000002,"15min":1.986},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":12.82,"5min":47.814,"15min":47.814},"last":1.036}]},{"osd":4,"last update":"Thu Feb 22 19:49:54 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":0.83999999999999997,"15min":0.878},"min":{"1min":0.27900000000000003,"5min":0.23799999999999999,"15min":0.223},"max":{"1min":10.347,"5min":10.347,"15min":10.347},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":0.73999999999999999,"15min":0.94399999999999995},"min":{"1min":0.32800000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":10.407},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.56399999999999995,"15min":0.67300000000000004},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.247},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":5.1150000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.64800000000000002,"15min":0.89000000000000001},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.26400000000000001},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":10.436},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Feb 22 19:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.57799999999999996,"15min":0.79200000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":2.5510000000000002,"5min":2.9689999999999999,"15min":6.7119999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.8300000000000001,"5min":0.93899999999999995,"15min":1.026},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29399999999999998},"max":{"1min":7.181,"5min":7.181,"15min":7.6379999999999999},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":2888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3450077184,"data_stored":3450077184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2201292800,"data_stored":2201288728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2468741120,"data_stored":2468741120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2142638080,"data_stored":2142634008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1492389888,"data_stored":1492385816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":589824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2426339328,"data_stored":2426339328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:50:20.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set .rgw.root pgp_num 32 2024-02-22T19:50:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:20.570 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:20.577 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:20.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:20.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:20.791 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:21.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:19.900762+0000 mgr.y (mgr.14535) 811 : cluster [DBG] pgmap v1120: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 490 op/s; 513/181446 objects degraded (0.283%); 43513/181446 objects misplaced (23.981%); 5.0 MiB/s, 79 objects/s recovering 2024-02-22T19:50:22.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:19.900762+0000 mgr.y (mgr.14535) 811 : cluster [DBG] pgmap v1120: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 490 op/s; 513/181446 objects degraded (0.283%); 43513/181446 objects misplaced (23.981%); 5.0 MiB/s, 79 objects/s recovering 2024-02-22T19:50:22.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:19.900762+0000 mgr.y (mgr.14535) 811 : cluster [DBG] pgmap v1120: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 490 op/s; 513/181446 objects degraded (0.283%); 43513/181446 objects misplaced (23.981%); 5.0 MiB/s, 79 objects/s recovering 2024-02-22T19:50:22.172 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:22.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:22.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:22.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:22.353 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:23.024 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:23.303 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:23.305 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:23.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:23.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:23.422 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:50:23.422 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:50:23.485 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:23.618 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:21.901348+0000 mgr.y (mgr.14535) 812 : cluster [DBG] pgmap v1121: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 627 op/s; 675/176556 objects degraded (0.382%); 42245/176556 objects misplaced (23.927%); 6.2 MiB/s, 99 objects/s recovering 2024-02-22T19:50:23.618 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:22.636196+0000 mon.a (mon.0) 1935 : cluster [WRN] Health check update: Degraded data redundancy: 675/176556 objects degraded (0.382%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:23.618 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:22.883511+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.53:0/2679221184' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:50:23.618 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:22.883827+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:50:23.663 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:23.668 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:23.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:23.675 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:50:23.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:23.847 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:23.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:21.901348+0000 mgr.y (mgr.14535) 812 : cluster [DBG] pgmap v1121: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 627 op/s; 675/176556 objects degraded (0.382%); 42245/176556 objects misplaced (23.927%); 6.2 MiB/s, 99 objects/s recovering 2024-02-22T19:50:23.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:22.636196+0000 mon.a (mon.0) 1935 : cluster [WRN] Health check update: Degraded data redundancy: 675/176556 objects degraded (0.382%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:23.876 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:22.883511+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.53:0/2679221184' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:50:23.877 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:22.883827+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:50:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:21.901348+0000 mgr.y (mgr.14535) 812 : cluster [DBG] pgmap v1121: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 627 op/s; 675/176556 objects degraded (0.382%); 42245/176556 objects misplaced (23.927%); 6.2 MiB/s, 99 objects/s recovering 2024-02-22T19:50:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:22.636196+0000 mon.a (mon.0) 1935 : cluster [WRN] Health check update: Degraded data redundancy: 675/176556 objects degraded (0.382%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:22.883511+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.53:0/2679221184' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:50:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:22.883827+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:50:24.125 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:24.134 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:24.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:24.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:24.296 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:23.644164+0000 mon.a (mon.0) 1937 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:23.652210+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:50:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:23.652375+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e555: 8 total, 8 up, 7 in 2024-02-22T19:50:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:23.644164+0000 mon.a (mon.0) 1937 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:23.652210+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:50:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:23.652375+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e555: 8 total, 8 up, 7 in 2024-02-22T19:50:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:50:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:50:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:23.644164+0000 mon.a (mon.0) 1937 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:23.652210+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:50:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:23.652375+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e555: 8 total, 8 up, 7 in 2024-02-22T19:50:25.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:50:25.595 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:50:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:50:25.803 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:25.828 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:25.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:25.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:23.901847+0000 mgr.y (mgr.14535) 813 : cluster [DBG] pgmap v1123: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 627 op/s; 675/176556 objects degraded (0.382%); 42245/176556 objects misplaced (23.927%); 6.2 MiB/s, 99 objects/s recovering 2024-02-22T19:50:25.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:23.901847+0000 mgr.y (mgr.14535) 813 : cluster [DBG] pgmap v1123: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 627 op/s; 675/176556 objects degraded (0.382%); 42245/176556 objects misplaced (23.927%); 6.2 MiB/s, 99 objects/s recovering 2024-02-22T19:50:25.908 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:23.901847+0000 mgr.y (mgr.14535) 813 : cluster [DBG] pgmap v1123: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 627 op/s; 675/176556 objects degraded (0.382%); 42245/176556 objects misplaced (23.927%); 6.2 MiB/s, 99 objects/s recovering 2024-02-22T19:50:25.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:26.351 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:26.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:26.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:26.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:26.530 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:26.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:25.592442+0000 mgr.y (mgr.14535) 814 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:26.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:25.592442+0000 mgr.y (mgr.14535) 814 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:25.592442+0000 mgr.y (mgr.14535) 814 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:27.605 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:27.620 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:27.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:27.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:27.703 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:25.902779+0000 mgr.y (mgr.14535) 815 : cluster [DBG] pgmap v1124: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 625 op/s; 795/167403 objects degraded (0.475%); 40008/167403 objects misplaced (23.899%); 6.3 MiB/s, 100 objects/s recovering 2024-02-22T19:50:27.703 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:27.639730+0000 mon.a (mon.0) 1940 : cluster [WRN] Health check update: Degraded data redundancy: 795/167403 objects degraded (0.475%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:27.703 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:27.661995+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e556: 8 total, 8 up, 7 in 2024-02-22T19:50:27.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:27.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:27.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:25.902779+0000 mgr.y (mgr.14535) 815 : cluster [DBG] pgmap v1124: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 625 op/s; 795/167403 objects degraded (0.475%); 40008/167403 objects misplaced (23.899%); 6.3 MiB/s, 100 objects/s recovering 2024-02-22T19:50:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:27.639730+0000 mon.a (mon.0) 1940 : cluster [WRN] Health check update: Degraded data redundancy: 795/167403 objects degraded (0.475%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:27.661995+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e556: 8 total, 8 up, 7 in 2024-02-22T19:50:28.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:25.902779+0000 mgr.y (mgr.14535) 815 : cluster [DBG] pgmap v1124: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 625 op/s; 795/167403 objects degraded (0.475%); 40008/167403 objects misplaced (23.899%); 6.3 MiB/s, 100 objects/s recovering 2024-02-22T19:50:28.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:27.639730+0000 mon.a (mon.0) 1940 : cluster [WRN] Health check update: Degraded data redundancy: 795/167403 objects degraded (0.475%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:28.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:27.661995+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e556: 8 total, 8 up, 7 in 2024-02-22T19:50:28.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:28.117 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:50:28.118 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:50:28.132 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:28.988 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:50:29.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:27.903253+0000 mgr.y (mgr.14535) 816 : cluster [DBG] pgmap v1126: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 585 op/s; 795/167403 objects degraded (0.475%); 40008/167403 objects misplaced (23.899%); 5.8 MiB/s, 92 objects/s recovering 2024-02-22T19:50:29.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:27.903253+0000 mgr.y (mgr.14535) 816 : cluster [DBG] pgmap v1126: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 585 op/s; 795/167403 objects degraded (0.475%); 40008/167403 objects misplaced (23.899%); 5.8 MiB/s, 92 objects/s recovering 2024-02-22T19:50:29.038 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:50:29.084 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:50:29.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:27.903253+0000 mgr.y (mgr.14535) 816 : cluster [DBG] pgmap v1126: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 585 op/s; 795/167403 objects degraded (0.475%); 40008/167403 objects misplaced (23.899%); 5.8 MiB/s, 92 objects/s recovering 2024-02-22T19:50:29.154 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:50:29.169 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:50:29.205 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:50:29.246 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:50:29.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:50:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:50:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:29.340 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:29.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:29.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:29.507 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:29] "GET /metrics HTTP/1.1" 200 176734 "" "Prometheus/2.33.4" 2024-02-22T19:50:29.591 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:29.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:50:29.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:28.768944+0000 osd.5 (osd.5) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:29.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:28.768999+0000 osd.5 (osd.5) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:29.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:28.937367+0000 mon.b (mon.1) 111 : audit [DBG] from='client.? 172.21.15.53:0/131375433' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:29.965 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:29.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:50:29.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:28.768944+0000 osd.5 (osd.5) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:28.768999+0000 osd.5 (osd.5) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:30.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:28.937367+0000 mon.b (mon.1) 111 : audit [DBG] from='client.? 172.21.15.53:0/131375433' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:30.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:28.768944+0000 osd.5 (osd.5) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:30.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:28.768999+0000 osd.5 (osd.5) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:30.011 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:28.937367+0000 mon.b (mon.1) 111 : audit [DBG] from='client.? 172.21.15.53:0/131375433' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:30.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:30.197 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:30.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:30.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:30.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:30.386 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:30.437 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:30.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:30.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:30.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:30.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:30.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:30.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:30.925 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:30.957 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:30.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:31.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:29.743890+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e557: 8 total, 8 up, 7 in 2024-02-22T19:50:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:29.908949+0000 mgr.y (mgr.14535) 817 : cluster [DBG] pgmap v1128: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 733 op/s; 795/162846 objects degraded (0.488%); 38905/162846 objects misplaced (23.891%); 7.5 MiB/s, 120 objects/s recovering 2024-02-22T19:50:31.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:29.743890+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e557: 8 total, 8 up, 7 in 2024-02-22T19:50:31.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:29.908949+0000 mgr.y (mgr.14535) 817 : cluster [DBG] pgmap v1128: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 733 op/s; 795/162846 objects degraded (0.488%); 38905/162846 objects misplaced (23.891%); 7.5 MiB/s, 120 objects/s recovering 2024-02-22T19:50:31.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:29.743890+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e557: 8 total, 8 up, 7 in 2024-02-22T19:50:31.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:29.908949+0000 mgr.y (mgr.14535) 817 : cluster [DBG] pgmap v1128: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 733 op/s; 795/162846 objects degraded (0.488%); 38905/162846 objects misplaced (23.891%); 7.5 MiB/s, 120 objects/s recovering 2024-02-22T19:50:31.122 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:31.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:31.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:31.339 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:31.381 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:31.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:31.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:31.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:31.736 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:31.786 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:31.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:31.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:31.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:32.620 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:32.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 3 pgp_num to 32 2024-02-22T19:50:32.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:32.757 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:32.981 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:31.909637+0000 mgr.y (mgr.14535) 818 : cluster [DBG] pgmap v1129: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.2 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 754 op/s; 927/158460 objects degraded (0.585%); 37747/158460 objects misplaced (23.821%); 7.9 MiB/s, 126 objects/s recovering 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:32.325827+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.53:0/1219110374' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:32.646755+0000 mon.a (mon.0) 1944 : cluster [WRN] Health check update: Degraded data redundancy: 927/158460 objects degraded (0.585%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:32.656213+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.53:0/1219110374' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]': finished 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:32.656270+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e558: 8 total, 8 up, 7 in 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:31.909637+0000 mgr.y (mgr.14535) 818 : cluster [DBG] pgmap v1129: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.2 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 754 op/s; 927/158460 objects degraded (0.585%); 37747/158460 objects misplaced (23.821%); 7.9 MiB/s, 126 objects/s recovering 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:32.325827+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.53:0/1219110374' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:32.646755+0000 mon.a (mon.0) 1944 : cluster [WRN] Health check update: Degraded data redundancy: 927/158460 objects degraded (0.585%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:32.656213+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.53:0/1219110374' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]': finished 2024-02-22T19:50:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:32.656270+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e558: 8 total, 8 up, 7 in 2024-02-22T19:50:32.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:32.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:33.085 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:33.164 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:33.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:50:33.305 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:33.311 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:33.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:33.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:31.909637+0000 mgr.y (mgr.14535) 818 : cluster [DBG] pgmap v1129: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.2 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 754 op/s; 927/158460 objects degraded (0.585%); 37747/158460 objects misplaced (23.821%); 7.9 MiB/s, 126 objects/s recovering 2024-02-22T19:50:33.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:32.325827+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.53:0/1219110374' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-22T19:50:33.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:32.646755+0000 mon.a (mon.0) 1944 : cluster [WRN] Health check update: Degraded data redundancy: 927/158460 objects degraded (0.585%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:33.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:32.656213+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.53:0/1219110374' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]': finished 2024-02-22T19:50:33.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:32.656270+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e558: 8 total, 8 up, 7 in 2024-02-22T19:50:33.364 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:50:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:50:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:50:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:50:33.364 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:50:33.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:33.476 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:34.286 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:34.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:34.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:34.439 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:34.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:50:35.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:33.910070+0000 mgr.y (mgr.14535) 819 : cluster [DBG] pgmap v1131: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.2 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 819 B/s rd, 0 B/s wr, 477 op/s; 927/158460 objects degraded (0.585%); 37747/158460 objects misplaced (23.821%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:35.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:50:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:50:35.156 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:35.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:33.910070+0000 mgr.y (mgr.14535) 819 : cluster [DBG] pgmap v1131: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.2 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 819 B/s rd, 0 B/s wr, 477 op/s; 927/158460 objects degraded (0.585%); 37747/158460 objects misplaced (23.821%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:35.324 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:33.910070+0000 mgr.y (mgr.14535) 819 : cluster [DBG] pgmap v1131: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.2 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 819 B/s rd, 0 B/s wr, 477 op/s; 927/158460 objects degraded (0.585%); 37747/158460 objects misplaced (23.821%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:35.325 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:35.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:35.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:35.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:35.847 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:35.850 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:35.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:35.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:50:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:50:35.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:36.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:36.079 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:36.180 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:34.477740+0000 osd.3 (osd.3) 358 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:34.477797+0000 osd.3 (osd.3) 359 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:35.009976+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e559: 8 total, 8 up, 7 in 2024-02-22T19:50:36.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:35.604212+0000 mgr.y (mgr.14535) 820 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:36.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:34.477740+0000 osd.3 (osd.3) 358 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:34.477797+0000 osd.3 (osd.3) 359 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:35.009976+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e559: 8 total, 8 up, 7 in 2024-02-22T19:50:36.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:35.604212+0000 mgr.y (mgr.14535) 820 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:34.477740+0000 osd.3 (osd.3) 358 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:34.477797+0000 osd.3 (osd.3) 359 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:35.009976+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e559: 8 total, 8 up, 7 in 2024-02-22T19:50:36.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:35.604212+0000 mgr.y (mgr.14535) 820 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:36.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:36.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:36.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:36.876 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:37.047 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:37.063 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:37.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:37.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:37.260 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:35.911664+0000 mgr.y (mgr.14535) 821 : cluster [DBG] pgmap v1133: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 830 B/s rd, 0 B/s wr, 731 op/s; 1055/149331 objects degraded (0.706%); 35487/149331 objects misplaced (23.764%); 7.6 MiB/s, 121 objects/s recovering 2024-02-22T19:50:37.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:35.911664+0000 mgr.y (mgr.14535) 821 : cluster [DBG] pgmap v1133: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 830 B/s rd, 0 B/s wr, 731 op/s; 1055/149331 objects degraded (0.706%); 35487/149331 objects misplaced (23.764%); 7.6 MiB/s, 121 objects/s recovering 2024-02-22T19:50:37.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:35.911664+0000 mgr.y (mgr.14535) 821 : cluster [DBG] pgmap v1133: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 830 B/s rd, 0 B/s wr, 731 op/s; 1055/149331 objects degraded (0.706%); 35487/149331 objects misplaced (23.764%); 7.6 MiB/s, 121 objects/s recovering 2024-02-22T19:50:37.676 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:37.459829+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.53:0/3738727019' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:37.463295+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:37.648896+0000 mon.a (mon.0) 1949 : cluster [WRN] Health check update: Degraded data redundancy: 1055/149331 objects degraded (0.706%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:37.649362+0000 mon.a (mon.0) 1950 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:37.652826+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:37.652882+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e560: 8 total, 8 up, 7 in 2024-02-22T19:50:38.342 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:37.459829+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.53:0/3738727019' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:38.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:37.463295+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:38.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:37.648896+0000 mon.a (mon.0) 1949 : cluster [WRN] Health check update: Degraded data redundancy: 1055/149331 objects degraded (0.706%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:38.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:37.649362+0000 mon.a (mon.0) 1950 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:38.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:37.652826+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:50:38.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:37.652882+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e560: 8 total, 8 up, 7 in 2024-02-22T19:50:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:38.352 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:38.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:37.459829+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.53:0/3738727019' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:37.463295+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:50:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:37.648896+0000 mon.a (mon.0) 1949 : cluster [WRN] Health check update: Degraded data redundancy: 1055/149331 objects degraded (0.706%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:37.649362+0000 mon.a (mon.0) 1950 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:37.652826+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:50:38.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:37.652882+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e560: 8 total, 8 up, 7 in 2024-02-22T19:50:38.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:38.524 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105399 11.e 11:71b2d97f:::benchmark_data_smithi053_199129_object11466:head [delete] snapc 0=[] ondisk+write+known_if_redirected e525)", 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.754708+0000", 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.780451624000001, 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0556589759999999, 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.541 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105399 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754708+0000", 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754708+0000", 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754706+0000", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0280000000000003e-06 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754713+0000", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0295999999999999e-05 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754723+0000", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7810000000000002e-06 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754729+0000", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6988000000000002e-05 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754806+0000", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0175e-05 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.754816+0000", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0483140230000001 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.803130+0000", 2024-02-22T19:50:38.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.803233+0000", 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8838000000000006e-05 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.803322+0000", 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0069373500000000001 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810259+0000", 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0295000000000001e-05 2024-02-22T19:50:38.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810270+0000", 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0239000000000001e-05 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810290+0000", 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4646e-05 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810324+0000", 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2539e-05 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810367+0000", 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0556589759999999 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105409 11.e 11:7ab18d85:::benchmark_data_smithi053_199129_object11476:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.769635+0000", 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.765524233000001, 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0409237170000001, 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.547 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105409 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769635+0000", 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769635+0000", 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769634+0000", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1899999999999998e-06 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769641+0000", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6368999999999999e-05 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769657+0000", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9589999999999995e-06 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769665+0000", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4105999999999999e-05 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769729+0000", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5928000000000001e-05 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.769745+0000", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353935160000001 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.805139+0000", 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2694000000000006e-05 2024-02-22T19:50:38.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.805212+0000", 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7100000000000004e-05 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.805289+0000", 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051417750000000003 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810430+0000", 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3081999999999998e-05 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810474+0000", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2700999999999997e-05 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810506+0000", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5996e-05 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810532+0000", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6845000000000001e-05 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810559+0000", 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0409237170000001 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105420 11.e 11:7769d54c:::benchmark_data_smithi053_199129_object11487:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.817423+0000", 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.717736629000001, 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99960097400000003, 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105420 2024-02-22T19:50:38.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817423+0000", 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817423+0000", 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817421+0000", 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3139999999999998e-06 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817429+0000", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1950000000000001e-05 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817441+0000", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4290000000000004e-06 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817448+0000", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8072000000000002e-05 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817506+0000", 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0969999999999995e-06 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817515+0000", 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98971760200000003 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.807233+0000", 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114914 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.807348+0000", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000148295 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.807496+0000", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030723479999999999 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810568+0000", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0961000000000003e-05 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810609+0000", 2024-02-22T19:50:38.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0063415320000000004 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.816951+0000", 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0301e-05 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.816981+0000", 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2830999999999997e-05 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817024+0000", 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99960097400000003 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.558 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105426 11.e 11:72f5b0f4:::benchmark_data_smithi053_199129_object11493:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.817461+0000", 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.717698941999998, 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99972198700000003, 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105426 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817461+0000", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817461+0000", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817460+0000", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.895e-06 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817464+0000", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0678e-05 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817475+0000", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2140000000000004e-06 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817480+0000", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8718999999999998e-05 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817539+0000", 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7880000000000001e-06 2024-02-22T19:50:38.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817545+0000", 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99130663399999996 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.808852+0000", 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012742999999999999 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.808979+0000", 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105271 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.809085+0000", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00149117 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810576+0000", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0945999999999995e-05 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810647+0000", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064376060000000002 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817084+0000", 2024-02-22T19:50:38.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7788999999999999e-05 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817132+0000", 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0430000000000003e-05 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817183+0000", 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99972198700000003 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105427 11.e 11:77cecb34:::benchmark_data_smithi053_199129_object11494:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.817501+0000", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.717658512, 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99985556200000003, 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105427 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817501+0000", 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817501+0000", 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817500+0000", 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9659999999999998e-06 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817504+0000", 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.148e-06 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817511+0000", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8810000000000004e-06 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817516+0000", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7185000000000003e-05 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817563+0000", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5609999999999998e-06 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.817569+0000", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99310532600000001 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810674+0000", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7155000000000003e-05 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810741+0000", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0909999999999997e-05 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.810812+0000", 2024-02-22T19:50:38.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004042014 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.814854+0000", 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8766e-05 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.814903+0000", 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023858199999999999 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817289+0000", 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9335999999999998e-05 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817318+0000", 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8343999999999997e-05 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817357+0000", 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99985556200000003 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105468 11.e 11:7531340d:::benchmark_data_smithi053_199129_object11535:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.916080+0000", 2024-02-22T19:50:38.570 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.619079460999998, 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.90138597099999995, 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105468 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916080+0000", 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916080+0000", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916078+0000", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.362e-06 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916086+0000", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5508000000000001e-05 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916101+0000", 2024-02-22T19:50:38.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9260000000000003e-06 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916108+0000", 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00047945600000000001 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916588+0000", 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7950000000000004e-06 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.916593+0000", 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89506605699999997 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.811659+0000", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3600000000000001e-05 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.811723+0000", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2341000000000004e-05 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.811795+0000", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031445470000000001 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.814940+0000", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7880000000000007e-06 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.814948+0000", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024573070000000002 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817405+0000", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2045999999999998e-05 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817437+0000", 2024-02-22T19:50:38.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9113000000000001e-05 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817466+0000", 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.90138597099999995 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105477 11.e 11:742b7bdf:::benchmark_data_smithi053_199129_object11544:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.932144+0000", 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.603015155000001, 2024-02-22T19:50:38.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89024312699999997, 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105477 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932144+0000", 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932144+0000", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932143+0000", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0790000000000003e-06 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932150+0000", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.116e-05 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932161+0000", 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0270000000000003e-06 2024-02-22T19:50:38.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932166+0000", 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7766000000000003e-05 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932214+0000", 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8920000000000006e-06 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932223+0000", 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.88048919800000003 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.812712+0000", 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1755999999999996e-05 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.812774+0000", 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4672999999999995e-05 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.812849+0000", 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004063377 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.816912+0000", 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001690457 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818602+0000", 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037127330000000002 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822315+0000", 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8751999999999998e-05 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822344+0000", 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3770999999999998e-05 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822388+0000", 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89024312699999997 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105479 11.e 11:7370c346:::benchmark_data_smithi053_199129_object11546:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.582 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.932173+0000", 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.602986348999998, 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89036086699999994, 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105479 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932173+0000", 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932173+0000", 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932173+0000", 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932177+0000", 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6610000000000003e-06 2024-02-22T19:50:38.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932183+0000", 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0929999999999996e-06 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932187+0000", 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1879999999999996e-05 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932259+0000", 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2989999999999996e-06 2024-02-22T19:50:38.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932265+0000", 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.88153535199999999 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.813800+0000", 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4929000000000001e-05 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.813865+0000", 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105085 2024-02-22T19:50:38.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.813970+0000", 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032288350000000002 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817199+0000", 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001440394 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818639+0000", 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038355220000000001 2024-02-22T19:50:38.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822475+0000", 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4583000000000002e-05 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822499+0000", 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4875000000000001e-05 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822534+0000", 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89036086699999994 2024-02-22T19:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105480 11.e 11:78c4a1cb:::benchmark_data_smithi053_199129_object11547:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.932209+0000", 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.602950876000001, 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89226565599999996, 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105480 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.589 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932209+0000", 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932209+0000", 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932208+0000", 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4010000000000001e-06 2024-02-22T19:50:38.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932211+0000", 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932217+0000", 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9359999999999998e-06 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932222+0000", 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5061000000000006e-05 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932287+0000", 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5209999999999999e-06 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.932294+0000", 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.88268260600000004 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.814977+0000", 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5401000000000006e-05 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.815042+0000", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3847e-05 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.815116+0000", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002120832 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817237+0000", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052012990000000004 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822438+0000", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019743579999999998 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.824412+0000", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8658000000000002e-05 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.824451+0000", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3456999999999999e-05 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.824474+0000", 2024-02-22T19:50:38.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89226565599999996 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105484 11.e 11:78657e34:::benchmark_data_smithi053_199129_object11551:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.942134+0000", 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.593025417, 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.88241446499999998, 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105484 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.595 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942134+0000", 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942134+0000", 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942133+0000", 2024-02-22T19:50:38.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.374e-06 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942140+0000", 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2884999999999999e-05 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942153+0000", 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3720000000000001e-06 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942159+0000", 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9091000000000001e-05 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942208+0000", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6579999999999996e-06 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.942215+0000", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87376661 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.815982+0000", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0316000000000003e-05 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.816042+0000", 2024-02-22T19:50:38.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9864999999999996e-05 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.816112+0000", 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001361112 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817473+0000", 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051218360000000003 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822595+0000", 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001906785 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.824502+0000", 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4434e-05 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.824526+0000", 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2617e-05 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.824549+0000", 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.88241446499999998 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.600 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105492 11.e 11:75ca9c82:::benchmark_data_smithi053_199129_object11559:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.964248+0000", 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.570911170999999, 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86312398000000001, 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105492 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964248+0000", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964248+0000", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964246+0000", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4910000000000005e-06 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964255+0000", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6588000000000001e-05 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964272+0000", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5719999999999992e-06 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964281+0000", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1275e-05 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964332+0000", 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0961000000000001e-05 2024-02-22T19:50:38.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.964343+0000", 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85315822799999996 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817502+0000", 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8019e-05 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817570+0000", 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4902999999999998e-05 2024-02-22T19:50:38.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.817644+0000", 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047540109999999998 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822398+0000", 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038527600000000002 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.826251+0000", 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001073016 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827324+0000", 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7035000000000001e-05 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827351+0000", 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1141e-05 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827372+0000", 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86312398000000001 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105506 11.e 11:72c63725:::benchmark_data_smithi053_199129_object11573:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:36.999438+0000", 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.535721485000003, 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.82801452200000003, 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105506 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999438+0000", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999438+0000", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999437+0000", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9190000000000004e-06 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999443+0000", 2024-02-22T19:50:38.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0781000000000001e-05 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999453+0000", 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9609999999999996e-06 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999458+0000", 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6261000000000003e-05 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999505+0000", 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9599999999999995e-06 2024-02-22T19:50:38.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:36.999515+0000", 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.81915479999999996 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818669+0000", 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8688000000000001e-05 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818768+0000", 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2238999999999997e-05 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818840+0000", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037037989999999998 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822544+0000", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037539919999999998 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.826298+0000", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0011084230000000001 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827407+0000", 2024-02-22T19:50:38.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5337000000000002e-05 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827432+0000", 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0712000000000001e-05 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827453+0000", 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.82801452200000003 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105557 11.e 11:7634af4b:::benchmark_data_smithi053_199129_object11624:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.170694+0000", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.364465500000001, 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.65686243499999997, 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105557 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170694+0000", 2024-02-22T19:50:38.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170694+0000", 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170692+0000", 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8360000000000004e-06 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170702+0000", 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5807999999999998e-05 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170718+0000", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0909999999999998e-06 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170724+0000", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9270000000000002e-05 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170763+0000", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7229999999999998e-06 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.170770+0000", 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.64894335999999997 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.819713+0000", 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9615000000000003e-05 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.819793+0000", 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010721300000000001 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.819900+0000", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026503009999999999 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822551+0000", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037942539999999999 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.826345+0000", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0011420040000000001 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827487+0000", 2024-02-22T19:50:38.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4202999999999998e-05 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827511+0000", 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5596999999999997e-05 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827557+0000", 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.65686243499999997 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.618 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105572 11.e 11:77d72baf:::benchmark_data_smithi053_199129_object11639:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.229250+0000", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.305909219999997, 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.59841477600000004, 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105572 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229250+0000", 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229250+0000", 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229248+0000", 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1619e-05 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229260+0000", 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4224e-05 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229274+0000", 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9999999999999999e-06 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229281+0000", 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6829999999999997e-05 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229328+0000", 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1489999999999994e-06 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.229336+0000", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.59146008500000002 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.820796+0000", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010644799999999999 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.820903+0000", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012365400000000001 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.821026+0000", 2024-02-22T19:50:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026136839999999998 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.823640+0000", 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002736755 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.826377+0000", 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001210083 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827587+0000", 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9279000000000001e-05 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827626+0000", 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9104999999999999e-05 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.827665+0000", 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.59841477600000004 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105575 11.e 11:75593d73:::benchmark_data_smithi053_199129_object11642:head [delete] snapc 0=[] ondisk+write+known_if_redirected e526)", 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.236671+0000", 2024-02-22T19:50:38.624 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.298489001, 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.59278575499999997, 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105575 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236671+0000", 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236671+0000", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236669+0000", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0390000000000005e-06 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236677+0000", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4127e-05 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236691+0000", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2599999999999999e-06 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236698+0000", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4416000000000003e-05 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236743+0000", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0472000000000001e-05 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.236753+0000", 2024-02-22T19:50:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.58585936999999999 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822613+0000", 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4478000000000002e-05 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822677+0000", 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5419000000000003e-05 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,7", 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.822763+0000", 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00097403799999999999 2024-02-22T19:50:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.823737+0000", 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002663398 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.826400+0000", 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029901530000000002 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.829390+0000", 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0165999999999999e-05 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.829420+0000", 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6015e-05 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.829456+0000", 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.59278575499999997 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:2995 5.5 5:b4812045:::notify.4:head [watch ping cookie 94129597100544 gen 17] snapc 0=[] ondisk+write+known_if_redirected e523)", 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:39.475284+0000", 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.059875861000002, 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34494148499999999, 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2995 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475284+0000", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475284+0000", 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:50:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475283+0000", 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8600000000000001e-06 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475286+0000", 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7150000000000003e-06 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475292+0000", 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9660000000000004e-06 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475297+0000", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000147217 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475444+0000", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0234000000000001e-05 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475454+0000", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34455793099999998 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820012+0000", 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135783 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820148+0000", 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7243999999999994e-05 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820225+0000", 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34494148499999999 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:3002 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 17] snapc 0=[] ondisk+write+known_if_redirected e528)", 2024-02-22T19:50:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:39.475922+0000", 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.059237482, 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.445460095, 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 3002 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475922+0000", 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475922+0000", 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475921+0000", 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3559999999999998e-06 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475927+0000", 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2118e-05 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475939+0000", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4760000000000003e-06 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475946+0000", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1441999999999998e-05 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476017+0000", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1829e-05 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476029+0000", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34422191499999999 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820251+0000", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3617999999999999e-05 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820325+0000", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10105755800000001 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.921382+0000", 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.445460095 2024-02-22T19:50:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:3003 5.5 5:b4812045:::notify.4:head [watch reconnect cookie 94129597100544 gen 18] snapc 0=[] ondisk+write+known_if_redirected e528)", 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:39.475958+0000", 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.059201559000002, 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.44561640000000002, 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 3003 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475958+0000", 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475958+0000", 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475958+0000", 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8710000000000001e-06 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475960+0000", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5218999999999999e-05 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475976+0000", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.475981+0000", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7439000000000002e-05 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476058+0000", 2024-02-22T19:50:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1912e-05 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476070+0000", 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34459168499999998 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820662+0000", 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4869000000000004e-05 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.820737+0000", 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100837885 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.921574+0000", 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.44561640000000002 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:3006 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 94129597098240 gen 18] snapc 0=[] ondisk+write+known_if_redirected e528)", 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:39.476023+0000", 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.059136416000001, 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.44567595100000001, 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:50:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 3006 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476023+0000", 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476023+0000", 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476023+0000", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1e-06 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476026+0000", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8709999999999999e-06 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476033+0000", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.544e-06 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476037+0000", 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7148999999999998e-05 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476094+0000", 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1756000000000001e-05 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476106+0000", 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34493089599999999 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.821037+0000", 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4468999999999995e-05 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.821111+0000", 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100587705 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.921699+0000", 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.44567595100000001 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.24332.0:3007 5.5 5:b4812045:::notify.4:head [watch reconnect cookie 94129597100544 gen 19] snapc 0=[] ondisk+write+known_if_redirected e528)", 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:39.476052+0000", 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.059107134999998, 2024-02-22T19:50:38.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.44575484100000001, 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.24332", 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/1058598254", 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 3007 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476052+0000", 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:38.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476052+0000", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476052+0000", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7360000000000001e-06 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476055+0000", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6409999999999999e-06 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476060+0000", 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2690000000000001e-06 2024-02-22T19:50:38.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476065+0000", 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0736999999999997e-05 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476135+0000", 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5110000000000003e-06 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.476141+0000", 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.34523799199999999 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.821379+0000", 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2096999999999998e-05 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.821451+0000", 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100356284 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.921807+0000", 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.44575484100000001 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:38.651 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:50:38.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:38.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:38.753 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:39.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:50:39.704 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:37.915005+0000 mgr.y (mgr.14535) 822 : cluster [DBG] pgmap v1135: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 507 op/s; 1055/149331 objects degraded (0.706%); 35487/149331 objects misplaced (23.764%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:38.408894+0000 mon.a (mon.0) 1953 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:39.709 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:39.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:39.710 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:37.915005+0000 mgr.y (mgr.14535) 822 : cluster [DBG] pgmap v1135: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 507 op/s; 1055/149331 objects degraded (0.706%); 35487/149331 objects misplaced (23.764%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:39.710 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:38.408894+0000 mon.a (mon.0) 1953 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:39.710 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:50:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:39] "GET /metrics HTTP/1.1" 200 176739 "" "Prometheus/2.33.4" 2024-02-22T19:50:39.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:39.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:37.915005+0000 mgr.y (mgr.14535) 822 : cluster [DBG] pgmap v1135: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 3.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 507 op/s; 1055/149331 objects degraded (0.706%); 35487/149331 objects misplaced (23.764%); 5.1 MiB/s, 81 objects/s recovering 2024-02-22T19:50:39.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:38.408894+0000 mon.a (mon.0) 1953 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:39.892 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:39.934 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:39.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:39.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:40.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:40.133 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:40.173 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:40.179 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:40.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:40.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:40.343 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:40.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:40.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:40.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:40.561 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:40.736 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:40.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:40.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:40.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:40.924 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:41.431 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:50:41.465 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:50:41.471 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 5, 4, 6, 1] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:50:41.471 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:50:41.471 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-02-22T19:50:41.471 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:50:41.471 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:50:41.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-02-22T19:50:41.509 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:50:41.544 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:50:41.585 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:50:41.620 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:50:41.656 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:50:41.697 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:50:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.715 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.715 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.715 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:50:41.720 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:41.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:41.732 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:39.919309+0000 mgr.y (mgr.14535) 823 : cluster [DBG] pgmap v1136: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 636 op/s; 1055/144630 objects degraded (0.729%); 34401/144630 objects misplaced (23.786%); 6.6 MiB/s, 105 objects/s recovering 2024-02-22T19:50:41.732 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:41.386917+0000 mon.a (mon.0) 1954 : audit [DBG] from='client.? 172.21.15.53:0/224991345' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:41.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:41.738 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:39.919309+0000 mgr.y (mgr.14535) 823 : cluster [DBG] pgmap v1136: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 636 op/s; 1055/144630 objects degraded (0.729%); 34401/144630 objects misplaced (23.786%); 6.6 MiB/s, 105 objects/s recovering 2024-02-22T19:50:41.738 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:41.386917+0000 mon.a (mon.0) 1954 : audit [DBG] from='client.? 172.21.15.53:0/224991345' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:41.738 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:39.919309+0000 mgr.y (mgr.14535) 823 : cluster [DBG] pgmap v1136: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 636 op/s; 1055/144630 objects degraded (0.729%); 34401/144630 objects misplaced (23.786%); 6.6 MiB/s, 105 objects/s recovering 2024-02-22T19:50:41.738 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:41.386917+0000 mon.a (mon.0) 1954 : audit [DBG] from='client.? 172.21.15.53:0/224991345' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:41.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:41.906 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:42.276 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:42.286 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:42.287 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:42.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:42.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:42.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:42.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:50:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:42.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:50:42.879 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:43.249 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:50:43.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:43.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:50:43.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:43.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:41.922813+0000 mgr.y (mgr.14535) 824 : cluster [DBG] pgmap v1137: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 1181/139809 objects degraded (0.845%); 33180/139809 objects misplaced (23.732%); 7.9 MiB/s, 127 objects/s recovering 2024-02-22T19:50:43.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:42.652496+0000 mon.a (mon.0) 1955 : cluster [WRN] Health check update: Degraded data redundancy: 1181/139809 objects degraded (0.845%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:43.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:41.922813+0000 mgr.y (mgr.14535) 824 : cluster [DBG] pgmap v1137: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 1181/139809 objects degraded (0.845%); 33180/139809 objects misplaced (23.732%); 7.9 MiB/s, 127 objects/s recovering 2024-02-22T19:50:43.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:42.652496+0000 mon.a (mon.0) 1955 : cluster [WRN] Health check update: Degraded data redundancy: 1181/139809 objects degraded (0.845%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:43.629 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:43.816 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:50:43.816 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:50:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:41.922813+0000 mgr.y (mgr.14535) 824 : cluster [DBG] pgmap v1137: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 777 op/s; 1181/139809 objects degraded (0.845%); 33180/139809 objects misplaced (23.732%); 7.9 MiB/s, 127 objects/s recovering 2024-02-22T19:50:43.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:42.652496+0000 mon.a (mon.0) 1955 : cluster [WRN] Health check update: Degraded data redundancy: 1181/139809 objects degraded (0.845%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:44.100 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:44.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:44.104 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:44.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:44.853 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:50:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:50:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:45.050 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:45.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:45.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:45.218 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:45.396 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:45.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:45.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:45.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:43.924364+0000 mgr.y (mgr.14535) 825 : cluster [DBG] pgmap v1138: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 574 B/s rd, 0 B/s wr, 521 op/s; 1181/139809 objects degraded (0.845%); 33180/139809 objects misplaced (23.732%); 5.4 MiB/s, 85 objects/s recovering 2024-02-22T19:50:45.479 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:44.756939+0000 osd.4 (osd.4) 200 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:45.480 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:44.756992+0000 osd.4 (osd.4) 201 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:45.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:45.546 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:45.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:50:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:43.924364+0000 mgr.y (mgr.14535) 825 : cluster [DBG] pgmap v1138: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 574 B/s rd, 0 B/s wr, 521 op/s; 1181/139809 objects degraded (0.845%); 33180/139809 objects misplaced (23.732%); 5.4 MiB/s, 85 objects/s recovering 2024-02-22T19:50:45.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:44.756939+0000 osd.4 (osd.4) 200 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:45.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:44.756992+0000 osd.4 (osd.4) 201 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:45.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:43.924364+0000 mgr.y (mgr.14535) 825 : cluster [DBG] pgmap v1138: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 574 B/s rd, 0 B/s wr, 521 op/s; 1181/139809 objects degraded (0.845%); 33180/139809 objects misplaced (23.732%); 5.4 MiB/s, 85 objects/s recovering 2024-02-22T19:50:45.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:44.756939+0000 osd.4 (osd.4) 200 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:45.757 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:44.756992+0000 osd.4 (osd.4) 201 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:45.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:50:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:50:45.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:50:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:46.151 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:46.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:46.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:46.505 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:50:46.505 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "" 2024-02-22T19:50:46.505 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:50:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:45.459547+0000 mon.a (mon.0) 1956 : cluster [DBG] osdmap e561: 8 total, 8 up, 7 in 2024-02-22T19:50:46.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:45.616217+0000 mgr.y (mgr.14535) 826 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:46.531 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:46.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:46.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:46.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:46.718 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:46.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:45.459547+0000 mon.a (mon.0) 1956 : cluster [DBG] osdmap e561: 8 total, 8 up, 7 in 2024-02-22T19:50:46.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:45.616217+0000 mgr.y (mgr.14535) 826 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:46.765 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:46.771 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:46.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:45.459547+0000 mon.a (mon.0) 1956 : cluster [DBG] osdmap e561: 8 total, 8 up, 7 in 2024-02-22T19:50:46.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:45.616217+0000 mgr.y (mgr.14535) 826 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:46.873 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:46.884 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:46.942 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:47.213 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:47.224 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:47.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:47.245 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:47.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:47.380 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:47.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:45.924868+0000 mgr.y (mgr.14535) 827 : cluster [DBG] pgmap v1140: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.6 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 762 op/s; 1322/130434 objects degraded (1.014%); 30878/130434 objects misplaced (23.673%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:47.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:47.229109+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.53:0/49478231' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:50:47.519 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:47.231040+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:50:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:50:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:45.924868+0000 mgr.y (mgr.14535) 827 : cluster [DBG] pgmap v1140: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.6 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 762 op/s; 1322/130434 objects degraded (1.014%); 30878/130434 objects misplaced (23.673%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:47.229109+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.53:0/49478231' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:50:47.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:47.231040+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:50:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:45.924868+0000 mgr.y (mgr.14535) 827 : cluster [DBG] pgmap v1140: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.6 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 762 op/s; 1322/130434 objects degraded (1.014%); 30878/130434 objects misplaced (23.673%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:47.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:47.229109+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.53:0/49478231' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:50:47.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:47.231040+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:50:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:46.817583+0000 osd.2 (osd.2) 298 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:46.817640+0000 osd.2 (osd.2) 299 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:47.502360+0000 mon.a (mon.0) 1958 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:47.511147+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:50:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:47.511222+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e562: 8 total, 8 up, 7 in 2024-02-22T19:50:48.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:47.653401+0000 mon.a (mon.0) 1961 : cluster [WRN] Health check update: Degraded data redundancy: 1322/130434 objects degraded (1.014%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:49.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:46.817583+0000 osd.2 (osd.2) 298 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:49.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:46.817640+0000 osd.2 (osd.2) 299 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:49.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:47.502360+0000 mon.a (mon.0) 1958 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:49.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:47.511147+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:50:49.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:47.511222+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e562: 8 total, 8 up, 7 in 2024-02-22T19:50:49.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:47.653401+0000 mon.a (mon.0) 1961 : cluster [WRN] Health check update: Degraded data redundancy: 1322/130434 objects degraded (1.014%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:49.015 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:46.817583+0000 osd.2 (osd.2) 298 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:49.015 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:46.817640+0000 osd.2 (osd.2) 299 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:49.015 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:47.502360+0000 mon.a (mon.0) 1958 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:50:49.015 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:47.511147+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:50:49.015 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:47.511222+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e562: 8 total, 8 up, 7 in 2024-02-22T19:50:49.016 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:47.653401+0000 mon.a (mon.0) 1961 : cluster [WRN] Health check update: Degraded data redundancy: 1322/130434 objects degraded (1.014%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:49.234 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:50:49.234 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:50:49.546 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:47.925361+0000 mgr.y (mgr.14535) 828 : cluster [DBG] pgmap v1142: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.6 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 591 op/s; 1322/130434 objects degraded (1.014%); 30878/130434 objects misplaced (23.673%); 5.9 MiB/s, 94 objects/s recovering 2024-02-22T19:50:49.546 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:47.925361+0000 mgr.y (mgr.14535) 828 : cluster [DBG] pgmap v1142: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.6 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 591 op/s; 1322/130434 objects degraded (1.014%); 30878/130434 objects misplaced (23.673%); 5.9 MiB/s, 94 objects/s recovering 2024-02-22T19:50:49.547 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:50:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:49] "GET /metrics HTTP/1.1" 200 176739 "" "Prometheus/2.33.4" 2024-02-22T19:50:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:49.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:49.728 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:49.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:49.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:47.925361+0000 mgr.y (mgr.14535) 828 : cluster [DBG] pgmap v1142: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.6 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 591 op/s; 1322/130434 objects degraded (1.014%); 30878/130434 objects misplaced (23.673%); 5.9 MiB/s, 94 objects/s recovering 2024-02-22T19:50:49.910 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:50.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:50:50.315 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:50.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:50.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:50.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:50.505 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:50.551 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:50.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:50.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:50.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:50.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:50.831 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:50:50.863 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:50:50.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:50.902 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:50:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:50.940 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:50:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:51.002 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:50:51.043 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:50:51.104 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:50:51.133 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:50:51.139 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.139 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.139 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.139 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.140 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.140 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.140 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.140 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:50:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:51.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:51.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:51.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:51.771 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:49.926556+0000 mgr.y (mgr.14535) 829 : cluster [DBG] pgmap v1143: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 581 op/s; 1322/125865 objects degraded (1.050%); 29803/125865 objects misplaced (23.679%); 5.8 MiB/s, 93 objects/s recovering 2024-02-22T19:50:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:50.788499+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.53:0/2879294708' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:49.926556+0000 mgr.y (mgr.14535) 829 : cluster [DBG] pgmap v1143: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 581 op/s; 1322/125865 objects degraded (1.050%); 29803/125865 objects misplaced (23.679%); 5.8 MiB/s, 93 objects/s recovering 2024-02-22T19:50:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:50.788499+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.53:0/2879294708' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:49.926556+0000 mgr.y (mgr.14535) 829 : cluster [DBG] pgmap v1143: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 581 op/s; 1322/125865 objects degraded (1.050%); 29803/125865 objects misplaced (23.679%); 5.8 MiB/s, 93 objects/s recovering 2024-02-22T19:50:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:50.788499+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.53:0/2879294708' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:50:52.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:50:52.270 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 5, 4, 6, 1] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:50:52.270 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:50:52.271 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [2, 3, 7, 5, 4, 6, 1] 2024-02-22T19:50:52.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 5 2024-02-22T19:50:52.446 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:52.450 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:52.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:52.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:52.605 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:52.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:52.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:52.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:53.039 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:53.049 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:53.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:53.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:53.391 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:53.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:53.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:53.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:50:53.576 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:53.594 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:51.927955+0000 mgr.y (mgr.14535) 830 : cluster [DBG] pgmap v1144: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 775 op/s; 1482/121224 objects degraded (1.223%); 28591/121224 objects misplaced (23.585%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:53.594 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.024150+0000 osd.1 (osd.1) 301 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:53.594 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.024201+0000 osd.1 (osd.1) 302 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:53.595 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.582776+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e563: 8 total, 8 up, 7 in 2024-02-22T19:50:53.595 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.659724+0000 mon.a (mon.0) 1963 : cluster [WRN] Health check update: Degraded data redundancy: 1482/121224 objects degraded (1.223%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:53.595 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.681859+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e564: 8 total, 8 up, 7 in 2024-02-22T19:50:53.644 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:50:53.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:53.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:50:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:53.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:51.927955+0000 mgr.y (mgr.14535) 830 : cluster [DBG] pgmap v1144: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 775 op/s; 1482/121224 objects degraded (1.223%); 28591/121224 objects misplaced (23.585%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.024150+0000 osd.1 (osd.1) 301 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.024201+0000 osd.1 (osd.1) 302 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.582776+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e563: 8 total, 8 up, 7 in 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.659724+0000 mon.a (mon.0) 1963 : cluster [WRN] Health check update: Degraded data redundancy: 1482/121224 objects degraded (1.223%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.681859+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e564: 8 total, 8 up, 7 in 2024-02-22T19:50:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:53.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:51.927955+0000 mgr.y (mgr.14535) 830 : cluster [DBG] pgmap v1144: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 775 op/s; 1482/121224 objects degraded (1.223%); 28591/121224 objects misplaced (23.585%); 7.8 MiB/s, 124 objects/s recovering 2024-02-22T19:50:53.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.024150+0000 osd.1 (osd.1) 301 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:53.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.024201+0000 osd.1 (osd.1) 302 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:53.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.582776+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e563: 8 total, 8 up, 7 in 2024-02-22T19:50:53.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.659724+0000 mon.a (mon.0) 1963 : cluster [WRN] Health check update: Degraded data redundancy: 1482/121224 objects degraded (1.223%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:53.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.681859+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e564: 8 total, 8 up, 7 in 2024-02-22T19:50:53.990 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:53.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:53.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:53.995 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:54.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:113474 10.6 10:653140ba:::benchmark_data_smithi053_180506_object95:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:38.924933+0000", 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 315.17475929400001, 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.123098763, 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 113474 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924933+0000", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924933+0000", 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:50:54.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924932+0000", 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1629999999999998e-06 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924937+0000", 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9600000000000003e-06 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924944+0000", 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2339999999999996e-06 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924948+0000", 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7381e-05 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.924985+0000", 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9109000000000004e-05 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.925064+0000", 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8466000000000002e-05 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.925163+0000", 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026464800000000001 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.927809+0000", 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003479312 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.931288+0000", 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.11665671599999999 2024-02-22T19:50:54.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047945+0000", 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6978000000000003e-05 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.047992+0000", 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9952e-05 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.048032+0000", 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.123098763 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.109 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:113477 10.6 10:6e6adf7f:::benchmark_data_smithi053_180506_object98:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:45:38.929580+0000", 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 315.17011315299999, 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.118571258, 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 113477 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929580+0000", 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929580+0000", 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929578+0000", 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.596e-06 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929585+0000", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9794000000000001e-05 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929605+0000", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3900000000000004e-06 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929612+0000", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4569e-05 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929657+0000", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013838500000000001 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929795+0000", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000147109 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.929942+0000", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001439961 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.931382+0000", 2024-02-22T19:50:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017512820000000001 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:38.933133+0000", 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.114939236 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.048073+0000", 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2735000000000003e-05 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.048115+0000", 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5441000000000002e-05 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:45:39.048151+0000", 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.118571258 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:181075 10.a 10:50b96053:::benchmark_data_smithi053_180506_object67696:head [delete] snapc 0=[] ondisk+write+known_if_redirected e449)", 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:46:40.738334+0000", 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 253.36135823500001, 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.17640343, 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.115 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 181075 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738334+0000", 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738334+0000", 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738333+0000", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.848e-06 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738338+0000", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6590000000000001e-06 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738345+0000", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1870000000000003e-06 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738354+0000", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0534000000000002e-05 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738394+0000", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011422299999999999 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738509+0000", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014745900000000001 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.738656+0000", 2024-02-22T19:50:54.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019162980000000001 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.740572+0000", 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020607630000000002 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.742633+0000", 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.17201891999999999 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.914652+0000", 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7459000000000001e-05 2024-02-22T19:50:54.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.914699+0000", 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8430999999999998e-05 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:46:40.914738+0000", 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.17640343 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.15576.0:223076 10.a 10:5ddba273:::benchmark_data_smithi053_180506_object109697:head [delete] snapc 0=[] ondisk+write+known_if_redirected e462)", 2024-02-22T19:50:54.120 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:47:19.039333+0000", 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 215.06035919499999, 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.120513365, 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.15576", 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/146345831", 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 223076 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039333+0000", 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039333+0000", 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039332+0000", 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5690000000000001e-06 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039338+0000", 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0451e-05 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039349+0000", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9200000000000003e-06 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039354+0000", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9876000000000002e-05 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039404+0000", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012313 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039527+0000", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014478399999999999 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.039672+0000", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020016560000000001 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.041673+0000", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042370070000000001 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.045910+0000", 2024-02-22T19:50:54.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.113849189 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.159759+0000", 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6615000000000001e-05 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.159796+0000", 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0760000000000002e-05 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:47:19.159847+0000", 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.120513365 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89463 11.7 11:e37c34f6:::benchmark_data_smithi053_199129_object89462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.408624+0000", 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.69106902599999, 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99583211299999996, 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89463 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408624+0000", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408624+0000", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408622+0000", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100809 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408722+0000", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7229999999999999e-05 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408740+0000", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2840000000000006e-06 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408748+0000", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4552000000000002e-05 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408802+0000", 2024-02-22T19:50:54.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0182e-05 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.408813+0000", 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98604147099999995 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.394854+0000", 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018199899999999999 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.395036+0000", 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021620400000000001 2024-02-22T19:50:54.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.395252+0000", 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089683750000000007 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404221+0000", 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3247999999999999e-05 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404234+0000", 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012663799999999999 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404361+0000", 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4469999999999999e-05 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404415+0000", 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0708999999999999e-05 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404456+0000", 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.99583211299999996 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89466 11.7 11:e4a81301:::benchmark_data_smithi053_199129_object89465:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.417633+0000", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.682059795, 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98698154699999996, 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89466 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417633+0000", 2024-02-22T19:50:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417633+0000", 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417632+0000", 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5211999999999999e-05 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417687+0000", 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1559999999999999e-05 2024-02-22T19:50:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417698+0000", 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3100000000000003e-06 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417706+0000", 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5966000000000001e-05 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417742+0000", 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.007e-06 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.417749+0000", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.97967531500000005 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.397424+0000", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104446 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.397528+0000", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010173799999999999 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.397630+0000", 2024-02-22T19:50:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066905669999999997 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404321+0000", 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017920999999999999 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404500+0000", 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1813000000000002e-05 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404532+0000", 2024-02-22T19:50:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4987999999999997e-05 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404577+0000", 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7718e-05 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404614+0000", 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98698154699999996 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89517 11.7 11:e04f8ebd:::benchmark_data_smithi053_199129_object89516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.521921+0000", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.57777197599999, 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89320850100000004, 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89517 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.521921+0000", 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.521921+0000", 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999852 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.521906+0000", 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2825999999999998e-05 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.521969+0000", 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4404e-05 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.521983+0000", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.815e-06 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.521990+0000", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9128000000000001e-05 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.522039+0000", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-06 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.522048+0000", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87742525400000004 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.399474+0000", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012682000000000001 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.399601+0000", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000168138 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.399769+0000", 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045572980000000004 2024-02-22T19:50:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404326+0000", 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004177905 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408504+0000", 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065311329999999997 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415035+0000", 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039000000000001e-05 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415087+0000", 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2162000000000001e-05 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415129+0000", 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.89320850100000004 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89526 11.7 11:e1f429ad:::benchmark_data_smithi053_199129_object89525:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.540000+0000", 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.559692352, 2024-02-22T19:50:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87520691500000003, 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89526 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540000+0000", 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540000+0000", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.539998+0000", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6768999999999999e-05 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540045+0000", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3627e-05 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540058+0000", 2024-02-22T19:50:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8139999999999997e-06 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540064+0000", 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2888e-05 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540107+0000", 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.892e-06 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.540115+0000", 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.86074207300000005 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.400857+0000", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6252e-05 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.400943+0000", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015819299999999999 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.401101+0000", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033657119999999999 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404467+0000", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040823379999999996 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408550+0000", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066120299999999996 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415162+0000", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6094000000000001e-05 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415188+0000", 2024-02-22T19:50:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9598000000000001e-05 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415207+0000", 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87520691500000003 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89554 11.7 11:ea928ba5:::benchmark_data_smithi053_199129_object89553:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.586531+0000", 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.513161677, 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.83234146200000003, 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89554 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586531+0000", 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586531+0000", 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586529+0000", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7052999999999998e-05 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586576+0000", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9029e-05 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586595+0000", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4819999999999994e-06 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586603+0000", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6217000000000001e-05 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586659+0000", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.533e-06 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.586669+0000", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.81590080799999998 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.402570+0000", 2024-02-22T19:50:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116038 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.402686+0000", 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000156298 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.402842+0000", 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056202539999999999 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408462+0000", 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067685330000000002 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415231+0000", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003549001 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418780+0000", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1589999999999999e-05 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418831+0000", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0991000000000001e-05 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418872+0000", 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.83234146200000003 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89562 11.7 11:eac50361:::benchmark_data_smithi053_199129_object89561:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.606329+0000", 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.49336376700001, 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.81270851899999996, 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89562 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606329+0000", 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606329+0000", 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606327+0000", 2024-02-22T19:50:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6825000000000001e-05 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606364+0000", 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3517e-05 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606377+0000", 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.43e-06 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606385+0000", 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9274000000000002e-05 2024-02-22T19:50:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606444+0000", 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4280000000000001e-06 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.606450+0000", 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.79818904300000004 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404639+0000", 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110454 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404750+0000", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018028400000000001 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.404930+0000", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003580415 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408511+0000", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067370590000000001 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415248+0000", 2024-02-22T19:50:54.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037034530000000002 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418951+0000", 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7274999999999997e-05 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418998+0000", 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8985e-05 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419037+0000", 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.81270851899999996 2024-02-22T19:50:54.160 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89567 11.7 11:e7dd5834:::benchmark_data_smithi053_199129_object89566:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.614351+0000", 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.48534192699999, 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.80481371000000002, 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89567 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614351+0000", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614351+0000", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614349+0000", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7498999999999999e-05 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614416+0000", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.844e-05 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614435+0000", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2529999999999993e-06 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614444+0000", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5019999999999998e-05 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614499+0000", 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4412e-05 2024-02-22T19:50:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.614513+0000", 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.791957825 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.406471+0000", 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000117217 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.406588+0000", 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146775 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.406735+0000", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007017878 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.413753+0000", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001551991 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415305+0000", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003778254 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419083+0000", 2024-02-22T19:50:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5207999999999998e-05 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419129+0000", 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5932000000000001e-05 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419164+0000", 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.80481371000000002 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89606 11.7 11:efebbe53:::benchmark_data_smithi053_199129_object89605:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.684757+0000", 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.414935224, 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.73451909999999998, 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89606 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684757+0000", 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684757+0000", 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684755+0000", 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1604000000000003e-05 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684807+0000", 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1661000000000001e-05 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684819+0000", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.784e-06 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684824+0000", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1563999999999995e-05 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684886+0000", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3145999999999999e-05 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.684899+0000", 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.72367371300000005 2024-02-22T19:50:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408573+0000", 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012589799999999999 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408699+0000", 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018253900000000001 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.408881+0000", 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061110319999999997 2024-02-22T19:50:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.414992+0000", 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000346392 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415339+0000", 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038649280000000001 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419204+0000", 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9702e-05 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419243+0000", 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3306999999999998e-05 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419277+0000", 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.73451909999999998 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89617 11.7 11:e689d319:::benchmark_data_smithi053_199129_object89616:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.703764+0000", 2024-02-22T19:50:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.395928822, 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.72200328499999999, 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89617 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703764+0000", 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703764+0000", 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703762+0000", 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014313800000000001 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703905+0000", 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0049e-05 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703915+0000", 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3830000000000002e-06 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703919+0000", 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0259999999999997e-05 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703959+0000", 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.025e-05 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.703970+0000", 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.70656955600000004 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.410539+0000", 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011355 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.410653+0000", 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000168049 2024-02-22T19:50:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.410821+0000", 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0041749329999999996 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.414996+0000", 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003721505 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418717+0000", 2024-02-22T19:50:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0069638210000000002 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425681+0000", 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8522000000000001e-05 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425730+0000", 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7558999999999997e-05 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425767+0000", 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.72200328499999999 2024-02-22T19:50:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89624 11.7 11:e89d3397:::benchmark_data_smithi053_199129_object89623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.724312+0000", 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.375380709, 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.70159912800000002, 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89624 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724312+0000", 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724312+0000", 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724310+0000", 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4558999999999999e-05 2024-02-22T19:50:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724345+0000", 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4e-05 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724359+0000", 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1199999999999996e-06 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724366+0000", 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1637999999999997e-05 2024-02-22T19:50:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724407+0000", 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9209999999999997e-06 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.724413+0000", 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.68795055199999999 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.412364+0000", 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014268000000000001 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.412506+0000", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016939599999999999 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.412676+0000", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00246423 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415140+0000", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036110040000000001 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418751+0000", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070754420000000004 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425827+0000", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7475999999999998e-05 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425864+0000", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7080999999999999e-05 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425911+0000", 2024-02-22T19:50:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.70159912800000002 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89647 11.7 11:e9eca426:::benchmark_data_smithi053_199129_object89646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.781355+0000", 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.318337247, 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.64465730300000001, 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89647 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781355+0000", 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781355+0000", 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781353+0000", 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0637000000000002e-05 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781414+0000", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1841999999999999e-05 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781436+0000", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.67e-06 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781444+0000", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5878000000000003e-05 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781500+0000", 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7249999999999992e-06 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.781510+0000", 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.63227207299999999 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.413782+0000", 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012987799999999999 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.413912+0000", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3768000000000001e-05 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.414006+0000", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001206504 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415212+0000", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037065010000000001 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.418919+0000", 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070243850000000002 2024-02-22T19:50:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425943+0000", 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7081e-05 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425980+0000", 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2546000000000003e-05 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.426013+0000", 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.64465730300000001 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89700 11.7 11:efda5ed9:::benchmark_data_smithi053_199129_object89699:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:08.994293+0000", 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.105400086, 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.431820282, 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89700 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994293+0000", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994293+0000", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994291+0000", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2758999999999997e-05 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994384+0000", 2024-02-22T19:50:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0733e-05 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994395+0000", 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1629999999999999e-06 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994402+0000", 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0221000000000001e-05 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994432+0000", 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3990000000000003e-06 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.994437+0000", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.42091564599999998 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415353+0000", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0352000000000007e-05 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415423+0000", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120363 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.415544+0000", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0095067880000000004 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425050+0000", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00058402200000000001 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425634+0000", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00040937299999999999 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.426044+0000", 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6341999999999999e-05 2024-02-22T19:50:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.426080+0000", 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2688000000000003e-05 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.426113+0000", 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.431820282 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89706 11.7 11:ecf732a3:::benchmark_data_smithi053_199129_object89705:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:09.011767+0000", 2024-02-22T19:50:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.087925829, 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.42375313199999998, 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89706 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011767+0000", 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011767+0000", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011765+0000", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4033999999999998e-05 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011819+0000", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4598e-05 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011833+0000", 2024-02-22T19:50:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2699999999999999e-06 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011840+0000", 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3708999999999996e-05 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011904+0000", 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0411000000000001e-05 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.011915+0000", 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.40486213999999998 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.416777+0000", 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9300000000000002e-05 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.416866+0000", 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018290099999999999 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.417049+0000", 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0085928159999999996 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425642+0000", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015868600000000001 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425800+0000", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096312250000000002 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435432+0000", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6635999999999999e-05 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435478+0000", 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699000000000002e-05 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435520+0000", 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.42375313199999998 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89710 11.7 11:ef1a0aaf:::benchmark_data_smithi053_199129_object89709:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:09.024743+0000", 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.07494923, 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.41096789, 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89710 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024743+0000", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024743+0000", 2024-02-22T19:50:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024742+0000", 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108342 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024850+0000", 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4569999999999999e-05 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024865+0000", 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8770000000000004e-06 2024-02-22T19:50:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024871+0000", 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6606999999999993e-05 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024948+0000", 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5659999999999996e-06 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.024958+0000", 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.39434476699999998 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419302+0000", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000117219 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419420+0000", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000161588 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.419581+0000", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006066302 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425648+0000", 2024-02-22T19:50:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0074310579999999999 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.433079+0000", 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025383910000000001 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435617+0000", 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3392e-05 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435670+0000", 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0976000000000002e-05 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435711+0000", 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.41096789 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89711 11.7 11:ea29fc5a:::benchmark_data_smithi053_199129_object89710:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:09.029242+0000", 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.07045057099999, 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.406596281, 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89711 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029242+0000", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029242+0000", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029241+0000", 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2293999999999998e-05 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029293+0000", 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1401e-05 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029304+0000", 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1730000000000003e-06 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029309+0000", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0756e-05 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029360+0000", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.06e-05 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.029371+0000", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.39190813699999999 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.421279+0000", 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011277699999999999 2024-02-22T19:50:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.421392+0000", 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000164167 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.421556+0000", 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040965970000000004 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.425652+0000", 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0074656660000000001 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.433118+0000", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026427769999999998 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435761+0000", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2537000000000003e-05 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435803+0000", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4956999999999998e-05 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435838+0000", 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.406596281 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89730 11.7 11:e43a9990:::benchmark_data_smithi053_199129_object89729:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e503)", 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:09.170068+0000", 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 104.92962462200001, 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.26589954199999999, 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89730 2024-02-22T19:50:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170068+0000", 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170068+0000", 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170067+0000", 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3101e-05 2024-02-22T19:50:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170110+0000", 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2026e-05 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170122+0000", 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6540000000000004e-06 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170127+0000", 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4489e-05 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170172+0000", 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2216e-05 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.170184+0000", 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.25302921499999997 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.423213+0000", 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111084 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.423324+0000", 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016891200000000001 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4,7", 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.423493+0000", 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043093489999999996 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.427803+0000", 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053650110000000003 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.433168+0000", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027220349999999998 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435890+0000", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3278999999999997e-05 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435933+0000", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4579999999999998e-05 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:09.435968+0000", 2024-02-22T19:50:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.26589954199999999 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:50:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:54.730 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.847316+0000 osd.7 (osd.7) 331 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:54.731 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:52.847373+0000 osd.7 (osd.7) 332 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:54.731 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:53.718579+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e565: 8 total, 8 up, 7 in 2024-02-22T19:50:54.731 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:53.928473+0000 mgr.y (mgr.14535) 831 : cluster [DBG] pgmap v1148: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 512 op/s; 1482/121224 objects degraded (1.223%); 28591/121224 objects misplaced (23.585%); 5.1 MiB/s, 82 objects/s recovering 2024-02-22T19:50:54.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.847316+0000 osd.7 (osd.7) 331 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:54.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:52.847373+0000 osd.7 (osd.7) 332 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:54.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:53.718579+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e565: 8 total, 8 up, 7 in 2024-02-22T19:50:54.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:53.928473+0000 mgr.y (mgr.14535) 831 : cluster [DBG] pgmap v1148: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 512 op/s; 1482/121224 objects degraded (1.223%); 28591/121224 objects misplaced (23.585%); 5.1 MiB/s, 82 objects/s recovering 2024-02-22T19:50:54.855 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:50:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:50:54.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:50:55.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.847316+0000 osd.7 (osd.7) 331 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:50:55.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:52.847373+0000 osd.7 (osd.7) 332 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:50:55.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:53.718579+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e565: 8 total, 8 up, 7 in 2024-02-22T19:50:55.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:53.928473+0000 mgr.y (mgr.14535) 831 : cluster [DBG] pgmap v1148: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.5 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 512 op/s; 1482/121224 objects degraded (1.223%); 28591/121224 objects misplaced (23.585%); 5.1 MiB/s, 82 objects/s recovering 2024-02-22T19:50:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:50:55.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:50:56.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:55.214958+0000 mon.a (mon.0) 1966 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:50:56.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:55.629682+0000 mgr.y (mgr.14535) 832 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:56.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:55.214958+0000 mon.a (mon.0) 1966 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:50:56.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:55.629682+0000 mgr.y (mgr.14535) 832 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:56.075 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:55.214958+0000 mon.a (mon.0) 1966 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:50:56.076 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:55.629682+0000 mgr.y (mgr.14535) 832 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:50:56.076 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:50:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:50:56.160 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:50:56.169 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:56.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:50:56.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:50:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:55.932591+0000 mgr.y (mgr.14535) 833 : cluster [DBG] pgmap v1149: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.3 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 760 op/s; 1610/112179 objects degraded (1.435%); 26365/112179 objects misplaced (23.503%); 7.4 MiB/s, 118 objects/s recovering 2024-02-22T19:50:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:56.375901+0000 mon.a (mon.0) 1967 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:50:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:56.378793+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:50:57.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:56.427103+0000 mon.a (mon.0) 1969 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:57.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:55.932591+0000 mgr.y (mgr.14535) 833 : cluster [DBG] pgmap v1149: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.3 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 760 op/s; 1610/112179 objects degraded (1.435%); 26365/112179 objects misplaced (23.503%); 7.4 MiB/s, 118 objects/s recovering 2024-02-22T19:50:57.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:56.375901+0000 mon.a (mon.0) 1967 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:50:57.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:56.378793+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:50:57.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:56.427103+0000 mon.a (mon.0) 1969 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:57.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:55.932591+0000 mgr.y (mgr.14535) 833 : cluster [DBG] pgmap v1149: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.3 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 760 op/s; 1610/112179 objects degraded (1.435%); 26365/112179 objects misplaced (23.503%); 7.4 MiB/s, 118 objects/s recovering 2024-02-22T19:50:57.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:56.375901+0000 mon.a (mon.0) 1967 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:50:57.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:56.378793+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:50:57.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:56.427103+0000 mon.a (mon.0) 1969 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:50:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:57.663819+0000 mon.a (mon.0) 1970 : cluster [WRN] Health check update: Degraded data redundancy: 1610/112179 objects degraded (1.435%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:57.668266+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e566: 8 total, 8 up, 7 in 2024-02-22T19:50:58.110 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:57.663819+0000 mon.a (mon.0) 1970 : cluster [WRN] Health check update: Degraded data redundancy: 1610/112179 objects degraded (1.435%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:58.110 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:57.668266+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e566: 8 total, 8 up, 7 in 2024-02-22T19:50:58.115 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:57.663819+0000 mon.a (mon.0) 1970 : cluster [WRN] Health check update: Degraded data redundancy: 1610/112179 objects degraded (1.435%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T19:50:58.115 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:57.668266+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e566: 8 total, 8 up, 7 in 2024-02-22T19:50:58.779 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:50:58.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:58.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:50:58.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:50:58.903 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:50:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:59.039 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:50:59.047 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:59.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:50:59.076 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:50:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:50:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.25652.0:153265 11.4 e566/528 11:2a1ace35:::benchmark_data_smithi053_199129_object59332:head v 566'9450, mlcod=0'0)", 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:50:59.068612+0000", 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001454709, 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014840159999999999, 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068612+0000", 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068612+0000", 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068611+0000", 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1596e-05 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068623+0000", 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8206000000000002e-05 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068641+0000", 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0160000000000008e-06 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068649+0000", 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2155000000000003e-05 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068721+0000", 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1314e-05 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.068742+0000", 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013007700000000001 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.25652.0:153274 11.9 e566/546 11:99ae38e6:::benchmark_data_smithi053_199129_object59341:head v 566'9688, mlcod=566'9686)", 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:50:59.069031+0000", 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001036305, 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0011433190000000001, 2024-02-22T19:50:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069031+0000", 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069031+0000", 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:50:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069030+0000", 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8789999999999992e-06 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069039+0000", 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0635999999999999e-05 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069049+0000", 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7939999999999999e-06 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:50:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069055+0000", 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9678999999999995e-05 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069125+0000", 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0364e-05 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.069145+0000", 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114555 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:50:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-22T19:50:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 2 2024-02-22T19:50:59.084 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:50:59.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:57.932991+0000 mgr.y (mgr.14535) 834 : cluster [DBG] pgmap v1151: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.3 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 957 B/s rd, 0 B/s wr, 564 op/s; 1610/112179 objects degraded (1.435%); 26365/112179 objects misplaced (23.503%); 5.4 MiB/s, 86 objects/s recovering 2024-02-22T19:50:59.145 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:57.932991+0000 mgr.y (mgr.14535) 834 : cluster [DBG] pgmap v1151: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.3 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 957 B/s rd, 0 B/s wr, 564 op/s; 1610/112179 objects degraded (1.435%); 26365/112179 objects misplaced (23.503%); 5.4 MiB/s, 86 objects/s recovering 2024-02-22T19:50:59.145 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:50:59.143+0000 7fe2d794e700 -1 mon.a@0(leader).osd e566 definitely_dead 0 2024-02-22T19:50:59.146 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:57.932991+0000 mgr.y (mgr.14535) 834 : cluster [DBG] pgmap v1151: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.3 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 957 B/s rd, 0 B/s wr, 564 op/s; 1610/112179 objects degraded (1.435%); 26365/112179 objects misplaced (23.503%); 5.4 MiB/s, 86 objects/s recovering 2024-02-22T19:50:59.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:50:59.214 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:59.244 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:50:59.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:59.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:50:59.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:50:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:59.404 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:50:59] "GET /metrics HTTP/1.1" 200 176739 "" "Prometheus/2.33.4" 2024-02-22T19:50:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:50:59.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:59.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:50:59.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:50:59.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:50:59.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:50:59.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:50:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:50:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:50:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.5. 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:59.144099+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.53:0/3625870470' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:59.144248+0000 mon.a (mon.0) 1973 : cluster [INF] Client client.admin marked osd.5 out, while it was still marked up 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:59.506096+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.53:0/1877088277' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:59.506554+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:59.144099+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.53:0/3625870470' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:59.144248+0000 mon.a (mon.0) 1973 : cluster [INF] Client client.admin marked osd.5 out, while it was still marked up 2024-02-22T19:51:00.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:59.506096+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.53:0/1877088277' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:00.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:50:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:59.506554+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:00.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:59.144099+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.53:0/3625870470' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["5"]}]: dispatch 2024-02-22T19:51:00.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:59.144248+0000 mon.a (mon.0) 1973 : cluster [INF] Client client.admin marked osd.5 out, while it was still marked up 2024-02-22T19:51:00.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:59.506096+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.53:0/1877088277' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:00.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:50:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:59.506554+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:00.047 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:51:00.068 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:00.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:00.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:00.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:00.202 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:00.223 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:00.226 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:00.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:00.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:00.341 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:51:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:00.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:59.819019+0000 mon.a (mon.0) 1975 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:51:00.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:59.848062+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.53:0/3625870470' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["5"]}]': finished 2024-02-22T19:51:00.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:50:59.848137+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:00.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:59.848190+0000 mon.a (mon.0) 1978 : cluster [DBG] osdmap e567: 8 total, 8 up, 6 in 2024-02-22T19:51:00.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:50:59.933506+0000 mgr.y (mgr.14535) 835 : cluster [DBG] pgmap v1153: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 737 op/s; 1610/107502 objects degraded (1.498%); 25281/107502 objects misplaced (23.517%); 7.0 MiB/s, 111 objects/s recovering 2024-02-22T19:51:00.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.150828+0000 osd.6 (osd.6) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:00.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.150898+0000 osd.6 (osd.6) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:01.215 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:59.819019+0000 mon.a (mon.0) 1975 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:59.848062+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.53:0/3625870470' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["5"]}]': finished 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:50:59.848137+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:59.848190+0000 mon.a (mon.0) 1978 : cluster [DBG] osdmap e567: 8 total, 8 up, 6 in 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:50:59.933506+0000 mgr.y (mgr.14535) 835 : cluster [DBG] pgmap v1153: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 737 op/s; 1610/107502 objects degraded (1.498%); 25281/107502 objects misplaced (23.517%); 7.0 MiB/s, 111 objects/s recovering 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.150828+0000 osd.6 (osd.6) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:01.217 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.150898+0000 osd.6 (osd.6) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:59.819019+0000 mon.a (mon.0) 1975 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:59.848062+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.53:0/3625870470' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["5"]}]': finished 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:50:59.848137+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:59.848190+0000 mon.a (mon.0) 1978 : cluster [DBG] osdmap e567: 8 total, 8 up, 6 in 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:50:59.933506+0000 mgr.y (mgr.14535) 835 : cluster [DBG] pgmap v1153: 128 pgs: 1 active+recovering+undersized+remapped, 5 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 3 active+recovery_wait+degraded, 1 active+recovering+undersized+degraded+remapped, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 107 active+clean; 2.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 737 op/s; 1610/107502 objects degraded (1.498%); 25281/107502 objects misplaced (23.517%); 7.0 MiB/s, 111 objects/s recovering 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.150828+0000 osd.6 (osd.6) 166 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:01.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.150898+0000 osd.6 (osd.6) 167 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:01.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:01.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:01.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:01.355 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:01.562 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:01.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:01.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:01.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:01.695 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:01.846 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.846980+0000 mon.a (mon.0) 1979 : cluster [DBG] osdmap e568: 8 total, 8 up, 6 in 2024-02-22T19:51:01.846 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.848218+0000 osd.5 (osd.5) 211 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,548'33] MIN to 548'33 2024-02-22T19:51:01.846 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.848947+0000 osd.5 (osd.5) 212 : cluster [DBG] 11.7 starting backfill to osd.1 from (487'3617,566'9581] MIN to 566'9582 2024-02-22T19:51:01.846 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.851640+0000 osd.3 (osd.3) 360 : cluster [DBG] 5.3 starting backfill to osd.1 from (0'0,0'0] MAX to 529'21 2024-02-22T19:51:01.847 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:00.851755+0000 osd.3 (osd.3) 361 : cluster [INF] 5.3 continuing backfill to osd.6 from (326'14,529'21] MIN to 529'21 2024-02-22T19:51:01.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.846980+0000 mon.a (mon.0) 1979 : cluster [DBG] osdmap e568: 8 total, 8 up, 6 in 2024-02-22T19:51:01.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.848218+0000 osd.5 (osd.5) 211 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,548'33] MIN to 548'33 2024-02-22T19:51:01.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.848947+0000 osd.5 (osd.5) 212 : cluster [DBG] 11.7 starting backfill to osd.1 from (487'3617,566'9581] MIN to 566'9582 2024-02-22T19:51:01.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.851640+0000 osd.3 (osd.3) 360 : cluster [DBG] 5.3 starting backfill to osd.1 from (0'0,0'0] MAX to 529'21 2024-02-22T19:51:01.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:00.851755+0000 osd.3 (osd.3) 361 : cluster [INF] 5.3 continuing backfill to osd.6 from (326'14,529'21] MIN to 529'21 2024-02-22T19:51:01.983 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:01.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:01.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:02.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:02.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.846980+0000 mon.a (mon.0) 1979 : cluster [DBG] osdmap e568: 8 total, 8 up, 6 in 2024-02-22T19:51:02.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.848218+0000 osd.5 (osd.5) 211 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,548'33] MIN to 548'33 2024-02-22T19:51:02.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.848947+0000 osd.5 (osd.5) 212 : cluster [DBG] 11.7 starting backfill to osd.1 from (487'3617,566'9581] MIN to 566'9582 2024-02-22T19:51:02.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.851640+0000 osd.3 (osd.3) 360 : cluster [DBG] 5.3 starting backfill to osd.1 from (0'0,0'0] MAX to 529'21 2024-02-22T19:51:02.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:00.851755+0000 osd.3 (osd.3) 361 : cluster [INF] 5.3 continuing backfill to osd.6 from (326'14,529'21] MIN to 529'21 2024-02-22T19:51:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:02.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:02.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:02.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:02.439 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:02.665 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:02.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:02.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:02.820 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:03.025 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:51:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:51:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:51:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:51:03.025 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:51:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:03.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:03.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:03.055 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.852231+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e569: 8 total, 8 up, 6 in 2024-02-22T19:51:03.055 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.855526+0000 osd.3 (osd.3) 362 : cluster [INF] 11.a continuing backfill to osd.6 from (500'5329,566'9475] 11:5006b9bf:::benchmark_data_smithi053_199129_object50778:head to 566'9475 2024-02-22T19:51:03.055 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.856367+0000 osd.3 (osd.3) 363 : cluster [DBG] 6.c starting backfill to osd.7 from (0'0,0'0] MAX to 137'2 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.856485+0000 osd.7 (osd.7) 333 : cluster [INF] 11.5 continuing backfill to osd.6 from (491'4499,566'9766] MIN to 566'9766 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.856930+0000 osd.2 (osd.2) 300 : cluster [DBG] 5.6 starting backfill to osd.4 from (0'0,0'0] MAX to 415'30 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.857195+0000 osd.7 (osd.7) 334 : cluster [DBG] 5.4 starting backfill to osd.1 from (0'0,0'0] MAX to 423'13 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.863127+0000 osd.3 (osd.3) 364 : cluster [INF] 11.8 continuing backfill to osd.6 from (500'5296,566'9416] MIN to 566'9416 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:01.934107+0000 mgr.y (mgr.14535) 836 : cluster [DBG] pgmap v1156: 128 pgs: 4 remapped+peering, 19 peering, 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 1 active+recovering, 8 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 88 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 712 op/s; 4/103074 objects degraded (0.004%); 15977/103074 objects misplaced (15.501%); 7.2 MiB/s, 116 objects/s recovering 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:02.665412+0000 mon.a (mon.0) 1981 : cluster [WRN] Health check update: Degraded data redundancy: 1610/107502 objects degraded (1.498%), 5 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-22T19:51:03.056 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:02.691550+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e570: 8 total, 8 up, 6 in 2024-02-22T19:51:03.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:03.176 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.852231+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e569: 8 total, 8 up, 6 in 2024-02-22T19:51:03.176 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.855526+0000 osd.3 (osd.3) 362 : cluster [INF] 11.a continuing backfill to osd.6 from (500'5329,566'9475] 11:5006b9bf:::benchmark_data_smithi053_199129_object50778:head to 566'9475 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.856367+0000 osd.3 (osd.3) 363 : cluster [DBG] 6.c starting backfill to osd.7 from (0'0,0'0] MAX to 137'2 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.856485+0000 osd.7 (osd.7) 333 : cluster [INF] 11.5 continuing backfill to osd.6 from (491'4499,566'9766] MIN to 566'9766 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.856930+0000 osd.2 (osd.2) 300 : cluster [DBG] 5.6 starting backfill to osd.4 from (0'0,0'0] MAX to 415'30 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.857195+0000 osd.7 (osd.7) 334 : cluster [DBG] 5.4 starting backfill to osd.1 from (0'0,0'0] MAX to 423'13 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.863127+0000 osd.3 (osd.3) 364 : cluster [INF] 11.8 continuing backfill to osd.6 from (500'5296,566'9416] MIN to 566'9416 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:01.934107+0000 mgr.y (mgr.14535) 836 : cluster [DBG] pgmap v1156: 128 pgs: 4 remapped+peering, 19 peering, 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 1 active+recovering, 8 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 88 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 712 op/s; 4/103074 objects degraded (0.004%); 15977/103074 objects misplaced (15.501%); 7.2 MiB/s, 116 objects/s recovering 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:02.665412+0000 mon.a (mon.0) 1981 : cluster [WRN] Health check update: Degraded data redundancy: 1610/107502 objects degraded (1.498%), 5 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-22T19:51:03.177 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:02.691550+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e570: 8 total, 8 up, 6 in 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.852231+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e569: 8 total, 8 up, 6 in 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.855526+0000 osd.3 (osd.3) 362 : cluster [INF] 11.a continuing backfill to osd.6 from (500'5329,566'9475] 11:5006b9bf:::benchmark_data_smithi053_199129_object50778:head to 566'9475 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.856367+0000 osd.3 (osd.3) 363 : cluster [DBG] 6.c starting backfill to osd.7 from (0'0,0'0] MAX to 137'2 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.856485+0000 osd.7 (osd.7) 333 : cluster [INF] 11.5 continuing backfill to osd.6 from (491'4499,566'9766] MIN to 566'9766 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.856930+0000 osd.2 (osd.2) 300 : cluster [DBG] 5.6 starting backfill to osd.4 from (0'0,0'0] MAX to 415'30 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.857195+0000 osd.7 (osd.7) 334 : cluster [DBG] 5.4 starting backfill to osd.1 from (0'0,0'0] MAX to 423'13 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.863127+0000 osd.3 (osd.3) 364 : cluster [INF] 11.8 continuing backfill to osd.6 from (500'5296,566'9416] MIN to 566'9416 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:01.934107+0000 mgr.y (mgr.14535) 836 : cluster [DBG] pgmap v1156: 128 pgs: 4 remapped+peering, 19 peering, 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 1 active+recovering, 8 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 88 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 712 op/s; 4/103074 objects degraded (0.004%); 15977/103074 objects misplaced (15.501%); 7.2 MiB/s, 116 objects/s recovering 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:02.665412+0000 mon.a (mon.0) 1981 : cluster [WRN] Health check update: Degraded data redundancy: 1610/107502 objects degraded (1.498%), 5 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-22T19:51:03.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:02.691550+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e570: 8 total, 8 up, 6 in 2024-02-22T19:51:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:03.283 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:03.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:03.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:03.457 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:03.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:03.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:03.578 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:51:03.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:03.613 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:51:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:03.669 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:51:03.692 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:03.699 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:03.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:03.705 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:51:03.731 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:51:03.772 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:51:03.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:03.814 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:51:03.843 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:03.860 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:51:03.880 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:51:04.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:51:04.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:02.849353+0000 mon.a (mon.0) 1983 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-02-22T19:51:04.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:02.849396+0000 mon.a (mon.0) 1984 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:04.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:03.520489+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? 172.21.15.53:0/3595159111' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:04.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:02.849353+0000 mon.a (mon.0) 1983 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-02-22T19:51:04.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:02.849396+0000 mon.a (mon.0) 1984 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:04.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:03.520489+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? 172.21.15.53:0/3595159111' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:04.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:02.849353+0000 mon.a (mon.0) 1983 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-02-22T19:51:04.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:02.849396+0000 mon.a (mon.0) 1984 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:04.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:03.520489+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? 172.21.15.53:0/3595159111' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:04.294 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:04.301 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:04.301 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:04.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:51:04.402 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:04.727 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:51:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a[93504]: debug 2024-02-22T19:51:04.517+0000 7fb44dbeb700 -1 RGWWatcher::handle_error cookie 94129597101696 err (110) Connection timed out 2024-02-22T19:51:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:51:04.810 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:04.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:04.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:04.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:04.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:03.934619+0000 mgr.y (mgr.14535) 837 : cluster [DBG] pgmap v1158: 128 pgs: 4 remapped+peering, 19 peering, 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 1 active+recovering, 8 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 88 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 0 B/s wr, 361 op/s; 4/103074 objects degraded (0.004%); 15977/103074 objects misplaced (15.501%); 4.0 MiB/s, 64 objects/s recovering 2024-02-22T19:51:04.977 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:51:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:51:04.981 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:05.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:03.934619+0000 mgr.y (mgr.14535) 837 : cluster [DBG] pgmap v1158: 128 pgs: 4 remapped+peering, 19 peering, 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 1 active+recovering, 8 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 88 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 0 B/s wr, 361 op/s; 4/103074 objects degraded (0.004%); 15977/103074 objects misplaced (15.501%); 4.0 MiB/s, 64 objects/s recovering 2024-02-22T19:51:05.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:03.934619+0000 mgr.y (mgr.14535) 837 : cluster [DBG] pgmap v1158: 128 pgs: 4 remapped+peering, 19 peering, 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 1 active+recovering, 8 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 88 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 0 B/s wr, 361 op/s; 4/103074 objects degraded (0.004%); 15977/103074 objects misplaced (15.501%); 4.0 MiB/s, 64 objects/s recovering 2024-02-22T19:51:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:05.092 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:05.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:05.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:05.256 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:05.292 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:05.296 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:05.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:05.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:05.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:51:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:51:06.190 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:05.640781+0000 mgr.y (mgr.14535) 838 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:06.193 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:05.640781+0000 mgr.y (mgr.14535) 838 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:06.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:05.640781+0000 mgr.y (mgr.14535) 838 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:06.383 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:06.391 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:06.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:06.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:06.549 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:06.759 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:51:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:06.846 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:06.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:06.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:07.038 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:07.046 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:07.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:05.935623+0000 mgr.y (mgr.14535) 839 : cluster [DBG] pgmap v1159: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 341 B/s rd, 0 B/s wr, 614 op/s; 30/96447 objects degraded (0.031%); 18596/96447 objects misplaced (19.281%); 8.2 MiB/s, 131 objects/s recovering 2024-02-22T19:51:07.148 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:07.215 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:05.935623+0000 mgr.y (mgr.14535) 839 : cluster [DBG] pgmap v1159: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 341 B/s rd, 0 B/s wr, 614 op/s; 30/96447 objects degraded (0.031%); 18596/96447 objects misplaced (19.281%); 8.2 MiB/s, 131 objects/s recovering 2024-02-22T19:51:07.216 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:05.935623+0000 mgr.y (mgr.14535) 839 : cluster [DBG] pgmap v1159: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 341 B/s rd, 0 B/s wr, 614 op/s; 30/96447 objects degraded (0.031%); 18596/96447 objects misplaced (19.281%); 8.2 MiB/s, 131 objects/s recovering 2024-02-22T19:51:07.221 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:07.276 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:07.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:07.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:07.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:07.445 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:07.520 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:51:07.521 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:51:07.956 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:51:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:08.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:08.054 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:08.067 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 4, 6, 1] out_osds: [0, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:51:08.067 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:51:08.067 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-02-22T19:51:08.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 0 2024-02-22T19:51:08.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:08.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:06.890195+0000 mon.a (mon.0) 1985 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs inactive, 8 pgs peering) 2024-02-22T19:51:08.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:07.245954+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? 172.21.15.53:0/1562300150' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:51:08.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:07.675631+0000 mon.a (mon.0) 1987 : cluster [WRN] Health check update: Degraded data redundancy: 30/96447 objects degraded (0.031%), 7 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:06.890195+0000 mon.a (mon.0) 1985 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs inactive, 8 pgs peering) 2024-02-22T19:51:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:07.245954+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? 172.21.15.53:0/1562300150' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:51:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:07.675631+0000 mon.a (mon.0) 1987 : cluster [WRN] Health check update: Degraded data redundancy: 30/96447 objects degraded (0.031%), 7 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:06.890195+0000 mon.a (mon.0) 1985 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs inactive, 8 pgs peering) 2024-02-22T19:51:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:07.245954+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? 172.21.15.53:0/1562300150' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:51:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:07.675631+0000 mon.a (mon.0) 1987 : cluster [WRN] Health check update: Degraded data redundancy: 30/96447 objects degraded (0.031%), 7 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:08.258 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:08.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:08.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:08.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:08.436 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:09.124 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:09.128 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:09.129 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:09.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:51:09.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:07.905490+0000 mon.a (mon.0) 1988 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:07.934728+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? 172.21.15.53:0/1562300150' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:07.934780+0000 mon.a (mon.0) 1990 : cluster [DBG] osdmap e571: 8 total, 8 up, 6 in 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:07.937791+0000 mgr.y (mgr.14535) 840 : cluster [DBG] pgmap v1161: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 336 B/s rd, 0 B/s wr, 363 op/s; 30/96447 objects degraded (0.031%); 18596/96447 objects misplaced (19.281%); 5.4 MiB/s, 86 objects/s recovering 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:07.905490+0000 mon.a (mon.0) 1988 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:07.934728+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? 172.21.15.53:0/1562300150' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:07.934780+0000 mon.a (mon.0) 1990 : cluster [DBG] osdmap e571: 8 total, 8 up, 6 in 2024-02-22T19:51:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:07.937791+0000 mgr.y (mgr.14535) 840 : cluster [DBG] pgmap v1161: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 336 B/s rd, 0 B/s wr, 363 op/s; 30/96447 objects degraded (0.031%); 18596/96447 objects misplaced (19.281%); 5.4 MiB/s, 86 objects/s recovering 2024-02-22T19:51:09.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:07.905490+0000 mon.a (mon.0) 1988 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:09.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:07.934728+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? 172.21.15.53:0/1562300150' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:51:09.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:07.934780+0000 mon.a (mon.0) 1990 : cluster [DBG] osdmap e571: 8 total, 8 up, 6 in 2024-02-22T19:51:09.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:07.937791+0000 mgr.y (mgr.14535) 840 : cluster [DBG] pgmap v1161: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 336 B/s rd, 0 B/s wr, 363 op/s; 30/96447 objects degraded (0.031%); 18596/96447 objects misplaced (19.281%); 5.4 MiB/s, 86 objects/s recovering 2024-02-22T19:51:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:09.351 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:09.351 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:09.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:09.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:51:09.736 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:51:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:09] "GET /metrics HTTP/1.1" 200 176738 "" "Prometheus/2.33.4" 2024-02-22T19:51:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:09.758 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:09.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:09.860 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:09.910 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:09.952 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:09.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:09.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:10.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:10.123 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:10.294 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:10.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:10.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:10.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:10.452 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:10.971 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:09.938580+0000 mgr.y (mgr.14535) 841 : cluster [DBG] pgmap v1162: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 895 B/s rd, 0 B/s wr, 478 op/s; 30/91599 objects degraded (0.033%); 17471/91599 objects misplaced (19.073%); 6.1 MiB/s, 96 objects/s recovering 2024-02-22T19:51:11.218 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:11.224 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:11.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:09.938580+0000 mgr.y (mgr.14535) 841 : cluster [DBG] pgmap v1162: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 895 B/s rd, 0 B/s wr, 478 op/s; 30/91599 objects degraded (0.033%); 17471/91599 objects misplaced (19.073%); 6.1 MiB/s, 96 objects/s recovering 2024-02-22T19:51:11.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:09.938580+0000 mgr.y (mgr.14535) 841 : cluster [DBG] pgmap v1162: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 895 B/s rd, 0 B/s wr, 478 op/s; 30/91599 objects degraded (0.033%); 17471/91599 objects misplaced (19.073%); 6.1 MiB/s, 96 objects/s recovering 2024-02-22T19:51:11.375 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:11.446 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:11.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:11.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:11.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:51:12.035 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:12.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:12.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:12.141 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:12.205 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:12.463 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:51:12.463 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:51:12.536 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:51:12.536 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:51:12.536 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:51:12.539 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:51:12.606 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:51:12.648 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:51:12.698 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:51:12.763 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:51:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:51:12.800 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:12.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:12.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:12.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:13.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:11.939297+0000 mgr.y (mgr.14535) 842 : cluster [DBG] pgmap v1163: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 775 B/s rd, 0 B/s wr, 595 op/s; 30/86562 objects degraded (0.035%); 16345/86562 objects misplaced (18.882%); 7.2 MiB/s, 114 objects/s recovering 2024-02-22T19:51:13.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:12.393869+0000 mon.c (mon.2) 95 : audit [DBG] from='client.? 172.21.15.53:0/1748131505' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:13.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:12.677916+0000 mon.a (mon.0) 1991 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:13.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:12.677952+0000 mon.a (mon.0) 1992 : cluster [WRN] Health check update: Degraded data redundancy: 30/86562 objects degraded (0.035%), 7 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:13.127 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:12.685999+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e572: 8 total, 8 up, 6 in 2024-02-22T19:51:13.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:11.939297+0000 mgr.y (mgr.14535) 842 : cluster [DBG] pgmap v1163: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 775 B/s rd, 0 B/s wr, 595 op/s; 30/86562 objects degraded (0.035%); 16345/86562 objects misplaced (18.882%); 7.2 MiB/s, 114 objects/s recovering 2024-02-22T19:51:13.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:12.393869+0000 mon.c (mon.2) 95 : audit [DBG] from='client.? 172.21.15.53:0/1748131505' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:13.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:12.677916+0000 mon.a (mon.0) 1991 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:13.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:12.677952+0000 mon.a (mon.0) 1992 : cluster [WRN] Health check update: Degraded data redundancy: 30/86562 objects degraded (0.035%), 7 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:13.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:12.685999+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e572: 8 total, 8 up, 6 in 2024-02-22T19:51:13.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:51:13.126+0000 7fe2d794e700 -1 mon.a@0(leader).osd e572 definitely_dead 0 2024-02-22T19:51:13.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:11.939297+0000 mgr.y (mgr.14535) 842 : cluster [DBG] pgmap v1163: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 775 B/s rd, 0 B/s wr, 595 op/s; 30/86562 objects degraded (0.035%); 16345/86562 objects misplaced (18.882%); 7.2 MiB/s, 114 objects/s recovering 2024-02-22T19:51:13.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:12.393869+0000 mon.c (mon.2) 95 : audit [DBG] from='client.? 172.21.15.53:0/1748131505' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:13.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:12.677916+0000 mon.a (mon.0) 1991 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:13.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:12.677952+0000 mon.a (mon.0) 1992 : cluster [WRN] Health check update: Degraded data redundancy: 30/86562 objects degraded (0.035%), 7 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:13.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:12.685999+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e572: 8 total, 8 up, 6 in 2024-02-22T19:51:13.739 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:13.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:51:13.817 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:13.823 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:13.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:13.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:14.033 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:14.041 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:14.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:14.073 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.0. 2024-02-22T19:51:14.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:14.175 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:14.220 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:51:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:13.125710+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.53:0/2624895624' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:51:14.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:13.127151+0000 mon.a (mon.0) 1994 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:51:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:13.125710+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.53:0/2624895624' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:51:14.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:13.127151+0000 mon.a (mon.0) 1994 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:51:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:13.125710+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.53:0/2624895624' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:51:14.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:13.127151+0000 mon.a (mon.0) 1994 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-22T19:51:14.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:51:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:51:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:13.939807+0000 mgr.y (mgr.14535) 843 : cluster [DBG] pgmap v1165: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 412 op/s; 30/86562 objects degraded (0.035%); 16345/86562 objects misplaced (18.882%); 4.1 MiB/s, 66 objects/s recovering 2024-02-22T19:51:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:14.035223+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-22T19:51:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:14.035290+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e573: 8 total, 8 up, 7 in 2024-02-22T19:51:15.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:13.939807+0000 mgr.y (mgr.14535) 843 : cluster [DBG] pgmap v1165: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 412 op/s; 30/86562 objects degraded (0.035%); 16345/86562 objects misplaced (18.882%); 4.1 MiB/s, 66 objects/s recovering 2024-02-22T19:51:15.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:14.035223+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-22T19:51:15.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:14.035290+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e573: 8 total, 8 up, 7 in 2024-02-22T19:51:15.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:13.939807+0000 mgr.y (mgr.14535) 843 : cluster [DBG] pgmap v1165: 128 pgs: 2 active+remapped+backfill_toofull, 2 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 6 active+recovery_wait+degraded, 12 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 101 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 412 op/s; 30/86562 objects degraded (0.035%); 16345/86562 objects misplaced (18.882%); 4.1 MiB/s, 66 objects/s recovering 2024-02-22T19:51:15.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:14.035223+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-22T19:51:15.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:14.035290+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e573: 8 total, 8 up, 7 in 2024-02-22T19:51:15.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:51:15.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:51:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:51:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:16.040 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:16.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:16.077 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.033197+0000 mon.a (mon.0) 1997 : cluster [DBG] osdmap e574: 8 total, 8 up, 7 in 2024-02-22T19:51:16.077 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.039211+0000 osd.2 (osd.2) 301 : cluster [DBG] 11.1 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10094 2024-02-22T19:51:16.077 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.040217+0000 osd.4 (osd.4) 202 : cluster [DBG] 5.5 starting backfill to osd.0 from (0'0,0'0] MAX to 548'38 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.040347+0000 osd.4 (osd.4) 203 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,548'38] MIN to 548'38 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.041896+0000 osd.5 (osd.5) 213 : cluster [DBG] 5.7 starting backfill to osd.0 from (0'0,0'0] MAX to 570'34 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.042001+0000 osd.5 (osd.5) 214 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,570'34] MIN to 570'34 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.042643+0000 osd.5 (osd.5) 215 : cluster [DBG] 11.7 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10086 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.043403+0000 osd.7 (osd.7) 335 : cluster [DBG] 5.4 starting backfill to osd.0 from (0'0,0'0] MAX to 570'15 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.047865+0000 osd.3 (osd.3) 365 : cluster [DBG] 11.9 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10242 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.047896+0000 osd.3 (osd.3) 366 : cluster [DBG] 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 572'9924 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.057964+0000 osd.5 (osd.5) 216 : cluster [INF] 11.7 continuing backfill to osd.1 from (487'3617,572'10086] MIN to 572'10086 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.060539+0000 osd.3 (osd.3) 367 : cluster [DBG] 5.3 starting backfill to osd.0 from (0'0,0'0] MAX to 570'23 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.060668+0000 osd.3 (osd.3) 368 : cluster [INF] 5.3 continuing backfill to osd.6 from (326'14,570'23] MIN to 570'23 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.064617+0000 osd.3 (osd.3) 369 : cluster [INF] 11.8 continuing backfill to osd.6 from (510'5796,572'9924] MIN to 572'9924 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.067439+0000 osd.3 (osd.3) 370 : cluster [INF] 11.9 continuing backfill to osd.6 from (511'6182,572'10242] MIN to 572'10242 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.070924+0000 osd.2 (osd.2) 302 : cluster [INF] 11.1 continuing backfill to osd.6 from (499'4957,572'10094] MIN to 572'10094 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:15.651992+0000 mgr.y (mgr.14535) 844 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:16.078 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:16.034037+0000 mon.a (mon.0) 1998 : cluster [DBG] osdmap e575: 8 total, 8 up, 7 in 2024-02-22T19:51:16.141 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.033197+0000 mon.a (mon.0) 1997 : cluster [DBG] osdmap e574: 8 total, 8 up, 7 in 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.039211+0000 osd.2 (osd.2) 301 : cluster [DBG] 11.1 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10094 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.040217+0000 osd.4 (osd.4) 202 : cluster [DBG] 5.5 starting backfill to osd.0 from (0'0,0'0] MAX to 548'38 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.040347+0000 osd.4 (osd.4) 203 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,548'38] MIN to 548'38 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.041896+0000 osd.5 (osd.5) 213 : cluster [DBG] 5.7 starting backfill to osd.0 from (0'0,0'0] MAX to 570'34 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.042001+0000 osd.5 (osd.5) 214 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,570'34] MIN to 570'34 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.042643+0000 osd.5 (osd.5) 215 : cluster [DBG] 11.7 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10086 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.043403+0000 osd.7 (osd.7) 335 : cluster [DBG] 5.4 starting backfill to osd.0 from (0'0,0'0] MAX to 570'15 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.047865+0000 osd.3 (osd.3) 365 : cluster [DBG] 11.9 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10242 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.047896+0000 osd.3 (osd.3) 366 : cluster [DBG] 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 572'9924 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.057964+0000 osd.5 (osd.5) 216 : cluster [INF] 11.7 continuing backfill to osd.1 from (487'3617,572'10086] MIN to 572'10086 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.060539+0000 osd.3 (osd.3) 367 : cluster [DBG] 5.3 starting backfill to osd.0 from (0'0,0'0] MAX to 570'23 2024-02-22T19:51:16.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.060668+0000 osd.3 (osd.3) 368 : cluster [INF] 5.3 continuing backfill to osd.6 from (326'14,570'23] MIN to 570'23 2024-02-22T19:51:16.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.064617+0000 osd.3 (osd.3) 369 : cluster [INF] 11.8 continuing backfill to osd.6 from (510'5796,572'9924] MIN to 572'9924 2024-02-22T19:51:16.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.067439+0000 osd.3 (osd.3) 370 : cluster [INF] 11.9 continuing backfill to osd.6 from (511'6182,572'10242] MIN to 572'10242 2024-02-22T19:51:16.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.070924+0000 osd.2 (osd.2) 302 : cluster [INF] 11.1 continuing backfill to osd.6 from (499'4957,572'10094] MIN to 572'10094 2024-02-22T19:51:16.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:15.651992+0000 mgr.y (mgr.14535) 844 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:16.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:16.034037+0000 mon.a (mon.0) 1998 : cluster [DBG] osdmap e575: 8 total, 8 up, 7 in 2024-02-22T19:51:16.255 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:16.389 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:16.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.033197+0000 mon.a (mon.0) 1997 : cluster [DBG] osdmap e574: 8 total, 8 up, 7 in 2024-02-22T19:51:16.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.039211+0000 osd.2 (osd.2) 301 : cluster [DBG] 11.1 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10094 2024-02-22T19:51:16.389 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.040217+0000 osd.4 (osd.4) 202 : cluster [DBG] 5.5 starting backfill to osd.0 from (0'0,0'0] MAX to 548'38 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.040347+0000 osd.4 (osd.4) 203 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,548'38] MIN to 548'38 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.041896+0000 osd.5 (osd.5) 213 : cluster [DBG] 5.7 starting backfill to osd.0 from (0'0,0'0] MAX to 570'34 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.042001+0000 osd.5 (osd.5) 214 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,570'34] MIN to 570'34 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.042643+0000 osd.5 (osd.5) 215 : cluster [DBG] 11.7 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10086 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.043403+0000 osd.7 (osd.7) 335 : cluster [DBG] 5.4 starting backfill to osd.0 from (0'0,0'0] MAX to 570'15 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.047865+0000 osd.3 (osd.3) 365 : cluster [DBG] 11.9 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10242 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.047896+0000 osd.3 (osd.3) 366 : cluster [DBG] 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 572'9924 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.057964+0000 osd.5 (osd.5) 216 : cluster [INF] 11.7 continuing backfill to osd.1 from (487'3617,572'10086] MIN to 572'10086 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.060539+0000 osd.3 (osd.3) 367 : cluster [DBG] 5.3 starting backfill to osd.0 from (0'0,0'0] MAX to 570'23 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.060668+0000 osd.3 (osd.3) 368 : cluster [INF] 5.3 continuing backfill to osd.6 from (326'14,570'23] MIN to 570'23 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.064617+0000 osd.3 (osd.3) 369 : cluster [INF] 11.8 continuing backfill to osd.6 from (510'5796,572'9924] MIN to 572'9924 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.067439+0000 osd.3 (osd.3) 370 : cluster [INF] 11.9 continuing backfill to osd.6 from (511'6182,572'10242] MIN to 572'10242 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.070924+0000 osd.2 (osd.2) 302 : cluster [INF] 11.1 continuing backfill to osd.6 from (499'4957,572'10094] MIN to 572'10094 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:15.651992+0000 mgr.y (mgr.14535) 844 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:16.390 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:16.034037+0000 mon.a (mon.0) 1998 : cluster [DBG] osdmap e575: 8 total, 8 up, 7 in 2024-02-22T19:51:16.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:16.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:16.548 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:16.560 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:51:16.560 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-02-22T19:51:16.560 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-02-22T19:51:16.560 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-02-22T19:51:16.560 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:51:16.587 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:16.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:16.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:16.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:16.762 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:17.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:15.940394+0000 mgr.y (mgr.14535) 845 : cluster [DBG] pgmap v1168: 128 pgs: 1 active+remapped+backfill_toofull, 6 remapped+peering, 33 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 7 active+recovery_wait+degraded, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 67 active+clean; 1.6 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 689 op/s; 31/78645 objects degraded (0.039%); 17554/78645 objects misplaced (22.321%); 7.8 MiB/s, 106 objects/s recovering 2024-02-22T19:51:17.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:15.940394+0000 mgr.y (mgr.14535) 845 : cluster [DBG] pgmap v1168: 128 pgs: 1 active+remapped+backfill_toofull, 6 remapped+peering, 33 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 7 active+recovery_wait+degraded, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 67 active+clean; 1.6 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 689 op/s; 31/78645 objects degraded (0.039%); 17554/78645 objects misplaced (22.321%); 7.8 MiB/s, 106 objects/s recovering 2024-02-22T19:51:17.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:16.035510+0000 osd.3 (osd.3) 371 : cluster [DBG] 11.b starting backfill to osd.0 from (0'0,0'0] MAX to 572'10093 2024-02-22T19:51:17.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:16.043874+0000 osd.2 (osd.2) 303 : cluster [DBG] 11.2 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10031 2024-02-22T19:51:17.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:16.044067+0000 osd.1 (osd.1) 303 : cluster [DBG] 11.6 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10340 2024-02-22T19:51:17.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:15.940394+0000 mgr.y (mgr.14535) 845 : cluster [DBG] pgmap v1168: 128 pgs: 1 active+remapped+backfill_toofull, 6 remapped+peering, 33 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 7 active+recovery_wait+degraded, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 67 active+clean; 1.6 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 689 op/s; 31/78645 objects degraded (0.039%); 17554/78645 objects misplaced (22.321%); 7.8 MiB/s, 106 objects/s recovering 2024-02-22T19:51:17.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:16.035510+0000 osd.3 (osd.3) 371 : cluster [DBG] 11.b starting backfill to osd.0 from (0'0,0'0] MAX to 572'10093 2024-02-22T19:51:17.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:16.043874+0000 osd.2 (osd.2) 303 : cluster [DBG] 11.2 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10031 2024-02-22T19:51:17.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:16.044067+0000 osd.1 (osd.1) 303 : cluster [DBG] 11.6 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10340 2024-02-22T19:51:17.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:16.035510+0000 osd.3 (osd.3) 371 : cluster [DBG] 11.b starting backfill to osd.0 from (0'0,0'0] MAX to 572'10093 2024-02-22T19:51:17.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:16.043874+0000 osd.2 (osd.2) 303 : cluster [DBG] 11.2 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10031 2024-02-22T19:51:17.361 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:16.044067+0000 osd.1 (osd.1) 303 : cluster [DBG] 11.6 starting backfill to osd.0 from (0'0,0'0] MAX to 572'10340 2024-02-22T19:51:17.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:51:18.007 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:18.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:18.012 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:18.092 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:17.681416+0000 mon.a (mon.0) 1999 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:18.139 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:17.681456+0000 mon.a (mon.0) 2000 : cluster [WRN] Health check update: Degraded data redundancy: 31/78645 objects degraded (0.039%), 8 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:18.139 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:17.690898+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e576: 8 total, 8 up, 7 in 2024-02-22T19:51:18.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:18.140 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:17.681416+0000 mon.a (mon.0) 1999 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:18.140 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:17.681456+0000 mon.a (mon.0) 2000 : cluster [WRN] Health check update: Degraded data redundancy: 31/78645 objects degraded (0.039%), 8 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:18.140 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:17.690898+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e576: 8 total, 8 up, 7 in 2024-02-22T19:51:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:18.235 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:51:18.419 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:17.681416+0000 mon.a (mon.0) 1999 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:18.419 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:17.681456+0000 mon.a (mon.0) 2000 : cluster [WRN] Health check update: Degraded data redundancy: 31/78645 objects degraded (0.039%), 8 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:18.419 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:17.690898+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e576: 8 total, 8 up, 7 in 2024-02-22T19:51:18.460 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-02-22T19:51:18.580 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:18.595 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:18.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:18.622 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:18.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:18.759 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:19.506 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:19] "GET /metrics HTTP/1.1" 200 176738 "" "Prometheus/2.33.4" 2024-02-22T19:51:19.729 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:17.946165+0000 mgr.y (mgr.14535) 846 : cluster [DBG] pgmap v1171: 128 pgs: 1 active+remapped+backfill_toofull, 6 remapped+peering, 33 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 7 active+recovery_wait+degraded, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 67 active+clean; 1.6 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 614 op/s; 31/78645 objects degraded (0.039%); 17554/78645 objects misplaced (22.321%); 7.3 MiB/s, 89 objects/s recovering 2024-02-22T19:51:19.729 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:18.782218+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.53:0/4251103686' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:19.729 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:18.785866+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:19.730 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:19.734 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:19.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:19.753 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:51:19.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:19.925 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:17.946165+0000 mgr.y (mgr.14535) 846 : cluster [DBG] pgmap v1171: 128 pgs: 1 active+remapped+backfill_toofull, 6 remapped+peering, 33 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 7 active+recovery_wait+degraded, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 67 active+clean; 1.6 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 614 op/s; 31/78645 objects degraded (0.039%); 17554/78645 objects misplaced (22.321%); 7.3 MiB/s, 89 objects/s recovering 2024-02-22T19:51:19.925 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:18.782218+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.53:0/4251103686' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:19.925 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:18.785866+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:19.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:17.946165+0000 mgr.y (mgr.14535) 846 : cluster [DBG] pgmap v1171: 128 pgs: 1 active+remapped+backfill_toofull, 6 remapped+peering, 33 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 7 active+recovery_wait+degraded, 9 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 67 active+clean; 1.6 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 0 B/s wr, 614 op/s; 31/78645 objects degraded (0.039%); 17554/78645 objects misplaced (22.321%); 7.3 MiB/s, 89 objects/s recovering 2024-02-22T19:51:19.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:18.782218+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.53:0/4251103686' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:19.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:18.785866+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:20.096 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:20.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:20.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:20.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:20.256 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:20.402 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:20.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:20.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:20.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:20.756 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:20.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:20.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:20.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:19.706397+0000 mon.a (mon.0) 2003 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:20.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:19.718761+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:19.718825+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e577: 8 total, 8 up, 7 in 2024-02-22T19:51:20.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:19.946697+0000 mgr.y (mgr.14535) 847 : cluster [DBG] pgmap v1173: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_toofull, 2 remapped+peering, 1 peering, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 16 active+recovery_wait+degraded, 14 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 89 active+clean; 1.5 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 40 KiB/s rd, 1.0 KiB/s wr, 606 op/s; 73/74271 objects degraded (0.098%); 21388/74271 objects misplaced (28.797%); 5.4 MiB/s, 110 objects/s recovering 2024-02-22T19:51:20.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:20.936 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89285 11.f 11:fbfc3c80:::benchmark_data_smithi053_199129_object89284:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e501)", 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.355145+0000", 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.58310408899999, 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0371205990000001, 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-02-22T19:51:20.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89285 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355145+0000", 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355145+0000", 2024-02-22T19:51:20.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355143+0000", 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7442999999999997e-05 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355220+0000", 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249e-05 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355231+0000", 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.118e-06 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355236+0000", 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012367500000000001 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355360+0000", 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012633100000000001 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355486+0000", 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015458400000000001 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 1,2,4", 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.355640+0000", 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019591082999999999 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.375232+0000", 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1969999999999999e-06 2024-02-22T19:51:20.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.375239+0000", 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169528960000001 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.392192+0000", 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3763999999999994e-05 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.392265+0000", 2024-02-22T19:51:20.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0371205990000001 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89308 11.f 11:f33cecc0:::benchmark_data_smithi053_199129_object89307:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.413460+0000", 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.52478838600001, 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98961662299999997, 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.948 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89308 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413460+0000", 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413460+0000", 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413459+0000", 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4469999999999999e-05 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413514+0000", 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0409e-05 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413524+0000", 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6e-06 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413529+0000", 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2700999999999997e-05 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413561+0000", 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1609999999999992e-06 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.413571+0000", 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.97905920800000001 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.392630+0000", 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113448 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.392743+0000", 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018254600000000001 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.392926+0000", 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055073159999999999 2024-02-22T19:51:20.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398433+0000", 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012895800000000001 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398562+0000", 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044094490000000002 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.402972+0000", 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1567000000000002e-05 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403033+0000", 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3949999999999998e-05 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403077+0000", 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98961662299999997 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89314 11.f 11:f272b990:::benchmark_data_smithi053_199129_object89313:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.954 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.417457+0000", 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.52079203700001, 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98876401999999997, 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89314 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417457+0000", 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417457+0000", 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417455+0000", 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1526999999999998e-05 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417487+0000", 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4760000000000004e-06 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417496+0000", 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5290000000000002e-06 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417501+0000", 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6709000000000003e-05 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417548+0000", 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8210000000000008e-06 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.417557+0000", 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.97737342100000002 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.394930+0000", 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000129944 2024-02-22T19:51:20.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.395060+0000", 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019523500000000001 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.395255+0000", 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003234715 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398490+0000", 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001239609 2024-02-22T19:51:20.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.399729+0000", 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0063949130000000003 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406124+0000", 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7342000000000001e-05 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406182+0000", 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9158e-05 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406221+0000", 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98876401999999997 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89320 11.f 11:f9e040c8:::benchmark_data_smithi053_199129_object89319:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.429311+0000", 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.50893737800001, 2024-02-22T19:51:20.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.97715437599999999, 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89320 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.429311+0000", 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.429311+0000", 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.429310+0000", 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016404779999999999 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.430950+0000", 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1789e-05 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.430962+0000", 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4609999999999996e-06 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.430968+0000", 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2295000000000001e-05 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.431011+0000", 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.875e-06 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.431019+0000", 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96560378400000002 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.396622+0000", 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011628099999999999 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.396739+0000", 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000183866 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.396923+0000", 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016874279999999999 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398610+0000", 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001124407 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.399734+0000", 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066498210000000002 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406384+0000", 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6088999999999998e-05 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406440+0000", 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5551999999999999e-05 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406466+0000", 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.97715437599999999 2024-02-22T19:51:20.967 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89358 11.f 11:f25b2548:::benchmark_data_smithi053_199129_object89357:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.498082+0000", 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.44016687000001, 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.92155814700000005, 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89358 2024-02-22T19:51:20.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498082+0000", 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498082+0000", 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498081+0000", 2024-02-22T19:51:20.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5216000000000003e-05 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498116+0000", 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1126e-05 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498127+0000", 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0430000000000002e-06 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498134+0000", 2024-02-22T19:51:20.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8403999999999999e-05 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498192+0000", 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0350000000000001e-06 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.498200+0000", 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.90043567499999999 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398636+0000", 2024-02-22T19:51:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6069000000000006e-05 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398722+0000", 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0001132 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.398835+0000", 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042903070000000001 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403126+0000", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029395699999999999 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406065+0000", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013480417999999999 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419546+0000", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2018999999999998e-05 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419598+0000", 2024-02-22T19:51:20.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2506999999999998e-05 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419640+0000", 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.92155814700000005 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89361 11.f 11:fbb04fbf:::benchmark_data_smithi053_199129_object89360:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.512330+0000", 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.42591901099999, 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.90741241800000005, 2024-02-22T19:51:20.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89361 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512330+0000", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512330+0000", 2024-02-22T19:51:20.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512329+0000", 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9773e-05 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512368+0000", 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0343000000000001e-05 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512379+0000", 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.916e-06 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512384+0000", 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5166999999999997e-05 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512419+0000", 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8270000000000004e-06 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.512428+0000", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.88734430600000003 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.399772+0000", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116487 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.399889+0000", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016646200000000001 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.400055+0000", 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031128760000000001 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403168+0000", 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002907775 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406076+0000", 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013608395000000001 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419684+0000", 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2611000000000003e-05 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419717+0000", 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5627e-05 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419742+0000", 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.90741241800000005 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89383 11.f 11:f94e41ff:::benchmark_data_smithi053_199129_object89382:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.548395+0000", 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.389854297, 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87146422499999998, 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89383 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548395+0000", 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548395+0000", 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548393+0000", 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107763 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548501+0000", 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1199999999999999e-05 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548512+0000", 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4030000000000004e-06 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548518+0000", 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5890000000000003e-05 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548564+0000", 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.049e-06 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.548570+0000", 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.85272071100000002 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.401291+0000", 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012400700000000001 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.401415+0000", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017363400000000001 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.401589+0000", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047007259999999997 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406289+0000", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3999999999999998e-06 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406299+0000", 2024-02-22T19:51:20.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013481462 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419780+0000", 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3983999999999998e-05 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419824+0000", 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4582999999999997e-05 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.419859+0000", 2024-02-22T19:51:20.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.87146422499999998 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89394 11.f 11:f97441da:::benchmark_data_smithi053_199129_object89393:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.574486+0000", 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.36376261500001, 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.84811223899999999, 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89394 2024-02-22T19:51:20.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574486+0000", 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574486+0000", 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999976 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574484+0000", 2024-02-22T19:51:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1282000000000001e-05 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574535+0000", 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1177e-05 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574547+0000", 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3289999999999999e-06 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574553+0000", 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0892000000000001e-05 2024-02-22T19:51:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574594+0000", 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5450000000000003e-06 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.574602+0000", 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.828607015 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403209+0000", 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5339999999999994e-05 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403285+0000", 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014505599999999999 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.403430+0000", 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002876698 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.406306+0000", 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001399024 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.407705+0000", 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014794382 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.422500+0000", 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6583999999999999e-05 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.422556+0000", 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2082999999999997e-05 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.422598+0000", 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.84811223899999999 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:89398 11.f 11:f6c911b6:::benchmark_data_smithi053_199129_object89397:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e502)", 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:07.606166+0000", 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 133.33208241, 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.81657298099999998, 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89398 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606166+0000", 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606166+0000", 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606165+0000", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2229000000000001e-05 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606257+0000", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.802e-05 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606275+0000", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6580000000000002e-06 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:20.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606283+0000", 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6864999999999999e-05 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606340+0000", 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0066e-05 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:07.606350+0000", 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.79825077200000005 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:20.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.404600+0000", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5276000000000002e-05 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.404676+0000", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5562000000000004e-05 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,4", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.404771+0000", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002895009 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.407666+0000", 2024-02-22T19:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3565000000000005e-05 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.407740+0000", 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014913881 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.422654+0000", 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7318999999999999e-05 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.422701+0000", 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8349000000000002e-05 2024-02-22T19:51:20.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:08.422739+0000", 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.81657298099999998 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105661 11.5 11:afc6b7e6:::benchmark_data_smithi053_199129_object11728:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.818307+0000", 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.11994171400001, 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1249787570000001, 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105661 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818307+0000", 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818307+0000", 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818306+0000", 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2359999999999998e-06 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818313+0000", 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5539999999999998e-06 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818322+0000", 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.262e-06 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818326+0000", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1390000000000001e-05 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818377+0000", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5780000000000004e-06 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.818384+0000", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95483839999999998 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.773222+0000", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4626e-05 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.773237+0000", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.157375174 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.930612+0000", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013579600000000001 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.930748+0000", 2024-02-22T19:51:21.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017462700000000001 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,5,6", 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.930922+0000", 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010950086 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.941873+0000", 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6651999999999999e-05 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.941929+0000", 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0649000000000007e-05 2024-02-22T19:51:21.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.942010+0000", 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001081172 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.943091+0000", 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5124000000000003e-05 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.943146+0000", 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013978600000000001 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.943286+0000", 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1249787570000001 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105667 11.4 11:20bc671d:::benchmark_data_smithi053_199129_object11734:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.830372+0000", 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.107876855, 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2505681279999998, 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105667 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830372+0000", 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830372+0000", 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830371+0000", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4479999999999999e-06 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830377+0000", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0886999999999999e-05 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830388+0000", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7240000000000003e-06 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830395+0000", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7445999999999999e-05 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830452+0000", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1231e-05 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830464+0000", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94430476399999996 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.774768+0000", 2024-02-22T19:51:21.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8580000000000002e-06 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.774775+0000", 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2746586600000001 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.049434+0000", 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015303199999999999 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.049587+0000", 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018985200000000001 2024-02-22T19:51:21.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3,6", 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.049777+0000", 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078226040000000004 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.057599+0000", 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036091740000000001 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.061208+0000", 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001091666 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.062300+0000", 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015249842 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.077550+0000", 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5764999999999999e-05 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.077606+0000", 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003334399 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.080940+0000", 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2505681279999998 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105668 11.3 11:c932ff60:::benchmark_data_smithi053_199129_object11735:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.830408+0000", 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.10784044, 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2013116620000002, 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105668 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830408+0000", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830408+0000", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830408+0000", 2024-02-22T19:51:21.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8820000000000003e-06 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830412+0000", 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6580000000000002e-06 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830419+0000", 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9779999999999996e-06 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830424+0000", 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9695000000000003e-05 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830494+0000", 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1078e-05 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830505+0000", 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95186910700000005 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.782374+0000", 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0386e-05 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.782385+0000", 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1937821120000001 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.976167+0000", 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000159273 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.976326+0000", 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00018705999999999999 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,5,6", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.976513+0000", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057469549999999998 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.982260+0000", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046833542999999998 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029093+0000", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000117564 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029211+0000", 2024-02-22T19:51:21.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9089e-05 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029300+0000", 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1740000000000002e-05 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029362+0000", 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023581959999999999 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.031720+0000", 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2013116620000002 2024-02-22T19:51:21.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105672 11.4 11:2f5f99e7:::benchmark_data_smithi053_199129_object11739:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.830465+0000", 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.10778409700001, 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.253623358, 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105672 2024-02-22T19:51:21.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830465+0000", 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830465+0000", 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830464+0000", 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2839999999999998e-06 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830468+0000", 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.507e-06 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830473+0000", 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4039999999999997e-06 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830477+0000", 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9949e-05 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830517+0000", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4290000000000001e-06 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830526+0000", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94427733899999999 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.774803+0000", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.774809+0000", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2767767669999999 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.051586+0000", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014416200000000001 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.051730+0000", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017607300000000001 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3,6", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.051906+0000", 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058089159999999999 2024-02-22T19:51:21.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.057715+0000", 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035450870000000002 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.061260+0000", 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013780842999999999 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.075041+0000", 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059468560000000004 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.080988+0000", 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0347999999999999e-05 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.081049+0000", 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003039581 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.084088+0000", 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.253623358 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105675 11.3 11:ccda5dc0:::benchmark_data_smithi053_199129_object11742:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.830929+0000", 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.10732023, 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2028890350000001, 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105675 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830929+0000", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830929+0000", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830928+0000", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.224e-06 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830934+0000", 2024-02-22T19:51:21.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0395e-05 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830944+0000", 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8939999999999998e-06 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830950+0000", 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7376000000000002e-05 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.830997+0000", 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7540000000000001e-06 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.831007+0000", 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.95140137999999996 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.782409+0000", 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9049999999999997e-06 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.782416+0000", 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1959077009999999 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.978324+0000", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000130665 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.978455+0000", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000139946 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,5,6", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.978595+0000", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.050378955000000003 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.028974+0000", 2024-02-22T19:51:21.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000156322 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029130+0000", 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110803 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029241+0000", 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002542295 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.031783+0000", 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9188000000000003e-05 2024-02-22T19:51:21.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.031842+0000", 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019753259999999999 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.033818+0000", 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2028890350000001 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105678 11.4 11:28c992d9:::benchmark_data_smithi053_199129_object11745:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2024-02-22T19:51:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:37.834546+0000", 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.103703013, 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.252429528, 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105678 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834546+0000", 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834546+0000", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.999999 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834545+0000", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6879999999999998e-06 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834549+0000", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2650000000000002e-06 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834559+0000", 2024-02-22T19:51:21.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4780000000000002e-06 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834563+0000", 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9919999999999997e-05 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834603+0000", 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4059999999999999e-06 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:37.834611+0000", 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94023984999999999 2024-02-22T19:51:21.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.774850+0000", 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0680000000000003e-06 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.774859+0000", 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2781641859999999 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.053024+0000", 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012262699999999999 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.053146+0000", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000154217 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,3,6", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.053301+0000", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044233789999999999 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.057724+0000", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035821260000000001 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.061306+0000", 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01377485 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.075081+0000", 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090580290000000004 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.084139+0000", 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5087999999999998e-05 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.084204+0000", 2024-02-22T19:51:21.042 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:19.706397+0000 mon.a (mon.0) 2003 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:19.718761+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:19.718825+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e577: 8 total, 8 up, 7 in 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:19.946697+0000 mgr.y (mgr.14535) 847 : cluster [DBG] pgmap v1173: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_toofull, 2 remapped+peering, 1 peering, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 16 active+recovery_wait+degraded, 14 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 89 active+clean; 1.5 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 40 KiB/s rd, 1.0 KiB/s wr, 606 op/s; 73/74271 objects degraded (0.098%); 21388/74271 objects misplaced (28.797%); 5.4 MiB/s, 110 objects/s recovering 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:19.706397+0000 mon.a (mon.0) 2003 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:19.718761+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:19.718825+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e577: 8 total, 8 up, 7 in 2024-02-22T19:51:21.043 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:19.946697+0000 mgr.y (mgr.14535) 847 : cluster [DBG] pgmap v1173: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_toofull, 2 remapped+peering, 1 peering, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 16 active+recovery_wait+degraded, 14 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 89 active+clean; 1.5 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 40 KiB/s rd, 1.0 KiB/s wr, 606 op/s; 73/74271 objects degraded (0.098%); 21388/74271 objects misplaced (28.797%); 5.4 MiB/s, 110 objects/s recovering 2024-02-22T19:51:21.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027713859999999998 2024-02-22T19:51:21.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.086975+0000", 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.252429528 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105681 11.5 11:adaf6ec3:::benchmark_data_smithi053_199129_object11748:head [delete] snapc 0=[] ondisk+write+known_if_redirected e528)", 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:38.888493+0000", 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.049755523, 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.055014522, 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105681 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888493+0000", 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888493+0000", 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888491+0000", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0180000000000001e-05 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888501+0000", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3009e-05 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888514+0000", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3200000000000002e-06 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888522+0000", 2024-02-22T19:51:21.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5801000000000003e-05 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888578+0000", 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8150000000000006e-06 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.888585+0000", 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0443647199999999 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.932950+0000", 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000119978 2024-02-22T19:51:21.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.933070+0000", 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000147899 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,5,6", 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.933218+0000", 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0086685170000000006 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.941886+0000", 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4918000000000002e-05 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.941971+0000", 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000101444 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.942073+0000", 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001260988 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.943334+0000", 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8458000000000003e-05 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.943382+0000", 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000125582 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.943508+0000", 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.055014522 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:105691 11.3 11:c8004b5d:::benchmark_data_smithi053_199129_object11758:head [delete] snapc 0=[] ondisk+write+known_if_redirected e528)", 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:49:38.911210+0000", 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.027038377, 2024-02-22T19:51:21.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.125241159, 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105691 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911210+0000", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911210+0000", 2024-02-22T19:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911209+0000", 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5610000000000004e-06 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911216+0000", 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1911e-05 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911228+0000", 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0140000000000001e-06 2024-02-22T19:51:21.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911235+0000", 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0525999999999998e-05 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911275+0000", 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2369999999999997e-06 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:38.911282+0000", 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0692009819999999 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.980483+0000", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012191199999999999 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.980604+0000", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015181099999999999 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 2,5,6", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:39.980756+0000", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.048282802999999999 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029039+0000", 2024-02-22T19:51:21.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000120783 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029160+0000", 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111233 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.029271+0000", 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004795779 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.034067+0000", 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8309000000000001e-05 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.034115+0000", 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023365090000000001 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:49:40.036452+0000", 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.125241159 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:153760 11.5 11:a611cb5e:::benchmark_data_smithi053_199129_object59827:head [delete] snapc 0=[] ondisk+write+known_if_redirected e567)", 2024-02-22T19:51:21.056 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:50:59.877095+0000", 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 21.061154243000001, 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.030645764, 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 153760 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877095+0000", 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877095+0000", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999981 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877093+0000", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0450000000000003e-06 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877100+0000", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0769e-05 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877110+0000", 2024-02-22T19:51:21.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9009999999999994e-06 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877118+0000", 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4137999999999999e-05 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877212+0000", 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7380000000000006e-06 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.877222+0000", 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.98593939799999997 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:00.863162+0000", 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9349999999999994e-06 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:00.863170+0000", 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.024654304 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.887824+0000", 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015689400000000001 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.887981+0000", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00050784099999999998 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,5,6", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.888489+0000", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013332596 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.901821+0000", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056037939999999996 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907425+0000", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2120999999999999e-05 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907467+0000", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9834000000000007e-05 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907567+0000", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1801000000000001e-05 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907619+0000", 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012164500000000001 2024-02-22T19:51:21.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907740+0000", 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.030645764 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:153777 11.5 11:a7665d28:::benchmark_data_smithi053_199129_object59844:head [delete] snapc 0=[] ondisk+write+known_if_redirected e567)", 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:50:59.915734+0000", 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 21.022515048999999, 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.992247691, 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 153777 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915734+0000", 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915734+0000", 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999995 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915733+0000", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0680000000000001e-06 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915736+0000", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5539999999999998e-06 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915745+0000", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3490000000000002e-06 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915749+0000", 2024-02-22T19:51:21.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3050999999999998e-05 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915792+0000", 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5979999999999999e-06 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.915800+0000", 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.94738503399999996 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:00.863185+0000", 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4100000000000001e-06 2024-02-22T19:51:21.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:00.863189+0000", 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0313858760000001 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.894575+0000", 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1242000000000005e-05 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.894667+0000", 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012564600000000001 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,5,6", 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.894792+0000", 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096332299999999996 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.904425+0000", 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030758410000000002 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907501+0000", 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3447e-05 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907535+0000", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031340799999999999 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907848+0000", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7197999999999997e-05 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907905+0000", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6211999999999995e-05 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907982+0000", 2024-02-22T19:51:21.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.992247691 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.25652.0:153790 11.5 11:a43c0eb3:::benchmark_data_smithi053_199129_object59857:head [delete] snapc 0=[] ondisk+write+known_if_redirected e567)", 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-02-22T19:50:59.940642+0000", 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 20.997606822000002, 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.967450895, 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.25652", 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.53:0/922170238", 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 153790 2024-02-22T19:51:21.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940642+0000", 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940642+0000", 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4294967295.9999986 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940641+0000", 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6799999999999993e-06 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940648+0000", 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2332e-05 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940661+0000", 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8430000000000004e-06 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940667+0000", 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2515000000000001e-05 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940720+0000", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2350000000000002e-06 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:50:59.940727+0000", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.92247569299999999 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:00.863203+0000", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3699999999999997e-06 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-02-22T19:51:21.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:00.863207+0000", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0336645179999999 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.896872+0000", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012960599999999999 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.897001+0000", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000106449 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 3,5,6", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.897108+0000", 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010072678 2024-02-22T19:51:21.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907181+0000", 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060133799999999998 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907782+0000", 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3376e-05 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.907815+0000", 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019159299999999999 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.908007+0000", 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4802000000000001e-05 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.908032+0000", 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1292999999999997e-05 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-02-22T19:51:01.908093+0000", 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.967450895 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-22T19:51:21.077 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-22T19:51:21.077 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-22T19:51:21.176 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:21.183 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:21.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:21.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:21.361 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:21.430 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:21.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:21.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:21.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:21.546 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:51:21.578 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:51:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:21.606 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:51:21.637 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:51:21.651 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:21.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:21.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:21.687 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-22T19:51:21.690 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:51:21.729 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:51:21.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:21.776 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:51:21.824 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.841 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:51:21.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:21.494756+0000 mon.b (mon.1) 116 : audit [DBG] from='client.? 172.21.15.53:0/382049716' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:21.875 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:22.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:21.494756+0000 mon.b (mon.1) 116 : audit [DBG] from='client.? 172.21.15.53:0/382049716' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:22.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:21.494756+0000 mon.b (mon.1) 116 : audit [DBG] from='client.? 172.21.15.53:0/382049716' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:22.116 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:22.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:22.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:22.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:22.276 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:22.309 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:22.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:22.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:22.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:22.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:22.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:22.645 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:22.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:22.728 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:22.815 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:21.947436+0000 mgr.y (mgr.14535) 848 : cluster [DBG] pgmap v1174: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.4 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 33 KiB/s rd, 852 B/s wr, 507 op/s; 1536/70731 objects degraded (2.172%); 23946/70731 objects misplaced (33.855%); 3.9 MiB/s, 82 objects/s recovering 2024-02-22T19:51:22.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:51:22.942 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:21.947436+0000 mgr.y (mgr.14535) 848 : cluster [DBG] pgmap v1174: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.4 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 33 KiB/s rd, 852 B/s wr, 507 op/s; 1536/70731 objects degraded (2.172%); 23946/70731 objects misplaced (33.855%); 3.9 MiB/s, 82 objects/s recovering 2024-02-22T19:51:22.943 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:22.682967+0000 mon.a (mon.0) 2006 : cluster [WRN] Health check update: Degraded data redundancy: 73/74271 objects degraded (0.098%), 17 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:22.943 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:22.700140+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e578: 8 total, 8 up, 7 in 2024-02-22T19:51:22.943 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:21.947436+0000 mgr.y (mgr.14535) 848 : cluster [DBG] pgmap v1174: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.4 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 33 KiB/s rd, 852 B/s wr, 507 op/s; 1536/70731 objects degraded (2.172%); 23946/70731 objects misplaced (33.855%); 3.9 MiB/s, 82 objects/s recovering 2024-02-22T19:51:22.943 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:22.682967+0000 mon.a (mon.0) 2006 : cluster [WRN] Health check update: Degraded data redundancy: 73/74271 objects degraded (0.098%), 17 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:22.943 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:22.700140+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e578: 8 total, 8 up, 7 in 2024-02-22T19:51:23.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:22.682967+0000 mon.a (mon.0) 2006 : cluster [WRN] Health check update: Degraded data redundancy: 73/74271 objects degraded (0.098%), 17 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:23.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:22.700140+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e578: 8 total, 8 up, 7 in 2024-02-22T19:51:23.461 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 4, 6, 1, 0] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:51:23.462 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:51:23.462 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-02-22T19:51:23.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-22T19:51:24.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:51:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:24.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:24.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:24.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:25.034 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:51:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:51:25.038 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:25.080 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:25.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:25.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:25.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:25.314 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:25.315 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:23.948770+0000 mgr.y (mgr.14535) 849 : cluster [DBG] pgmap v1176: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.4 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 31 KiB/s rd, 818 B/s wr, 487 op/s; 1536/70731 objects degraded (2.172%); 23946/70731 objects misplaced (33.855%); 3.7 MiB/s, 79 objects/s recovering 2024-02-22T19:51:25.315 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:23.948770+0000 mgr.y (mgr.14535) 849 : cluster [DBG] pgmap v1176: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.4 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 31 KiB/s rd, 818 B/s wr, 487 op/s; 1536/70731 objects degraded (2.172%); 23946/70731 objects misplaced (33.855%); 3.7 MiB/s, 79 objects/s recovering 2024-02-22T19:51:25.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:25.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:25.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:23.948770+0000 mgr.y (mgr.14535) 849 : cluster [DBG] pgmap v1176: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.4 GiB data, 9.9 GiB used, 616 GiB / 626 GiB avail; 31 KiB/s rd, 818 B/s wr, 487 op/s; 1536/70731 objects degraded (2.172%); 23946/70731 objects misplaced (33.855%); 3.7 MiB/s, 79 objects/s recovering 2024-02-22T19:51:25.426 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:25.487 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:25.528 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:25.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:25.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:25.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:25.762 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:25.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:25.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:25.952 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:51:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:51:25.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:25.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:25.996 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:26.014 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:26.020 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:26.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:26.127 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:26.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:26.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:26.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:26.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:25.664379+0000 mgr.y (mgr.14535) 850 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:26.283 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:26.324 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:26.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:26.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:26.418 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:25.664379+0000 mgr.y (mgr.14535) 850 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:26.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:25.664379+0000 mgr.y (mgr.14535) 850 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:26.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:26.526 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:27.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:25.949783+0000 mgr.y (mgr.14535) 851 : cluster [DBG] pgmap v1177: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.2 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 25 KiB/s rd, 639 B/s wr, 791 op/s; 1323/60885 objects degraded (2.173%); 20550/60885 objects misplaced (33.752%); 6.6 MiB/s, 121 objects/s recovering 2024-02-22T19:51:27.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:25.949783+0000 mgr.y (mgr.14535) 851 : cluster [DBG] pgmap v1177: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.2 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 25 KiB/s rd, 639 B/s wr, 791 op/s; 1323/60885 objects degraded (2.173%); 20550/60885 objects misplaced (33.752%); 6.6 MiB/s, 121 objects/s recovering 2024-02-22T19:51:27.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:25.949783+0000 mgr.y (mgr.14535) 851 : cluster [DBG] pgmap v1177: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.2 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 25 KiB/s rd, 639 B/s wr, 791 op/s; 1323/60885 objects degraded (2.173%); 20550/60885 objects misplaced (33.752%); 6.6 MiB/s, 121 objects/s recovering 2024-02-22T19:51:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:27.686421+0000 mon.a (mon.0) 2008 : cluster [WRN] Health check update: Degraded data redundancy: 1323/60885 objects degraded (2.173%), 16 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:28.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:27.686421+0000 mon.a (mon.0) 2008 : cluster [WRN] Health check update: Degraded data redundancy: 1323/60885 objects degraded (2.173%), 16 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:28.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:27.686421+0000 mon.a (mon.0) 2008 : cluster [WRN] Health check update: Degraded data redundancy: 1323/60885 objects degraded (2.173%), 16 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:29.687 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:51:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:29] "GET /metrics HTTP/1.1" 200 176758 "" "Prometheus/2.33.4" 2024-02-22T19:51:29.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:27.950357+0000 mgr.y (mgr.14535) 852 : cluster [DBG] pgmap v1178: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.2 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 621 B/s rd, 0 B/s wr, 542 op/s; 1323/60885 objects degraded (2.173%); 20550/60885 objects misplaced (33.752%); 4.9 MiB/s, 79 objects/s recovering 2024-02-22T19:51:29.989 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:27.950357+0000 mgr.y (mgr.14535) 852 : cluster [DBG] pgmap v1178: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.2 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 621 B/s rd, 0 B/s wr, 542 op/s; 1323/60885 objects degraded (2.173%); 20550/60885 objects misplaced (33.752%); 4.9 MiB/s, 79 objects/s recovering 2024-02-22T19:51:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:27.950357+0000 mgr.y (mgr.14535) 852 : cluster [DBG] pgmap v1178: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.2 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 621 B/s rd, 0 B/s wr, 542 op/s; 1323/60885 objects degraded (2.173%); 20550/60885 objects misplaced (33.752%); 4.9 MiB/s, 79 objects/s recovering 2024-02-22T19:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:30.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:30.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:30.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:30.514 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:30.742 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:29.952964+0000 mgr.y (mgr.14535) 853 : cluster [DBG] pgmap v1179: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.1 GiB data, 9.3 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 637 op/s; 1323/55164 objects degraded (2.398%); 18994/55164 objects misplaced (34.432%); 5.4 MiB/s, 89 objects/s recovering 2024-02-22T19:51:31.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:29.952964+0000 mgr.y (mgr.14535) 853 : cluster [DBG] pgmap v1179: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.1 GiB data, 9.3 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 637 op/s; 1323/55164 objects degraded (2.398%); 18994/55164 objects misplaced (34.432%); 5.4 MiB/s, 89 objects/s recovering 2024-02-22T19:51:31.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:29.952964+0000 mgr.y (mgr.14535) 853 : cluster [DBG] pgmap v1179: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active+recovery_wait+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovering+degraded+remapped, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.1 GiB data, 9.3 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 637 op/s; 1323/55164 objects degraded (2.398%); 18994/55164 objects misplaced (34.432%); 5.4 MiB/s, 89 objects/s recovering 2024-02-22T19:51:31.950 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:51:31.950 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:51:32.610 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:32.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:32.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:32.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:32.783 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:32.831 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:32.841 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:32.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:32.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:32.984 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:33.011 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:33.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:33.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:33.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:31.954278+0000 mgr.y (mgr.14535) 854 : cluster [DBG] pgmap v1180: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.0 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 664 op/s; 1112/50799 objects degraded (2.189%); 17146/50799 objects misplaced (33.753%); 5.3 MiB/s, 87 objects/s recovering 2024-02-22T19:51:33.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:32.687967+0000 mon.a (mon.0) 2009 : cluster [WRN] Health check update: Degraded data redundancy: 1112/50799 objects degraded (2.189%), 15 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:33.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:33.167 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:33.190 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:33.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:33.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:31.954278+0000 mgr.y (mgr.14535) 854 : cluster [DBG] pgmap v1180: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.0 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 664 op/s; 1112/50799 objects degraded (2.189%); 17146/50799 objects misplaced (33.753%); 5.3 MiB/s, 87 objects/s recovering 2024-02-22T19:51:33.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:32.687967+0000 mon.a (mon.0) 2009 : cluster [WRN] Health check update: Degraded data redundancy: 1112/50799 objects degraded (2.189%), 15 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:31.954278+0000 mgr.y (mgr.14535) 854 : cluster [DBG] pgmap v1180: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.0 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 664 op/s; 1112/50799 objects degraded (2.189%); 17146/50799 objects misplaced (33.753%); 5.3 MiB/s, 87 objects/s recovering 2024-02-22T19:51:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:32.687967+0000 mon.a (mon.0) 2009 : cluster [WRN] Health check update: Degraded data redundancy: 1112/50799 objects degraded (2.189%), 15 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:33.298 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:33.371 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:33.425 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:33.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:33.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:33.529 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:33.616 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:35.066 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:33.787177+0000 osd.0 (osd.0) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:35.067 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:33.787229+0000 osd.0 (osd.0) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:35.067 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:33.954790+0000 mgr.y (mgr.14535) 855 : cluster [DBG] pgmap v1181: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.0 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 910 B/s rd, 0 B/s wr, 591 op/s; 1112/50799 objects degraded (2.189%); 17146/50799 objects misplaced (33.753%); 4.7 MiB/s, 77 objects/s recovering 2024-02-22T19:51:35.067 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:34.055367+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e579: 8 total, 8 up, 7 in 2024-02-22T19:51:35.067 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:51:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:51:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:33.787177+0000 osd.0 (osd.0) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:33.787229+0000 osd.0 (osd.0) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:33.954790+0000 mgr.y (mgr.14535) 855 : cluster [DBG] pgmap v1181: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.0 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 910 B/s rd, 0 B/s wr, 591 op/s; 1112/50799 objects degraded (2.189%); 17146/50799 objects misplaced (33.753%); 4.7 MiB/s, 77 objects/s recovering 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:34.055367+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e579: 8 total, 8 up, 7 in 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:34.933431+0000 mon.c (mon.2) 97 : audit [DBG] from='client.? 172.21.15.53:0/1899222585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:33.787177+0000 osd.0 (osd.0) 181 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:33.787229+0000 osd.0 (osd.0) 182 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:33.954790+0000 mgr.y (mgr.14535) 855 : cluster [DBG] pgmap v1181: 128 pgs: 1 active+remapped+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 1.0 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 910 B/s rd, 0 B/s wr, 591 op/s; 1112/50799 objects degraded (2.189%); 17146/50799 objects misplaced (33.753%); 4.7 MiB/s, 77 objects/s recovering 2024-02-22T19:51:35.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:34.055367+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e579: 8 total, 8 up, 7 in 2024-02-22T19:51:35.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:34.933431+0000 mon.c (mon.2) 97 : audit [DBG] from='client.? 172.21.15.53:0/1899222585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:51:35.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:34.933431+0000 mon.c (mon.2) 97 : audit [DBG] from='client.? 172.21.15.53:0/1899222585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-22T19:51:35.823 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:51:35.823 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:51:35.823 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 579, 2024-02-22T19:51:35.823 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "b6783888-d1b7-11ee-95bf-87774f69a715", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-02-22T19:22:59.746063+0000", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-02-22T19:51:34.025065+0000", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-02-22T19:28:53.274045+0000", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-02-22T19:51:13.994957+0000", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5806080, 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-02-22T19:51:35.824 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 18, 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 11, 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-02-22T19:51:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "pacific", 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "device_health_metrics", 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:26:54.342689+0000", 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.826 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 11, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 11, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 11, 2024-02-22T19:51:35.827 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 11, 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 11, 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "164", 2024-02-22T19:51:35.828 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "163", 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.830 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_devicehealth": {} 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "iscsi", 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.555543+0000", 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.833 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 3, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 3, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 3, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 3, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 3, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:51:35.834 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "242", 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.835 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.836 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.837 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.838 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".rgw.root", 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:28:59.986190+0000", 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.839 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:51:35.840 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "558", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.841 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.842 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.843 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 4, 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.log", 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:01.494051+0000", 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-02-22T19:51:35.845 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "297", 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.846 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.847 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:51:35.849 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 5, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.control", 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:03.518927+0000", 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.850 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 12, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 12, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 12, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 12, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 12, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "5.c", 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 422, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 421, 2024-02-22T19:51:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 421, 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "392'11" 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "422", 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "422", 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "420", 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 6, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "default.rgw.meta", 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:29:05.596341+0000", 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 22, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 22, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 22, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 22, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 22, 2024-02-22T19:51:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "6.16", 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 548, 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 547, 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 547, 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "548", 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "548", 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "546", 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_bias": 4 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw": {} 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 11, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "unique_pool_4", 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-02-22T19:47:39.311188+0000", 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 3, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 2, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-02-22T19:51:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 16, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 16, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 16, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 16, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 16, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-02-22T19:51:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "474", 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-02-22T19:51:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-02-22T19:51:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-02-22T19:51:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "rados": {} 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "4326a76a-6144-4859-9b40-5ff69edbbcab", 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 8, 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 573, 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 162271081 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/162271081", 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6802/162271081", 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6804/162271081", 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6803/162271081", 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "fc056bcd-1a3b-408c-b9d1-4992091df6c2", 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 574, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:51:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2973403044 2024-02-22T19:51:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6805/2973403044", 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6806/2973403044", 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6808/2973403044", 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6807/2973403044", 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:51:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "61f3f980-82cd-4e8e-9a3f-6d3f4559cc83", 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 18, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 574, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 936980935 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/936980935", 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6810/936980935", 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6812/936980935", 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6811/936980935", 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "3f285320-aa28-4fc1-b1c7-456c9aecd993", 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 26, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 574, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-02-22T19:51:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3878043918 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6813/3878043918", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6814/3878043918", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6816/3878043918", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6815/3878043918", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "773e8080-1028-48fe-8f6e-ed6beca60211", 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 32, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 573, 2024-02-22T19:51:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6800", 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6801", 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6803", 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6802", 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2373040976 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6800/2373040976", 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6801/2373040976", 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6803/2373040976", 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6802/2373040976", 2024-02-22T19:51:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d79dbfb8-8f92-4d65-92ab-4b00aaccdb07", 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 39, 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 573, 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6804", 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6805", 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6807", 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:51:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6806", 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2413553331 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6804/2413553331", 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6805/2413553331", 2024-02-22T19:51:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6807/2413553331", 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6806/2413553331", 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "ebbb9499-df1e-42b7-89bb-df6c5935f39e", 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 45, 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 573, 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6808", 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6809", 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6811", 2024-02-22T19:51:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6810", 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3043274732 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6808/3043274732", 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6809/3043274732", 2024-02-22T19:51:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6811/3043274732", 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6810/3043274732", 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "cde36201-cb77-4967-9402-6fdbe633b40a", 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-02-22T19:51:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 53, 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 573, 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6812", 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6813", 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6815", 2024-02-22T19:51:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.64:6814", 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2605082260 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.64:6812/2605082260", 2024-02-22T19:51:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.64:6813/2605082260", 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.64:6815/2605082260", 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.64:6814/2605082260", 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:51:33.787231+0000", 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:50:52.024203+0000", 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:50:46.817642+0000", 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-02-22T19:51:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:50:34.477799+0000", 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:50:44.756995+0000", 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:50:28.769002+0000", 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-02-22T19:51:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:51:00.150901+0000", 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138314316775423, 2024-02-22T19:51:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-02-22T19:50:52.847375+0000", 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [ 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "1.4", 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: 6 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.1", 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.2", 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: 7 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.e", 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 7, 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [ 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.0", 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: 2 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.3", 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: 7 2024-02-22T19:51:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.4", 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.5", 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: 3 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.7", 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: 7 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "5.9", 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "6.c", 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: 6, 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "6.e", 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.1", 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.2", 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: 7 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.3", 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.4", 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: 3 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.5", 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.6", 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.7", 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: 7 2024-02-22T19:51:35.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.8", 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.9", 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:51:35.916 INFO:teuthology.orchestra.run.smithi053.stdout: 7 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.a", 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.b", 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: 3, 2024-02-22T19:51:35.917 INFO:teuthology.orchestra.run.smithi053.stdout: 1, 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.d", 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: 5 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.e", 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-02-22T19:51:35.918 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: 3 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "11.f", 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: 7, 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": { 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4161439976": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:51:35.919 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2578736337": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/762516580": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3676085495": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/3516945073": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/828421682": "2024-02-23T19:23:54.856898+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/4127851809": "2024-02-23T19:29:23.167786+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/1906929179": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:0/2640420245": "2024-02-23T19:23:32.485117+0000", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "172.21.15.53:6800/2854906057": "2024-02-23T19:23:32.485117+0000" 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-02-22T19:51:35.920 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "ruleset-failure-domain": "osd", 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-02-22T19:51:35.921 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-02-22T19:51:35.922 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-02-22T19:51:35.922 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-02-22T19:51:35.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-02-22T19:51:35.922 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:51:36.035 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:36.047 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:36.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:36.081 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:51:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:51:36.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:35.675114+0000 mgr.y (mgr.14535) 856 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:36.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:36.217 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:36.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:36.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:36.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-22T19:51:36.336 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:35.675114+0000 mgr.y (mgr.14535) 856 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:36.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:35.675114+0000 mgr.y (mgr.14535) 856 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:36.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:36.437 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:36.523 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:36.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:36.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:36.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:36.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:36.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:36.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:36.903 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:37.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:35.955781+0000 mgr.y (mgr.14535) 857 : cluster [DBG] pgmap v1183: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 832 MiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 680 op/s; 894/40494 objects degraded (2.208%); 14034/40494 objects misplaced (34.657%); 2.7 MiB/s, 57 objects/s recovering 2024-02-22T19:51:37.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:35.955781+0000 mgr.y (mgr.14535) 857 : cluster [DBG] pgmap v1183: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 832 MiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 680 op/s; 894/40494 objects degraded (2.208%); 14034/40494 objects misplaced (34.657%); 2.7 MiB/s, 57 objects/s recovering 2024-02-22T19:51:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:35.955781+0000 mgr.y (mgr.14535) 857 : cluster [DBG] pgmap v1183: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 832 MiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 680 op/s; 894/40494 objects degraded (2.208%); 14034/40494 objects misplaced (34.657%); 2.7 MiB/s, 57 objects/s recovering 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:51:37.695 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:51:37.715 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:51:37.931 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:37.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:37.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:38.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:38.149 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:38.149 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:38.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:37.111295+0000 mon.c (mon.2) 98 : audit [DBG] from='client.? 172.21.15.53:0/21947566' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:37.339386+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.53:0/1853798061' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:37.340153+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:37.689118+0000 mon.a (mon.0) 2012 : cluster [WRN] Health check update: Degraded data redundancy: 894/40494 objects degraded (2.208%), 15 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:37.689728+0000 mon.a (mon.0) 2013 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:37.696365+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:51:38.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:37.696427+0000 mon.a (mon.0) 2015 : cluster [DBG] osdmap e580: 8 total, 8 up, 7 in 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:37.111295+0000 mon.c (mon.2) 98 : audit [DBG] from='client.? 172.21.15.53:0/21947566' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:37.339386+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.53:0/1853798061' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:37.340153+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:37.689118+0000 mon.a (mon.0) 2012 : cluster [WRN] Health check update: Degraded data redundancy: 894/40494 objects degraded (2.208%), 15 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:37.689728+0000 mon.a (mon.0) 2013 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:37.696365+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:51:38.257 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:37.696427+0000 mon.a (mon.0) 2015 : cluster [DBG] osdmap e580: 8 total, 8 up, 7 in 2024-02-22T19:51:38.333 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:38.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:37.111295+0000 mon.c (mon.2) 98 : audit [DBG] from='client.? 172.21.15.53:0/21947566' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:38.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:37.339386+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.53:0/1853798061' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:51:38.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:37.340153+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:51:38.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:37.689118+0000 mon.a (mon.0) 2012 : cluster [WRN] Health check update: Degraded data redundancy: 894/40494 objects degraded (2.208%), 15 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:38.605 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:37.689728+0000 mon.a (mon.0) 2013 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:51:38.605 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:37.696365+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:51:38.605 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:37.696427+0000 mon.a (mon.0) 2015 : cluster [DBG] osdmap e580: 8 total, 8 up, 7 in 2024-02-22T19:51:38.901 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:38.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:38.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:39.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:39.091 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:39.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:37.956340+0000 mgr.y (mgr.14535) 858 : cluster [DBG] pgmap v1185: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 832 MiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 611 op/s; 894/40494 objects degraded (2.208%); 14034/40494 objects misplaced (34.657%); 1.6 MiB/s, 42 objects/s recovering 2024-02-22T19:51:39.258 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:37.956340+0000 mgr.y (mgr.14535) 858 : cluster [DBG] pgmap v1185: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 832 MiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 611 op/s; 894/40494 objects degraded (2.208%); 14034/40494 objects misplaced (34.657%); 1.6 MiB/s, 42 objects/s recovering 2024-02-22T19:51:39.293 INFO:tasks.thrashosds.thrasher:j is {'epoch': 579, 'fsid': 'b6783888-d1b7-11ee-95bf-87774f69a715', 'created': '2024-02-22T19:22:59.746063+0000', 'modified': '2024-02-22T19:51:34.025065+0000', 'last_up_change': '2024-02-22T19:28:53.274045+0000', 'last_in_change': '2024-02-22T19:51:13.994957+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-22T19:26:54.342689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '163', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-22T19:28:59.555543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-22T19:28:59.986190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '558', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-22T19:29:01.494051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '297', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-22T19:29:03.518927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 12, 'pg_placement_num': 12, 'pg_placement_num_target': 12, 'pg_num_target': 12, 'pg_num_pending': 12, 'last_pg_merge_meta': {'source_pgid': '5.c', 'ready_epoch': 422, 'last_epoch_started': 421, 'last_epoch_clean': 421, 'source_version': "0'0", 'target_version': "392'11"}, 'last_change': '422', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '422', 'last_force_op_resend_preluminous': '420', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-22T19:29:05.596341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '6.16', 'ready_epoch': 548, 'last_epoch_started': 547, 'last_epoch_clean': 547, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '548', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '548', 'last_force_op_resend_preluminous': '546', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-02-22T19:47:39.311188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '474', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '4326a76a-6144-4859-9b40-5ff69edbbcab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 573, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6801', 'nonce': 162271081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 162271081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 162271081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6803', 'nonce': 162271081}]}, 'public_addr': '172.21.15.53:6801/162271081', 'cluster_addr': '172.21.15.53:6802/162271081', 'heartbeat_back_addr': '172.21.15.53:6804/162271081', 'heartbeat_front_addr': '172.21.15.53:6803/162271081', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'fc056bcd-1a3b-408c-b9d1-4992091df6c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 574, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6805', 'nonce': 2973403044}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2973403044}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2973403044}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6807', 'nonce': 2973403044}]}, 'public_addr': '172.21.15.53:6805/2973403044', 'cluster_addr': '172.21.15.53:6806/2973403044', 'heartbeat_back_addr': '172.21.15.53:6808/2973403044', 'heartbeat_front_addr': '172.21.15.53:6807/2973403044', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61f3f980-82cd-4e8e-9a3f-6d3f4559cc83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 574, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6809', 'nonce': 936980935}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 936980935}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 936980935}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6811', 'nonce': 936980935}]}, 'public_addr': '172.21.15.53:6809/936980935', 'cluster_addr': '172.21.15.53:6810/936980935', 'heartbeat_back_addr': '172.21.15.53:6812/936980935', 'heartbeat_front_addr': '172.21.15.53:6811/936980935', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3f285320-aa28-4fc1-b1c7-456c9aecd993', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 574, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6813', 'nonce': 3878043918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 3878043918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 3878043918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6815', 'nonce': 3878043918}]}, 'public_addr': '172.21.15.53:6813/3878043918', 'cluster_addr': '172.21.15.53:6814/3878043918', 'heartbeat_back_addr': '172.21.15.53:6816/3878043918', 'heartbeat_front_addr': '172.21.15.53:6815/3878043918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '773e8080-1028-48fe-8f6e-ed6beca60211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 573, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2373040976}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6801', 'nonce': 2373040976}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 2373040976}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2373040976}]}, 'public_addr': '172.21.15.64:6800/2373040976', 'cluster_addr': '172.21.15.64:6801/2373040976', 'heartbeat_back_addr': '172.21.15.64:6803/2373040976', 'heartbeat_front_addr': '172.21.15.64:6802/2373040976', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd79dbfb8-8f92-4d65-92ab-4b00aaccdb07', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 573, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6804', 'nonce': 2413553331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6805', 'nonce': 2413553331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 2413553331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2413553331}]}, 'public_addr': '172.21.15.64:6804/2413553331', 'cluster_addr': '172.21.15.64:6805/2413553331', 'heartbeat_back_addr': '172.21.15.64:6807/2413553331', 'heartbeat_front_addr': '172.21.15.64:6806/2413553331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebbb9499-df1e-42b7-89bb-df6c5935f39e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 573, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6808', 'nonce': 3043274732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6809', 'nonce': 3043274732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 3043274732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 3043274732}]}, 'public_addr': '172.21.15.64:6808/3043274732', 'cluster_addr': '172.21.15.64:6809/3043274732', 'heartbeat_back_addr': '172.21.15.64:6811/3043274732', 'heartbeat_front_addr': '172.21.15.64:6810/3043274732', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'cde36201-cb77-4967-9402-6fdbe633b40a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 573, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6812', 'nonce': 2605082260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6813', 'nonce': 2605082260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6815', 'nonce': 2605082260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 2605082260}]}, 'public_addr': '172.21.15.64:6812/2605082260', 'cluster_addr': '172.21.15.64:6813/2605082260', 'heartbeat_back_addr': '172.21.15.64:6815/2605082260', 'heartbeat_front_addr': '172.21.15.64:6814/2605082260', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:51:33.787231+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:50:52.024203+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:50:46.817642+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:50:34.477799+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:50:44.756995+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-22T19:50:28.769002+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:51:00.150901+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-22T19:50:52.847375+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.4', 'osds': [3, 5, 6]}, {'pgid': '2.1', 'osds': [5, 3, 4]}, {'pgid': '11.2', 'osds': [1, 5, 7]}], 'pg_upmap_items': [{'pgid': '3.e', 'mappings': [{'from': 7, 'to': 2}]}], 'pg_temp': [{'pgid': '5.0', 'osds': [3, 4, 2]}, {'pgid': '5.3', 'osds': [3, 5, 7]}, {'pgid': '5.4', 'osds': [7, 2, 5]}, {'pgid': '5.5', 'osds': [4, 7, 3]}, {'pgid': '5.7', 'osds': [5, 4, 7]}, {'pgid': '5.9', 'osds': [7, 4, 1]}, {'pgid': '6.c', 'osds': [3, 6, 5]}, {'pgid': '6.e', 'osds': [4, 2, 5]}, {'pgid': '11.1', 'osds': [2, 3, 4]}, {'pgid': '11.2', 'osds': [2, 3, 7]}, {'pgid': '11.3', 'osds': [7, 5]}, {'pgid': '11.4', 'osds': [7, 2, 3]}, {'pgid': '11.5', 'osds': [7, 3, 5]}, {'pgid': '11.6', 'osds': [1, 2, 4]}, {'pgid': '11.7', 'osds': [5, 4, 7]}, {'pgid': '11.8', 'osds': [3, 2, 5]}, {'pgid': '11.9', 'osds': [3, 1, 7]}, {'pgid': '11.a', 'osds': [3, 1, 5]}, {'pgid': '11.b', 'osds': [3, 1, 4]}, {'pgid': '11.d', 'osds': [2, 4, 5]}, {'pgid': '11.e', 'osds': [4, 7, 3]}, {'pgid': '11.f', 'osds': [7, 2, 4]}], 'primary_temp': [], 'blocklist': {'172.21.15.53:0/4161439976': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/2578736337': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/762516580': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:6800/3676085495': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:6800/3516945073': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/828421682': '2024-02-23T19:23:54.856898+0000', '172.21.15.53:0/4127851809': '2024-02-23T19:29:23.167786+0000', '172.21.15.53:0/1906929179': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:0/2640420245': '2024-02-23T19:23:32.485117+0000', '172.21.15.53:6800/2854906057': '2024-02-23T19:23:32.485117+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-22T19:51:39.293 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 11.2 2024-02-22T19:51:39.294 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd rm-pg-upmap 11.2 2024-02-22T19:51:39.374 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:39.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:39.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:39.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:39.550 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:37.956340+0000 mgr.y (mgr.14535) 858 : cluster [DBG] pgmap v1185: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 832 MiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 611 op/s; 894/40494 objects degraded (2.208%); 14034/40494 objects misplaced (34.657%); 1.6 MiB/s, 42 objects/s recovering 2024-02-22T19:51:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:39.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:39.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:39.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:39.744 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:39.760 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:51:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:39] "GET /metrics HTTP/1.1" 200 176750 "" "Prometheus/2.33.4" 2024-02-22T19:51:39.778 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:39.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:39.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:39.851 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:39.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:40.155 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:40.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:40.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:40.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:40.348 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:40.414 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:40.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:40.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:40.612 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:40.861 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:41.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:51:41.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:39.957580+0000 mgr.y (mgr.14535) 859 : cluster [DBG] pgmap v1186: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 713 MiB data, 8.2 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 668 op/s; 894/34788 objects degraded (2.570%); 12736/34788 objects misplaced (36.610%); 536 KiB/s, 35 objects/s recovering 2024-02-22T19:51:41.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:39.957580+0000 mgr.y (mgr.14535) 859 : cluster [DBG] pgmap v1186: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 713 MiB data, 8.2 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 668 op/s; 894/34788 objects degraded (2.570%); 12736/34788 objects misplaced (36.610%); 536 KiB/s, 35 objects/s recovering 2024-02-22T19:51:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:39.957580+0000 mgr.y (mgr.14535) 859 : cluster [DBG] pgmap v1186: 128 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 2 active+recovering+remapped, 14 active+recovery_wait+degraded, 15 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 92 active+clean; 713 MiB data, 8.2 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 668 op/s; 894/34788 objects degraded (2.570%); 12736/34788 objects misplaced (36.610%); 536 KiB/s, 35 objects/s recovering 2024-02-22T19:51:41.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:51:42.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:41.143138+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e581: 8 total, 8 up, 7 in 2024-02-22T19:51:42.259 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:41.143138+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e581: 8 total, 8 up, 7 in 2024-02-22T19:51:42.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:41.143138+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e581: 8 total, 8 up, 7 in 2024-02-22T19:51:43.506 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:41.960510+0000 mgr.y (mgr.14535) 860 : cluster [DBG] pgmap v1188: 128 pgs: 2 active+remapped+backfill_toofull, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 12 active+recovery_wait+degraded, 1 active+recovering, 14 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 93 active+clean; 623 MiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 857 op/s; 675/30498 objects degraded (2.213%); 11728/30498 objects misplaced (38.455%); 543 KiB/s, 43 objects/s recovering 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:42.141233+0000 mon.a (mon.0) 2017 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:42.167918+0000 mon.a (mon.0) 2018 : cluster [DBG] osdmap e582: 8 total, 8 up, 7 in 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:42.691139+0000 mon.a (mon.0) 2019 : cluster [WRN] Health check update: Degraded data redundancy: 675/30498 objects degraded (2.213%), 13 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:41.960510+0000 mgr.y (mgr.14535) 860 : cluster [DBG] pgmap v1188: 128 pgs: 2 active+remapped+backfill_toofull, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 12 active+recovery_wait+degraded, 1 active+recovering, 14 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 93 active+clean; 623 MiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 857 op/s; 675/30498 objects degraded (2.213%); 11728/30498 objects misplaced (38.455%); 543 KiB/s, 43 objects/s recovering 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:42.141233+0000 mon.a (mon.0) 2017 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:42.167918+0000 mon.a (mon.0) 2018 : cluster [DBG] osdmap e582: 8 total, 8 up, 7 in 2024-02-22T19:51:43.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:42.691139+0000 mon.a (mon.0) 2019 : cluster [WRN] Health check update: Degraded data redundancy: 675/30498 objects degraded (2.213%), 13 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:43.607 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:41.960510+0000 mgr.y (mgr.14535) 860 : cluster [DBG] pgmap v1188: 128 pgs: 2 active+remapped+backfill_toofull, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 12 active+recovery_wait+degraded, 1 active+recovering, 14 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 93 active+clean; 623 MiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 857 op/s; 675/30498 objects degraded (2.213%); 11728/30498 objects misplaced (38.455%); 543 KiB/s, 43 objects/s recovering 2024-02-22T19:51:43.607 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:42.141233+0000 mon.a (mon.0) 2017 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:43.607 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:42.167918+0000 mon.a (mon.0) 2018 : cluster [DBG] osdmap e582: 8 total, 8 up, 7 in 2024-02-22T19:51:43.607 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:42.691139+0000 mon.a (mon.0) 2019 : cluster [WRN] Health check update: Degraded data redundancy: 675/30498 objects degraded (2.213%), 13 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:43.626 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:51:43.626 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:51:43.626 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:51:43.626 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:51:43.626 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:51:44.506 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:43.192552+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e583: 8 total, 8 up, 7 in 2024-02-22T19:51:44.507 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:43.192552+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e583: 8 total, 8 up, 7 in 2024-02-22T19:51:44.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:43.192552+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e583: 8 total, 8 up, 7 in 2024-02-22T19:51:44.909 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:44.917 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:44.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:45.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:45.067 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:51:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:51:45.071 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:45.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:45.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:45.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:45.324 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:45.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:43.961067+0000 mgr.y (mgr.14535) 861 : cluster [DBG] pgmap v1191: 128 pgs: 2 active+remapped+backfill_toofull, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 12 active+recovery_wait+degraded, 1 active+recovering, 14 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 93 active+clean; 623 MiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 555 op/s; 675/30498 objects degraded (2.213%); 11728/30498 objects misplaced (38.455%); 96 KiB/s, 28 objects/s recovering 2024-02-22T19:51:45.758 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:43.961067+0000 mgr.y (mgr.14535) 861 : cluster [DBG] pgmap v1191: 128 pgs: 2 active+remapped+backfill_toofull, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 12 active+recovery_wait+degraded, 1 active+recovering, 14 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 93 active+clean; 623 MiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 555 op/s; 675/30498 objects degraded (2.213%); 11728/30498 objects misplaced (38.455%); 96 KiB/s, 28 objects/s recovering 2024-02-22T19:51:45.759 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:43.961067+0000 mgr.y (mgr.14535) 861 : cluster [DBG] pgmap v1191: 128 pgs: 2 active+remapped+backfill_toofull, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+remapped, 1 active+recovering+remapped, 12 active+recovery_wait+degraded, 1 active+recovering, 14 active+remapped+backfill_wait, 1 active+recovery_wait+degraded+remapped, 93 active+clean; 623 MiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 555 op/s; 675/30498 objects degraded (2.213%); 11728/30498 objects misplaced (38.455%); 96 KiB/s, 28 objects/s recovering 2024-02-22T19:51:46.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:51:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:51:46.132 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:46.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:46.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:46.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-22T19:51:46.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:46.285 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:46.314 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:46.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:46.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:46.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:46.477 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:45.696624+0000 mgr.y (mgr.14535) 862 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:46.478 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:45.696624+0000 mgr.y (mgr.14535) 862 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:46.560 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:46.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:45.696624+0000 mgr.y (mgr.14535) 862 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:46.616 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:46.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:46.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:46.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:46.805 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:46.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:46.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:46.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:46.958 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:47.000 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:47.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:47.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:47.199 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:47.308 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:47.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:47.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:47.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:47.448 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:47.485 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:47.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:47.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:45.961856+0000 mgr.y (mgr.14535) 863 : cluster [DBG] pgmap v1192: 128 pgs: 5 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 7 active+remapped+backfill_wait, 2 active+remapped, 110 active+clean; 463 MiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 666 op/s; 6203/22806 objects misplaced (27.199%); 8.9 MiB/s, 183 objects/s recovering 2024-02-22T19:51:47.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:45.961856+0000 mgr.y (mgr.14535) 863 : cluster [DBG] pgmap v1192: 128 pgs: 5 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 7 active+remapped+backfill_wait, 2 active+remapped, 110 active+clean; 463 MiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 666 op/s; 6203/22806 objects misplaced (27.199%); 8.9 MiB/s, 183 objects/s recovering 2024-02-22T19:51:47.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:47.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:45.961856+0000 mgr.y (mgr.14535) 863 : cluster [DBG] pgmap v1192: 128 pgs: 5 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 7 active+remapped+backfill_wait, 2 active+remapped, 110 active+clean; 463 MiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 666 op/s; 6203/22806 objects misplaced (27.199%); 8.9 MiB/s, 183 objects/s recovering 2024-02-22T19:51:47.624 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:47.940 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:47.989 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:48.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:48.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:48.010 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:48.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:48.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:47.345764+0000 mon.a (mon.0) 2021 : cluster [DBG] osdmap e584: 8 total, 8 up, 7 in 2024-02-22T19:51:48.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:47.692020+0000 mon.a (mon.0) 2022 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:48.361 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:47.345764+0000 mon.a (mon.0) 2021 : cluster [DBG] osdmap e584: 8 total, 8 up, 7 in 2024-02-22T19:51:48.362 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:47.692020+0000 mon.a (mon.0) 2022 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:48.362 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:47.692057+0000 mon.a (mon.0) 2023 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:48.362 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:48.169286+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.53:0/891787856' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]: dispatch 2024-02-22T19:51:48.362 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:48.169688+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]: dispatch 2024-02-22T19:51:48.374 INFO:teuthology.orchestra.run.smithi053.stderr:clear 11.2 pg_upmap mapping 2024-02-22T19:51:48.665 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:47.692057+0000 mon.a (mon.0) 2023 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:48.665 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:48.169286+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.53:0/891787856' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]: dispatch 2024-02-22T19:51:48.666 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:48.169688+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]: dispatch 2024-02-22T19:51:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:47.345764+0000 mon.a (mon.0) 2021 : cluster [DBG] osdmap e584: 8 total, 8 up, 7 in 2024-02-22T19:51:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:47.692020+0000 mon.a (mon.0) 2022 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:47.692057+0000 mon.a (mon.0) 2023 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-22T19:51:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:48.169286+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.53:0/891787856' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]: dispatch 2024-02-22T19:51:48.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:48.169688+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]: dispatch 2024-02-22T19:51:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:49.340 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:49.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:49.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:49.508 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:49.542 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:49.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:49.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:49.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:47.965453+0000 mgr.y (mgr.14535) 864 : cluster [DBG] pgmap v1194: 128 pgs: 5 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 7 active+remapped+backfill_wait, 2 active+remapped, 110 active+clean; 463 MiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 427 op/s; 6203/22806 objects misplaced (27.199%); 8.9 MiB/s, 173 objects/s recovering 2024-02-22T19:51:49.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:48.347044+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]': finished 2024-02-22T19:51:49.557 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:48.347118+0000 mon.a (mon.0) 2026 : cluster [DBG] osdmap e585: 8 total, 8 up, 7 in 2024-02-22T19:51:49.562 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:47.965453+0000 mgr.y (mgr.14535) 864 : cluster [DBG] pgmap v1194: 128 pgs: 5 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 7 active+remapped+backfill_wait, 2 active+remapped, 110 active+clean; 463 MiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 427 op/s; 6203/22806 objects misplaced (27.199%); 8.9 MiB/s, 173 objects/s recovering 2024-02-22T19:51:49.562 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:48.347044+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]': finished 2024-02-22T19:51:49.562 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:48.347118+0000 mon.a (mon.0) 2026 : cluster [DBG] osdmap e585: 8 total, 8 up, 7 in 2024-02-22T19:51:49.562 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:51:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:49] "GET /metrics HTTP/1.1" 200 176750 "" "Prometheus/2.33.4" 2024-02-22T19:51:49.652 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:51:49.701 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:47.965453+0000 mgr.y (mgr.14535) 864 : cluster [DBG] pgmap v1194: 128 pgs: 5 active+remapped+backfilling, 1 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 7 active+remapped+backfill_wait, 2 active+remapped, 110 active+clean; 463 MiB data, 7.5 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 427 op/s; 6203/22806 objects misplaced (27.199%); 8.9 MiB/s, 173 objects/s recovering 2024-02-22T19:51:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:48.347044+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap", "pgid": "11.2"}]': finished 2024-02-22T19:51:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:48.347118+0000 mon.a (mon.0) 2026 : cluster [DBG] osdmap e585: 8 total, 8 up, 7 in 2024-02-22T19:51:50.418 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:51:50.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:50.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:51:50.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:50.569 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:50.600 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:50.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:50.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:50.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:50.740 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:49.363919+0000 mon.a (mon.0) 2027 : cluster [DBG] osdmap e586: 8 total, 8 up, 7 in 2024-02-22T19:51:50.741 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:49.499873+0000 mon.a (mon.0) 2028 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:51:50.743 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:49.363919+0000 mon.a (mon.0) 2027 : cluster [DBG] osdmap e586: 8 total, 8 up, 7 in 2024-02-22T19:51:50.756 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:49.499873+0000 mon.a (mon.0) 2028 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:51:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:49.363919+0000 mon.a (mon.0) 2027 : cluster [DBG] osdmap e586: 8 total, 8 up, 7 in 2024-02-22T19:51:50.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:49.499873+0000 mon.a (mon.0) 2028 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:51:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:50.777 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:50.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:50.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:50.976 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:51.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:49.966147+0000 mgr.y (mgr.14535) 865 : cluster [DBG] pgmap v1197: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active, 1 active+recovering+remapped, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 393 MiB data, 6.7 GiB used, 619 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 614 op/s; 4622/19452 objects misplaced (23.761%); 15 MiB/s, 279 objects/s recovering 2024-02-22T19:51:51.407 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:50.365813+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e587: 8 total, 8 up, 7 in 2024-02-22T19:51:51.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:49.966147+0000 mgr.y (mgr.14535) 865 : cluster [DBG] pgmap v1197: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active, 1 active+recovering+remapped, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 393 MiB data, 6.7 GiB used, 619 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 614 op/s; 4622/19452 objects misplaced (23.761%); 15 MiB/s, 279 objects/s recovering 2024-02-22T19:51:51.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:50.365813+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e587: 8 total, 8 up, 7 in 2024-02-22T19:51:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:49.966147+0000 mgr.y (mgr.14535) 865 : cluster [DBG] pgmap v1197: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active, 1 active+recovering+remapped, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 393 MiB data, 6.7 GiB used, 619 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 614 op/s; 4622/19452 objects misplaced (23.761%); 15 MiB/s, 279 objects/s recovering 2024-02-22T19:51:51.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:50.365813+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e587: 8 total, 8 up, 7 in 2024-02-22T19:51:51.861 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:51.867 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:51.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:51.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:52.000 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:52.028 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:52.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:52.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:52.166 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:52.198 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:52.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:52.201 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:52.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:52.331 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:52.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:52.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:52.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:52.505 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:52.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:52.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:52.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:52.688 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:52.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:52.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:52.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:52.876 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:52.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:52.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:52.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:53.013 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:53.041 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:53.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:53.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:53.231 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:53.300 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:51:53.301 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:51:53.311 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:53.311 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:51:53.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:53.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:53.358 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:51:53.397 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:51:53.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:51:53.437 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:51:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:51:53.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:53.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:51:53.492 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:51:53.541 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:51:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.565 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.565 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:51:53.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:53.718 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:54.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:51.968474+0000 mgr.y (mgr.14535) 866 : cluster [DBG] pgmap v1199: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 347 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 401 op/s; 3585/17256 objects misplaced (20.775%); 9.3 MiB/s, 174 objects/s recovering 2024-02-22T19:51:54.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:52.706437+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e588: 8 total, 8 up, 7 in 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:52.854724+0000 osd.5 (osd.5) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:52.854786+0000 osd.5 (osd.5) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:53.122646+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.53:0/590646325' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:53.294236+0000 osd.3 (osd.3) 372 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:53.294287+0000 osd.3 (osd.3) 373 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:51.968474+0000 mgr.y (mgr.14535) 866 : cluster [DBG] pgmap v1199: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 347 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 401 op/s; 3585/17256 objects misplaced (20.775%); 9.3 MiB/s, 174 objects/s recovering 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:52.706437+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e588: 8 total, 8 up, 7 in 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:52.854724+0000 osd.5 (osd.5) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:52.854786+0000 osd.5 (osd.5) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:53.122646+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.53:0/590646325' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:53.294236+0000 osd.3 (osd.3) 372 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:54.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:53.294287+0000 osd.3 (osd.3) 373 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:51.968474+0000 mgr.y (mgr.14535) 866 : cluster [DBG] pgmap v1199: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 347 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 401 op/s; 3585/17256 objects misplaced (20.775%); 9.3 MiB/s, 174 objects/s recovering 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:52.706437+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e588: 8 total, 8 up, 7 in 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:52.854724+0000 osd.5 (osd.5) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:52.854786+0000 osd.5 (osd.5) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:53.122646+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.53:0/590646325' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:53.294236+0000 osd.3 (osd.3) 372 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:53.294287+0000 osd.3 (osd.3) 373 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:54.791 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:51:55.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:53.740055+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e589: 8 total, 8 up, 7 in 2024-02-22T19:51:55.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:53.974237+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.53:0/200586281' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:55.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:53.975161+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:55.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:54.527705+0000 mon.a (mon.0) 2033 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:51:55.105 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:51:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:51:55.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:53.740055+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e589: 8 total, 8 up, 7 in 2024-02-22T19:51:55.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:53.974237+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.53:0/200586281' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:55.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:53.975161+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:55.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:54.527705+0000 mon.a (mon.0) 2033 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:51:55.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:53.740055+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e589: 8 total, 8 up, 7 in 2024-02-22T19:51:55.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:53.974237+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.53:0/200586281' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:53.975161+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:51:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:54.527705+0000 mon.a (mon.0) 2033 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:51:55.950 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:55.968 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:55.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:56.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:51:56.095 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:53.973744+0000 mgr.y (mgr.14535) 867 : cluster [DBG] pgmap v1202: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 347 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 402 op/s; 3585/17256 objects misplaced (20.775%); 9.4 MiB/s, 175 objects/s recovering 2024-02-22T19:51:56.095 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:54.103201+0000 osd.7 (osd.7) 336 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:54.103254+0000 osd.7 (osd.7) 337 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:54.729512+0000 mon.a (mon.0) 2034 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:54.763705+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:54.763771+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e590: 8 total, 8 up, 7 in 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:55.707212+0000 mgr.y (mgr.14535) 868 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:53.973744+0000 mgr.y (mgr.14535) 867 : cluster [DBG] pgmap v1202: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 347 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 402 op/s; 3585/17256 objects misplaced (20.775%); 9.4 MiB/s, 175 objects/s recovering 2024-02-22T19:51:56.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:54.103201+0000 osd.7 (osd.7) 336 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:56.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:54.103254+0000 osd.7 (osd.7) 337 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:56.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:54.729512+0000 mon.a (mon.0) 2034 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:51:56.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:54.763705+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:56.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:54.763771+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e590: 8 total, 8 up, 7 in 2024-02-22T19:51:56.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:55.707212+0000 mgr.y (mgr.14535) 868 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:56.104 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:53.973744+0000 mgr.y (mgr.14535) 867 : cluster [DBG] pgmap v1202: 128 pgs: 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 1 active+recovering+remapped, 1 active, 6 active+remapped+backfill_wait, 2 active+remapped, 112 active+clean; 347 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 402 op/s; 3585/17256 objects misplaced (20.775%); 9.4 MiB/s, 175 objects/s recovering 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:54.103201+0000 osd.7 (osd.7) 336 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:54.103254+0000 osd.7 (osd.7) 337 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:54.729512+0000 mon.a (mon.0) 2034 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:54.763705+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:54.763771+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e590: 8 total, 8 up, 7 in 2024-02-22T19:51:56.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:55.707212+0000 mgr.y (mgr.14535) 868 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:51:56.117 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:56.142 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:51:56.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:56.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:51:56.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:51:56.349 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:56.873 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 4, 6, 1, 0] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:51:56.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:51:56.874 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [2, 3, 7, 4, 6, 1, 0] 2024-02-22T19:51:56.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd out 4 2024-02-22T19:51:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:55.975113+0000 mgr.y (mgr.14535) 869 : cluster [DBG] pgmap v1204: 128 pgs: 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 5 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 178 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 912 B/s rd, 0 B/s wr, 538 op/s; 1669/9132 objects misplaced (18.276%); 1.9 MiB/s, 62 objects/s recovering 2024-02-22T19:51:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:51:56.436109+0000 mon.a (mon.0) 2037 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:51:57.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:55.975113+0000 mgr.y (mgr.14535) 869 : cluster [DBG] pgmap v1204: 128 pgs: 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 5 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 178 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 912 B/s rd, 0 B/s wr, 538 op/s; 1669/9132 objects misplaced (18.276%); 1.9 MiB/s, 62 objects/s recovering 2024-02-22T19:51:57.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:51:56.436109+0000 mon.a (mon.0) 2037 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:51:57.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:55.975113+0000 mgr.y (mgr.14535) 869 : cluster [DBG] pgmap v1204: 128 pgs: 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 5 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 178 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 912 B/s rd, 0 B/s wr, 538 op/s; 1669/9132 objects misplaced (18.276%); 1.9 MiB/s, 62 objects/s recovering 2024-02-22T19:51:57.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:51:56.436109+0000 mon.a (mon.0) 2037 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:51:57.724 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:51:57.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:57.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:51:57.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:57.852 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:51:57.852 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:51:57.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:56.798645+0000 mon.a (mon.0) 2038 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:57.915 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:51:57.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:57.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:51:58.022 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:51:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:58.235 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:56.798645+0000 mon.a (mon.0) 2038 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:51:58.254 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:58.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:51:58.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:56.798645+0000 mon.a (mon.0) 2038 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:51:58.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:51:58.388 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:58.427 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:51:58.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:58.433 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:51:58.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:51:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:51:58.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:51:58.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:51:58.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-22T19:51:58.857 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:51:58.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:51:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:51:59.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:57.976164+0000 mgr.y (mgr.14535) 870 : cluster [DBG] pgmap v1205: 128 pgs: 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 5 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 178 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 451 op/s; 1669/9132 objects misplaced (18.276%); 1.2 MiB/s, 38 objects/s recovering 2024-02-22T19:51:59.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:57.976164+0000 mgr.y (mgr.14535) 870 : cluster [DBG] pgmap v1205: 128 pgs: 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 5 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 178 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 451 op/s; 1669/9132 objects misplaced (18.276%); 1.2 MiB/s, 38 objects/s recovering 2024-02-22T19:51:59.194 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:57.976164+0000 mgr.y (mgr.14535) 870 : cluster [DBG] pgmap v1205: 128 pgs: 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 5 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 178 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 451 op/s; 1669/9132 objects misplaced (18.276%); 1.2 MiB/s, 38 objects/s recovering 2024-02-22T19:51:59.505 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:51:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:51:59] "GET /metrics HTTP/1.1" 200 176740 "" "Prometheus/2.33.4" 2024-02-22T19:51:59.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set noscrub 2024-02-22T19:51:59.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-22T19:52:00.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:51:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:58.888046+0000 mon.a (mon.0) 2039 : cluster [DBG] osdmap e591: 8 total, 8 up, 7 in 2024-02-22T19:52:00.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:51:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:58.888046+0000 mon.a (mon.0) 2039 : cluster [DBG] osdmap e591: 8 total, 8 up, 7 in 2024-02-22T19:52:00.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:51:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:58.888046+0000 mon.a (mon.0) 2039 : cluster [DBG] osdmap e591: 8 total, 8 up, 7 in 2024-02-22T19:52:00.962 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Removed 93930 objects 2024-02-22T19:52:00.962 INFO:tasks.radosbench.radosbench.0.smithi053.stdout:Clean up completed and total clean up time :164.39 2024-02-22T19:52:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:59.891690+0000 mon.a (mon.0) 2040 : cluster [DBG] osdmap e592: 8 total, 8 up, 7 in 2024-02-22T19:52:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:51:59.976589+0000 mgr.y (mgr.14535) 871 : cluster [DBG] pgmap v1208: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+recovering+remapped, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 3 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 77 MiB data, 5.8 GiB used, 620 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 722 op/s; 1325/4275 objects misplaced (30.994%); 1.3 MiB/s, 50 objects/s recovering 2024-02-22T19:52:01.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:59.891690+0000 mon.a (mon.0) 2040 : cluster [DBG] osdmap e592: 8 total, 8 up, 7 in 2024-02-22T19:52:01.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:51:59.976589+0000 mgr.y (mgr.14535) 871 : cluster [DBG] pgmap v1208: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+recovering+remapped, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 3 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 77 MiB data, 5.8 GiB used, 620 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 722 op/s; 1325/4275 objects misplaced (30.994%); 1.3 MiB/s, 50 objects/s recovering 2024-02-22T19:52:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:59.891690+0000 mon.a (mon.0) 2040 : cluster [DBG] osdmap e592: 8 total, 8 up, 7 in 2024-02-22T19:52:01.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:51:59.976589+0000 mgr.y (mgr.14535) 871 : cluster [DBG] pgmap v1208: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+recovering+remapped, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 3 active+remapped+backfill_wait, 2 active+remapped, 114 active+clean; 77 MiB data, 5.8 GiB used, 620 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 722 op/s; 1325/4275 objects misplaced (30.994%); 1.3 MiB/s, 50 objects/s recovering 2024-02-22T19:52:01.461 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:52:01.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:01.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:52:01.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:52:01.595 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:02.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:00.896346+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e593: 8 total, 8 up, 7 in 2024-02-22T19:52:02.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:00.896346+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e593: 8 total, 8 up, 7 in 2024-02-22T19:52:02.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:00.896346+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e593: 8 total, 8 up, 7 in 2024-02-22T19:52:02.555 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-02-22T19:52:02.555 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-02-22T19:52:02.575 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:02.584 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:02.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:52:02.345+0000 7fe2d794e700 -1 mon.a@0(leader).osd e594 definitely_dead 0 2024-02-22T19:52:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.4. 2024-02-22T19:52:03.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:01.900417+0000 mon.a (mon.0) 2042 : cluster [DBG] osdmap e594: 8 total, 8 up, 7 in 2024-02-22T19:52:03.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:01.976960+0000 mgr.y (mgr.14535) 872 : cluster [DBG] pgmap v1211: 128 pgs: 2 peering, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+remapped, 115 active+clean; 6.2 MiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 689 op/s; 16/876 objects misplaced (1.826%); 1.5 MiB/s, 53 objects/s recovering 2024-02-22T19:52:03.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:02.346926+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? 172.21.15.53:0/1022609874' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:52:03.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:02.347063+0000 mon.a (mon.0) 2044 : cluster [INF] Client client.admin marked osd.4 out, while it was still marked up 2024-02-22T19:52:03.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:01.900417+0000 mon.a (mon.0) 2042 : cluster [DBG] osdmap e594: 8 total, 8 up, 7 in 2024-02-22T19:52:03.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:01.976960+0000 mgr.y (mgr.14535) 872 : cluster [DBG] pgmap v1211: 128 pgs: 2 peering, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+remapped, 115 active+clean; 6.2 MiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 689 op/s; 16/876 objects misplaced (1.826%); 1.5 MiB/s, 53 objects/s recovering 2024-02-22T19:52:03.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:02.346926+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? 172.21.15.53:0/1022609874' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:52:03.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:02.347063+0000 mon.a (mon.0) 2044 : cluster [INF] Client client.admin marked osd.4 out, while it was still marked up 2024-02-22T19:52:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:52:03.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:01.900417+0000 mon.a (mon.0) 2042 : cluster [DBG] osdmap e594: 8 total, 8 up, 7 in 2024-02-22T19:52:03.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:01.976960+0000 mgr.y (mgr.14535) 872 : cluster [DBG] pgmap v1211: 128 pgs: 2 peering, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+remapped, 115 active+clean; 6.2 MiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 689 op/s; 16/876 objects misplaced (1.826%); 1.5 MiB/s, 53 objects/s recovering 2024-02-22T19:52:03.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:02.346926+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? 172.21.15.53:0/1022609874' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-02-22T19:52:03.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:02.347063+0000 mon.a (mon.0) 2044 : cluster [INF] Client client.admin marked osd.4 out, while it was still marked up 2024-02-22T19:52:03.350 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:03.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:52:03.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:52:03.480 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:03.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:02.588396+0000 osd.4 (osd.4) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:03.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:02.588437+0000 osd.4 (osd.4) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:03.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:02.897929+0000 mon.a (mon.0) 2045 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:03.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:02.906833+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? 172.21.15.53:0/1022609874' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-02-22T19:52:03.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:02.906891+0000 mon.a (mon.0) 2047 : cluster [DBG] osdmap e595: 8 total, 8 up, 6 in 2024-02-22T19:52:03.936 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:52:03.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:03.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:52:04.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7 2024-02-22T19:52:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:02.588396+0000 osd.4 (osd.4) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:02.588437+0000 osd.4 (osd.4) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:02.897929+0000 mon.a (mon.0) 2045 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:02.906833+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? 172.21.15.53:0/1022609874' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-02-22T19:52:04.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:02.906891+0000 mon.a (mon.0) 2047 : cluster [DBG] osdmap e595: 8 total, 8 up, 6 in 2024-02-22T19:52:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:04.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:02.588396+0000 osd.4 (osd.4) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:04.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:02.588437+0000 osd.4 (osd.4) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:04.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:02.897929+0000 mon.a (mon.0) 2045 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:04.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:02.906833+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? 172.21.15.53:0/1022609874' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-02-22T19:52:04.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:02.906891+0000 mon.a (mon.0) 2047 : cluster [DBG] osdmap e595: 8 total, 8 up, 6 in 2024-02-22T19:52:04.461 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:52:04.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:04.466 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7' 2024-02-22T19:52:04.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:52:04.597 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:04.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:52:04.929 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:52:04.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:04.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:52:05.042 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:52:05.081 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:03.908233+0000 mon.a (mon.0) 2048 : cluster [DBG] osdmap e596: 8 total, 8 up, 6 in 2024-02-22T19:52:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:03.911188+0000 osd.4 (osd.4) 206 : cluster [INF] 5.5 continuing backfill to osd.0 from (387'17,574'40] MIN to 574'40 2024-02-22T19:52:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:03.911219+0000 osd.4 (osd.4) 207 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,574'40] MIN to 574'40 2024-02-22T19:52:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:03.911276+0000 osd.4 (osd.4) 208 : cluster [DBG] 5.5 starting backfill to osd.6 from (387'17,548'38] MIN to 574'40 2024-02-22T19:52:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:03.977370+0000 mgr.y (mgr.14535) 873 : cluster [DBG] pgmap v1214: 128 pgs: 2 peering, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+remapped, 115 active+clean; 6.2 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 283 op/s; 16/876 objects misplaced (1.826%); 1.3 MiB/s, 35 objects/s recovering 2024-02-22T19:52:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.909328+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e597: 8 total, 8 up, 6 in 2024-02-22T19:52:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:03.908233+0000 mon.a (mon.0) 2048 : cluster [DBG] osdmap e596: 8 total, 8 up, 6 in 2024-02-22T19:52:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:03.911188+0000 osd.4 (osd.4) 206 : cluster [INF] 5.5 continuing backfill to osd.0 from (387'17,574'40] MIN to 574'40 2024-02-22T19:52:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:03.911219+0000 osd.4 (osd.4) 207 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,574'40] MIN to 574'40 2024-02-22T19:52:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:03.911276+0000 osd.4 (osd.4) 208 : cluster [DBG] 5.5 starting backfill to osd.6 from (387'17,548'38] MIN to 574'40 2024-02-22T19:52:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:03.977370+0000 mgr.y (mgr.14535) 873 : cluster [DBG] pgmap v1214: 128 pgs: 2 peering, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+remapped, 115 active+clean; 6.2 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 283 op/s; 16/876 objects misplaced (1.826%); 1.3 MiB/s, 35 objects/s recovering 2024-02-22T19:52:05.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.909328+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e597: 8 total, 8 up, 6 in 2024-02-22T19:52:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:03.908233+0000 mon.a (mon.0) 2048 : cluster [DBG] osdmap e596: 8 total, 8 up, 6 in 2024-02-22T19:52:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:03.911188+0000 osd.4 (osd.4) 206 : cluster [INF] 5.5 continuing backfill to osd.0 from (387'17,574'40] MIN to 574'40 2024-02-22T19:52:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:03.911219+0000 osd.4 (osd.4) 207 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,574'40] MIN to 574'40 2024-02-22T19:52:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:03.911276+0000 osd.4 (osd.4) 208 : cluster [DBG] 5.5 starting backfill to osd.6 from (387'17,548'38] MIN to 574'40 2024-02-22T19:52:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:03.977370+0000 mgr.y (mgr.14535) 873 : cluster [DBG] pgmap v1214: 128 pgs: 2 peering, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+remapped, 115 active+clean; 6.2 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 283 op/s; 16/876 objects misplaced (1.826%); 1.3 MiB/s, 35 objects/s recovering 2024-02-22T19:52:05.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.909328+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e597: 8 total, 8 up, 6 in 2024-02-22T19:52:05.731 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:52:05.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:05.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:52:05.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:52:05.862 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:06.082 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.909409+0000 osd.7 (osd.7) 338 : cluster [DBG] 5.9 starting backfill to osd.1 from (0'0,0'0] MAX to 583'27 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.909620+0000 osd.1 (osd.1) 304 : cluster [DBG] 6.e starting backfill to osd.7 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.910441+0000 osd.1 (osd.1) 305 : cluster [DBG] 11.e starting backfill to osd.0 from (0'0,0'0] MAX to 592'11574 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.910806+0000 osd.3 (osd.3) 374 : cluster [DBG] 5.0 starting backfill to osd.0 from (0'0,0'0] MAX to 582'22 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.911568+0000 osd.5 (osd.5) 219 : cluster [INF] 5.7 continuing backfill to osd.0 from (382'23,574'35] MIN to 574'35 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.911599+0000 osd.5 (osd.5) 220 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,574'35] MIN to 574'35 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.914204+0000 osd.1 (osd.1) 306 : cluster [DBG] 11.7 starting backfill to osd.7 from (487'3617,585'11421] MIN to 592'11794 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.914496+0000 osd.1 (osd.1) 307 : cluster [DBG] 11.c starting backfill to osd.7 from (0'0,0'0] MAX to 592'11836 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.917640+0000 osd.2 (osd.2) 304 : cluster [DBG] 5.6 starting backfill to osd.1 from (0'0,0'0] MAX to 575'33 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:04.917664+0000 osd.2 (osd.2) 305 : cluster [DBG] 11.d starting backfill to osd.1 from (0'0,492'4761] MIN to 593'11718 2024-02-22T19:52:06.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:05.065701+0000 osd.2 (osd.2) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:06.084 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:05.065761+0000 osd.2 (osd.2) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:06.084 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:05.717158+0000 mgr.y (mgr.14535) 874 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.909409+0000 osd.7 (osd.7) 338 : cluster [DBG] 5.9 starting backfill to osd.1 from (0'0,0'0] MAX to 583'27 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.909620+0000 osd.1 (osd.1) 304 : cluster [DBG] 6.e starting backfill to osd.7 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.910441+0000 osd.1 (osd.1) 305 : cluster [DBG] 11.e starting backfill to osd.0 from (0'0,0'0] MAX to 592'11574 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.910806+0000 osd.3 (osd.3) 374 : cluster [DBG] 5.0 starting backfill to osd.0 from (0'0,0'0] MAX to 582'22 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.911568+0000 osd.5 (osd.5) 219 : cluster [INF] 5.7 continuing backfill to osd.0 from (382'23,574'35] MIN to 574'35 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.911599+0000 osd.5 (osd.5) 220 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,574'35] MIN to 574'35 2024-02-22T19:52:06.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.914204+0000 osd.1 (osd.1) 306 : cluster [DBG] 11.7 starting backfill to osd.7 from (487'3617,585'11421] MIN to 592'11794 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.914496+0000 osd.1 (osd.1) 307 : cluster [DBG] 11.c starting backfill to osd.7 from (0'0,0'0] MAX to 592'11836 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.917640+0000 osd.2 (osd.2) 304 : cluster [DBG] 5.6 starting backfill to osd.1 from (0'0,0'0] MAX to 575'33 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:04.917664+0000 osd.2 (osd.2) 305 : cluster [DBG] 11.d starting backfill to osd.1 from (0'0,492'4761] MIN to 593'11718 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:05.065701+0000 osd.2 (osd.2) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:05.065761+0000 osd.2 (osd.2) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:05.717158+0000 mgr.y (mgr.14535) 874 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.909409+0000 osd.7 (osd.7) 338 : cluster [DBG] 5.9 starting backfill to osd.1 from (0'0,0'0] MAX to 583'27 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.909620+0000 osd.1 (osd.1) 304 : cluster [DBG] 6.e starting backfill to osd.7 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.910441+0000 osd.1 (osd.1) 305 : cluster [DBG] 11.e starting backfill to osd.0 from (0'0,0'0] MAX to 592'11574 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.910806+0000 osd.3 (osd.3) 374 : cluster [DBG] 5.0 starting backfill to osd.0 from (0'0,0'0] MAX to 582'22 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.911568+0000 osd.5 (osd.5) 219 : cluster [INF] 5.7 continuing backfill to osd.0 from (382'23,574'35] MIN to 574'35 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.911599+0000 osd.5 (osd.5) 220 : cluster [INF] 5.7 continuing backfill to osd.1 from (382'23,574'35] MIN to 574'35 2024-02-22T19:52:06.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.914204+0000 osd.1 (osd.1) 306 : cluster [DBG] 11.7 starting backfill to osd.7 from (487'3617,585'11421] MIN to 592'11794 2024-02-22T19:52:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.914496+0000 osd.1 (osd.1) 307 : cluster [DBG] 11.c starting backfill to osd.7 from (0'0,0'0] MAX to 592'11836 2024-02-22T19:52:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.917640+0000 osd.2 (osd.2) 304 : cluster [DBG] 5.6 starting backfill to osd.1 from (0'0,0'0] MAX to 575'33 2024-02-22T19:52:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:04.917664+0000 osd.2 (osd.2) 305 : cluster [DBG] 11.d starting backfill to osd.1 from (0'0,492'4761] MIN to 593'11718 2024-02-22T19:52:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:05.065701+0000 osd.2 (osd.2) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:05.065761+0000 osd.2 (osd.2) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:06.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:05.717158+0000 mgr.y (mgr.14535) 874 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:06.988 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:05.942544+0000 mon.a (mon.0) 2050 : cluster [DBG] osdmap e598: 8 total, 8 up, 6 in 2024-02-22T19:52:06.988 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:05.977742+0000 mgr.y (mgr.14535) 875 : cluster [DBG] pgmap v1217: 128 pgs: 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 6 remapped+peering, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 6 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+recovering, 108 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 25 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 48 KiB/s, 31 objects/s recovering 2024-02-22T19:52:06.988 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:06.555537+0000 mon.a (mon.0) 2051 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:06.988 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:05.942544+0000 mon.a (mon.0) 2050 : cluster [DBG] osdmap e598: 8 total, 8 up, 6 in 2024-02-22T19:52:06.988 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:05.977742+0000 mgr.y (mgr.14535) 875 : cluster [DBG] pgmap v1217: 128 pgs: 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 6 remapped+peering, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 6 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+recovering, 108 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 25 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 48 KiB/s, 31 objects/s recovering 2024-02-22T19:52:06.988 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:06.555537+0000 mon.a (mon.0) 2051 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:07.016 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:52:07.022 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:07.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:52:07.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:52:07.154 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:52:07.258 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:07.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:52:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:05.942544+0000 mon.a (mon.0) 2050 : cluster [DBG] osdmap e598: 8 total, 8 up, 6 in 2024-02-22T19:52:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:05.977742+0000 mgr.y (mgr.14535) 875 : cluster [DBG] pgmap v1217: 128 pgs: 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 6 remapped+peering, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 6 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+recovering, 108 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 25 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 48 KiB/s, 31 objects/s recovering 2024-02-22T19:52:07.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:06.555537+0000 mon.a (mon.0) 2051 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:07.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:52:07.387 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:07.414 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:52:07.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:07.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:52:07.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:52:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:52:07.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:07.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:52:07.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:52:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:07.722 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:52:07.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:07.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:52:07.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6 2024-02-22T19:52:07.854 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:07.873 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:52:07.876 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:07.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6' 2024-02-22T19:52:07.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:52:08.004 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:06.939928+0000 mon.a (mon.0) 2052 : cluster [WRN] Health check update: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:52:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:06.949626+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e599: 8 total, 8 up, 6 in 2024-02-22T19:52:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:07.709637+0000 mon.a (mon.0) 2054 : cluster [DBG] osdmap e600: 8 total, 8 up, 6 in 2024-02-22T19:52:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:06.939928+0000 mon.a (mon.0) 2052 : cluster [WRN] Health check update: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:52:08.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:06.949626+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e599: 8 total, 8 up, 6 in 2024-02-22T19:52:08.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:07.709637+0000 mon.a (mon.0) 2054 : cluster [DBG] osdmap e600: 8 total, 8 up, 6 in 2024-02-22T19:52:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:06.939928+0000 mon.a (mon.0) 2052 : cluster [WRN] Health check update: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-02-22T19:52:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:06.949626+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e599: 8 total, 8 up, 6 in 2024-02-22T19:52:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:07.709637+0000 mon.a (mon.0) 2054 : cluster [DBG] osdmap e600: 8 total, 8 up, 6 in 2024-02-22T19:52:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-02-22T19:52:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:07.978137+0000 mgr.y (mgr.14535) 876 : cluster [DBG] pgmap v1220: 128 pgs: 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 6 remapped+peering, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 6 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+recovering, 108 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 25 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 48 KiB/s, 31 objects/s recovering 2024-02-22T19:52:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:08.097354+0000 mon.c (mon.2) 101 : audit [DBG] from='client.? 172.21.15.53:0/4186009549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:52:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:08.308789+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? 172.21.15.53:0/1782460040' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:52:09.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:08.405744+0000 mon.a (mon.0) 2056 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:09.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:07.978137+0000 mgr.y (mgr.14535) 876 : cluster [DBG] pgmap v1220: 128 pgs: 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 6 remapped+peering, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 6 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+recovering, 108 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 25 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 48 KiB/s, 31 objects/s recovering 2024-02-22T19:52:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:08.097354+0000 mon.c (mon.2) 101 : audit [DBG] from='client.? 172.21.15.53:0/4186009549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:52:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:08.308789+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? 172.21.15.53:0/1782460040' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:52:09.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:08.405744+0000 mon.a (mon.0) 2056 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:09.136 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:52:09.136 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:52:09.136 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:52:09.136 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:52:09.136 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-22T19:52:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:07.978137+0000 mgr.y (mgr.14535) 876 : cluster [DBG] pgmap v1220: 128 pgs: 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 6 remapped+peering, 1 active+recovering+remapped, 2 active+recovery_wait+degraded, 6 active+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 2 active+recovering, 108 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 25 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 48 KiB/s, 31 objects/s recovering 2024-02-22T19:52:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:08.097354+0000 mon.c (mon.2) 101 : audit [DBG] from='client.? 172.21.15.53:0/4186009549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:52:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:08.308789+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? 172.21.15.53:0/1782460040' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-22T19:52:09.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:08.405744+0000 mon.a (mon.0) 2056 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:09.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:09] "GET /metrics HTTP/1.1" 200 176730 "" "Prometheus/2.33.4" 2024-02-22T19:52:10.144 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:08.948230+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:10.144 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:08.956097+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? 172.21.15.53:0/1782460040' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:52:10.144 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:08.956165+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e601: 8 total, 8 up, 6 in 2024-02-22T19:52:10.145 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:08.948230+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:10.145 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:08.956097+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? 172.21.15.53:0/1782460040' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:52:10.145 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:08.956165+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e601: 8 total, 8 up, 6 in 2024-02-22T19:52:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:08.948230+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:08.956097+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? 172.21.15.53:0/1782460040' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-22T19:52:10.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:08.956165+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e601: 8 total, 8 up, 6 in 2024-02-22T19:52:10.863 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:52:10.869 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:10.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:52:10.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:52:10.997 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:11.018 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:52:11.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:11.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:52:11.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0 2024-02-22T19:52:11.149 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:09.978876+0000 mgr.y (mgr.14535) 877 : cluster [DBG] pgmap v1222: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+recovering+remapped, 9 active+remapped+backfill_toofull, 116 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%); 26 B/s, 17 objects/s recovering 2024-02-22T19:52:11.149 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:09.978876+0000 mgr.y (mgr.14535) 877 : cluster [DBG] pgmap v1222: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+recovering+remapped, 9 active+remapped+backfill_toofull, 116 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%); 26 B/s, 17 objects/s recovering 2024-02-22T19:52:11.152 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:11.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:09.978876+0000 mgr.y (mgr.14535) 877 : cluster [DBG] pgmap v1222: 128 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+remapped, 1 active+recovering+remapped, 9 active+remapped+backfill_toofull, 116 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%); 26 B/s, 17 objects/s recovering 2024-02-22T19:52:11.409 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:52:11.409 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:52:11.409 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-02-22T19:52:11.409 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:52:12.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:10.953425+0000 mon.a (mon.0) 2060 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:12.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:10.959816+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e602: 8 total, 8 up, 6 in 2024-02-22T19:52:12.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:10.953425+0000 mon.a (mon.0) 2060 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:12.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:10.959816+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e602: 8 total, 8 up, 6 in 2024-02-22T19:52:12.105 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:52:12.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:12.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0' 2024-02-22T19:52:12.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-22T19:52:12.210 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:52:12.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:10.953425+0000 mon.a (mon.0) 2060 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:12.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:10.959816+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e602: 8 total, 8 up, 6 in 2024-02-22T19:52:12.237 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:52:12.258 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:12.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:52:12.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5 2024-02-22T19:52:12.386 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:12.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-22T19:52:12.406 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:52:12.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:12.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5' 2024-02-22T19:52:12.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1 2024-02-22T19:52:12.523 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:12.582 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:12.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-22T19:52:13.114 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:11.961560+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e603: 8 total, 8 up, 6 in 2024-02-22T19:52:13.114 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:11.979265+0000 mgr.y (mgr.14535) 878 : cluster [DBG] pgmap v1225: 128 pgs: 2 active+clean+remapped, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 7 active+remapped+backfill_toofull, 117 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%); 24 B/s, 30 objects/s recovering 2024-02-22T19:52:13.115 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:12.706405+0000 mon.a (mon.0) 2063 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-22T19:52:13.115 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:12.710541+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e604: 8 total, 8 up, 6 in 2024-02-22T19:52:13.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:11.961560+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e603: 8 total, 8 up, 6 in 2024-02-22T19:52:13.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:11.979265+0000 mgr.y (mgr.14535) 878 : cluster [DBG] pgmap v1225: 128 pgs: 2 active+clean+remapped, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 7 active+remapped+backfill_toofull, 117 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%); 24 B/s, 30 objects/s recovering 2024-02-22T19:52:13.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:12.706405+0000 mon.a (mon.0) 2063 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-22T19:52:13.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:12.710541+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e604: 8 total, 8 up, 6 in 2024-02-22T19:52:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:11.961560+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e603: 8 total, 8 up, 6 in 2024-02-22T19:52:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:11.979265+0000 mgr.y (mgr.14535) 878 : cluster [DBG] pgmap v1225: 128 pgs: 2 active+clean+remapped, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 7 active+remapped+backfill_toofull, 117 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%); 24 B/s, 30 objects/s recovering 2024-02-22T19:52:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:12.706405+0000 mon.a (mon.0) 2063 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-22T19:52:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:12.710541+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e604: 8 total, 8 up, 6 in 2024-02-22T19:52:13.533 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:52:13.536 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:13.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1' 2024-02-22T19:52:13.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:52:13.668 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:14.357 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:15.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:52:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:52:15.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:13.979622+0000 mgr.y (mgr.14535) 879 : cluster [DBG] pgmap v1227: 128 pgs: 2 active+clean+remapped, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 7 active+remapped+backfill_toofull, 117 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1018 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 20 B/s, 25 objects/s recovering 2024-02-22T19:52:15.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:14.395512+0000 mon.a (mon.0) 2065 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:15.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:14.757477+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:52:15.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:14.758241+0000 mon.a (mon.0) 2067 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:52:15.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:14.763149+0000 mon.a (mon.0) 2068 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:15.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:15.385861+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? 172.21.15.53:0/3600729455' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:13.979622+0000 mgr.y (mgr.14535) 879 : cluster [DBG] pgmap v1227: 128 pgs: 2 active+clean+remapped, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 7 active+remapped+backfill_toofull, 117 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1018 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 20 B/s, 25 objects/s recovering 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:14.395512+0000 mon.a (mon.0) 2065 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:14.757477+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:14.758241+0000 mon.a (mon.0) 2067 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:14.763149+0000 mon.a (mon.0) 2068 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:15.385861+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? 172.21.15.53:0/3600729455' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:52:15.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:13.979622+0000 mgr.y (mgr.14535) 879 : cluster [DBG] pgmap v1227: 128 pgs: 2 active+clean+remapped, 1 active+recovering+remapped, 1 active+recovering+undersized+remapped, 7 active+remapped+backfill_toofull, 117 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1018 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 20 B/s, 25 objects/s recovering 2024-02-22T19:52:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:14.395512+0000 mon.a (mon.0) 2065 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:14.757477+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:52:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:14.758241+0000 mon.a (mon.0) 2067 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:52:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:14.763149+0000 mon.a (mon.0) 2068 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:15.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:15.385861+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? 172.21.15.53:0/3600729455' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-22T19:52:15.919 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:52:15.923 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:15.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:52:16.023 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'unique_pool_4' removed 2024-02-22T19:52:16.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3 2024-02-22T19:52:16.055 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:16.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:52:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:52:16.190 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:16.204 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:52:16.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:16.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3' 2024-02-22T19:52:16.260 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 7, 6, 1, 0] out_osds: [5, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-22T19:52:16.260 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-22T19:52:16.260 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-02-22T19:52:16.260 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-22T19:52:16.260 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-22T19:52:16.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-02-22T19:52:16.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4 2024-02-22T19:52:16.338 INFO:teuthology.orchestra.run.smithi064.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:16.358 INFO:teuthology.orchestra.run.smithi064.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:52:16.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:16.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi064 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4' 2024-02-22T19:52:16.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2 2024-02-22T19:52:16.488 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:15.727204+0000 mgr.y (mgr.14535) 880 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:16.488 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:15.764364+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.53:0/3600729455' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:52:16.489 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:15.764431+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e605: 8 total, 8 up, 6 in 2024-02-22T19:52:16.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:15.727204+0000 mgr.y (mgr.14535) 880 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:16.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:15.764364+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.53:0/3600729455' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:52:16.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:15.764431+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e605: 8 total, 8 up, 6 in 2024-02-22T19:52:16.491 INFO:teuthology.orchestra.run.smithi053.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-22T19:52:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:15.727204+0000 mgr.y (mgr.14535) 880 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:15.764364+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.53:0/3600729455' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-02-22T19:52:16.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:15.764431+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e605: 8 total, 8 up, 6 in 2024-02-22T19:52:16.616 INFO:teuthology.orchestra.run.smithi053.stderr:Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:52:16.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-22T19:52:16.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi053 with status 125: 'sudo docker kill -s 1 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2' 2024-02-22T19:52:16.624 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-02-22T19:52:16.634 INFO:tasks.thrashosds:joining thrashosds 2024-02-22T19:52:17.273 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:17.309 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:52:17.309 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-02-22T19:52:17.310 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-02-22T19:52:17.310 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-02-22T19:52:17.310 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:52:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:15.979993+0000 mgr.y (mgr.14535) 881 : cluster [DBG] pgmap v1229: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 815 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:16.761466+0000 mon.a (mon.0) 2072 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:17.754 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:16.761499+0000 mon.a (mon.0) 2073 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-02-22T19:52:17.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:15.979993+0000 mgr.y (mgr.14535) 881 : cluster [DBG] pgmap v1229: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 815 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:17.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:16.761466+0000 mon.a (mon.0) 2072 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:17.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:16.761499+0000 mon.a (mon.0) 2073 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-02-22T19:52:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:15.979993+0000 mgr.y (mgr.14535) 881 : cluster [DBG] pgmap v1229: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 815 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:16.761466+0000 mon.a (mon.0) 2072 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:17.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:16.761499+0000 mon.a (mon.0) 2073 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-02-22T19:52:17.858 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:18.771 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:17.712204+0000 mon.a (mon.0) 2074 : cluster [DBG] osdmap e606: 8 total, 8 up, 6 in 2024-02-22T19:52:18.771 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:17.712204+0000 mon.a (mon.0) 2074 : cluster [DBG] osdmap e606: 8 total, 8 up, 6 in 2024-02-22T19:52:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:17.712204+0000 mon.a (mon.0) 2074 : cluster [DBG] osdmap e606: 8 total, 8 up, 6 in 2024-02-22T19:52:19.431 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:19] "GET /metrics HTTP/1.1" 200 176730 "" "Prometheus/2.33.4" 2024-02-22T19:52:19.721 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:17.980380+0000 mgr.y (mgr.14535) 882 : cluster [DBG] pgmap v1231: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:19.721 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:19.113520+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? 172.21.15.53:0/33703401' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:52:19.721 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:19.553122+0000 mon.a (mon.0) 2076 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:19.723 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-02-22T19:52:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:17.980380+0000 mgr.y (mgr.14535) 882 : cluster [DBG] pgmap v1231: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:19.113520+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? 172.21.15.53:0/33703401' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:52:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:19.553122+0000 mon.a (mon.0) 2076 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:17.980380+0000 mgr.y (mgr.14535) 882 : cluster [DBG] pgmap v1231: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:19.113520+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? 172.21.15.53:0/33703401' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:52:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:19.553122+0000 mon.a (mon.0) 2076 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:21.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:19.714343+0000 mon.a (mon.0) 2077 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:21.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:19.718290+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.53:0/33703401' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:52:21.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:19.718350+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e607: 8 total, 8 up, 6 in 2024-02-22T19:52:21.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:19.714343+0000 mon.a (mon.0) 2077 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:21.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:19.718290+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.53:0/33703401' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:52:21.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:19.718350+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e607: 8 total, 8 up, 6 in 2024-02-22T19:52:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:19.714343+0000 mon.a (mon.0) 2077 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:19.718290+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.53:0/33703401' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-22T19:52:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:19.718350+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e607: 8 total, 8 up, 6 in 2024-02-22T19:52:21.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-22T19:52:21.818 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:19.980797+0000 mgr.y (mgr.14535) 883 : cluster [DBG] pgmap v1233: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:21.818 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:19.980797+0000 mgr.y (mgr.14535) 883 : cluster [DBG] pgmap v1233: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:22.004 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-02-22T19:52:22.021 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-02-22T19:52:22.038 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-02-22T19:52:22.055 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-02-22T19:52:22.073 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-02-22T19:52:22.091 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-02-22T19:52:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:19.980797+0000 mgr.y (mgr.14535) 883 : cluster [DBG] pgmap v1233: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:22.110 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-02-22T19:52:22.129 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-22T19:52:22.217 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-02-22T19:52:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-22T19:52:22.217 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-02-22T19:52:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:21.986848+0000 mon.a (mon.0) 2080 : audit [DBG] from='client.? 172.21.15.53:0/3931055529' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:52:23.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:22.713712+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e608: 8 total, 8 up, 6 in 2024-02-22T19:52:23.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:21.986848+0000 mon.a (mon.0) 2080 : audit [DBG] from='client.? 172.21.15.53:0/3931055529' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:52:23.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:22.713712+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e608: 8 total, 8 up, 6 in 2024-02-22T19:52:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:21.986848+0000 mon.a (mon.0) 2080 : audit [DBG] from='client.? 172.21.15.53:0/3931055529' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-22T19:52:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:22.713712+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e608: 8 total, 8 up, 6 in 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:21.981262+0000 mgr.y (mgr.14535) 884 : cluster [DBG] pgmap v1234: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 823 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:22.690097+0000 osd.1 (osd.1) 308 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:22.690148+0000 osd.1 (osd.1) 309 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:23.406952+0000 mon.a (mon.0) 2082 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:21.981262+0000 mgr.y (mgr.14535) 884 : cluster [DBG] pgmap v1234: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 823 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:22.690097+0000 osd.1 (osd.1) 308 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:22.690148+0000 osd.1 (osd.1) 309 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:23.999 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:23.406952+0000 mon.a (mon.0) 2082 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:21.981262+0000 mgr.y (mgr.14535) 884 : cluster [DBG] pgmap v1234: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 823 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:22.690097+0000 osd.1 (osd.1) 308 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:22.690148+0000 osd.1 (osd.1) 309 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:23.406952+0000 mon.a (mon.0) 2082 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:24.350 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:24.846 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:24.218273+0000 osd.6 (osd.6) 168 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:24.846 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:24.218332+0000 osd.6 (osd.6) 169 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:24.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:24.218273+0000 osd.6 (osd.6) 168 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:24.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:24.218332+0000 osd.6 (osd.6) 169 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:52:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:52:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:24.218273+0000 osd.6 (osd.6) 168 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:24.218332+0000 osd.6 (osd.6) 169 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:25.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset noscrub 2024-02-22T19:52:25.835 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:23.981586+0000 mgr.y (mgr.14535) 885 : cluster [DBG] pgmap v1236: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 816 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:25.835 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:24.741030+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e609: 8 total, 8 up, 6 in 2024-02-22T19:52:25.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:23.981586+0000 mgr.y (mgr.14535) 885 : cluster [DBG] pgmap v1236: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 816 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:25.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:24.741030+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e609: 8 total, 8 up, 6 in 2024-02-22T19:52:26.083 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:26.092 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:26.092 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:26.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:52:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:52:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:23.981586+0000 mgr.y (mgr.14535) 885 : cluster [DBG] pgmap v1236: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 816 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:24.741030+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e609: 8 total, 8 up, 6 in 2024-02-22T19:52:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:25.729981+0000 mgr.y (mgr.14535) 886 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:27.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:25.926730+0000 mon.a (mon.0) 2084 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 7]}]: dispatch 2024-02-22T19:52:27.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:25.926931+0000 mon.a (mon.0) 2085 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [1, 2]}]: dispatch 2024-02-22T19:52:27.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:25.927114+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-02-22T19:52:27.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:25.729981+0000 mgr.y (mgr.14535) 886 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:27.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:25.926730+0000 mon.a (mon.0) 2084 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 7]}]: dispatch 2024-02-22T19:52:27.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:25.926931+0000 mon.a (mon.0) 2085 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [1, 2]}]: dispatch 2024-02-22T19:52:27.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:25.927114+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-02-22T19:52:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:25.729981+0000 mgr.y (mgr.14535) 886 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:25.926730+0000 mon.a (mon.0) 2084 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 7]}]: dispatch 2024-02-22T19:52:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:25.926931+0000 mon.a (mon.0) 2085 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [1, 2]}]: dispatch 2024-02-22T19:52:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:25.927114+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-02-22T19:52:27.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-22T19:52:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:25.982066+0000 mgr.y (mgr.14535) 887 : cluster [DBG] pgmap v1238: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 817 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:26.748289+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 7]}]': finished 2024-02-22T19:52:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:26.748379+0000 mon.a (mon.0) 2088 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [1, 2]}]': finished 2024-02-22T19:52:28.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:26.748439+0000 mon.a (mon.0) 2089 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-02-22T19:52:28.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:26.748490+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e610: 8 total, 8 up, 6 in 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:25.982066+0000 mgr.y (mgr.14535) 887 : cluster [DBG] pgmap v1238: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 817 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:26.748289+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 7]}]': finished 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:26.748379+0000 mon.a (mon.0) 2088 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [1, 2]}]': finished 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:26.748439+0000 mon.a (mon.0) 2089 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:26.748490+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e610: 8 total, 8 up, 6 in 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:25.982066+0000 mgr.y (mgr.14535) 887 : cluster [DBG] pgmap v1238: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 817 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:26.748289+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 7]}]': finished 2024-02-22T19:52:28.143 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:26.748379+0000 mon.a (mon.0) 2088 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [1, 2]}]': finished 2024-02-22T19:52:28.144 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:26.748439+0000 mon.a (mon.0) 2089 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-02-22T19:52:28.144 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:26.748490+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e610: 8 total, 8 up, 6 in 2024-02-22T19:52:28.749 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:29.051 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:27.783539+0000 mon.a (mon.0) 2091 : cluster [DBG] osdmap e611: 8 total, 8 up, 6 in 2024-02-22T19:52:29.051 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:27.982392+0000 mgr.y (mgr.14535) 888 : cluster [DBG] pgmap v1241: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 971 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:29.051 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:27.783539+0000 mon.a (mon.0) 2091 : cluster [DBG] osdmap e611: 8 total, 8 up, 6 in 2024-02-22T19:52:29.051 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:27.982392+0000 mgr.y (mgr.14535) 888 : cluster [DBG] pgmap v1241: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 971 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:27.783539+0000 mon.a (mon.0) 2091 : cluster [DBG] osdmap e611: 8 total, 8 up, 6 in 2024-02-22T19:52:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:27.982392+0000 mgr.y (mgr.14535) 888 : cluster [DBG] pgmap v1241: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 971 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:29.338 INFO:tasks.thrashosds.thrasher:inning osd 2024-02-22T19:52:29.338 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-02-22T19:52:29.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 5 2024-02-22T19:52:29.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:52:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:29] "GET /metrics HTTP/1.1" 200 174314 "" "Prometheus/2.33.4" 2024-02-22T19:52:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-02-22T19:52:30.096 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:28.918436+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.53:0/4112521973' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:52:30.097 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:28.918768+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:52:30.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:28.918436+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.53:0/4112521973' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:52:30.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:28.918768+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:52:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:28.918436+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.53:0/4112521973' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:52:30.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:28.918768+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-22T19:52:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:29.786250+0000 mon.a (mon.0) 2093 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:29.793163+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:52:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:29.793233+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e612: 8 total, 8 up, 6 in 2024-02-22T19:52:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:29.982784+0000 mgr.y (mgr.14535) 889 : cluster [DBG] pgmap v1243: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 976 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:29.786250+0000 mon.a (mon.0) 2093 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:29.793163+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:52:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:29.793233+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e612: 8 total, 8 up, 6 in 2024-02-22T19:52:31.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:29.982784+0000 mgr.y (mgr.14535) 889 : cluster [DBG] pgmap v1243: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 976 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:29.786250+0000 mon.a (mon.0) 2093 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-22T19:52:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:29.793163+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-22T19:52:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:29.793233+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e612: 8 total, 8 up, 6 in 2024-02-22T19:52:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:29.982784+0000 mgr.y (mgr.14535) 889 : cluster [DBG] pgmap v1243: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 976 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%) 2024-02-22T19:52:31.968 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:30.786397+0000 osd.3 (osd.3) 375 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:52:31.969 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:30.789894+0000 osd.3 (osd.3) 376 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:52:31.969 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:30.786397+0000 osd.3 (osd.3) 375 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:52:31.969 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:30.789894+0000 osd.3 (osd.3) 376 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:52:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:30.786397+0000 osd.3 (osd.3) 375 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:52:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:30.789894+0000 osd.3 (osd.3) 376 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:52:32.301 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:32.301 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:32.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:52:32.462+0000 7fe2d794e700 -1 mon.a@0(leader).osd e612 definitely_dead 0 2024-02-22T19:52:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.730335+0000 osd.0 (osd.0) 183 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:52:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.733467+0000 osd.0 (osd.0) 184 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:52:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.765238+0000 osd.3 (osd.3) 377 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.816607+0000 osd.3 (osd.3) 378 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.950690+0000 osd.6 (osd.6) 170 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.953578+0000 osd.6 (osd.6) 171 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:31.983294+0000 mgr.y (mgr.14535) 890 : cluster [DBG] pgmap v1244: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 36 B/s, 0 objects/s recovering 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:32.463866+0000 mon.a (mon.0) 2096 : audit [DBG] from='client.? 172.21.15.53:0/172176547' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["5"]}]: dispatch 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.730335+0000 osd.0 (osd.0) 183 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.733467+0000 osd.0 (osd.0) 184 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.765238+0000 osd.3 (osd.3) 377 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:52:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.816607+0000 osd.3 (osd.3) 378 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:52:33.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.950690+0000 osd.6 (osd.6) 170 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:52:33.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.953578+0000 osd.6 (osd.6) 171 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:52:33.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:31.983294+0000 mgr.y (mgr.14535) 890 : cluster [DBG] pgmap v1244: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 36 B/s, 0 objects/s recovering 2024-02-22T19:52:33.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:32.463866+0000 mon.a (mon.0) 2096 : audit [DBG] from='client.? 172.21.15.53:0/172176547' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["5"]}]: dispatch 2024-02-22T19:52:33.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-22T19:52:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.730335+0000 osd.0 (osd.0) 183 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:52:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.733467+0000 osd.0 (osd.0) 184 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:52:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.765238+0000 osd.3 (osd.3) 377 : cluster [DBG] 6.11 scrub starts 2024-02-22T19:52:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.816607+0000 osd.3 (osd.3) 378 : cluster [DBG] 6.11 scrub ok 2024-02-22T19:52:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.950690+0000 osd.6 (osd.6) 170 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:52:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.953578+0000 osd.6 (osd.6) 171 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:52:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:31.983294+0000 mgr.y (mgr.14535) 890 : cluster [DBG] pgmap v1244: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 36 B/s, 0 objects/s recovering 2024-02-22T19:52:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:32.463866+0000 mon.a (mon.0) 2096 : audit [DBG] from='client.? 172.21.15.53:0/172176547' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["5"]}]: dispatch 2024-02-22T19:52:33.206 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.5. 2024-02-22T19:52:33.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-22T19:52:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:32.713863+0000 osd.1 (osd.1) 310 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:52:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:32.724229+0000 osd.1 (osd.1) 311 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:52:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:32.733937+0000 osd.3 (osd.3) 379 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:52:34.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:32.737485+0000 osd.3 (osd.3) 380 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:52:34.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:32.801788+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? 172.21.15.53:0/172176547' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["5"]}]': finished 2024-02-22T19:52:34.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:32.801847+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e613: 8 total, 8 up, 7 in 2024-02-22T19:52:34.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:32.713863+0000 osd.1 (osd.1) 310 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:52:34.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:32.724229+0000 osd.1 (osd.1) 311 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:52:34.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:32.733937+0000 osd.3 (osd.3) 379 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:52:34.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:32.737485+0000 osd.3 (osd.3) 380 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:52:34.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:32.801788+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? 172.21.15.53:0/172176547' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["5"]}]': finished 2024-02-22T19:52:34.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:32.801847+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e613: 8 total, 8 up, 7 in 2024-02-22T19:52:34.202 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:32.713863+0000 osd.1 (osd.1) 310 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:52:34.202 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:32.724229+0000 osd.1 (osd.1) 311 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:52:34.202 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:32.733937+0000 osd.3 (osd.3) 379 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:52:34.202 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:32.737485+0000 osd.3 (osd.3) 380 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:52:34.202 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:32.801788+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? 172.21.15.53:0/172176547' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["5"]}]': finished 2024-02-22T19:52:34.202 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:32.801847+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e613: 8 total, 8 up, 7 in 2024-02-22T19:52:34.220 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-02-22T19:52:34.220 INFO:tasks.thrashosds.thrasher:inning osd 2024-02-22T19:52:34.220 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-02-22T19:52:34.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd in 4 2024-02-22T19:52:34.294 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:35.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:52:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.713904+0000 osd.0 (osd.0) 185 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.714918+0000 osd.0 (osd.0) 186 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.763806+0000 osd.1 (osd.1) 312 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.766149+0000 osd.1 (osd.1) 313 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.803181+0000 mon.a (mon.0) 2099 : cluster [DBG] osdmap e614: 8 total, 8 up, 7 in 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.811539+0000 osd.2 (osd.2) 308 : cluster [DBG] 5.6 starting backfill to osd.5 from (0'0,0'0] MAX to 597'35 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:33.983826+0000 mgr.y (mgr.14535) 891 : cluster [DBG] pgmap v1247: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 36 B/s, 0 objects/s recovering 2024-02-22T19:52:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:34.803795+0000 mon.a (mon.0) 2100 : cluster [DBG] osdmap e615: 8 total, 8 up, 7 in 2024-02-22T19:52:35.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.713904+0000 osd.0 (osd.0) 185 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:52:35.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.714918+0000 osd.0 (osd.0) 186 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:52:35.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.763806+0000 osd.1 (osd.1) 312 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:52:35.244 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.766149+0000 osd.1 (osd.1) 313 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.803181+0000 mon.a (mon.0) 2099 : cluster [DBG] osdmap e614: 8 total, 8 up, 7 in 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.811539+0000 osd.2 (osd.2) 308 : cluster [DBG] 5.6 starting backfill to osd.5 from (0'0,0'0] MAX to 597'35 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:33.983826+0000 mgr.y (mgr.14535) 891 : cluster [DBG] pgmap v1247: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 36 B/s, 0 objects/s recovering 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:34.803795+0000 mon.a (mon.0) 2100 : cluster [DBG] osdmap e615: 8 total, 8 up, 7 in 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.713904+0000 osd.0 (osd.0) 185 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.714918+0000 osd.0 (osd.0) 186 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.763806+0000 osd.1 (osd.1) 312 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.766149+0000 osd.1 (osd.1) 313 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.803181+0000 mon.a (mon.0) 2099 : cluster [DBG] osdmap e614: 8 total, 8 up, 7 in 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.811539+0000 osd.2 (osd.2) 308 : cluster [DBG] 5.6 starting backfill to osd.5 from (0'0,0'0] MAX to 597'35 2024-02-22T19:52:35.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:33.983826+0000 mgr.y (mgr.14535) 891 : cluster [DBG] pgmap v1247: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 7/576 objects misplaced (1.215%); 36 B/s, 0 objects/s recovering 2024-02-22T19:52:35.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:34.803795+0000 mon.a (mon.0) 2100 : cluster [DBG] osdmap e615: 8 total, 8 up, 7 in 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:34.802818+0000 osd.1 (osd.1) 314 : cluster [DBG] 6.e starting backfill to osd.5 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:34.804132+0000 osd.7 (osd.7) 339 : cluster [DBG] 5.4 starting backfill to osd.5 from (0'0,0'0] MAX to 596'17 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:34.805738+0000 osd.3 (osd.3) 381 : cluster [DBG] 6.c starting backfill to osd.5 from (0'0,0'0] MAX to 137'2 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:34.806256+0000 osd.0 (osd.0) 187 : cluster [DBG] 5.7 starting backfill to osd.5 from (0'0,0'0] MAX to 601'38 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:34.806843+0000 osd.0 (osd.0) 188 : cluster [DBG] 5.3 starting backfill to osd.5 from (0'0,0'0] MAX to 596'25 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.074557+0000 osd.5 (osd.5) 221 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.077447+0000 osd.5 (osd.5) 222 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:35.737517+0000 mgr.y (mgr.14535) 892 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:36.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.805564+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e616: 8 total, 8 up, 7 in 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:34.802818+0000 osd.1 (osd.1) 314 : cluster [DBG] 6.e starting backfill to osd.5 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:34.804132+0000 osd.7 (osd.7) 339 : cluster [DBG] 5.4 starting backfill to osd.5 from (0'0,0'0] MAX to 596'17 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:34.805738+0000 osd.3 (osd.3) 381 : cluster [DBG] 6.c starting backfill to osd.5 from (0'0,0'0] MAX to 137'2 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:34.806256+0000 osd.0 (osd.0) 187 : cluster [DBG] 5.7 starting backfill to osd.5 from (0'0,0'0] MAX to 601'38 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:34.806843+0000 osd.0 (osd.0) 188 : cluster [DBG] 5.3 starting backfill to osd.5 from (0'0,0'0] MAX to 596'25 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.074557+0000 osd.5 (osd.5) 221 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.077447+0000 osd.5 (osd.5) 222 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:35.737517+0000 mgr.y (mgr.14535) 892 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.805564+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e616: 8 total, 8 up, 7 in 2024-02-22T19:52:36.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:52:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:34.802818+0000 osd.1 (osd.1) 314 : cluster [DBG] 6.e starting backfill to osd.5 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:34.804132+0000 osd.7 (osd.7) 339 : cluster [DBG] 5.4 starting backfill to osd.5 from (0'0,0'0] MAX to 596'17 2024-02-22T19:52:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:34.805738+0000 osd.3 (osd.3) 381 : cluster [DBG] 6.c starting backfill to osd.5 from (0'0,0'0] MAX to 137'2 2024-02-22T19:52:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:34.806256+0000 osd.0 (osd.0) 187 : cluster [DBG] 5.7 starting backfill to osd.5 from (0'0,0'0] MAX to 601'38 2024-02-22T19:52:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:34.806843+0000 osd.0 (osd.0) 188 : cluster [DBG] 5.3 starting backfill to osd.5 from (0'0,0'0] MAX to 596'25 2024-02-22T19:52:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.074557+0000 osd.5 (osd.5) 221 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:52:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.077447+0000 osd.5 (osd.5) 222 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:52:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:35.737517+0000 mgr.y (mgr.14535) 892 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.805564+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e616: 8 total, 8 up, 7 in 2024-02-22T19:52:37.020 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:37.036 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:37.036 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:37.072 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.734034+0000 osd.3 (osd.3) 382 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.737757+0000 osd.3 (osd.3) 383 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.812686+0000 osd.1 (osd.1) 315 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.814475+0000 osd.1 (osd.1) 316 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.980627+0000 osd.6 (osd.6) 172 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.982244+0000 osd.6 (osd.6) 173 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:35.984181+0000 mgr.y (mgr.14535) 893 : cluster [DBG] pgmap v1250: 112 pgs: 1 peering, 4 remapped+peering, 2 active+recovering, 2 active+remapped+backfill_toofull, 103 active+clean; 5.7 KiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 11/576 objects misplaced (1.910%); 309 B/s, 8 objects/s recovering 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.807259+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e617: 8 total, 8 up, 7 in 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.734034+0000 osd.3 (osd.3) 382 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.737757+0000 osd.3 (osd.3) 383 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.812686+0000 osd.1 (osd.1) 315 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:52:37.073 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.814475+0000 osd.1 (osd.1) 316 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:52:37.074 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.980627+0000 osd.6 (osd.6) 172 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:52:37.074 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.982244+0000 osd.6 (osd.6) 173 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:52:37.074 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:35.984181+0000 mgr.y (mgr.14535) 893 : cluster [DBG] pgmap v1250: 112 pgs: 1 peering, 4 remapped+peering, 2 active+recovering, 2 active+remapped+backfill_toofull, 103 active+clean; 5.7 KiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 11/576 objects misplaced (1.910%); 309 B/s, 8 objects/s recovering 2024-02-22T19:52:37.074 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.807259+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e617: 8 total, 8 up, 7 in 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.734034+0000 osd.3 (osd.3) 382 : cluster [DBG] 4.0 scrub starts 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.737757+0000 osd.3 (osd.3) 383 : cluster [DBG] 4.0 scrub ok 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.812686+0000 osd.1 (osd.1) 315 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.814475+0000 osd.1 (osd.1) 316 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.980627+0000 osd.6 (osd.6) 172 : cluster [DBG] 4.1f scrub starts 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.982244+0000 osd.6 (osd.6) 173 : cluster [DBG] 4.1f scrub ok 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:35.984181+0000 mgr.y (mgr.14535) 893 : cluster [DBG] pgmap v1250: 112 pgs: 1 peering, 4 remapped+peering, 2 active+recovering, 2 active+remapped+backfill_toofull, 103 active+clean; 5.7 KiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 11/576 objects misplaced (1.910%); 309 B/s, 8 objects/s recovering 2024-02-22T19:52:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.807259+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e617: 8 total, 8 up, 7 in 2024-02-22T19:52:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.611397+0000 osd.0 (osd.0) 189 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:52:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.613212+0000 osd.0 (osd.0) 190 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:52:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.774175+0000 osd.1 (osd.1) 317 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:52:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.835050+0000 osd.1 (osd.1) 318 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:52:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:37.717963+0000 mon.a (mon.0) 2103 : cluster [DBG] osdmap e618: 8 total, 8 up, 7 in 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.611397+0000 osd.0 (osd.0) 189 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.613212+0000 osd.0 (osd.0) 190 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.774175+0000 osd.1 (osd.1) 317 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.835050+0000 osd.1 (osd.1) 318 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:37.717963+0000 mon.a (mon.0) 2103 : cluster [DBG] osdmap e618: 8 total, 8 up, 7 in 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.611397+0000 osd.0 (osd.0) 189 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.613212+0000 osd.0 (osd.0) 190 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.774175+0000 osd.1 (osd.1) 317 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:52:38.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.835050+0000 osd.1 (osd.1) 318 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:52:38.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:37.717963+0000 mon.a (mon.0) 2103 : cluster [DBG] osdmap e618: 8 total, 8 up, 7 in 2024-02-22T19:52:38.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-22T19:52:38.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.849319+0000 osd.2 (osd.2) 309 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:52:38.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:36.852206+0000 osd.2 (osd.2) 310 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:52:38.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:37.858307+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? 172.21.15.53:0/531060696' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:52:38.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:37.984812+0000 mgr.y (mgr.14535) 894 : cluster [DBG] pgmap v1253: 112 pgs: 1 peering, 4 remapped+peering, 2 active+recovering, 2 active+remapped+backfill_toofull, 103 active+clean; 5.7 KiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 11/576 objects misplaced (1.910%); 309 B/s, 8 objects/s recovering 2024-02-22T19:52:38.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.025935+0000 osd.5 (osd.5) 223 : cluster [DBG] 1.a scrub starts 2024-02-22T19:52:38.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.029027+0000 osd.5 (osd.5) 224 : cluster [DBG] 1.a scrub ok 2024-02-22T19:52:38.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.568409+0000 osd.0 (osd.0) 191 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:52:38.834 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.569304+0000 osd.0 (osd.0) 192 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:52:38.836 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-02-22T19:52:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.849319+0000 osd.2 (osd.2) 309 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:52:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:36.852206+0000 osd.2 (osd.2) 310 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:52:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:37.858307+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? 172.21.15.53:0/531060696' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:52:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:37.984812+0000 mgr.y (mgr.14535) 894 : cluster [DBG] pgmap v1253: 112 pgs: 1 peering, 4 remapped+peering, 2 active+recovering, 2 active+remapped+backfill_toofull, 103 active+clean; 5.7 KiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 11/576 objects misplaced (1.910%); 309 B/s, 8 objects/s recovering 2024-02-22T19:52:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.025935+0000 osd.5 (osd.5) 223 : cluster [DBG] 1.a scrub starts 2024-02-22T19:52:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.029027+0000 osd.5 (osd.5) 224 : cluster [DBG] 1.a scrub ok 2024-02-22T19:52:39.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.568409+0000 osd.0 (osd.0) 191 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:52:39.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.569304+0000 osd.0 (osd.0) 192 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:52:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.849319+0000 osd.2 (osd.2) 309 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:52:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:36.852206+0000 osd.2 (osd.2) 310 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:52:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:37.858307+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? 172.21.15.53:0/531060696' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-22T19:52:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:37.984812+0000 mgr.y (mgr.14535) 894 : cluster [DBG] pgmap v1253: 112 pgs: 1 peering, 4 remapped+peering, 2 active+recovering, 2 active+remapped+backfill_toofull, 103 active+clean; 5.7 KiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 11/576 objects misplaced (1.910%); 309 B/s, 8 objects/s recovering 2024-02-22T19:52:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.025935+0000 osd.5 (osd.5) 223 : cluster [DBG] 1.a scrub starts 2024-02-22T19:52:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.029027+0000 osd.5 (osd.5) 224 : cluster [DBG] 1.a scrub ok 2024-02-22T19:52:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.568409+0000 osd.0 (osd.0) 191 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:52:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.569304+0000 osd.0 (osd.0) 192 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:52:39.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:39] "GET /metrics HTTP/1.1" 200 174320 "" "Prometheus/2.33.4" 2024-02-22T19:52:40.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.721804+0000 osd.1 (osd.1) 319 : cluster [DBG] 4.f scrub starts 2024-02-22T19:52:40.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.723521+0000 osd.1 (osd.1) 320 : cluster [DBG] 4.f scrub ok 2024-02-22T19:52:40.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.784652+0000 osd.3 (osd.3) 384 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.788255+0000 osd.3 (osd.3) 385 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.800741+0000 osd.2 (osd.2) 311 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.804274+0000 osd.2 (osd.2) 312 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.824957+0000 mon.a (mon.0) 2105 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:38.833265+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.53:0/531060696' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.833366+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e619: 8 total, 8 up, 7 in 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.982787+0000 osd.5 (osd.5) 225 : cluster [DBG] 6.a scrub starts 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:38.993891+0000 osd.5 (osd.5) 226 : cluster [DBG] 6.a scrub ok 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:39.561558+0000 osd.0 (osd.0) 193 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:52:40.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:39.564775+0000 osd.0 (osd.0) 194 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:52:40.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.721804+0000 osd.1 (osd.1) 319 : cluster [DBG] 4.f scrub starts 2024-02-22T19:52:40.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.723521+0000 osd.1 (osd.1) 320 : cluster [DBG] 4.f scrub ok 2024-02-22T19:52:40.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.784652+0000 osd.3 (osd.3) 384 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:52:40.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.788255+0000 osd.3 (osd.3) 385 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:52:40.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.800741+0000 osd.2 (osd.2) 311 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.804274+0000 osd.2 (osd.2) 312 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.824957+0000 mon.a (mon.0) 2105 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:38.833265+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.53:0/531060696' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.833366+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e619: 8 total, 8 up, 7 in 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.982787+0000 osd.5 (osd.5) 225 : cluster [DBG] 6.a scrub starts 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:38.993891+0000 osd.5 (osd.5) 226 : cluster [DBG] 6.a scrub ok 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:39.561558+0000 osd.0 (osd.0) 193 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:52:40.236 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:39.564775+0000 osd.0 (osd.0) 194 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.721804+0000 osd.1 (osd.1) 319 : cluster [DBG] 4.f scrub starts 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.723521+0000 osd.1 (osd.1) 320 : cluster [DBG] 4.f scrub ok 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.784652+0000 osd.3 (osd.3) 384 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.788255+0000 osd.3 (osd.3) 385 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.800741+0000 osd.2 (osd.2) 311 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.804274+0000 osd.2 (osd.2) 312 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.824957+0000 mon.a (mon.0) 2105 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:38.833265+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.53:0/531060696' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.833366+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e619: 8 total, 8 up, 7 in 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.982787+0000 osd.5 (osd.5) 225 : cluster [DBG] 6.a scrub starts 2024-02-22T19:52:40.237 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:38.993891+0000 osd.5 (osd.5) 226 : cluster [DBG] 6.a scrub ok 2024-02-22T19:52:40.238 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:39.561558+0000 osd.0 (osd.0) 193 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:52:40.238 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:39.564775+0000 osd.0 (osd.0) 194 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:52:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:39.727361+0000 osd.1 (osd.1) 321 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:52:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:39.728974+0000 osd.1 (osd.1) 322 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:52:41.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:39.985466+0000 mgr.y (mgr.14535) 895 : cluster [DBG] pgmap v1255: 112 pgs: 1 peering, 4 active+remapped+backfill_toofull, 107 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.546640+0000 osd.0 (osd.0) 195 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.549306+0000 osd.0 (osd.0) 196 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:39.727361+0000 osd.1 (osd.1) 321 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:39.728974+0000 osd.1 (osd.1) 322 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:39.985466+0000 mgr.y (mgr.14535) 895 : cluster [DBG] pgmap v1255: 112 pgs: 1 peering, 4 active+remapped+backfill_toofull, 107 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.546640+0000 osd.0 (osd.0) 195 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:52:41.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.549306+0000 osd.0 (osd.0) 196 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:52:41.016 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:41.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:39.727361+0000 osd.1 (osd.1) 321 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:52:41.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:39.728974+0000 osd.1 (osd.1) 322 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:52:41.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:39.985466+0000 mgr.y (mgr.14535) 895 : cluster [DBG] pgmap v1255: 112 pgs: 1 peering, 4 active+remapped+backfill_toofull, 107 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:41.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.546640+0000 osd.0 (osd.0) 195 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:52:41.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.549306+0000 osd.0 (osd.0) 196 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:52:41.860 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.4. 2024-02-22T19:52:42.249 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.742043+0000 osd.3 (osd.3) 386 : cluster [DBG] 3.b scrub starts 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.745495+0000 osd.3 (osd.3) 387 : cluster [DBG] 3.b scrub ok 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.830616+0000 mon.a (mon.0) 2108 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.942149+0000 osd.5 (osd.5) 227 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:40.945730+0000 osd.5 (osd.5) 228 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:41.806580+0000 mon.c (mon.2) 103 : audit [DBG] from='client.? 172.21.15.53:0/1700633245' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:41.806895+0000 mon.a (mon.0) 2109 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:52:41.806+0000 7fe2d794e700 -1 mon.a@0(leader).osd e619 definitely_dead 0 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.742043+0000 osd.3 (osd.3) 386 : cluster [DBG] 3.b scrub starts 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.745495+0000 osd.3 (osd.3) 387 : cluster [DBG] 3.b scrub ok 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.830616+0000 mon.a (mon.0) 2108 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:42.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.942149+0000 osd.5 (osd.5) 227 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:52:42.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:40.945730+0000 osd.5 (osd.5) 228 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:52:42.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:41.806580+0000 mon.c (mon.2) 103 : audit [DBG] from='client.? 172.21.15.53:0/1700633245' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:52:42.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:41.806895+0000 mon.a (mon.0) 2109 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.742043+0000 osd.3 (osd.3) 386 : cluster [DBG] 3.b scrub starts 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.745495+0000 osd.3 (osd.3) 387 : cluster [DBG] 3.b scrub ok 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.830616+0000 mon.a (mon.0) 2108 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.942149+0000 osd.5 (osd.5) 227 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:40.945730+0000 osd.5 (osd.5) 228 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:41.806580+0000 mon.c (mon.2) 103 : audit [DBG] from='client.? 172.21.15.53:0/1700633245' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:52:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:41.806895+0000 mon.a (mon.0) 2109 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-02-22T19:52:42.676 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-02-22T19:52:42.676 INFO:tasks.thrashosds.thrasher:fixing pg num pool device_health_metrics 2024-02-22T19:52:42.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set device_health_metrics pgp_num 11 2024-02-22T19:52:43.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:41.687628+0000 osd.2 (osd.2) 313 : cluster [DBG] 5.a scrub starts 2024-02-22T19:52:43.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:41.690517+0000 osd.2 (osd.2) 314 : cluster [DBG] 5.a scrub ok 2024-02-22T19:52:43.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:41.778836+0000 osd.3 (osd.3) 388 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:41.781724+0000 osd.3 (osd.3) 389 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:41.840116+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:41.840189+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:41.986036+0000 mgr.y (mgr.14535) 896 : cluster [DBG] pgmap v1257: 112 pgs: 4 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 988 B/s rd, 0 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:42.719653+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:41.687628+0000 osd.2 (osd.2) 313 : cluster [DBG] 5.a scrub starts 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:41.690517+0000 osd.2 (osd.2) 314 : cluster [DBG] 5.a scrub ok 2024-02-22T19:52:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:41.778836+0000 osd.3 (osd.3) 388 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:52:43.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:41.781724+0000 osd.3 (osd.3) 389 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:52:43.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:41.840116+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-02-22T19:52:43.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:41.840189+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-22T19:52:43.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:41.986036+0000 mgr.y (mgr.14535) 896 : cluster [DBG] pgmap v1257: 112 pgs: 4 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 988 B/s rd, 0 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:43.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:42.719653+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-22T19:52:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:41.687628+0000 osd.2 (osd.2) 313 : cluster [DBG] 5.a scrub starts 2024-02-22T19:52:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:41.690517+0000 osd.2 (osd.2) 314 : cluster [DBG] 5.a scrub ok 2024-02-22T19:52:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:41.778836+0000 osd.3 (osd.3) 388 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:52:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:41.781724+0000 osd.3 (osd.3) 389 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:52:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:41.840116+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-02-22T19:52:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:41.840189+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-22T19:52:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:41.986036+0000 mgr.y (mgr.14535) 896 : cluster [DBG] pgmap v1257: 112 pgs: 4 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 988 B/s rd, 0 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:42.719653+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-22T19:52:43.725 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:43.725 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:43.750 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:42.718418+0000 osd.1 (osd.1) 323 : cluster [DBG] 6.e starting backfill to osd.4 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:42.719750+0000 osd.4 (osd.4) 209 : cluster [INF] 5.5 continuing backfill to osd.0 from (387'17,596'42] MIN to 596'42 2024-02-22T19:52:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:42.719786+0000 osd.4 (osd.4) 210 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,596'42] MIN to 596'42 2024-02-22T19:52:44.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:42.756647+0000 osd.0 (osd.0) 197 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:42.756695+0000 osd.0 (osd.0) 198 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:43.720267+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:43.722759+0000 osd.7 (osd.7) 340 : cluster [DBG] 5.9 starting backfill to osd.4 from (0'0,0'0] MAX to 601'30 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:42.718418+0000 osd.1 (osd.1) 323 : cluster [DBG] 6.e starting backfill to osd.4 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:42.719750+0000 osd.4 (osd.4) 209 : cluster [INF] 5.5 continuing backfill to osd.0 from (387'17,596'42] MIN to 596'42 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:42.719786+0000 osd.4 (osd.4) 210 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,596'42] MIN to 596'42 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:42.756647+0000 osd.0 (osd.0) 197 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:42.756695+0000 osd.0 (osd.0) 198 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:43.720267+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-22T19:52:44.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:43.722759+0000 osd.7 (osd.7) 340 : cluster [DBG] 5.9 starting backfill to osd.4 from (0'0,0'0] MAX to 601'30 2024-02-22T19:52:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:42.718418+0000 osd.1 (osd.1) 323 : cluster [DBG] 6.e starting backfill to osd.4 from (0'0,0'0] MAX to 85'2 2024-02-22T19:52:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:42.719750+0000 osd.4 (osd.4) 209 : cluster [INF] 5.5 continuing backfill to osd.0 from (387'17,596'42] MIN to 596'42 2024-02-22T19:52:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:42.719786+0000 osd.4 (osd.4) 210 : cluster [INF] 5.5 continuing backfill to osd.1 from (387'17,596'42] MIN to 596'42 2024-02-22T19:52:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:42.756647+0000 osd.0 (osd.0) 197 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:42.756695+0000 osd.0 (osd.0) 198 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:44.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:43.720267+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-22T19:52:44.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:43.722759+0000 osd.7 (osd.7) 340 : cluster [DBG] 5.9 starting backfill to osd.4 from (0'0,0'0] MAX to 601'30 2024-02-22T19:52:45.103 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:52:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:52:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:43.721686+0000 osd.3 (osd.3) 390 : cluster [DBG] 5.0 starting backfill to osd.4 from (0'0,0'0] MAX to 601'25 2024-02-22T19:52:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:43.986402+0000 mgr.y (mgr.14535) 897 : cluster [DBG] pgmap v1260: 112 pgs: 4 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 992 B/s rd, 0 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:44.721363+0000 mon.a (mon.0) 2114 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-22T19:52:45.246 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:43.721686+0000 osd.3 (osd.3) 390 : cluster [DBG] 5.0 starting backfill to osd.4 from (0'0,0'0] MAX to 601'25 2024-02-22T19:52:45.246 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:43.986402+0000 mgr.y (mgr.14535) 897 : cluster [DBG] pgmap v1260: 112 pgs: 4 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 992 B/s rd, 0 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:45.246 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:44.721363+0000 mon.a (mon.0) 2114 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-22T19:52:45.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:43.721686+0000 osd.3 (osd.3) 390 : cluster [DBG] 5.0 starting backfill to osd.4 from (0'0,0'0] MAX to 601'25 2024-02-22T19:52:45.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:43.986402+0000 mgr.y (mgr.14535) 897 : cluster [DBG] pgmap v1260: 112 pgs: 4 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 992 B/s rd, 0 op/s; 9/576 objects misplaced (1.562%); 0 B/s, 1 objects/s recovering 2024-02-22T19:52:45.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:44.721363+0000 mon.a (mon.0) 2114 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-22T19:52:45.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-22T19:52:46.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:52:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:52:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:46.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:45.722845+0000 mon.a (mon.0) 2115 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-22T19:52:46.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:45.749119+0000 osd.1 (osd.1) 324 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:52:46.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:45.749878+0000 osd.1 (osd.1) 325 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:52:46.728 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:46.263317+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.53:0/3295882487' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "11"}]: dispatch 2024-02-22T19:52:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 1 pgp_num to 11 2024-02-22T19:52:47.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:45.722845+0000 mon.a (mon.0) 2115 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-22T19:52:47.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:45.749119+0000 osd.1 (osd.1) 324 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:52:47.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:45.749878+0000 osd.1 (osd.1) 325 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:52:47.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:46.263317+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.53:0/3295882487' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "11"}]: dispatch 2024-02-22T19:52:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:45.722845+0000 mon.a (mon.0) 2115 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-22T19:52:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:45.749119+0000 osd.1 (osd.1) 324 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:52:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:45.749878+0000 osd.1 (osd.1) 325 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:52:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:46.263317+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.53:0/3295882487' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "11"}]: dispatch 2024-02-22T19:52:47.623 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.meta 2024-02-22T19:52:47.624 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd pool set default.rgw.meta pgp_num 22 2024-02-22T19:52:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:45.748118+0000 mgr.y (mgr.14535) 898 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:45.986770+0000 mgr.y (mgr.14535) 899 : cluster [DBG] pgmap v1263: 112 pgs: 1 peering, 1 active+clean+remapped, 1 active+remapped+backfill_wait, 3 active+recovering, 2 active+recovery_wait+degraded, 3 active+remapped+backfill_toofull, 101 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 509 B/s, 9 objects/s recovering 2024-02-22T19:52:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.586677+0000 osd.0 (osd.0) 199 : cluster [DBG] 4.b scrub starts 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.588609+0000 osd.0 (osd.0) 200 : cluster [DBG] 4.b scrub ok 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.591265+0000 osd.2 (osd.2) 315 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.606271+0000 osd.2 (osd.2) 316 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.719790+0000 mon.a (mon.0) 2117 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.719816+0000 mon.a (mon.0) 2118 : cluster [WRN] Health check failed: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded (PG_DEGRADED) 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:46.728267+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? 172.21.15.53:0/3295882487' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "11"}]': finished 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.728314+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.890752+0000 osd.5 (osd.5) 229 : cluster [DBG] 6.d scrub starts 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.011020+0000 osd.5 (osd.5) 230 : cluster [DBG] 6.d scrub ok 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.570746+0000 osd.0 (osd.0) 201 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.571980+0000 osd.0 (osd.0) 202 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:52:48.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.720217+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:45.748118+0000 mgr.y (mgr.14535) 898 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:45.986770+0000 mgr.y (mgr.14535) 899 : cluster [DBG] pgmap v1263: 112 pgs: 1 peering, 1 active+clean+remapped, 1 active+remapped+backfill_wait, 3 active+recovering, 2 active+recovery_wait+degraded, 3 active+remapped+backfill_toofull, 101 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 509 B/s, 9 objects/s recovering 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.586677+0000 osd.0 (osd.0) 199 : cluster [DBG] 4.b scrub starts 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.588609+0000 osd.0 (osd.0) 200 : cluster [DBG] 4.b scrub ok 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.591265+0000 osd.2 (osd.2) 315 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.606271+0000 osd.2 (osd.2) 316 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.719790+0000 mon.a (mon.0) 2117 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.719816+0000 mon.a (mon.0) 2118 : cluster [WRN] Health check failed: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded (PG_DEGRADED) 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:46.728267+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? 172.21.15.53:0/3295882487' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "11"}]': finished 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.728314+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.890752+0000 osd.5 (osd.5) 229 : cluster [DBG] 6.d scrub starts 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.011020+0000 osd.5 (osd.5) 230 : cluster [DBG] 6.d scrub ok 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.570746+0000 osd.0 (osd.0) 201 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:52:48.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.571980+0000 osd.0 (osd.0) 202 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:52:48.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.720217+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-22T19:52:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:45.748118+0000 mgr.y (mgr.14535) 898 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:45.986770+0000 mgr.y (mgr.14535) 899 : cluster [DBG] pgmap v1263: 112 pgs: 1 peering, 1 active+clean+remapped, 1 active+remapped+backfill_wait, 3 active+recovering, 2 active+recovery_wait+degraded, 3 active+remapped+backfill_toofull, 101 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 509 B/s, 9 objects/s recovering 2024-02-22T19:52:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.586677+0000 osd.0 (osd.0) 199 : cluster [DBG] 4.b scrub starts 2024-02-22T19:52:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.588609+0000 osd.0 (osd.0) 200 : cluster [DBG] 4.b scrub ok 2024-02-22T19:52:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.591265+0000 osd.2 (osd.2) 315 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.606271+0000 osd.2 (osd.2) 316 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.719790+0000 mon.a (mon.0) 2117 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.719816+0000 mon.a (mon.0) 2118 : cluster [WRN] Health check failed: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded (PG_DEGRADED) 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:46.728267+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? 172.21.15.53:0/3295882487' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "device_health_metrics", "var": "pgp_num", "val": "11"}]': finished 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.728314+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.890752+0000 osd.5 (osd.5) 229 : cluster [DBG] 6.d scrub starts 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.011020+0000 osd.5 (osd.5) 230 : cluster [DBG] 6.d scrub ok 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.570746+0000 osd.0 (osd.0) 201 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.571980+0000 osd.0 (osd.0) 202 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:52:48.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.720217+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-22T19:52:48.708 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:48.709 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:48.960 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:49.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.945102+0000 osd.7 (osd.7) 341 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:52:49.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:46.965728+0000 osd.7 (osd.7) 342 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:52:49.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.872392+0000 osd.4 (osd.4) 211 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:52:49.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.875833+0000 osd.4 (osd.4) 212 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:52:49.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.945102+0000 osd.7 (osd.7) 341 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:52:49.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:46.965728+0000 osd.7 (osd.7) 342 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:52:49.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.872392+0000 osd.4 (osd.4) 211 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:52:49.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.875833+0000 osd.4 (osd.4) 212 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:52:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.945102+0000 osd.7 (osd.7) 341 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:52:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:46.965728+0000 osd.7 (osd.7) 342 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:52:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.872392+0000 osd.4 (osd.4) 211 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:52:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.875833+0000 osd.4 (osd.4) 212 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:52:49.438 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:49] "GET /metrics HTTP/1.1" 200 174320 "" "Prometheus/2.33.4" 2024-02-22T19:52:49.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-22T19:52:49.738 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.941450+0000 osd.7 (osd.7) 343 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.944765+0000 osd.7 (osd.7) 344 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:47.987287+0000 mgr.y (mgr.14535) 900 : cluster [DBG] pgmap v1266: 112 pgs: 1 peering, 1 active+clean+remapped, 1 active+remapped+backfill_wait, 3 active+recovering, 2 active+recovery_wait+degraded, 3 active+remapped+backfill_toofull, 101 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 509 B/s, 9 objects/s recovering 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:48.579507+0000 osd.2 (osd.2) 317 : cluster [DBG] 3.e deep-scrub starts 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:48.582620+0000 osd.2 (osd.2) 318 : cluster [DBG] 3.e deep-scrub ok 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:48.748000+0000 osd.1 (osd.1) 326 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:48.748964+0000 osd.1 (osd.1) 327 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:48.883135+0000 osd.4 (osd.4) 213 : cluster [DBG] 4.e scrub starts 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:48.884608+0000 osd.4 (osd.4) 214 : cluster [DBG] 4.e scrub ok 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.557065+0000 osd.0 (osd.0) 203 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:52:49.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:49.582637+0000 mon.a (mon.0) 2122 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:50.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.941450+0000 osd.7 (osd.7) 343 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.944765+0000 osd.7 (osd.7) 344 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:47.987287+0000 mgr.y (mgr.14535) 900 : cluster [DBG] pgmap v1266: 112 pgs: 1 peering, 1 active+clean+remapped, 1 active+remapped+backfill_wait, 3 active+recovering, 2 active+recovery_wait+degraded, 3 active+remapped+backfill_toofull, 101 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 509 B/s, 9 objects/s recovering 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:48.579507+0000 osd.2 (osd.2) 317 : cluster [DBG] 3.e deep-scrub starts 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:48.582620+0000 osd.2 (osd.2) 318 : cluster [DBG] 3.e deep-scrub ok 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:48.748000+0000 osd.1 (osd.1) 326 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:48.748964+0000 osd.1 (osd.1) 327 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:48.883135+0000 osd.4 (osd.4) 213 : cluster [DBG] 4.e scrub starts 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:48.884608+0000 osd.4 (osd.4) 214 : cluster [DBG] 4.e scrub ok 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.557065+0000 osd.0 (osd.0) 203 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:52:50.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:49.582637+0000 mon.a (mon.0) 2122 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.941450+0000 osd.7 (osd.7) 343 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:52:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.944765+0000 osd.7 (osd.7) 344 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:52:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:47.987287+0000 mgr.y (mgr.14535) 900 : cluster [DBG] pgmap v1266: 112 pgs: 1 peering, 1 active+clean+remapped, 1 active+remapped+backfill_wait, 3 active+recovering, 2 active+recovery_wait+degraded, 3 active+remapped+backfill_toofull, 101 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 7/576 objects degraded (1.215%); 11/576 objects misplaced (1.910%); 509 B/s, 9 objects/s recovering 2024-02-22T19:52:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:48.579507+0000 osd.2 (osd.2) 317 : cluster [DBG] 3.e deep-scrub starts 2024-02-22T19:52:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:48.582620+0000 osd.2 (osd.2) 318 : cluster [DBG] 3.e deep-scrub ok 2024-02-22T19:52:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:48.748000+0000 osd.1 (osd.1) 326 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:52:50.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:48.748964+0000 osd.1 (osd.1) 327 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:52:50.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:48.883135+0000 osd.4 (osd.4) 213 : cluster [DBG] 4.e scrub starts 2024-02-22T19:52:50.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:48.884608+0000 osd.4 (osd.4) 214 : cluster [DBG] 4.e scrub ok 2024-02-22T19:52:50.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.557065+0000 osd.0 (osd.0) 203 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:52:50.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:49.582637+0000 mon.a (mon.0) 2122 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:50.687 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:50.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.636070+0000 osd.0 (osd.0) 204 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.708838+0000 osd.1 (osd.1) 328 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.806829+0000 osd.1 (osd.1) 329 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.887715+0000 osd.5 (osd.5) 231 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.888738+0000 osd.5 (osd.5) 232 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.636070+0000 osd.0 (osd.0) 204 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.708838+0000 osd.1 (osd.1) 328 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.806829+0000 osd.1 (osd.1) 329 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:52:50.940 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.887715+0000 osd.5 (osd.5) 231 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:52:50.941 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.888738+0000 osd.5 (osd.5) 232 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:52:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.636070+0000 osd.0 (osd.0) 204 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:52:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.708838+0000 osd.1 (osd.1) 328 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:52:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.806829+0000 osd.1 (osd.1) 329 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:52:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.887715+0000 osd.5 (osd.5) 231 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:52:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.888738+0000 osd.5 (osd.5) 232 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:52:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:49.987817+0000 mgr.y (mgr.14535) 901 : cluster [DBG] pgmap v1267: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 971 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 270 B/s, 3 objects/s recovering 2024-02-22T19:52:52.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.599571+0000 osd.0 (osd.0) 205 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.601359+0000 osd.0 (osd.0) 206 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.736439+0000 mon.a (mon.0) 2123 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded) 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.875484+0000 osd.5 (osd.5) 233 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.877702+0000 osd.5 (osd.5) 234 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.934931+0000 osd.6 (osd.6) 174 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:50.935778+0000 osd.6 (osd.6) 175 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:49.987817+0000 mgr.y (mgr.14535) 901 : cluster [DBG] pgmap v1267: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 971 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 270 B/s, 3 objects/s recovering 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.599571+0000 osd.0 (osd.0) 205 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.601359+0000 osd.0 (osd.0) 206 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.736439+0000 mon.a (mon.0) 2123 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded) 2024-02-22T19:52:52.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.875484+0000 osd.5 (osd.5) 233 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:52:52.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.877702+0000 osd.5 (osd.5) 234 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:52:52.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.934931+0000 osd.6 (osd.6) 174 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:52:52.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:50.935778+0000 osd.6 (osd.6) 175 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:49.987817+0000 mgr.y (mgr.14535) 901 : cluster [DBG] pgmap v1267: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 971 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 270 B/s, 3 objects/s recovering 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.599571+0000 osd.0 (osd.0) 205 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.601359+0000 osd.0 (osd.0) 206 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.736439+0000 mon.a (mon.0) 2123 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/576 objects degraded (1.215%), 2 pgs degraded) 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.875484+0000 osd.5 (osd.5) 233 : cluster [DBG] 4.15 scrub starts 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.877702+0000 osd.5 (osd.5) 234 : cluster [DBG] 4.15 scrub ok 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.934931+0000 osd.6 (osd.6) 174 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:52:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:50.935778+0000 osd.6 (osd.6) 175 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:52:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 6 pgp_num to 22 2024-02-22T19:52:52.809 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:52.809 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:52.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:51.582265+0000 osd.0 (osd.0) 207 : cluster [DBG] 6.10 deep-scrub starts 2024-02-22T19:52:52.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:51.597843+0000 osd.0 (osd.0) 208 : cluster [DBG] 6.10 deep-scrub ok 2024-02-22T19:52:52.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:51.673195+0000 osd.1 (osd.1) 330 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:52:52.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:51.674855+0000 osd.1 (osd.1) 331 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:52:52.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:51.976241+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.53:0/143014978' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-22T19:52:52.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:51.976552+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-22T19:52:52.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:51.582265+0000 osd.0 (osd.0) 207 : cluster [DBG] 6.10 deep-scrub starts 2024-02-22T19:52:52.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:51.597843+0000 osd.0 (osd.0) 208 : cluster [DBG] 6.10 deep-scrub ok 2024-02-22T19:52:52.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:51.673195+0000 osd.1 (osd.1) 330 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:52:52.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:51.674855+0000 osd.1 (osd.1) 331 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:52:52.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:51.976241+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.53:0/143014978' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-22T19:52:52.833 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:51.976552+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-22T19:52:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:51.582265+0000 osd.0 (osd.0) 207 : cluster [DBG] 6.10 deep-scrub starts 2024-02-22T19:52:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:51.597843+0000 osd.0 (osd.0) 208 : cluster [DBG] 6.10 deep-scrub ok 2024-02-22T19:52:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:51.673195+0000 osd.1 (osd.1) 330 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:52:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:51.674855+0000 osd.1 (osd.1) 331 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:52:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:51.976241+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.53:0/143014978' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-22T19:52:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:51.976552+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-22T19:52:53.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-22T19:52:54.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:51.988527+0000 mgr.y (mgr.14535) 902 : cluster [DBG] pgmap v1268: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 52 B/s, 2 objects/s recovering 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.570661+0000 osd.2 (osd.2) 319 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.574191+0000 osd.2 (osd.2) 320 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.596096+0000 osd.0 (osd.0) 209 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.596921+0000 osd.0 (osd.0) 210 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:52.752320+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]': finished 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.752391+0000 mon.a (mon.0) 2126 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.804283+0000 osd.4 (osd.4) 215 : cluster [DBG] 1.5 deep-scrub starts 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.807868+0000 osd.4 (osd.4) 216 : cluster [DBG] 1.5 deep-scrub ok 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:53.406766+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.684109+0000 osd.5 (osd.5) 235 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.684197+0000 osd.5 (osd.5) 236 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:54.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:51.988527+0000 mgr.y (mgr.14535) 902 : cluster [DBG] pgmap v1268: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 52 B/s, 2 objects/s recovering 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.570661+0000 osd.2 (osd.2) 319 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.574191+0000 osd.2 (osd.2) 320 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.596096+0000 osd.0 (osd.0) 209 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.596921+0000 osd.0 (osd.0) 210 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:52.752320+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]': finished 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.752391+0000 mon.a (mon.0) 2126 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.804283+0000 osd.4 (osd.4) 215 : cluster [DBG] 1.5 deep-scrub starts 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.807868+0000 osd.4 (osd.4) 216 : cluster [DBG] 1.5 deep-scrub ok 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:53.406766+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.684109+0000 osd.5 (osd.5) 235 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:54.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.684197+0000 osd.5 (osd.5) 236 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:51.988527+0000 mgr.y (mgr.14535) 902 : cluster [DBG] pgmap v1268: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 52 B/s, 2 objects/s recovering 2024-02-22T19:52:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.570661+0000 osd.2 (osd.2) 319 : cluster [DBG] 4.1c scrub starts 2024-02-22T19:52:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.574191+0000 osd.2 (osd.2) 320 : cluster [DBG] 4.1c scrub ok 2024-02-22T19:52:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.596096+0000 osd.0 (osd.0) 209 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:52:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.596921+0000 osd.0 (osd.0) 210 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:52.752320+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]': finished 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.752391+0000 mon.a (mon.0) 2126 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.804283+0000 osd.4 (osd.4) 215 : cluster [DBG] 1.5 deep-scrub starts 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.807868+0000 osd.4 (osd.4) 216 : cluster [DBG] 1.5 deep-scrub ok 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:53.406766+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.684109+0000 osd.5 (osd.5) 235 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:52:54.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.684197+0000 osd.5 (osd.5) 236 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:52:54.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-02-22T19:52:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:54.810 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.954407+0000 osd.7 (osd.7) 345 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:52.957925+0000 osd.7 (osd.7) 346 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.598380+0000 osd.0 (osd.0) 211 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.617714+0000 osd.0 (osd.0) 212 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.774240+0000 osd.4 (osd.4) 217 : cluster [DBG] 4.d scrub starts 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.777721+0000 osd.4 (osd.4) 218 : cluster [DBG] 4.d scrub ok 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.915759+0000 osd.5 (osd.5) 237 : cluster [DBG] 3.c scrub starts 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.916861+0000 osd.5 (osd.5) 238 : cluster [DBG] 3.c scrub ok 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.989047+0000 mgr.y (mgr.14535) 903 : cluster [DBG] pgmap v1270: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 704 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 45 B/s, 1 objects/s recovering 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.408636+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.954407+0000 osd.7 (osd.7) 345 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:52.957925+0000 osd.7 (osd.7) 346 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.598380+0000 osd.0 (osd.0) 211 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:52:54.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.617714+0000 osd.0 (osd.0) 212 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:52:54.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.774240+0000 osd.4 (osd.4) 217 : cluster [DBG] 4.d scrub starts 2024-02-22T19:52:54.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.777721+0000 osd.4 (osd.4) 218 : cluster [DBG] 4.d scrub ok 2024-02-22T19:52:54.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.915759+0000 osd.5 (osd.5) 237 : cluster [DBG] 3.c scrub starts 2024-02-22T19:52:54.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.916861+0000 osd.5 (osd.5) 238 : cluster [DBG] 3.c scrub ok 2024-02-22T19:52:54.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.989047+0000 mgr.y (mgr.14535) 903 : cluster [DBG] pgmap v1270: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 704 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 45 B/s, 1 objects/s recovering 2024-02-22T19:52:54.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.408636+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-22T19:52:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:52:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.954407+0000 osd.7 (osd.7) 345 : cluster [DBG] 2.0 scrub starts 2024-02-22T19:52:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:52.957925+0000 osd.7 (osd.7) 346 : cluster [DBG] 2.0 scrub ok 2024-02-22T19:52:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.598380+0000 osd.0 (osd.0) 211 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:52:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.617714+0000 osd.0 (osd.0) 212 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:52:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.774240+0000 osd.4 (osd.4) 217 : cluster [DBG] 4.d scrub starts 2024-02-22T19:52:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.777721+0000 osd.4 (osd.4) 218 : cluster [DBG] 4.d scrub ok 2024-02-22T19:52:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.915759+0000 osd.5 (osd.5) 237 : cluster [DBG] 3.c scrub starts 2024-02-22T19:52:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.916861+0000 osd.5 (osd.5) 238 : cluster [DBG] 3.c scrub ok 2024-02-22T19:52:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.989047+0000 mgr.y (mgr.14535) 903 : cluster [DBG] pgmap v1270: 112 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 108 active+clean; 5.7 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 704 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%); 45 B/s, 1 objects/s recovering 2024-02-22T19:52:55.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.408636+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-22T19:52:55.598 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:55.852 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:53.928658+0000 osd.7 (osd.7) 347 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.043963+0000 osd.7 (osd.7) 348 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.729836+0000 osd.4 (osd.4) 219 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.734336+0000 osd.4 (osd.4) 220 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.861862+0000 osd.3 (osd.3) 391 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.876813+0000 osd.3 (osd.3) 392 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:53.928658+0000 osd.7 (osd.7) 347 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.043963+0000 osd.7 (osd.7) 348 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.729836+0000 osd.4 (osd.4) 219 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.734336+0000 osd.4 (osd.4) 220 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.861862+0000 osd.3 (osd.3) 391 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:52:55.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.876813+0000 osd.3 (osd.3) 392 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:52:56.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:52:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:53.928658+0000 osd.7 (osd.7) 347 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:52:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.043963+0000 osd.7 (osd.7) 348 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:52:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.729836+0000 osd.4 (osd.4) 219 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:52:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.734336+0000 osd.4 (osd.4) 220 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:52:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.861862+0000 osd.3 (osd.3) 391 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:52:56.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.876813+0000 osd.3 (osd.3) 392 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:52:56.896 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.976982+0000 osd.7 (osd.7) 349 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:54.991056+0000 osd.7 (osd.7) 350 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.641265+0000 osd.0 (osd.0) 213 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.642328+0000 osd.0 (osd.0) 214 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.716363+0000 osd.4 (osd.4) 221 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.719298+0000 osd.4 (osd.4) 222 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:55.758771+0000 mgr.y (mgr.14535) 904 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.885071+0000 osd.3 (osd.3) 393 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.888556+0000 osd.3 (osd.3) 394 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:52:56.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.989679+0000 mgr.y (mgr.14535) 905 : cluster [DBG] pgmap v1272: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%); 41 B/s, 1 objects/s recovering 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.976982+0000 osd.7 (osd.7) 349 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:54.991056+0000 osd.7 (osd.7) 350 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.641265+0000 osd.0 (osd.0) 213 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.642328+0000 osd.0 (osd.0) 214 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.716363+0000 osd.4 (osd.4) 221 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.719298+0000 osd.4 (osd.4) 222 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:55.758771+0000 mgr.y (mgr.14535) 904 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:56.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.885071+0000 osd.3 (osd.3) 393 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:52:56.899 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.888556+0000 osd.3 (osd.3) 394 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:52:56.899 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.989679+0000 mgr.y (mgr.14535) 905 : cluster [DBG] pgmap v1272: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%); 41 B/s, 1 objects/s recovering 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.976982+0000 osd.7 (osd.7) 349 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:54.991056+0000 osd.7 (osd.7) 350 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.641265+0000 osd.0 (osd.0) 213 : cluster [DBG] 3.17 scrub starts 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.642328+0000 osd.0 (osd.0) 214 : cluster [DBG] 3.17 scrub ok 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.716363+0000 osd.4 (osd.4) 221 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.719298+0000 osd.4 (osd.4) 222 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:52:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:55.758771+0000 mgr.y (mgr.14535) 904 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:52:57.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.885071+0000 osd.3 (osd.3) 393 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:52:57.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.888556+0000 osd.3 (osd.3) 394 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:52:57.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.989679+0000 mgr.y (mgr.14535) 905 : cluster [DBG] pgmap v1272: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%); 41 B/s, 1 objects/s recovering 2024-02-22T19:52:57.385 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-02-22T19:52:57.385 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:52:57.761 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a: {} 2024-02-22T19:52:57.775 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b: {} 2024-02-22T19:52:57.786 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c: {} 2024-02-22T19:52:57.787 INFO:teuthology.orchestra.run.smithi053.stderr:mon.a: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-22T19:52:57.787 INFO:teuthology.orchestra.run.smithi053.stderr:mon.b: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-22T19:52:57.787 INFO:teuthology.orchestra.run.smithi053.stderr:mon.c: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.963447+0000 osd.7 (osd.7) 351 : cluster [DBG] 3.f scrub starts 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:55.964727+0000 osd.7 (osd.7) 352 : cluster [DBG] 3.f scrub ok 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:56.601690+0000 osd.2 (osd.2) 321 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:56.604582+0000 osd.2 (osd.2) 322 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:56.807214+0000 osd.5 (osd.5) 239 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:56.810217+0000 osd.5 (osd.5) 240 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:56.888825+0000 osd.3 (osd.3) 395 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:52:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:56.891970+0000 osd.3 (osd.3) 396 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:52:58.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:57.748560+0000 mon.a (mon.0) 2129 : audit [DBG] from='client.? 172.21.15.53:0/3871322506' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.963447+0000 osd.7 (osd.7) 351 : cluster [DBG] 3.f scrub starts 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:55.964727+0000 osd.7 (osd.7) 352 : cluster [DBG] 3.f scrub ok 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:56.601690+0000 osd.2 (osd.2) 321 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:56.604582+0000 osd.2 (osd.2) 322 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:56.807214+0000 osd.5 (osd.5) 239 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:56.810217+0000 osd.5 (osd.5) 240 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:56.888825+0000 osd.3 (osd.3) 395 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:56.891970+0000 osd.3 (osd.3) 396 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:57.748560+0000 mon.a (mon.0) 2129 : audit [DBG] from='client.? 172.21.15.53:0/3871322506' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:52:58.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.963447+0000 osd.7 (osd.7) 351 : cluster [DBG] 3.f scrub starts 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:55.964727+0000 osd.7 (osd.7) 352 : cluster [DBG] 3.f scrub ok 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:56.601690+0000 osd.2 (osd.2) 321 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:56.604582+0000 osd.2 (osd.2) 322 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:56.807214+0000 osd.5 (osd.5) 239 : cluster [DBG] 2.1 deep-scrub starts 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:56.810217+0000 osd.5 (osd.5) 240 : cluster [DBG] 2.1 deep-scrub ok 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:56.888825+0000 osd.3 (osd.3) 395 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:56.891970+0000 osd.3 (osd.3) 396 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:52:58.192 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:57.748560+0000 mon.a (mon.0) 2129 : audit [DBG] from='client.? 172.21.15.53:0/3871322506' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T19:52:58.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 0 1 2024-02-22T19:52:58.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:57.707617+0000 osd.4 (osd.4) 223 : cluster [DBG] 4.c scrub starts 2024-02-22T19:52:58.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:57.711239+0000 osd.4 (osd.4) 224 : cluster [DBG] 4.c scrub ok 2024-02-22T19:52:58.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:57.793370+0000 osd.5 (osd.5) 241 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:52:58.983 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:57.796247+0000 osd.5 (osd.5) 242 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:52:58.984 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:57.990175+0000 mgr.y (mgr.14535) 906 : cluster [DBG] pgmap v1273: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:52:58.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:57.707617+0000 osd.4 (osd.4) 223 : cluster [DBG] 4.c scrub starts 2024-02-22T19:52:58.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:57.711239+0000 osd.4 (osd.4) 224 : cluster [DBG] 4.c scrub ok 2024-02-22T19:52:58.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:57.793370+0000 osd.5 (osd.5) 241 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:52:58.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:57.796247+0000 osd.5 (osd.5) 242 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:52:58.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:57.990175+0000 mgr.y (mgr.14535) 906 : cluster [DBG] pgmap v1273: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:52:59.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-22T19:52:59.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:57.707617+0000 osd.4 (osd.4) 223 : cluster [DBG] 4.c scrub starts 2024-02-22T19:52:59.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:57.711239+0000 osd.4 (osd.4) 224 : cluster [DBG] 4.c scrub ok 2024-02-22T19:52:59.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:57.793370+0000 osd.5 (osd.5) 241 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:52:59.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:57.796247+0000 osd.5 (osd.5) 242 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:52:59.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:57.990175+0000 mgr.y (mgr.14535) 906 : cluster [DBG] pgmap v1273: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:52:59.086 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:52:59.289 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:52:59.595 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:52:59] "GET /metrics HTTP/1.1" 200 174316 "" "Prometheus/2.33.4" 2024-02-22T19:52:59.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:58.620744+0000 osd.2 (osd.2) 323 : cluster [DBG] 6.f deep-scrub starts 2024-02-22T19:52:59.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:58.807216+0000 osd.2 (osd.2) 324 : cluster [DBG] 6.f deep-scrub ok 2024-02-22T19:52:59.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:52:59.590554+0000 mon.a (mon.0) 2130 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:52:59.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:58.620744+0000 osd.2 (osd.2) 323 : cluster [DBG] 6.f deep-scrub starts 2024-02-22T19:52:59.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:58.807216+0000 osd.2 (osd.2) 324 : cluster [DBG] 6.f deep-scrub ok 2024-02-22T19:52:59.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:52:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:52:59.590554+0000 mon.a (mon.0) 2130 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:53:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:58.620744+0000 osd.2 (osd.2) 323 : cluster [DBG] 6.f deep-scrub starts 2024-02-22T19:53:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:58.807216+0000 osd.2 (osd.2) 324 : cluster [DBG] 6.f deep-scrub ok 2024-02-22T19:53:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:52:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:52:59.590554+0000 mon.a (mon.0) 2130 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:53:00.787 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:58.902609+0000 osd.7 (osd.7) 353 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:53:00.787 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:58.905550+0000 osd.7 (osd.7) 354 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:53:00.788 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:52:59.990794+0000 mgr.y (mgr.14535) 907 : cluster [DBG] pgmap v1274: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:00.911 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:58.902609+0000 osd.7 (osd.7) 353 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:53:00.911 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:58.905550+0000 osd.7 (osd.7) 354 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:53:00.911 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:52:59.990794+0000 mgr.y (mgr.14535) 907 : cluster [DBG] pgmap v1274: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:01.201 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:01.201 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:58.902609+0000 osd.7 (osd.7) 353 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:53:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:58.905550+0000 osd.7 (osd.7) 354 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:53:01.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:52:59.990794+0000 mgr.y (mgr.14535) 907 : cluster [DBG] pgmap v1274: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-02-22T19:53:01.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:00.979553+0000 osd.3 (osd.3) 397 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:53:01.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:01.079202+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.53:0/2549481092' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:01.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:01.079524+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:01.969 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:01.086321+0000 osd.3 (osd.3) 398 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:53:01.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:00.979553+0000 osd.3 (osd.3) 397 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:01.079202+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.53:0/2549481092' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:01.079524+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:01.086321+0000 osd.3 (osd.3) 398 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:00.979553+0000 osd.3 (osd.3) 397 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:01.079202+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.53:0/2549481092' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:01.079524+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:02.090 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:01.086321+0000 osd.3 (osd.3) 398 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:53:02.318 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:02.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 0 1 2024-02-22T19:53:02.779 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:03.047 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:01.705975+0000 osd.5 (osd.5) 243 : cluster [DBG] 3.1d deep-scrub starts 2024-02-22T19:53:03.047 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:01.706832+0000 osd.5 (osd.5) 244 : cluster [DBG] 3.1d deep-scrub ok 2024-02-22T19:53:03.047 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:01.794811+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:53:03.048 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:01.794915+0000 mon.a (mon.0) 2133 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-22T19:53:03.048 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:01.991201+0000 mgr.y (mgr.14535) 908 : cluster [DBG] pgmap v1276: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:03.048 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:02.315314+0000 osd.3 (osd.3) 399 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:03.048 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:02.315410+0000 osd.3 (osd.3) 400 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:03.048 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:02.725604+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-22T19:53:03.049 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:01.705975+0000 osd.5 (osd.5) 243 : cluster [DBG] 3.1d deep-scrub starts 2024-02-22T19:53:03.049 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:01.706832+0000 osd.5 (osd.5) 244 : cluster [DBG] 3.1d deep-scrub ok 2024-02-22T19:53:03.049 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:01.794811+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:53:03.049 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:01.794915+0000 mon.a (mon.0) 2133 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-22T19:53:03.050 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:01.991201+0000 mgr.y (mgr.14535) 908 : cluster [DBG] pgmap v1276: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:03.050 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:02.315314+0000 osd.3 (osd.3) 399 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:03.050 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:02.315410+0000 osd.3 (osd.3) 400 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:03.050 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:02.725604+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-22T19:53:03.085 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:01.705975+0000 osd.5 (osd.5) 243 : cluster [DBG] 3.1d deep-scrub starts 2024-02-22T19:53:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:01.706832+0000 osd.5 (osd.5) 244 : cluster [DBG] 3.1d deep-scrub ok 2024-02-22T19:53:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:01.794811+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:53:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:01.794915+0000 mon.a (mon.0) 2133 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-22T19:53:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:01.991201+0000 mgr.y (mgr.14535) 908 : cluster [DBG] pgmap v1276: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:02.315314+0000 osd.3 (osd.3) 399 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:02.315410+0000 osd.3 (osd.3) 400 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:02.725604+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-22T19:53:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:03.994 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:02.927958+0000 osd.7 (osd.7) 355 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:53:04.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:03.002838+0000 osd.7 (osd.7) 356 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:53:04.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:02.927958+0000 osd.7 (osd.7) 355 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:53:04.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:03.002838+0000 osd.7 (osd.7) 356 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:53:04.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:02.927958+0000 osd.7 (osd.7) 355 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:53:04.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:03.002838+0000 osd.7 (osd.7) 356 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:53:04.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-22T19:53:04.892 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:53:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:53:04.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:03.950834+0000 osd.6 (osd.6) 176 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:53:04.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:03.952679+0000 osd.6 (osd.6) 177 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:53:04.895 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-02-22T19:53:04.909 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:05.146 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:03.991685+0000 mgr.y (mgr.14535) 909 : cluster [DBG] pgmap v1278: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:05.146 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:04.744608+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? 172.21.15.53:0/3404910303' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:03.950834+0000 osd.6 (osd.6) 176 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:03.952679+0000 osd.6 (osd.6) 177 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:03.991685+0000 mgr.y (mgr.14535) 909 : cluster [DBG] pgmap v1278: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:04.744608+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? 172.21.15.53:0/3404910303' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:03.950834+0000 osd.6 (osd.6) 176 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:03.952679+0000 osd.6 (osd.6) 177 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:03.991685+0000 mgr.y (mgr.14535) 909 : cluster [DBG] pgmap v1278: 112 pgs: 3 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 3/576 objects misplaced (0.521%) 2024-02-22T19:53:05.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:04.744608+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? 172.21.15.53:0/3404910303' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-22T19:53:05.753 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 1 1 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:04.890922+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? 172.21.15.53:0/3404910303' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:04.890984+0000 mon.a (mon.0) 2137 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:04.960936+0000 osd.6 (osd.6) 178 : cluster [DBG] 3.a deep-scrub starts 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:04.963826+0000 osd.6 (osd.6) 179 : cluster [DBG] 3.a deep-scrub ok 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:05.464187+0000 osd.2 (osd.2) 325 : cluster [DBG] 5.b scrub starts 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:05.467279+0000 osd.2 (osd.2) 326 : cluster [DBG] 5.b scrub ok 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:05.662716+0000 osd.5 (osd.5) 245 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:05.663787+0000 osd.5 (osd.5) 246 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:04.890922+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? 172.21.15.53:0/3404910303' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:53:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:04.890984+0000 mon.a (mon.0) 2137 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-22T19:53:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:04.960936+0000 osd.6 (osd.6) 178 : cluster [DBG] 3.a deep-scrub starts 2024-02-22T19:53:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:04.963826+0000 osd.6 (osd.6) 179 : cluster [DBG] 3.a deep-scrub ok 2024-02-22T19:53:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:05.464187+0000 osd.2 (osd.2) 325 : cluster [DBG] 5.b scrub starts 2024-02-22T19:53:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:05.467279+0000 osd.2 (osd.2) 326 : cluster [DBG] 5.b scrub ok 2024-02-22T19:53:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:05.662716+0000 osd.5 (osd.5) 245 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:53:06.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:05.663787+0000 osd.5 (osd.5) 246 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:53:06.007 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:04.890922+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? 172.21.15.53:0/3404910303' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:04.890984+0000 mon.a (mon.0) 2137 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:04.960936+0000 osd.6 (osd.6) 178 : cluster [DBG] 3.a deep-scrub starts 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:04.963826+0000 osd.6 (osd.6) 179 : cluster [DBG] 3.a deep-scrub ok 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:05.464187+0000 osd.2 (osd.2) 325 : cluster [DBG] 5.b scrub starts 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:05.467279+0000 osd.2 (osd.2) 326 : cluster [DBG] 5.b scrub ok 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:05.662716+0000 osd.5 (osd.5) 245 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:53:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:05.663787+0000 osd.5 (osd.5) 246 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:53:06.044 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:06.520 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:06.520 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:05.760267+0000 mgr.y (mgr.14535) 910 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:05.896603+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-22T19:53:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:05.992036+0000 mgr.y (mgr.14535) 911 : cluster [DBG] pgmap v1281: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 87 B/s, 0 objects/s recovering 2024-02-22T19:53:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.036060+0000 osd.3 (osd.3) 401 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:53:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.039075+0000 osd.3 (osd.3) 402 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:53:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.632153+0000 osd.5 (osd.5) 247 : cluster [DBG] 6.7 deep-scrub starts 2024-02-22T19:53:07.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-22T19:53:07.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:05.760267+0000 mgr.y (mgr.14535) 910 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:07.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:05.896603+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-22T19:53:07.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:05.992036+0000 mgr.y (mgr.14535) 911 : cluster [DBG] pgmap v1281: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 87 B/s, 0 objects/s recovering 2024-02-22T19:53:07.227 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.036060+0000 osd.3 (osd.3) 401 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.039075+0000 osd.3 (osd.3) 402 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.632153+0000 osd.5 (osd.5) 247 : cluster [DBG] 6.7 deep-scrub starts 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:05.760267+0000 mgr.y (mgr.14535) 910 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:05.896603+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:05.992036+0000 mgr.y (mgr.14535) 911 : cluster [DBG] pgmap v1281: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 87 B/s, 0 objects/s recovering 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.036060+0000 osd.3 (osd.3) 401 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.039075+0000 osd.3 (osd.3) 402 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:53:07.228 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.632153+0000 osd.5 (osd.5) 247 : cluster [DBG] 6.7 deep-scrub starts 2024-02-22T19:53:07.462 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.762003+0000 osd.5 (osd.5) 248 : cluster [DBG] 6.7 deep-scrub ok 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.899689+0000 mon.a (mon.0) 2139 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.908472+0000 mon.a (mon.0) 2140 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.942717+0000 osd.6 (osd.6) 180 : cluster [DBG] 4.a scrub starts 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:06.946173+0000 osd.6 (osd.6) 181 : cluster [DBG] 4.a scrub ok 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.131361+0000 osd.7 (osd.7) 357 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.131411+0000 osd.7 (osd.7) 358 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:07.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.726536+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.762003+0000 osd.5 (osd.5) 248 : cluster [DBG] 6.7 deep-scrub ok 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.899689+0000 mon.a (mon.0) 2139 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.908472+0000 mon.a (mon.0) 2140 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.942717+0000 osd.6 (osd.6) 180 : cluster [DBG] 4.a scrub starts 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:06.946173+0000 osd.6 (osd.6) 181 : cluster [DBG] 4.a scrub ok 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.131361+0000 osd.7 (osd.7) 357 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:08.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.131411+0000 osd.7 (osd.7) 358 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:08.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.726536+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.762003+0000 osd.5 (osd.5) 248 : cluster [DBG] 6.7 deep-scrub ok 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.899689+0000 mon.a (mon.0) 2139 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.908472+0000 mon.a (mon.0) 2140 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.942717+0000 osd.6 (osd.6) 180 : cluster [DBG] 4.a scrub starts 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:06.946173+0000 osd.6 (osd.6) 181 : cluster [DBG] 4.a scrub ok 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.131361+0000 osd.7 (osd.7) 357 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.131411+0000 osd.7 (osd.7) 358 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:08.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.726536+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-22T19:53:08.919 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-02-22T19:53:09.241 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:09.241 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.682640+0000 osd.0 (osd.0) 215 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.685693+0000 osd.0 (osd.0) 216 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.717331+0000 osd.4 (osd.4) 225 : cluster [DBG] 6.e scrub starts 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.773088+0000 osd.4 (osd.4) 226 : cluster [DBG] 6.e scrub ok 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.902418+0000 osd.7 (osd.7) 359 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.905363+0000 osd.7 (osd.7) 360 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:07.976636+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? 172.21.15.53:0/1033516367' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.984963+0000 osd.3 (osd.3) 403 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.988025+0000 osd.3 (osd.3) 404 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:53:09.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:07.992494+0000 mgr.y (mgr.14535) 912 : cluster [DBG] pgmap v1284: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 92 B/s, 0 objects/s recovering 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.620927+0000 osd.5 (osd.5) 249 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.622346+0000 osd.5 (osd.5) 250 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.682640+0000 osd.0 (osd.0) 215 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.685693+0000 osd.0 (osd.0) 216 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.717331+0000 osd.4 (osd.4) 225 : cluster [DBG] 6.e scrub starts 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.773088+0000 osd.4 (osd.4) 226 : cluster [DBG] 6.e scrub ok 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.902418+0000 osd.7 (osd.7) 359 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.905363+0000 osd.7 (osd.7) 360 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:07.976636+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? 172.21.15.53:0/1033516367' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.984963+0000 osd.3 (osd.3) 403 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.988025+0000 osd.3 (osd.3) 404 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:53:09.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:07.992494+0000 mgr.y (mgr.14535) 912 : cluster [DBG] pgmap v1284: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 92 B/s, 0 objects/s recovering 2024-02-22T19:53:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.620927+0000 osd.5 (osd.5) 249 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:53:09.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:08 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.622346+0000 osd.5 (osd.5) 250 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.682640+0000 osd.0 (osd.0) 215 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.685693+0000 osd.0 (osd.0) 216 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.717331+0000 osd.4 (osd.4) 225 : cluster [DBG] 6.e scrub starts 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.773088+0000 osd.4 (osd.4) 226 : cluster [DBG] 6.e scrub ok 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.902418+0000 osd.7 (osd.7) 359 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.905363+0000 osd.7 (osd.7) 360 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:07.976636+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? 172.21.15.53:0/1033516367' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.984963+0000 osd.3 (osd.3) 403 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.988025+0000 osd.3 (osd.3) 404 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:07.992494+0000 mgr.y (mgr.14535) 912 : cluster [DBG] pgmap v1284: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 92 B/s, 0 objects/s recovering 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.620927+0000 osd.5 (osd.5) 249 : cluster [DBG] 3.1c scrub starts 2024-02-22T19:53:09.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:08 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.622346+0000 osd.5 (osd.5) 250 : cluster [DBG] 3.1c scrub ok 2024-02-22T19:53:09.668 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:09] "GET /metrics HTTP/1.1" 200 174318 "" "Prometheus/2.33.4" 2024-02-22T19:53:09.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 1 1 2024-02-22T19:53:09.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-22T19:53:09.979 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:08.915361+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? 172.21.15.53:0/1033516367' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:53:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.915441+0000 mon.a (mon.0) 2144 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-22T19:53:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.920485+0000 osd.7 (osd.7) 361 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:53:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.924023+0000 osd.7 (osd.7) 362 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:53:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.958244+0000 osd.6 (osd.6) 182 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:53:10.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:08.959032+0000 osd.6 (osd.6) 183 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:53:10.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:08.915361+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? 172.21.15.53:0/1033516367' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:53:10.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.915441+0000 mon.a (mon.0) 2144 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-22T19:53:10.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.920485+0000 osd.7 (osd.7) 361 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:53:10.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.924023+0000 osd.7 (osd.7) 362 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:53:10.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.958244+0000 osd.6 (osd.6) 182 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:53:10.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:08.959032+0000 osd.6 (osd.6) 183 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:53:10.187 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:08.915361+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? 172.21.15.53:0/1033516367' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:53:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.915441+0000 mon.a (mon.0) 2144 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-22T19:53:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.920485+0000 osd.7 (osd.7) 361 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:53:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.924023+0000 osd.7 (osd.7) 362 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:53:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.958244+0000 osd.6 (osd.6) 182 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:53:10.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:08.959032+0000 osd.6 (osd.6) 183 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:53:11.078 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:09.769190+0000 osd.4 (osd.4) 227 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:53:11.078 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:09.772657+0000 osd.4 (osd.4) 228 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:53:11.078 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:09.993061+0000 mgr.y (mgr.14535) 913 : cluster [DBG] pgmap v1286: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:09.769190+0000 osd.4 (osd.4) 227 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:53:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:09.772657+0000 osd.4 (osd.4) 228 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:53:11.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:09.993061+0000 mgr.y (mgr.14535) 913 : cluster [DBG] pgmap v1286: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:11.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:09.769190+0000 osd.4 (osd.4) 227 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:53:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:09.772657+0000 osd.4 (osd.4) 228 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:53:11.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:09.993061+0000 mgr.y (mgr.14535) 913 : cluster [DBG] pgmap v1286: 112 pgs: 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:11.874 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:11.875 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:12.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-22T19:53:12.731 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-02-22T19:53:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:11.917507+0000 osd.3 (osd.3) 405 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:53:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:11.920420+0000 osd.3 (osd.3) 406 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:53:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:11.993636+0000 mgr.y (mgr.14535) 914 : cluster [DBG] pgmap v1287: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:12.012766+0000 osd.7 (osd.7) 363 : cluster [DBG] 3.13 deep-scrub starts 2024-02-22T19:53:13.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:12.015752+0000 osd.7 (osd.7) 364 : cluster [DBG] 3.13 deep-scrub ok 2024-02-22T19:53:13.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:12.496938+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.53:0/4024951524' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:53:13.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:12.728322+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? 172.21.15.53:0/4024951524' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:53:13.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:12.728392+0000 mon.a (mon.0) 2147 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-22T19:53:13.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:11.917507+0000 osd.3 (osd.3) 405 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:53:13.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:11.920420+0000 osd.3 (osd.3) 406 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:11.993636+0000 mgr.y (mgr.14535) 914 : cluster [DBG] pgmap v1287: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:12.012766+0000 osd.7 (osd.7) 363 : cluster [DBG] 3.13 deep-scrub starts 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:12.015752+0000 osd.7 (osd.7) 364 : cluster [DBG] 3.13 deep-scrub ok 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:12.496938+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.53:0/4024951524' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:12.728322+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? 172.21.15.53:0/4024951524' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:12.728392+0000 mon.a (mon.0) 2147 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:11.917507+0000 osd.3 (osd.3) 405 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:11.920420+0000 osd.3 (osd.3) 406 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:11.993636+0000 mgr.y (mgr.14535) 914 : cluster [DBG] pgmap v1287: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:12.012766+0000 osd.7 (osd.7) 363 : cluster [DBG] 3.13 deep-scrub starts 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:12.015752+0000 osd.7 (osd.7) 364 : cluster [DBG] 3.13 deep-scrub ok 2024-02-22T19:53:13.357 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:12.496938+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.53:0/4024951524' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-22T19:53:13.358 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:12.728322+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? 172.21.15.53:0/4024951524' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-22T19:53:13.358 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:12.728392+0000 mon.a (mon.0) 2147 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-22T19:53:13.379 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 2 1 2024-02-22T19:53:13.661 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:14.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:12.751348+0000 osd.0 (osd.0) 217 : cluster [DBG] 5.5 deep-scrub starts 2024-02-22T19:53:14.191 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:12.763650+0000 osd.0 (osd.0) 218 : cluster [DBG] 5.5 deep-scrub ok 2024-02-22T19:53:14.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:12.751348+0000 osd.0 (osd.0) 217 : cluster [DBG] 5.5 deep-scrub starts 2024-02-22T19:53:14.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:12.763650+0000 osd.0 (osd.0) 218 : cluster [DBG] 5.5 deep-scrub ok 2024-02-22T19:53:14.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:12.751348+0000 osd.0 (osd.0) 217 : cluster [DBG] 5.5 deep-scrub starts 2024-02-22T19:53:14.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:12.763650+0000 osd.0 (osd.0) 218 : cluster [DBG] 5.5 deep-scrub ok 2024-02-22T19:53:14.424 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:14.424 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:15.029 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:53:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:53:15.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-22T19:53:15.332 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:13.835324+0000 osd.3 (osd.3) 407 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:13.838866+0000 osd.3 (osd.3) 408 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:13.988338+0000 osd.6 (osd.6) 184 : cluster [DBG] 3.10 deep-scrub starts 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:13.989190+0000 osd.6 (osd.6) 185 : cluster [DBG] 3.10 deep-scrub ok 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:13.994139+0000 mgr.y (mgr.14535) 915 : cluster [DBG] pgmap v1289: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:14.768198+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:14.802095+0000 osd.4 (osd.4) 229 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:14.802161+0000 osd.4 (osd.4) 230 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:14.814435+0000 osd.4 (osd.4) 231 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:53:15.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:14.817314+0000 osd.4 (osd.4) 232 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:53:15.403 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:13.835324+0000 osd.3 (osd.3) 407 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:13.838866+0000 osd.3 (osd.3) 408 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:13.988338+0000 osd.6 (osd.6) 184 : cluster [DBG] 3.10 deep-scrub starts 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:13.989190+0000 osd.6 (osd.6) 185 : cluster [DBG] 3.10 deep-scrub ok 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:13.994139+0000 mgr.y (mgr.14535) 915 : cluster [DBG] pgmap v1289: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:14.768198+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:14.802095+0000 osd.4 (osd.4) 229 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:14.802161+0000 osd.4 (osd.4) 230 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:14.814435+0000 osd.4 (osd.4) 231 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:14.817314+0000 osd.4 (osd.4) 232 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:13.835324+0000 osd.3 (osd.3) 407 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:13.838866+0000 osd.3 (osd.3) 408 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:13.988338+0000 osd.6 (osd.6) 184 : cluster [DBG] 3.10 deep-scrub starts 2024-02-22T19:53:15.404 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:13.989190+0000 osd.6 (osd.6) 185 : cluster [DBG] 3.10 deep-scrub ok 2024-02-22T19:53:15.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:13.994139+0000 mgr.y (mgr.14535) 915 : cluster [DBG] pgmap v1289: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:15.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:14.768198+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:53:15.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:14.802095+0000 osd.4 (osd.4) 229 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:15.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:14.802161+0000 osd.4 (osd.4) 230 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:15.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:14.814435+0000 osd.4 (osd.4) 231 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:53:15.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:14.817314+0000 osd.4 (osd.4) 232 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:53:15.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:53:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:53:16.321 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:14.711172+0000 osd.0 (osd.0) 219 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:14.713140+0000 osd.0 (osd.0) 220 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:15.058680+0000 mon.a (mon.0) 2149 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:15.306275+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:15.306913+0000 mon.a (mon.0) 2151 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:15.311813+0000 mon.a (mon.0) 2152 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:15.820489+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.53:0/295958904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:15.820760+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:16.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:14.711172+0000 osd.0 (osd.0) 219 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:14.713140+0000 osd.0 (osd.0) 220 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:15.058680+0000 mon.a (mon.0) 2149 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:15.306275+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:15.306913+0000 mon.a (mon.0) 2151 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:15.311813+0000 mon.a (mon.0) 2152 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:15.820489+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.53:0/295958904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:16.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:15.820760+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:14.711172+0000 osd.0 (osd.0) 219 : cluster [DBG] 4.16 scrub starts 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:14.713140+0000 osd.0 (osd.0) 220 : cluster [DBG] 4.16 scrub ok 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:15.058680+0000 mon.a (mon.0) 2149 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:15.306275+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:15.306913+0000 mon.a (mon.0) 2151 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:15.311813+0000 mon.a (mon.0) 2152 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:15.820489+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.53:0/295958904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:16.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:15.820760+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:16.944 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:16.945 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:17.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 2 1 2024-02-22T19:53:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:15.763296+0000 mgr.y (mgr.14535) 916 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:15.994574+0000 mgr.y (mgr.14535) 917 : cluster [DBG] pgmap v1291: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:16.316796+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:53:17.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:16.316900+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-22T19:53:17.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:15.763296+0000 mgr.y (mgr.14535) 916 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:15.994574+0000 mgr.y (mgr.14535) 917 : cluster [DBG] pgmap v1291: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:16.316796+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:53:17.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:16.316900+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-22T19:53:17.550 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:15.763296+0000 mgr.y (mgr.14535) 916 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:17.550 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:15.994574+0000 mgr.y (mgr.14535) 917 : cluster [DBG] pgmap v1291: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:17.550 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:16.316796+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:53:17.550 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:16.316900+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-22T19:53:17.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-22T19:53:17.848 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:16.939391+0000 osd.7 (osd.7) 365 : cluster [DBG] 1.9 deep-scrub starts 2024-02-22T19:53:18.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:16.942706+0000 osd.7 (osd.7) 366 : cluster [DBG] 1.9 deep-scrub ok 2024-02-22T19:53:18.687 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:16.939391+0000 osd.7 (osd.7) 365 : cluster [DBG] 1.9 deep-scrub starts 2024-02-22T19:53:18.687 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:16.942706+0000 osd.7 (osd.7) 366 : cluster [DBG] 1.9 deep-scrub ok 2024-02-22T19:53:18.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:16.939391+0000 osd.7 (osd.7) 365 : cluster [DBG] 1.9 deep-scrub starts 2024-02-22T19:53:18.687 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:16.942706+0000 osd.7 (osd.7) 366 : cluster [DBG] 1.9 deep-scrub ok 2024-02-22T19:53:19.487 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:19.487 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:17.995059+0000 mgr.y (mgr.14535) 918 : cluster [DBG] pgmap v1293: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:18.210377+0000 osd.2 (osd.2) 327 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:18.210428+0000 osd.2 (osd.2) 328 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:18.322387+0000 mon.a (mon.0) 2156 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-22T19:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:18.846277+0000 osd.4 (osd.4) 233 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:18.849882+0000 osd.4 (osd.4) 234 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:53:19.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:19] "GET /metrics HTTP/1.1" 200 174318 "" "Prometheus/2.33.4" 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:17.995059+0000 mgr.y (mgr.14535) 918 : cluster [DBG] pgmap v1293: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:18.210377+0000 osd.2 (osd.2) 327 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:18.210428+0000 osd.2 (osd.2) 328 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:18.322387+0000 mon.a (mon.0) 2156 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:18.846277+0000 osd.4 (osd.4) 233 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:18.849882+0000 osd.4 (osd.4) 234 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:17.995059+0000 mgr.y (mgr.14535) 918 : cluster [DBG] pgmap v1293: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:18.210377+0000 osd.2 (osd.2) 327 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:18.210428+0000 osd.2 (osd.2) 328 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:18.322387+0000 mon.a (mon.0) 2156 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:18.846277+0000 osd.4 (osd.4) 233 : cluster [DBG] 4.9 scrub starts 2024-02-22T19:53:19.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:18.849882+0000 osd.4 (osd.4) 234 : cluster [DBG] 4.9 scrub ok 2024-02-22T19:53:20.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-22T19:53:20.333 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-02-22T19:53:20.443 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:19.364912+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.53:0/1113826950' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:20.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:19.365287+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:20.703 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:19.364912+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.53:0/1113826950' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:20.703 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:19.365287+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:20.704 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:19.364912+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.53:0/1113826950' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:20.704 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:19.365287+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-22T19:53:21.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 3 1 2024-02-22T19:53:21.458 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:19.867410+0000 osd.7 (osd.7) 367 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:53:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:19.995677+0000 mgr.y (mgr.14535) 919 : cluster [DBG] pgmap v1295: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:20.024102+0000 osd.7 (osd.7) 368 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:53:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:20.328774+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:53:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:20.328885+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-22T19:53:21.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:20.903341+0000 osd.4 (osd.4) 235 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:20.906224+0000 osd.4 (osd.4) 236 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:19.867410+0000 osd.7 (osd.7) 367 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:19.995677+0000 mgr.y (mgr.14535) 919 : cluster [DBG] pgmap v1295: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:20.024102+0000 osd.7 (osd.7) 368 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:20.328774+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:20.328885+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:20.903341+0000 osd.4 (osd.4) 235 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:53:21.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:20.906224+0000 osd.4 (osd.4) 236 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:53:21.544 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:19.867410+0000 osd.7 (osd.7) 367 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:53:21.544 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:19.995677+0000 mgr.y (mgr.14535) 919 : cluster [DBG] pgmap v1295: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:21.544 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:20.024102+0000 osd.7 (osd.7) 368 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:53:21.544 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:20.328774+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-22T19:53:21.544 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:20.328885+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-22T19:53:21.545 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:20.903341+0000 osd.4 (osd.4) 235 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:53:21.545 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:20.906224+0000 osd.4 (osd.4) 236 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:53:22.024 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:22.024 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:22.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:20.877797+0000 osd.7 (osd.7) 369 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:53:22.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:20.881131+0000 osd.7 (osd.7) 370 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:53:22.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-22T19:53:22.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:20.877797+0000 osd.7 (osd.7) 369 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:53:22.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:20.881131+0000 osd.7 (osd.7) 370 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:53:22.717 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:20.877797+0000 osd.7 (osd.7) 369 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:53:22.717 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:20.881131+0000 osd.7 (osd.7) 370 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:53:22.903 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.3 to 1 (10000) 2024-02-22T19:53:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:21.996303+0000 mgr.y (mgr.14535) 920 : cluster [DBG] pgmap v1297: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:22.738336+0000 osd.3 (osd.3) 409 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:22.741983+0000 osd.3 (osd.3) 410 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:23.402763+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? 172.21.15.53:0/1053219410' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:21.996303+0000 mgr.y (mgr.14535) 920 : cluster [DBG] pgmap v1297: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:22.738336+0000 osd.3 (osd.3) 409 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:22.741983+0000 osd.3 (osd.3) 410 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:53:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:23.402763+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? 172.21.15.53:0/1053219410' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:24.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:21.996303+0000 mgr.y (mgr.14535) 920 : cluster [DBG] pgmap v1297: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:24.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:22.738336+0000 osd.3 (osd.3) 409 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:53:24.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:22.741983+0000 osd.3 (osd.3) 410 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:53:24.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:23.402763+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? 172.21.15.53:0/1053219410' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:24.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 3 1 2024-02-22T19:53:24.573 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:24.573 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:24.739 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:23.732391+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? 172.21.15.53:0/1053219410' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:53:24.739 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:23.732470+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-22T19:53:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:23.732391+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? 172.21.15.53:0/1053219410' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:53:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:23.732470+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-22T19:53:24.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:53:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:53:25.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:23.732391+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? 172.21.15.53:0/1053219410' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:53:25.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:23.732470+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-22T19:53:25.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-22T19:53:25.512 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:26.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:23.840009+0000 osd.7 (osd.7) 371 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:53:26.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:23.843224+0000 osd.7 (osd.7) 372 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:23.996820+0000 mgr.y (mgr.14535) 921 : cluster [DBG] pgmap v1299: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:24.716248+0000 osd.3 (osd.3) 411 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:24.719822+0000 osd.3 (osd.3) 412 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:23.840009+0000 osd.7 (osd.7) 371 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:23.843224+0000 osd.7 (osd.7) 372 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:23.996820+0000 mgr.y (mgr.14535) 921 : cluster [DBG] pgmap v1299: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:24.716248+0000 osd.3 (osd.3) 411 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:53:26.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:24.719822+0000 osd.3 (osd.3) 412 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:53:26.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:53:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:53:26.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:23.840009+0000 osd.7 (osd.7) 371 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:53:26.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:23.843224+0000 osd.7 (osd.7) 372 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:53:26.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:23.996820+0000 mgr.y (mgr.14535) 921 : cluster [DBG] pgmap v1299: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:26.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:24.716248+0000 osd.3 (osd.3) 411 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:53:26.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:24.719822+0000 osd.3 (osd.3) 412 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:53:26.750 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-02-22T19:53:26.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:25.788839+0000 osd.7 (osd.7) 373 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:53:26.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:25.791727+0000 osd.7 (osd.7) 374 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:53:26.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:25.920661+0000 osd.4 (osd.4) 237 : cluster [DBG] 4.13 deep-scrub starts 2024-02-22T19:53:26.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:25.925875+0000 osd.4 (osd.4) 238 : cluster [DBG] 4.13 deep-scrub ok 2024-02-22T19:53:26.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:26.717662+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.53:0/1645293394' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:26.927 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:26.717991+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:25.788839+0000 osd.7 (osd.7) 373 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:25.791727+0000 osd.7 (osd.7) 374 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:25.920661+0000 osd.4 (osd.4) 237 : cluster [DBG] 4.13 deep-scrub starts 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:25.925875+0000 osd.4 (osd.4) 238 : cluster [DBG] 4.13 deep-scrub ok 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:26.717662+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.53:0/1645293394' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:26.717991+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:25.788839+0000 osd.7 (osd.7) 373 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:25.791727+0000 osd.7 (osd.7) 374 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:25.920661+0000 osd.4 (osd.4) 237 : cluster [DBG] 4.13 deep-scrub starts 2024-02-22T19:53:27.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:25.925875+0000 osd.4 (osd.4) 238 : cluster [DBG] 4.13 deep-scrub ok 2024-02-22T19:53:27.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:26.717662+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.53:0/1645293394' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:27.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:26.717991+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-22T19:53:27.178 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:27.178 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:27.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 4 1 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:25.768930+0000 mgr.y (mgr.14535) 922 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:25.997481+0000 mgr.y (mgr.14535) 923 : cluster [DBG] pgmap v1300: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:26.651783+0000 osd.3 (osd.3) 413 : cluster [DBG] 6.c scrub starts 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:26.677998+0000 osd.3 (osd.3) 414 : cluster [DBG] 6.c scrub ok 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:26.746372+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:26.746450+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:26.752011+0000 mon.a (mon.0) 2166 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-22T19:53:27.756 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:26.752221+0000 mon.a (mon.0) 2167 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-22T19:53:27.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-22T19:53:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:25.768930+0000 mgr.y (mgr.14535) 922 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:25.997481+0000 mgr.y (mgr.14535) 923 : cluster [DBG] pgmap v1300: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:26.651783+0000 osd.3 (osd.3) 413 : cluster [DBG] 6.c scrub starts 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:26.677998+0000 osd.3 (osd.3) 414 : cluster [DBG] 6.c scrub ok 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:26.746372+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:26.746450+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:26.752011+0000 mon.a (mon.0) 2166 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-22T19:53:28.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:26.752221+0000 mon.a (mon.0) 2167 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-22T19:53:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.b/config 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:25.768930+0000 mgr.y (mgr.14535) 922 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:25.997481+0000 mgr.y (mgr.14535) 923 : cluster [DBG] pgmap v1300: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:26.651783+0000 osd.3 (osd.3) 413 : cluster [DBG] 6.c scrub starts 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:26.677998+0000 osd.3 (osd.3) 414 : cluster [DBG] 6.c scrub ok 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:26.746372+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:26.746450+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:26.752011+0000 mon.a (mon.0) 2166 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-22T19:53:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:26.752221+0000 mon.a (mon.0) 2167 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:27.747917+0000 mon.a (mon.0) 2168 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:27.748025+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:27.748094+0000 mon.a (mon.0) 2170 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:27.806566+0000 osd.7 (osd.7) 375 : cluster [DBG] 3.d scrub starts 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:27.810049+0000 osd.7 (osd.7) 376 : cluster [DBG] 3.d scrub ok 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:27.883180+0000 osd.6 (osd.6) 186 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:27.883231+0000 osd.6 (osd.6) 187 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:27.997935+0000 mgr.y (mgr.14535) 924 : cluster [DBG] pgmap v1303: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:28.992 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:27.747917+0000 mon.a (mon.0) 2168 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:27.748025+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:27.748094+0000 mon.a (mon.0) 2170 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:27.806566+0000 osd.7 (osd.7) 375 : cluster [DBG] 3.d scrub starts 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:27.810049+0000 osd.7 (osd.7) 376 : cluster [DBG] 3.d scrub ok 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:27.883180+0000 osd.6 (osd.6) 186 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:27.883231+0000 osd.6 (osd.6) 187 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:28.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:27.997935+0000 mgr.y (mgr.14535) 924 : cluster [DBG] pgmap v1303: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:27.747917+0000 mon.a (mon.0) 2168 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:27.748025+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:27.748094+0000 mon.a (mon.0) 2170 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:27.806566+0000 osd.7 (osd.7) 375 : cluster [DBG] 3.d scrub starts 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:27.810049+0000 osd.7 (osd.7) 376 : cluster [DBG] 3.d scrub ok 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:27.883180+0000 osd.6 (osd.6) 186 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:29.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:27.883231+0000 osd.6 (osd.6) 187 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:29.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:27.997935+0000 mgr.y (mgr.14535) 924 : cluster [DBG] pgmap v1303: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%) 2024-02-22T19:53:29.698 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-02-22T19:53:29.698 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-22T19:53:29.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:53:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:29] "GET /metrics HTTP/1.1" 200 174309 "" "Prometheus/2.33.4" 2024-02-22T19:53:29.768 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-02-22T19:53:30.042 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:28.763208+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-22T19:53:30.042 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:29.684277+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.53:0/625156802' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:53:30.118 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:28.763208+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-22T19:53:30.118 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:29.684277+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.53:0/625156802' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:53:30.119 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:28.763208+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-22T19:53:30.119 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:29.684277+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.53:0/625156802' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:53:30.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 4 1 2024-02-22T19:53:30.856 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:29.565856+0000 osd.1 (osd.1) 332 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:29.565912+0000 osd.1 (osd.1) 333 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:29.765199+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.53:0/625156802' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:53:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:29.765261+0000 mon.a (mon.0) 2174 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-22T19:53:31.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:29.998348+0000 mgr.y (mgr.14535) 925 : cluster [DBG] pgmap v1306: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%) 2024-02-22T19:53:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:29.565856+0000 osd.1 (osd.1) 332 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:29.565912+0000 osd.1 (osd.1) 333 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:29.765199+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.53:0/625156802' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:53:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:29.765261+0000 mon.a (mon.0) 2174 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-22T19:53:31.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:29.998348+0000 mgr.y (mgr.14535) 925 : cluster [DBG] pgmap v1306: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%) 2024-02-22T19:53:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:29.565856+0000 osd.1 (osd.1) 332 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:29.565912+0000 osd.1 (osd.1) 333 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:29.765199+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.53:0/625156802' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:53:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:29.765261+0000 mon.a (mon.0) 2174 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-22T19:53:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:29.998348+0000 mgr.y (mgr.14535) 925 : cluster [DBG] pgmap v1306: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%) 2024-02-22T19:53:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:30.762830+0000 mon.a (mon.0) 2175 : cluster [WRN] Health check failed: Degraded data redundancy: 8/576 objects degraded (1.389%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:53:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:30.806506+0000 osd.4 (osd.4) 239 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:53:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:30.827115+0000 osd.4 (osd.4) 240 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:53:32.105 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:30.762830+0000 mon.a (mon.0) 2175 : cluster [WRN] Health check failed: Degraded data redundancy: 8/576 objects degraded (1.389%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:53:32.105 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:30.806506+0000 osd.4 (osd.4) 239 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:53:32.105 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:30.827115+0000 osd.4 (osd.4) 240 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:53:32.106 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:30.762830+0000 mon.a (mon.0) 2175 : cluster [WRN] Health check failed: Degraded data redundancy: 8/576 objects degraded (1.389%), 1 pg degraded (PG_DEGRADED) 2024-02-22T19:53:32.106 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:30.806506+0000 osd.4 (osd.4) 239 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:53:32.106 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:30.827115+0000 osd.4 (osd.4) 240 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:53:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:31.712479+0000 osd.3 (osd.3) 415 : cluster [DBG] 6.b scrub starts 2024-02-22T19:53:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:31.864712+0000 osd.3 (osd.3) 416 : cluster [DBG] 6.b scrub ok 2024-02-22T19:53:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:31.998863+0000 mgr.y (mgr.14535) 926 : cluster [DBG] pgmap v1307: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 974 B/s rd, 0 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%); 20 B/s, 1 objects/s recovering 2024-02-22T19:53:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:32.733115+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-22T19:53:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:31.712479+0000 osd.3 (osd.3) 415 : cluster [DBG] 6.b scrub starts 2024-02-22T19:53:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:31.864712+0000 osd.3 (osd.3) 416 : cluster [DBG] 6.b scrub ok 2024-02-22T19:53:33.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:31.998863+0000 mgr.y (mgr.14535) 926 : cluster [DBG] pgmap v1307: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 974 B/s rd, 0 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%); 20 B/s, 1 objects/s recovering 2024-02-22T19:53:33.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:32.733115+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-22T19:53:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:31.712479+0000 osd.3 (osd.3) 415 : cluster [DBG] 6.b scrub starts 2024-02-22T19:53:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:31.864712+0000 osd.3 (osd.3) 416 : cluster [DBG] 6.b scrub ok 2024-02-22T19:53:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:31.998863+0000 mgr.y (mgr.14535) 926 : cluster [DBG] pgmap v1307: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 974 B/s rd, 0 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%); 20 B/s, 1 objects/s recovering 2024-02-22T19:53:33.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:32.733115+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-22T19:53:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-02-22T19:53:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:32.803010+0000 osd.4 (osd.4) 241 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:53:34.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:32.806688+0000 osd.4 (osd.4) 242 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:53:34.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:32.827273+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? 172.21.15.53:0/3321612139' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:53:34.199 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:32.803010+0000 osd.4 (osd.4) 241 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:53:34.199 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:32.806688+0000 osd.4 (osd.4) 242 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:53:34.199 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:32.827273+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? 172.21.15.53:0/3321612139' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:53:34.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:32.803010+0000 osd.4 (osd.4) 241 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:53:34.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:32.806688+0000 osd.4 (osd.4) 242 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:53:34.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:32.827273+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? 172.21.15.53:0/3321612139' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-22T19:53:34.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 5 1 2024-02-22T19:53:34.896 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:35.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:33.775578+0000 osd.3 (osd.3) 417 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:53:35.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:33.777792+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? 172.21.15.53:0/3321612139' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:53:35.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:33.777877+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-22T19:53:35.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:33.779820+0000 osd.3 (osd.3) 418 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:33.785492+0000 osd.4 (osd.4) 243 : cluster [DBG] 3.9 deep-scrub starts 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:33.788572+0000 osd.4 (osd.4) 244 : cluster [DBG] 3.9 deep-scrub ok 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:33.999408+0000 mgr.y (mgr.14535) 927 : cluster [DBG] pgmap v1310: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 977 B/s rd, 0 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%); 20 B/s, 1 objects/s recovering 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:33.775578+0000 osd.3 (osd.3) 417 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:33.777792+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? 172.21.15.53:0/3321612139' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:33.777877+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:33.779820+0000 osd.3 (osd.3) 418 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:33.785492+0000 osd.4 (osd.4) 243 : cluster [DBG] 3.9 deep-scrub starts 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:33.788572+0000 osd.4 (osd.4) 244 : cluster [DBG] 3.9 deep-scrub ok 2024-02-22T19:53:35.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:33.999408+0000 mgr.y (mgr.14535) 927 : cluster [DBG] pgmap v1310: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 977 B/s rd, 0 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%); 20 B/s, 1 objects/s recovering 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:33.775578+0000 osd.3 (osd.3) 417 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:33.777792+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? 172.21.15.53:0/3321612139' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:33.777877+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:33.779820+0000 osd.3 (osd.3) 418 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:33.785492+0000 osd.4 (osd.4) 243 : cluster [DBG] 3.9 deep-scrub starts 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:33.788572+0000 osd.4 (osd.4) 244 : cluster [DBG] 3.9 deep-scrub ok 2024-02-22T19:53:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:33.999408+0000 mgr.y (mgr.14535) 927 : cluster [DBG] pgmap v1310: 112 pgs: 1 activating+degraded, 2 active+remapped+backfill_toofull, 109 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 977 B/s rd, 0 op/s; 8/576 objects degraded (1.389%); 2/576 objects misplaced (0.347%); 20 B/s, 1 objects/s recovering 2024-02-22T19:53:36.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:53:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:53:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:35.776398+0000 mgr.y (mgr.14535) 928 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:35.790565+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-22T19:53:37.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:35.999900+0000 mgr.y (mgr.14535) 929 : cluster [DBG] pgmap v1312: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%); 36 B/s, 2 objects/s recovering 2024-02-22T19:53:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:35.776398+0000 mgr.y (mgr.14535) 928 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:35.790565+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-22T19:53:37.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:35.999900+0000 mgr.y (mgr.14535) 929 : cluster [DBG] pgmap v1312: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%); 36 B/s, 2 objects/s recovering 2024-02-22T19:53:37.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:35.776398+0000 mgr.y (mgr.14535) 928 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:37.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:35.790565+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-22T19:53:37.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:35.999900+0000 mgr.y (mgr.14535) 929 : cluster [DBG] pgmap v1312: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 2/576 objects misplaced (0.347%); 36 B/s, 2 objects/s recovering 2024-02-22T19:53:37.742 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-02-22T19:53:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:36.787605+0000 mon.a (mon.0) 2181 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/576 objects degraded (1.389%), 1 pg degraded) 2024-02-22T19:53:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:36.795417+0000 mon.a (mon.0) 2182 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-22T19:53:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:36.880417+0000 mon.a (mon.0) 2183 : audit [INF] from='client.? 172.21.15.53:0/1914583671' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:37.736464+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? 172.21.15.53:0/1914583671' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:53:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:37.736557+0000 mon.a (mon.0) 2185 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-22T19:53:38.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:36.787605+0000 mon.a (mon.0) 2181 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/576 objects degraded (1.389%), 1 pg degraded) 2024-02-22T19:53:38.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:36.795417+0000 mon.a (mon.0) 2182 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-22T19:53:38.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:36.880417+0000 mon.a (mon.0) 2183 : audit [INF] from='client.? 172.21.15.53:0/1914583671' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:38.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:37.736464+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? 172.21.15.53:0/1914583671' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:53:38.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:37.736557+0000 mon.a (mon.0) 2185 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-22T19:53:38.204 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:36.787605+0000 mon.a (mon.0) 2181 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/576 objects degraded (1.389%), 1 pg degraded) 2024-02-22T19:53:38.204 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:36.795417+0000 mon.a (mon.0) 2182 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-22T19:53:38.204 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:36.880417+0000 mon.a (mon.0) 2183 : audit [INF] from='client.? 172.21.15.53:0/1914583671' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:38.204 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:37.736464+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? 172.21.15.53:0/1914583671' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:53:38.204 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:37.736557+0000 mon.a (mon.0) 2185 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-22T19:53:38.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 5 1 2024-02-22T19:53:38.889 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:39.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:38.000279+0000 mgr.y (mgr.14535) 930 : cluster [DBG] pgmap v1315: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 26 B/s, 1 objects/s recovering 2024-02-22T19:53:39.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:38.000279+0000 mgr.y (mgr.14535) 930 : cluster [DBG] pgmap v1315: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 26 B/s, 1 objects/s recovering 2024-02-22T19:53:39.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:38.000279+0000 mgr.y (mgr.14535) 930 : cluster [DBG] pgmap v1315: 112 pgs: 2 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2/576 objects misplaced (0.347%); 26 B/s, 1 objects/s recovering 2024-02-22T19:53:39.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:53:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:39] "GET /metrics HTTP/1.1" 200 174312 "" "Prometheus/2.33.4" 2024-02-22T19:53:40.483 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:40.052122+0000 mon.a (mon.0) 2186 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:53:40.484 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:40.052122+0000 mon.a (mon.0) 2186 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:53:40.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:40.052122+0000 mon.a (mon.0) 2186 : cluster [WRN] Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-22T19:53:41.112 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-02-22T19:53:41.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:40.000969+0000 mgr.y (mgr.14535) 931 : cluster [DBG] pgmap v1316: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%); 18 B/s, 1 objects/s recovering 2024-02-22T19:53:41.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:40.953242+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.53:0/3833584672' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:41.492 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:40.953620+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:41.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:40.000969+0000 mgr.y (mgr.14535) 931 : cluster [DBG] pgmap v1316: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%); 18 B/s, 1 objects/s recovering 2024-02-22T19:53:41.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:40.953242+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.53:0/3833584672' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:41.493 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:40.953620+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:40.000969+0000 mgr.y (mgr.14535) 931 : cluster [DBG] pgmap v1316: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%); 18 B/s, 1 objects/s recovering 2024-02-22T19:53:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:40.953242+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.53:0/3833584672' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:40.953620+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-22T19:53:41.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 6 1 2024-02-22T19:53:42.086 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:40.776506+0000 osd.0 (osd.0) 221 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:53:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:40.789769+0000 osd.0 (osd.0) 222 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:53:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:41.108602+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:53:42.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:41.108684+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-22T19:53:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:40.776506+0000 osd.0 (osd.0) 221 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:53:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:40.789769+0000 osd.0 (osd.0) 222 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:53:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:41.108602+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:53:42.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:41.108684+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-22T19:53:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:40.776506+0000 osd.0 (osd.0) 221 : cluster [DBG] 5.3 scrub starts 2024-02-22T19:53:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:40.789769+0000 osd.0 (osd.0) 222 : cluster [DBG] 5.3 scrub ok 2024-02-22T19:53:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:41.108602+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-22T19:53:42.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:41.108684+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:42.001342+0000 mgr.y (mgr.14535) 932 : cluster [DBG] pgmap v1318: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%); 0 B/s, 0 objects/s recovering 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:42.737958+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:42.955133+0000 osd.4 (osd.4) 245 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:42.958124+0000 osd.4 (osd.4) 246 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:42.001342+0000 mgr.y (mgr.14535) 932 : cluster [DBG] pgmap v1318: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%); 0 B/s, 0 objects/s recovering 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:42.737958+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:42.955133+0000 osd.4 (osd.4) 245 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:53:43.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:42.958124+0000 osd.4 (osd.4) 246 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:53:44.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:42.001342+0000 mgr.y (mgr.14535) 932 : cluster [DBG] pgmap v1318: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%); 0 B/s, 0 objects/s recovering 2024-02-22T19:53:44.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:42.737958+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-22T19:53:44.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:42.955133+0000 osd.4 (osd.4) 245 : cluster [DBG] 3.3 scrub starts 2024-02-22T19:53:44.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:42.958124+0000 osd.4 (osd.4) 246 : cluster [DBG] 3.3 scrub ok 2024-02-22T19:53:44.748 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.6 to 1 (10000) 2024-02-22T19:53:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:43.812960+0000 osd.6 (osd.6) 188 : cluster [DBG] 4.1f deep-scrub starts 2024-02-22T19:53:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:43.816996+0000 osd.6 (osd.6) 189 : cluster [DBG] 4.1f deep-scrub ok 2024-02-22T19:53:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:43.973801+0000 osd.1 (osd.1) 334 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:53:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:43.975301+0000 osd.1 (osd.1) 335 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:53:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:44.314369+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.53:0/3094465062' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:45.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:44.314681+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:45.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:43.812960+0000 osd.6 (osd.6) 188 : cluster [DBG] 4.1f deep-scrub starts 2024-02-22T19:53:45.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:43.816996+0000 osd.6 (osd.6) 189 : cluster [DBG] 4.1f deep-scrub ok 2024-02-22T19:53:45.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:43.973801+0000 osd.1 (osd.1) 334 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:53:45.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:43.975301+0000 osd.1 (osd.1) 335 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:53:45.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:44.314369+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.53:0/3094465062' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:45.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:44.314681+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:45.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:53:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:43.812960+0000 osd.6 (osd.6) 188 : cluster [DBG] 4.1f deep-scrub starts 2024-02-22T19:53:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:43.816996+0000 osd.6 (osd.6) 189 : cluster [DBG] 4.1f deep-scrub ok 2024-02-22T19:53:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:43.973801+0000 osd.1 (osd.1) 334 : cluster [DBG] 4.7 scrub starts 2024-02-22T19:53:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:43.975301+0000 osd.1 (osd.1) 335 : cluster [DBG] 4.7 scrub ok 2024-02-22T19:53:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:44.314369+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.53:0/3094465062' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:44.314681+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:45.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 6 1 2024-02-22T19:53:45.752 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:46.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:44.001768+0000 mgr.y (mgr.14535) 933 : cluster [DBG] pgmap v1320: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%); 0 B/s, 0 objects/s recovering 2024-02-22T19:53:46.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:44.711290+0000 osd.0 (osd.0) 223 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:53:46.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:44.713379+0000 osd.0 (osd.0) 224 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:53:46.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:44.744609+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:53:46.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:44.744701+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-22T19:53:46.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:44.001768+0000 mgr.y (mgr.14535) 933 : cluster [DBG] pgmap v1320: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%); 0 B/s, 0 objects/s recovering 2024-02-22T19:53:46.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:44.711290+0000 osd.0 (osd.0) 223 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:53:46.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:44.713379+0000 osd.0 (osd.0) 224 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:53:46.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:44.744609+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:53:46.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:44.744701+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-22T19:53:46.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:53:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:53:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:44.001768+0000 mgr.y (mgr.14535) 933 : cluster [DBG] pgmap v1320: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%); 0 B/s, 0 objects/s recovering 2024-02-22T19:53:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:44.711290+0000 osd.0 (osd.0) 223 : cluster [DBG] 4.1e scrub starts 2024-02-22T19:53:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:44.713379+0000 osd.0 (osd.0) 224 : cluster [DBG] 4.1e scrub ok 2024-02-22T19:53:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:44.744609+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:53:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:44.744701+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-22T19:53:46.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:45.076106+0000 osd.2 (osd.2) 329 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:53:46.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:45.079791+0000 osd.2 (osd.2) 330 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:53:46.837 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:45.677479+0000 osd.0 (osd.0) 225 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:53:46.838 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:45.679850+0000 osd.0 (osd.0) 226 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:53:46.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:45.076106+0000 osd.2 (osd.2) 329 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:53:46.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:45.079791+0000 osd.2 (osd.2) 330 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:53:46.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:45.677479+0000 osd.0 (osd.0) 225 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:53:46.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:45.679850+0000 osd.0 (osd.0) 226 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:53:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:45.076106+0000 osd.2 (osd.2) 329 : cluster [DBG] 5.6 scrub starts 2024-02-22T19:53:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:45.079791+0000 osd.2 (osd.2) 330 : cluster [DBG] 5.6 scrub ok 2024-02-22T19:53:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:45.677479+0000 osd.0 (osd.0) 225 : cluster [DBG] 4.3 scrub starts 2024-02-22T19:53:47.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:45.679850+0000 osd.0 (osd.0) 226 : cluster [DBG] 4.3 scrub ok 2024-02-22T19:53:47.743 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-02-22T19:53:48.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:45.787212+0000 mgr.y (mgr.14535) 934 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:48.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:46.002277+0000 mgr.y (mgr.14535) 935 : cluster [DBG] pgmap v1322: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:48.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:46.026297+0000 osd.2 (osd.2) 331 : cluster [DBG] 5.a scrub starts 2024-02-22T19:53:48.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:46.029364+0000 osd.2 (osd.2) 332 : cluster [DBG] 5.a scrub ok 2024-02-22T19:53:48.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:46.925867+0000 osd.1 (osd.1) 336 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:46.927737+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:47.561745+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? 172.21.15.53:0/2265166514' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:47.739328+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.53:0/2265166514' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:47.739397+0000 mon.a (mon.0) 2196 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:45.787212+0000 mgr.y (mgr.14535) 934 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:46.002277+0000 mgr.y (mgr.14535) 935 : cluster [DBG] pgmap v1322: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:46.026297+0000 osd.2 (osd.2) 331 : cluster [DBG] 5.a scrub starts 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:46.029364+0000 osd.2 (osd.2) 332 : cluster [DBG] 5.a scrub ok 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:46.925867+0000 osd.1 (osd.1) 336 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:46.927737+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:47.561745+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? 172.21.15.53:0/2265166514' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:47.739328+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.53:0/2265166514' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:53:48.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:47.739397+0000 mon.a (mon.0) 2196 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:45.787212+0000 mgr.y (mgr.14535) 934 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:46.002277+0000 mgr.y (mgr.14535) 935 : cluster [DBG] pgmap v1322: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:46.026297+0000 osd.2 (osd.2) 331 : cluster [DBG] 5.a scrub starts 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:46.029364+0000 osd.2 (osd.2) 332 : cluster [DBG] 5.a scrub ok 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:46.925867+0000 osd.1 (osd.1) 336 : cluster [DBG] 4.2 scrub starts 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:46.927737+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.2 scrub ok 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:47.561745+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? 172.21.15.53:0/2265166514' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:47.739328+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.53:0/2265166514' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-22T19:53:48.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:47.739397+0000 mon.a (mon.0) 2196 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-22T19:53:48.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd reweight 7 1 2024-02-22T19:53:48.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:47.643316+0000 osd.0 (osd.0) 227 : cluster [DBG] 4.b scrub starts 2024-02-22T19:53:48.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:47.645323+0000 osd.0 (osd.0) 228 : cluster [DBG] 4.b scrub ok 2024-02-22T19:53:48.791 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:49.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:47.643316+0000 osd.0 (osd.0) 227 : cluster [DBG] 4.b scrub starts 2024-02-22T19:53:49.017 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:47.645323+0000 osd.0 (osd.0) 228 : cluster [DBG] 4.b scrub ok 2024-02-22T19:53:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:47.643316+0000 osd.0 (osd.0) 227 : cluster [DBG] 4.b scrub starts 2024-02-22T19:53:49.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:47.645323+0000 osd.0 (osd.0) 228 : cluster [DBG] 4.b scrub ok 2024-02-22T19:53:49.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:49] "GET /metrics HTTP/1.1" 200 174312 "" "Prometheus/2.33.4" 2024-02-22T19:53:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:48.002879+0000 mgr.y (mgr.14535) 936 : cluster [DBG] pgmap v1324: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:48.678663+0000 osd.0 (osd.0) 229 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:48.678717+0000 osd.0 (osd.0) 230 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:48.757144+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-22T19:53:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:49.654616+0000 osd.5 (osd.5) 251 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:53:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:49.657645+0000 osd.5 (osd.5) 252 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:53:50.169 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:48.002879+0000 mgr.y (mgr.14535) 936 : cluster [DBG] pgmap v1324: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:48.678663+0000 osd.0 (osd.0) 229 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:48.678717+0000 osd.0 (osd.0) 230 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:48.757144+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:49.654616+0000 osd.5 (osd.5) 251 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:49.657645+0000 osd.5 (osd.5) 252 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:48.002879+0000 mgr.y (mgr.14535) 936 : cluster [DBG] pgmap v1324: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:48.678663+0000 osd.0 (osd.0) 229 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:48.678717+0000 osd.0 (osd.0) 230 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:48.757144+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:49.654616+0000 osd.5 (osd.5) 251 : cluster [DBG] 1.8 scrub starts 2024-02-22T19:53:50.170 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:49.657645+0000 osd.5 (osd.5) 252 : cluster [DBG] 1.8 scrub ok 2024-02-22T19:53:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:49.927541+0000 osd.1 (osd.1) 338 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:53:51.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:49.929898+0000 osd.1 (osd.1) 339 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:53:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:49.927541+0000 osd.1 (osd.1) 338 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:53:51.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:49.929898+0000 osd.1 (osd.1) 339 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:53:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:49.927541+0000 osd.1 (osd.1) 338 : cluster [DBG] 4.4 scrub starts 2024-02-22T19:53:51.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:50 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:49.929898+0000 osd.1 (osd.1) 339 : cluster [DBG] 4.4 scrub ok 2024-02-22T19:53:51.772 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.7 to 1 (10000) 2024-02-22T19:53:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:50.003517+0000 mgr.y (mgr.14535) 937 : cluster [DBG] pgmap v1326: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:50.898779+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? 172.21.15.53:0/3344435513' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:53:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:51.649344+0000 osd.5 (osd.5) 253 : cluster [DBG] 6.a scrub starts 2024-02-22T19:53:52.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:50.003517+0000 mgr.y (mgr.14535) 937 : cluster [DBG] pgmap v1326: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:52.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:50.898779+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? 172.21.15.53:0/3344435513' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:53:52.179 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:51.649344+0000 osd.5 (osd.5) 253 : cluster [DBG] 6.a scrub starts 2024-02-22T19:53:52.179 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:50.003517+0000 mgr.y (mgr.14535) 937 : cluster [DBG] pgmap v1326: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:52.179 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:50.898779+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? 172.21.15.53:0/3344435513' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:53:52.179 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:51.649344+0000 osd.5 (osd.5) 253 : cluster [DBG] 6.a scrub starts 2024-02-22T19:53:52.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd primary-affinity 7 1 2024-02-22T19:53:52.885 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:50.997183+0000 osd.2 (osd.2) 333 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:53:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:51.000075+0000 osd.2 (osd.2) 334 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:53:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:51.591929+0000 osd.0 (osd.0) 231 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:53:53.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:51.593279+0000 osd.0 (osd.0) 232 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:51.703580+0000 osd.5 (osd.5) 254 : cluster [DBG] 6.a scrub ok 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:51.765618+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? 172.21.15.53:0/3344435513' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:51.765745+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:52.004003+0000 mgr.y (mgr.14535) 938 : cluster [DBG] pgmap v1328: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:50.997183+0000 osd.2 (osd.2) 333 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:51.000075+0000 osd.2 (osd.2) 334 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:51.591929+0000 osd.0 (osd.0) 231 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:51.593279+0000 osd.0 (osd.0) 232 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:51.703580+0000 osd.5 (osd.5) 254 : cluster [DBG] 6.a scrub ok 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:51.765618+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? 172.21.15.53:0/3344435513' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:51.765745+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-22T19:53:53.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:52.004003+0000 mgr.y (mgr.14535) 938 : cluster [DBG] pgmap v1328: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:50.997183+0000 osd.2 (osd.2) 333 : cluster [DBG] 5.8 scrub starts 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:51.000075+0000 osd.2 (osd.2) 334 : cluster [DBG] 5.8 scrub ok 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:51.591929+0000 osd.0 (osd.0) 231 : cluster [DBG] 3.1f scrub starts 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:51.593279+0000 osd.0 (osd.0) 232 : cluster [DBG] 3.1f scrub ok 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:51.703580+0000 osd.5 (osd.5) 254 : cluster [DBG] 6.a scrub ok 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:51.765618+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? 172.21.15.53:0/3344435513' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:53:53.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:51.765745+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-22T19:53:53.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:52.004003+0000 mgr.y (mgr.14535) 938 : cluster [DBG] pgmap v1328: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:52.961424+0000 osd.6 (osd.6) 190 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:53:54.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:52.962467+0000 osd.6 (osd.6) 191 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:53:54.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:52.961424+0000 osd.6 (osd.6) 190 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:53:54.201 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:52.962467+0000 osd.6 (osd.6) 191 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:53:54.201 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:52.961424+0000 osd.6 (osd.6) 190 : cluster [DBG] 1.6 scrub starts 2024-02-22T19:53:54.201 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:52.962467+0000 osd.6 (osd.6) 191 : cluster [DBG] 1.6 scrub ok 2024-02-22T19:53:55.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:54.004427+0000 mgr.y (mgr.14535) 939 : cluster [DBG] pgmap v1329: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:55.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:53:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:53:55.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:54.004427+0000 mgr.y (mgr.14535) 939 : cluster [DBG] pgmap v1329: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:55.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:54.004427+0000 mgr.y (mgr.14535) 939 : cluster [DBG] pgmap v1329: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:55.817 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-02-22T19:53:56.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:53:56.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:54.589552+0000 osd.3 (osd.3) 419 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:53:56.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:54.593333+0000 osd.3 (osd.3) 420 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:53:56.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:54.920235+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? 172.21.15.53:0/1846201432' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:53:56.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:54.993542+0000 osd.1 (osd.1) 340 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:53:56.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:54.994719+0000 osd.1 (osd.1) 341 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:53:56.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:55.503103+0000 osd.0 (osd.0) 233 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:53:56.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:54.589552+0000 osd.3 (osd.3) 419 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:53:56.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:54.593333+0000 osd.3 (osd.3) 420 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:53:56.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:54.920235+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? 172.21.15.53:0/1846201432' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:53:56.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:54.993542+0000 osd.1 (osd.1) 340 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:53:56.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:54.994719+0000 osd.1 (osd.1) 341 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:53:56.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:55.503103+0000 osd.0 (osd.0) 233 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:53:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:54.589552+0000 osd.3 (osd.3) 419 : cluster [DBG] 4.14 scrub starts 2024-02-22T19:53:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:54.593333+0000 osd.3 (osd.3) 420 : cluster [DBG] 4.14 scrub ok 2024-02-22T19:53:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:54.920235+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? 172.21.15.53:0/1846201432' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-22T19:53:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:54.993542+0000 osd.1 (osd.1) 340 : cluster [DBG] 1.3 scrub starts 2024-02-22T19:53:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:54.994719+0000 osd.1 (osd.1) 341 : cluster [DBG] 1.3 scrub ok 2024-02-22T19:53:56.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:55.503103+0000 osd.0 (osd.0) 233 : cluster [DBG] 6.9 scrub starts 2024-02-22T19:53:56.575 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-02-22T19:53:56.576 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-02-22T19:53:56.576 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-02-22T19:53:56.576 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-02-22T19:53:56.576 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-22T19:53:56.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:55.590780+0000 osd.0 (osd.0) 234 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:55.798192+0000 mgr.y (mgr.14535) 940 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:55.811540+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? 172.21.15.53:0/1846201432' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:55.811617+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.005001+0000 mgr.y (mgr.14535) 941 : cluster [DBG] pgmap v1331: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.528917+0000 osd.0 (osd.0) 235 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.530976+0000 osd.0 (osd.0) 236 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:53:56.820 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:55.590780+0000 osd.0 (osd.0) 234 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:53:56.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:55.798192+0000 mgr.y (mgr.14535) 940 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:56.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:55.811540+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? 172.21.15.53:0/1846201432' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:53:56.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:55.811617+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-22T19:53:56.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.005001+0000 mgr.y (mgr.14535) 941 : cluster [DBG] pgmap v1331: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:56.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.528917+0000 osd.0 (osd.0) 235 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:53:56.821 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.530976+0000 osd.0 (osd.0) 236 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:53:56.865 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:55.590780+0000 osd.0 (osd.0) 234 : cluster [DBG] 6.9 scrub ok 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:55.798192+0000 mgr.y (mgr.14535) 940 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:55.811540+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? 172.21.15.53:0/1846201432' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:55.811617+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.005001+0000 mgr.y (mgr.14535) 941 : cluster [DBG] pgmap v1331: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.528917+0000 osd.0 (osd.0) 235 : cluster [DBG] 4.6 scrub starts 2024-02-22T19:53:57.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.530976+0000 osd.0 (osd.0) 236 : cluster [DBG] 4.6 scrub ok 2024-02-22T19:53:57.981 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.679403+0000 osd.7 (osd.7) 377 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:53:57.981 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.695226+0000 osd.7 (osd.7) 378 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:53:57.981 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.790581+0000 osd.4 (osd.4) 247 : cluster [DBG] 4.e scrub starts 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:56.792019+0000 osd.4 (osd.4) 248 : cluster [DBG] 4.e scrub ok 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:57.741910+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.679403+0000 osd.7 (osd.7) 377 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.695226+0000 osd.7 (osd.7) 378 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.790581+0000 osd.4 (osd.4) 247 : cluster [DBG] 4.e scrub starts 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:56.792019+0000 osd.4 (osd.4) 248 : cluster [DBG] 4.e scrub ok 2024-02-22T19:53:57.982 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:57.741910+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-22T19:53:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.679403+0000 osd.7 (osd.7) 377 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:53:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.695226+0000 osd.7 (osd.7) 378 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:53:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.790581+0000 osd.4 (osd.4) 247 : cluster [DBG] 4.e scrub starts 2024-02-22T19:53:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:56.792019+0000 osd.4 (osd.4) 248 : cluster [DBG] 4.e scrub ok 2024-02-22T19:53:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:57.741910+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-22T19:53:58.642 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:53:58.642 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":658,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","created":"2024-02-22T19:22:59.746063+0000","modified":"2024-02-22T19:53:57.737597+0000","last_up_change":"2024-02-22T19:28:53.274045+0000","last_in_change":"2024-02-22T19:52:41.835736+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-22T19:26:54.342689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"625","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"163","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-22T19:28:59.555543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-22T19:28:59.986190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"558","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-22T19:29:01.494051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"297","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-22T19:29:03.518927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":12,"pg_placement_num":12,"pg_placement_num_target":12,"pg_num_target":12,"pg_num_pending":12,"last_pg_merge_meta":{"source_pgid":"5.c","ready_epoch":422,"last_epoch_started":421,"last_epoch_clean":421,"source_version":"0'0","target_version":"392'11"},"last_change":"422","last_force_op_resend":"0","last_force_op_resend_prenautilus":"422","last_force_op_resend_preluminous":"420","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-22T19:29:05.596341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":22,"pg_placement_num":22,"pg_placement_num_target":22,"pg_num_target":22,"pg_num_pending":22,"last_pg_merge_meta":{"source_pgid":"6.16","ready_epoch":548,"last_epoch_started":547,"last_epoch_clean":547,"source_version":"0'0","target_version":"0'0"},"last_change":"627","last_force_op_resend":"0","last_force_op_resend_prenautilus":"548","last_force_op_resend_preluminous":"546","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"4326a76a-6144-4859-9b40-5ff69edbbcab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":648,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6801","nonce":162271081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":162271081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":162271081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":162271081}]},"public_addr":"172.21.15.53:6801/162271081","cluster_addr":"172.21.15.53:6802/162271081","heartbeat_back_addr":"172.21.15.53:6804/162271081","heartbeat_front_addr":"172.21.15.53:6803/162271081","state":["exists","up"]},{"osd":1,"uuid":"fc056bcd-1a3b-408c-b9d1-4992091df6c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":621,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6805","nonce":2973403044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2973403044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2973403044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2973403044}]},"public_addr":"172.21.15.53:6805/2973403044","cluster_addr":"172.21.15.53:6806/2973403044","heartbeat_back_addr":"172.21.15.53:6808/2973403044","heartbeat_front_addr":"172.21.15.53:6807/2973403044","state":["exists","up"]},{"osd":2,"uuid":"61f3f980-82cd-4e8e-9a3f-6d3f4559cc83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":643,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6809","nonce":936980935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":936980935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":936980935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":936980935}]},"public_addr":"172.21.15.53:6809/936980935","cluster_addr":"172.21.15.53:6810/936980935","heartbeat_back_addr":"172.21.15.53:6812/936980935","heartbeat_front_addr":"172.21.15.53:6811/936980935","state":["exists","up"]},{"osd":3,"uuid":"3f285320-aa28-4fc1-b1c7-456c9aecd993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":632,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":3878043918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3878043918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3878043918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6815","nonce":3878043918}]},"public_addr":"172.21.15.53:6813/3878043918","cluster_addr":"172.21.15.53:6814/3878043918","heartbeat_back_addr":"172.21.15.53:6816/3878043918","heartbeat_front_addr":"172.21.15.53:6815/3878043918","state":["exists","up"]},{"osd":4,"uuid":"773e8080-1028-48fe-8f6e-ed6beca60211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":643,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2373040976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6801","nonce":2373040976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":2373040976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2373040976}]},"public_addr":"172.21.15.64:6800/2373040976","cluster_addr":"172.21.15.64:6801/2373040976","heartbeat_back_addr":"172.21.15.64:6803/2373040976","heartbeat_front_addr":"172.21.15.64:6802/2373040976","state":["exists","up"]},{"osd":5,"uuid":"d79dbfb8-8f92-4d65-92ab-4b00aaccdb07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":620,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2413553331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6805","nonce":2413553331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":2413553331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2413553331}]},"public_addr":"172.21.15.64:6804/2413553331","cluster_addr":"172.21.15.64:6805/2413553331","heartbeat_back_addr":"172.21.15.64:6807/2413553331","heartbeat_front_addr":"172.21.15.64:6806/2413553331","state":["exists","up"]},{"osd":6,"uuid":"ebbb9499-df1e-42b7-89bb-df6c5935f39e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":620,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3043274732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6809","nonce":3043274732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":3043274732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3043274732}]},"public_addr":"172.21.15.64:6808/3043274732","cluster_addr":"172.21.15.64:6809/3043274732","heartbeat_back_addr":"172.21.15.64:6811/3043274732","heartbeat_front_addr":"172.21.15.64:6810/3043274732","state":["exists","up"]},{"osd":7,"uuid":"cde36201-cb77-4967-9402-6fdbe633b40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":623,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2605082260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6813","nonce":2605082260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6815","nonce":2605082260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2605082260}]},"public_addr":"172.21.15.64:6812/2605082260","cluster_addr":"172.21.15.64:6813/2605082260","heartbeat_back_addr":"172.21.15.64:6815/2605082260","heartbeat_front_addr":"172.21.15.64:6814/2605082260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:48.678720+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:29.565915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:18.210430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:02.315414+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:14.802166+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:52:53.684202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:27.883233+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:07.131414+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.4","osds":[3,5,6]},{"pgid":"2.1","osds":[5,3,4]}],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":1,"to":7}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_temp":[{"pgid":"5.0","osds":[3,1,0]}],"primary_temp":[],"blocklist":{"172.21.15.53:0/4161439976":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/2578736337":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/762516580":"2024-02-23T19:23:54.856898+0000","172.21.15.53:6800/3676085495":"2024-02-23T19:29:23.167786+0000","172.21.15.53:6800/3516945073":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/828421682":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/4127851809":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/1906929179":"2024-02-23T19:23:32.485117+0000","172.21.15.53:0/2640420245":"2024-02-23T19:23:32.485117+0000","172.21.15.53:6800/2854906057":"2024-02-23T19:23:32.485117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:57.516947+0000 osd.3 (osd.3) 421 : cluster [DBG] 4.0 deep-scrub starts 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:57.524218+0000 osd.3 (osd.3) 422 : cluster [DBG] 4.0 deep-scrub ok 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:58.005352+0000 mgr.y (mgr.14535) 942 : cluster [DBG] pgmap v1333: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 820 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:58.454513+0000 osd.0 (osd.0) 237 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:58.473440+0000 osd.0 (osd.0) 238 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:53:58.641608+0000 mon.a (mon.0) 2205 : audit [DBG] from='client.? 172.21.15.53:0/449031653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:57.516947+0000 osd.3 (osd.3) 421 : cluster [DBG] 4.0 deep-scrub starts 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:57.524218+0000 osd.3 (osd.3) 422 : cluster [DBG] 4.0 deep-scrub ok 2024-02-22T19:53:58.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:58.005352+0000 mgr.y (mgr.14535) 942 : cluster [DBG] pgmap v1333: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 820 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:58.934 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:58.454513+0000 osd.0 (osd.0) 237 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:53:58.934 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:58.473440+0000 osd.0 (osd.0) 238 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:53:58.934 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:53:58.641608+0000 mon.a (mon.0) 2205 : audit [DBG] from='client.? 172.21.15.53:0/449031653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:53:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:57.516947+0000 osd.3 (osd.3) 421 : cluster [DBG] 4.0 deep-scrub starts 2024-02-22T19:53:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:57.524218+0000 osd.3 (osd.3) 422 : cluster [DBG] 4.0 deep-scrub ok 2024-02-22T19:53:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:58.005352+0000 mgr.y (mgr.14535) 942 : cluster [DBG] pgmap v1333: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 820 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:53:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:58.454513+0000 osd.0 (osd.0) 237 : cluster [DBG] 6.10 scrub starts 2024-02-22T19:53:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:58.473440+0000 osd.0 (osd.0) 238 : cluster [DBG] 6.10 scrub ok 2024-02-22T19:53:59.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:53:58.641608+0000 mon.a (mon.0) 2205 : audit [DBG] from='client.? 172.21.15.53:0/449031653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:53:59.335 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-22T19:53:59.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T19:53:59.615 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:53:59.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:53:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:53:59] "GET /metrics HTTP/1.1" 200 174317 "" "Prometheus/2.33.4" 2024-02-22T19:54:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:59.412891+0000 osd.0 (osd.0) 239 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:54:00.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:53:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:53:59.413860+0000 osd.0 (osd.0) 240 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:54:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:59.412891+0000 osd.0 (osd.0) 239 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:54:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:53:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:53:59.413860+0000 osd.0 (osd.0) 240 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:54:00.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:59.412891+0000 osd.0 (osd.0) 239 : cluster [DBG] 3.1b scrub starts 2024-02-22T19:54:00.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:53:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:53:59.413860+0000 osd.0 (osd.0) 240 : cluster [DBG] 3.1b scrub ok 2024-02-22T19:54:01.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:00.006058+0000 mgr.y (mgr.14535) 943 : cluster [DBG] pgmap v1334: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:01.112 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:00.006058+0000 mgr.y (mgr.14535) 943 : cluster [DBG] pgmap v1334: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:01.112 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:00.006058+0000 mgr.y (mgr.14535) 943 : cluster [DBG] pgmap v1334: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:54:01.545 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":658,"fsid":"b6783888-d1b7-11ee-95bf-87774f69a715","created":"2024-02-22T19:22:59.746063+0000","modified":"2024-02-22T19:53:57.737597+0000","last_up_change":"2024-02-22T19:28:53.274045+0000","last_in_change":"2024-02-22T19:52:41.835736+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-22T19:26:54.342689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"625","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"163","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-22T19:28:59.555543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-22T19:28:59.986190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"558","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-22T19:29:01.494051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"297","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-22T19:29:03.518927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":12,"pg_placement_num":12,"pg_placement_num_target":12,"pg_num_target":12,"pg_num_pending":12,"last_pg_merge_meta":{"source_pgid":"5.c","ready_epoch":422,"last_epoch_started":421,"last_epoch_clean":421,"source_version":"0'0","target_version":"392'11"},"last_change":"422","last_force_op_resend":"0","last_force_op_resend_prenautilus":"422","last_force_op_resend_preluminous":"420","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-22T19:29:05.596341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":22,"pg_placement_num":22,"pg_placement_num_target":22,"pg_num_target":22,"pg_num_pending":22,"last_pg_merge_meta":{"source_pgid":"6.16","ready_epoch":548,"last_epoch_started":547,"last_epoch_clean":547,"source_version":"0'0","target_version":"0'0"},"last_change":"627","last_force_op_resend":"0","last_force_op_resend_prenautilus":"548","last_force_op_resend_preluminous":"546","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"4326a76a-6144-4859-9b40-5ff69edbbcab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":648,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6801","nonce":162271081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":162271081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":162271081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":162271081}]},"public_addr":"172.21.15.53:6801/162271081","cluster_addr":"172.21.15.53:6802/162271081","heartbeat_back_addr":"172.21.15.53:6804/162271081","heartbeat_front_addr":"172.21.15.53:6803/162271081","state":["exists","up"]},{"osd":1,"uuid":"fc056bcd-1a3b-408c-b9d1-4992091df6c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":621,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6805","nonce":2973403044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2973403044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2973403044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2973403044}]},"public_addr":"172.21.15.53:6805/2973403044","cluster_addr":"172.21.15.53:6806/2973403044","heartbeat_back_addr":"172.21.15.53:6808/2973403044","heartbeat_front_addr":"172.21.15.53:6807/2973403044","state":["exists","up"]},{"osd":2,"uuid":"61f3f980-82cd-4e8e-9a3f-6d3f4559cc83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":643,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6809","nonce":936980935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":936980935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":936980935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":936980935}]},"public_addr":"172.21.15.53:6809/936980935","cluster_addr":"172.21.15.53:6810/936980935","heartbeat_back_addr":"172.21.15.53:6812/936980935","heartbeat_front_addr":"172.21.15.53:6811/936980935","state":["exists","up"]},{"osd":3,"uuid":"3f285320-aa28-4fc1-b1c7-456c9aecd993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":632,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6813","nonce":3878043918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3878043918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3878043918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6815","nonce":3878043918}]},"public_addr":"172.21.15.53:6813/3878043918","cluster_addr":"172.21.15.53:6814/3878043918","heartbeat_back_addr":"172.21.15.53:6816/3878043918","heartbeat_front_addr":"172.21.15.53:6815/3878043918","state":["exists","up"]},{"osd":4,"uuid":"773e8080-1028-48fe-8f6e-ed6beca60211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":643,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2373040976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6801","nonce":2373040976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":2373040976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2373040976}]},"public_addr":"172.21.15.64:6800/2373040976","cluster_addr":"172.21.15.64:6801/2373040976","heartbeat_back_addr":"172.21.15.64:6803/2373040976","heartbeat_front_addr":"172.21.15.64:6802/2373040976","state":["exists","up"]},{"osd":5,"uuid":"d79dbfb8-8f92-4d65-92ab-4b00aaccdb07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":620,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2413553331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6805","nonce":2413553331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":2413553331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2413553331}]},"public_addr":"172.21.15.64:6804/2413553331","cluster_addr":"172.21.15.64:6805/2413553331","heartbeat_back_addr":"172.21.15.64:6807/2413553331","heartbeat_front_addr":"172.21.15.64:6806/2413553331","state":["exists","up"]},{"osd":6,"uuid":"ebbb9499-df1e-42b7-89bb-df6c5935f39e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":620,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3043274732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6809","nonce":3043274732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":3043274732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3043274732}]},"public_addr":"172.21.15.64:6808/3043274732","cluster_addr":"172.21.15.64:6809/3043274732","heartbeat_back_addr":"172.21.15.64:6811/3043274732","heartbeat_front_addr":"172.21.15.64:6810/3043274732","state":["exists","up"]},{"osd":7,"uuid":"cde36201-cb77-4967-9402-6fdbe633b40a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":623,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2605082260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6813","nonce":2605082260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6815","nonce":2605082260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2605082260}]},"public_addr":"172.21.15.64:6812/2605082260","cluster_addr":"172.21.15.64:6813/2605082260","heartbeat_back_addr":"172.21.15.64:6815/2605082260","heartbeat_front_addr":"172.21.15.64:6814/2605082260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:48.678720+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:29.565915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:18.210430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:02.315414+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:14.802166+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:52:53.684202+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:27.883233+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T19:53:07.131414+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.4","osds":[3,5,6]},{"pgid":"2.1","osds":[5,3,4]}],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":1,"to":7}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_temp":[{"pgid":"5.0","osds":[3,1,0]}],"primary_temp":[],"blocklist":{"172.21.15.53:0/4161439976":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/2578736337":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/762516580":"2024-02-23T19:23:54.856898+0000","172.21.15.53:6800/3676085495":"2024-02-23T19:29:23.167786+0000","172.21.15.53:6800/3516945073":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/828421682":"2024-02-23T19:23:54.856898+0000","172.21.15.53:0/4127851809":"2024-02-23T19:29:23.167786+0000","172.21.15.53:0/1906929179":"2024-02-23T19:23:32.485117+0000","172.21.15.53:0/2640420245":"2024-02-23T19:23:32.485117+0000","172.21.15.53:6800/2854906057":"2024-02-23T19:23:32.485117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T19:54:02.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:00.819766+0000 osd.4 (osd.4) 249 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:54:02.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:00.823577+0000 osd.4 (osd.4) 250 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:54:02.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:01.544746+0000 mon.a (mon.0) 2206 : audit [DBG] from='client.? 172.21.15.53:0/1914868042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:54:02.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:00.819766+0000 osd.4 (osd.4) 249 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:54:02.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:00.823577+0000 osd.4 (osd.4) 250 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:54:02.178 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:01.544746+0000 mon.a (mon.0) 2206 : audit [DBG] from='client.? 172.21.15.53:0/1914868042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:54:02.178 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:00.819766+0000 osd.4 (osd.4) 249 : cluster [DBG] 4.1 deep-scrub starts 2024-02-22T19:54:02.178 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:00.823577+0000 osd.4 (osd.4) 250 : cluster [DBG] 4.1 deep-scrub ok 2024-02-22T19:54:02.178 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:01 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:01.544746+0000 mon.a (mon.0) 2206 : audit [DBG] from='client.? 172.21.15.53:0/1914868042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T19:54:02.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-22T19:54:02.478 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:03.988 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:02.006761+0000 mgr.y (mgr.14535) 944 : cluster [DBG] pgmap v1335: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:03.988 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:03.124100+0000 osd.6 (osd.6) 192 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:03.127096+0000 osd.6 (osd.6) 193 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:03.645621+0000 osd.5 (osd.5) 255 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:03.649320+0000 osd.5 (osd.5) 256 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:02.006761+0000 mgr.y (mgr.14535) 944 : cluster [DBG] pgmap v1335: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:03.124100+0000 osd.6 (osd.6) 192 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:03.127096+0000 osd.6 (osd.6) 193 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:03.645621+0000 osd.5 (osd.5) 255 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:54:03.989 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:03 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:03.649320+0000 osd.5 (osd.5) 256 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:54:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:02.006761+0000 mgr.y (mgr.14535) 944 : cluster [DBG] pgmap v1335: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:03.124100+0000 osd.6 (osd.6) 192 : cluster [DBG] 5.2 scrub starts 2024-02-22T19:54:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:03.127096+0000 osd.6 (osd.6) 193 : cluster [DBG] 5.2 scrub ok 2024-02-22T19:54:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:03.645621+0000 osd.5 (osd.5) 255 : cluster [DBG] 4.8 scrub starts 2024-02-22T19:54:04.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:03.649320+0000 osd.5 (osd.5) 256 : cluster [DBG] 4.8 scrub ok 2024-02-22T19:54:04.859 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:04.641564+0000 osd.5 (osd.5) 257 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:54:04.859 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:04.650583+0000 osd.5 (osd.5) 258 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:54:04.860 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:04.641564+0000 osd.5 (osd.5) 257 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:54:04.860 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:04.650583+0000 osd.5 (osd.5) 258 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:54:04.880 INFO:teuthology.orchestra.run.smithi053.stdout:34359738707 2024-02-22T19:54:04.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-22T19:54:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:04.641564+0000 osd.5 (osd.5) 257 : cluster [DBG] 5.7 scrub starts 2024-02-22T19:54:05.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:04.650583+0000 osd.5 (osd.5) 258 : cluster [DBG] 5.7 scrub ok 2024-02-22T19:54:05.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:54:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:54:05.162 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:04.007120+0000 mgr.y (mgr.14535) 945 : cluster [DBG] pgmap v1336: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 624 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:04.587514+0000 osd.3 (osd.3) 423 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:54:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:04.590972+0000 osd.3 (osd.3) 424 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:54:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:05.355225+0000 osd.0 (osd.0) 241 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:54:06.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:05.370607+0000 osd.0 (osd.0) 242 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:54:06.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:04.007120+0000 mgr.y (mgr.14535) 945 : cluster [DBG] pgmap v1336: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 624 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:06.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:04.587514+0000 osd.3 (osd.3) 423 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:54:06.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:04.590972+0000 osd.3 (osd.3) 424 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:54:06.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:05.355225+0000 osd.0 (osd.0) 241 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:54:06.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:05 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:05.370607+0000 osd.0 (osd.0) 242 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:54:06.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:54:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:54:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:04.007120+0000 mgr.y (mgr.14535) 945 : cluster [DBG] pgmap v1336: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 624 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:04.587514+0000 osd.3 (osd.3) 423 : cluster [DBG] 4.17 scrub starts 2024-02-22T19:54:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:04.590972+0000 osd.3 (osd.3) 424 : cluster [DBG] 4.17 scrub ok 2024-02-22T19:54:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:05.355225+0000 osd.0 (osd.0) 241 : cluster [DBG] 6.0 scrub starts 2024-02-22T19:54:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:05 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:05.370607+0000 osd.0 (osd.0) 242 : cluster [DBG] 6.0 scrub ok 2024-02-22T19:54:06.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:05.165601+0000 osd.2 (osd.2) 335 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:54:06.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:05.180662+0000 osd.2 (osd.2) 336 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:54:06.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:05.165601+0000 osd.2 (osd.2) 335 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:54:06.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:06 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:05.180662+0000 osd.2 (osd.2) 336 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:54:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:05.165601+0000 osd.2 (osd.2) 335 : cluster [DBG] 6.14 scrub starts 2024-02-22T19:54:07.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:06 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:05.180662+0000 osd.2 (osd.2) 336 : cluster [DBG] 6.14 scrub ok 2024-02-22T19:54:07.657 INFO:teuthology.orchestra.run.smithi053.stdout:55834575183 2024-02-22T19:54:07.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-22T19:54:07.949 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:08.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:05.801126+0000 mgr.y (mgr.14535) 946 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:08.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:06.007704+0000 mgr.y (mgr.14535) 947 : cluster [DBG] pgmap v1337: 112 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:08.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:06.172986+0000 osd.2 (osd.2) 337 : cluster [DBG] 4.1c deep-scrub starts 2024-02-22T19:54:08.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:06.176721+0000 osd.2 (osd.2) 338 : cluster [DBG] 4.1c deep-scrub ok 2024-02-22T19:54:08.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:05.801126+0000 mgr.y (mgr.14535) 946 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:08.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:06.007704+0000 mgr.y (mgr.14535) 947 : cluster [DBG] pgmap v1337: 112 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:08.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:06.172986+0000 osd.2 (osd.2) 337 : cluster [DBG] 4.1c deep-scrub starts 2024-02-22T19:54:08.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:06.176721+0000 osd.2 (osd.2) 338 : cluster [DBG] 4.1c deep-scrub ok 2024-02-22T19:54:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:05.801126+0000 mgr.y (mgr.14535) 946 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:06.007704+0000 mgr.y (mgr.14535) 947 : cluster [DBG] pgmap v1337: 112 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:06.172986+0000 osd.2 (osd.2) 337 : cluster [DBG] 4.1c deep-scrub starts 2024-02-22T19:54:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:07 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:06.176721+0000 osd.2 (osd.2) 338 : cluster [DBG] 4.1c deep-scrub ok 2024-02-22T19:54:09.387 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:09] "GET /metrics HTTP/1.1" 200 174328 "" "Prometheus/2.33.4" 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:08.005851+0000 osd.1 (osd.1) 342 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:08.006942+0000 osd.1 (osd.1) 343 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:08.008061+0000 mgr.y (mgr.14535) 948 : cluster [DBG] pgmap v1338: 112 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 997 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:08.652437+0000 osd.7 (osd.7) 379 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:08.655680+0000 osd.7 (osd.7) 380 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:09.687628+0000 osd.5 (osd.5) 259 : cluster [DBG] 1.a scrub starts 2024-02-22T19:54:10.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:09.690841+0000 osd.5 (osd.5) 260 : cluster [DBG] 1.a scrub ok 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:08.005851+0000 osd.1 (osd.1) 342 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:08.006942+0000 osd.1 (osd.1) 343 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:08.008061+0000 mgr.y (mgr.14535) 948 : cluster [DBG] pgmap v1338: 112 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 997 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:08.652437+0000 osd.7 (osd.7) 379 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:08.655680+0000 osd.7 (osd.7) 380 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:09.687628+0000 osd.5 (osd.5) 259 : cluster [DBG] 1.a scrub starts 2024-02-22T19:54:10.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:09 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:09.690841+0000 osd.5 (osd.5) 260 : cluster [DBG] 1.a scrub ok 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:08.005851+0000 osd.1 (osd.1) 342 : cluster [DBG] 3.4 deep-scrub starts 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:08.006942+0000 osd.1 (osd.1) 343 : cluster [DBG] 3.4 deep-scrub ok 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:08.008061+0000 mgr.y (mgr.14535) 948 : cluster [DBG] pgmap v1338: 112 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_toofull, 110 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 997 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:08.652437+0000 osd.7 (osd.7) 379 : cluster [DBG] 3.0 scrub starts 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:08.655680+0000 osd.7 (osd.7) 380 : cluster [DBG] 3.0 scrub ok 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:09.687628+0000 osd.5 (osd.5) 259 : cluster [DBG] 1.a scrub starts 2024-02-22T19:54:10.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:09 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:09.690841+0000 osd.5 (osd.5) 260 : cluster [DBG] 1.a scrub ok 2024-02-22T19:54:10.460 INFO:teuthology.orchestra.run.smithi053.stdout:77309411657 2024-02-22T19:54:10.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-22T19:54:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:11.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:09.977429+0000 osd.1 (osd.1) 344 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:54:11.028 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:10 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:09.977429+0000 osd.1 (osd.1) 344 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:54:11.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:10 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:09.977429+0000 osd.1 (osd.1) 344 : cluster [DBG] 6.4 scrub starts 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:10.008605+0000 mgr.y (mgr.14535) 949 : cluster [DBG] pgmap v1339: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:10.030284+0000 osd.1 (osd.1) 345 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:10.084610+0000 osd.2 (osd.2) 339 : cluster [DBG] 4.f scrub starts 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:10.088156+0000 osd.2 (osd.2) 340 : cluster [DBG] 4.f scrub ok 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:10.008605+0000 mgr.y (mgr.14535) 949 : cluster [DBG] pgmap v1339: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:10.030284+0000 osd.1 (osd.1) 345 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:10.084610+0000 osd.2 (osd.2) 339 : cluster [DBG] 4.f scrub starts 2024-02-22T19:54:11.939 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:11 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:10.088156+0000 osd.2 (osd.2) 340 : cluster [DBG] 4.f scrub ok 2024-02-22T19:54:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:10.008605+0000 mgr.y (mgr.14535) 949 : cluster [DBG] pgmap v1339: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:10.030284+0000 osd.1 (osd.1) 345 : cluster [DBG] 6.4 scrub ok 2024-02-22T19:54:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:10.084610+0000 osd.2 (osd.2) 339 : cluster [DBG] 4.f scrub starts 2024-02-22T19:54:12.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:11 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:10.088156+0000 osd.2 (osd.2) 340 : cluster [DBG] 4.f scrub ok 2024-02-22T19:54:12.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:12.009289+0000 mgr.y (mgr.14535) 950 : cluster [DBG] pgmap v1340: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:12.856 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:12.009289+0000 mgr.y (mgr.14535) 950 : cluster [DBG] pgmap v1340: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:13.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:12 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:12.009289+0000 mgr.y (mgr.14535) 950 : cluster [DBG] pgmap v1340: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:13.253 INFO:teuthology.orchestra.run.smithi053.stdout:111669150020 2024-02-22T19:54:13.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-22T19:54:13.537 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:14.085 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:13.371841+0000 osd.0 (osd.0) 243 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:54:14.085 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:13.374812+0000 osd.0 (osd.0) 244 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:54:14.085 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:13.591560+0000 osd.5 (osd.5) 261 : cluster [DBG] 6.d scrub starts 2024-02-22T19:54:14.086 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:13.371841+0000 osd.0 (osd.0) 243 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:54:14.086 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:13.374812+0000 osd.0 (osd.0) 244 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:54:14.086 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:13 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:13.591560+0000 osd.5 (osd.5) 261 : cluster [DBG] 6.d scrub starts 2024-02-22T19:54:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:13.371841+0000 osd.0 (osd.0) 243 : cluster [DBG] 3.1 scrub starts 2024-02-22T19:54:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:13.374812+0000 osd.0 (osd.0) 244 : cluster [DBG] 3.1 scrub ok 2024-02-22T19:54:14.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:13 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:13.591560+0000 osd.5 (osd.5) 261 : cluster [DBG] 6.d scrub starts 2024-02-22T19:54:15.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:54:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:54:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:13.718439+0000 osd.5 (osd.5) 262 : cluster [DBG] 6.d scrub ok 2024-02-22T19:54:15.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:13.834709+0000 osd.4 (osd.4) 251 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:54:15.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:13.838176+0000 osd.4 (osd.4) 252 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:54:15.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:14 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:14.009753+0000 mgr.y (mgr.14535) 951 : cluster [DBG] pgmap v1341: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:15.193 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:13.718439+0000 osd.5 (osd.5) 262 : cluster [DBG] 6.d scrub ok 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:13.834709+0000 osd.4 (osd.4) 251 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:13.838176+0000 osd.4 (osd.4) 252 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:14.009753+0000 mgr.y (mgr.14535) 951 : cluster [DBG] pgmap v1341: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:13.718439+0000 osd.5 (osd.5) 262 : cluster [DBG] 6.d scrub ok 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:13.834709+0000 osd.4 (osd.4) 251 : cluster [DBG] 4.18 scrub starts 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:13.838176+0000 osd.4 (osd.4) 252 : cluster [DBG] 4.18 scrub ok 2024-02-22T19:54:15.194 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:14.009753+0000 mgr.y (mgr.14535) 951 : cluster [DBG] pgmap v1341: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:15.864 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:14.857258+0000 osd.4 (osd.4) 253 : cluster [DBG] 4.d scrub starts 2024-02-22T19:54:15.864 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:14.860906+0000 osd.4 (osd.4) 254 : cluster [DBG] 4.d scrub ok 2024-02-22T19:54:15.864 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:15.316545+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:54:15.864 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:15.631018+0000 mon.a (mon.0) 2208 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:54:15.864 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:15.631690+0000 mon.a (mon.0) 2209 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:54:15.864 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:15.635767+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:15.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:14.857258+0000 osd.4 (osd.4) 253 : cluster [DBG] 4.d scrub starts 2024-02-22T19:54:15.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:14.860906+0000 osd.4 (osd.4) 254 : cluster [DBG] 4.d scrub ok 2024-02-22T19:54:15.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:15.316545+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:54:15.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:15.631018+0000 mon.a (mon.0) 2208 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:54:15.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:15.631690+0000 mon.a (mon.0) 2209 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:54:15.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:15 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:15.635767+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:16.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:54:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:14.857258+0000 osd.4 (osd.4) 253 : cluster [DBG] 4.d scrub starts 2024-02-22T19:54:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:14.860906+0000 osd.4 (osd.4) 254 : cluster [DBG] 4.d scrub ok 2024-02-22T19:54:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:15.316545+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:54:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:15.631018+0000 mon.a (mon.0) 2208 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:54:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:15.631690+0000 mon.a (mon.0) 2209 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:54:16.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:15 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:15.635767+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:16.302 INFO:teuthology.orchestra.run.smithi053.stdout:137438953795 2024-02-22T19:54:16.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-22T19:54:16.589 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:17.089 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:15.105702+0000 osd.1 (osd.1) 346 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:54:17.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:15.204275+0000 osd.1 (osd.1) 347 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:54:17.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:15.616645+0000 osd.7 (osd.7) 381 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:54:17.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:15.629678+0000 osd.7 (osd.7) 382 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:54:17.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:15.807278+0000 mgr.y (mgr.14535) 952 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:17.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:16.010510+0000 mgr.y (mgr.14535) 953 : cluster [DBG] pgmap v1342: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:17.090 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:15.105702+0000 osd.1 (osd.1) 346 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:54:17.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:15.204275+0000 osd.1 (osd.1) 347 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:54:17.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:15.616645+0000 osd.7 (osd.7) 381 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:54:17.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:15.629678+0000 osd.7 (osd.7) 382 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:54:17.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:15.807278+0000 mgr.y (mgr.14535) 952 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:17.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:16 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:16.010510+0000 mgr.y (mgr.14535) 953 : cluster [DBG] pgmap v1342: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:15.105702+0000 osd.1 (osd.1) 346 : cluster [DBG] 6.1 scrub starts 2024-02-22T19:54:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:15.204275+0000 osd.1 (osd.1) 347 : cluster [DBG] 6.1 scrub ok 2024-02-22T19:54:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:15.616645+0000 osd.7 (osd.7) 381 : cluster [DBG] 5.9 scrub starts 2024-02-22T19:54:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:15.629678+0000 osd.7 (osd.7) 382 : cluster [DBG] 5.9 scrub ok 2024-02-22T19:54:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:15.807278+0000 mgr.y (mgr.14535) 952 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:17.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:16 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:16.010510+0000 mgr.y (mgr.14535) 953 : cluster [DBG] pgmap v1342: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:16.861622+0000 osd.4 (osd.4) 255 : cluster [DBG] 6.e scrub starts 2024-02-22T19:54:18.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:17 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:16.914328+0000 osd.4 (osd.4) 256 : cluster [DBG] 6.e scrub ok 2024-02-22T19:54:18.160 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:16.861622+0000 osd.4 (osd.4) 255 : cluster [DBG] 6.e scrub starts 2024-02-22T19:54:18.160 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:16.914328+0000 osd.4 (osd.4) 256 : cluster [DBG] 6.e scrub ok 2024-02-22T19:54:18.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:16.861622+0000 osd.4 (osd.4) 255 : cluster [DBG] 6.e scrub starts 2024-02-22T19:54:18.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:17 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:16.914328+0000 osd.4 (osd.4) 256 : cluster [DBG] 6.e scrub ok 2024-02-22T19:54:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:17.055188+0000 osd.1 (osd.1) 348 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:54:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:17.056956+0000 osd.1 (osd.1) 349 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:54:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.010887+0000 mgr.y (mgr.14535) 954 : cluster [DBG] pgmap v1343: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.356390+0000 osd.6 (osd.6) 194 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:54:19.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.357299+0000 osd.6 (osd.6) 195 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:54:19.159 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:17.055188+0000 osd.1 (osd.1) 348 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:17.056956+0000 osd.1 (osd.1) 349 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.010887+0000 mgr.y (mgr.14535) 954 : cluster [DBG] pgmap v1343: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.356390+0000 osd.6 (osd.6) 194 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.357299+0000 osd.6 (osd.6) 195 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:17.055188+0000 osd.1 (osd.1) 348 : cluster [DBG] 4.12 scrub starts 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:17.056956+0000 osd.1 (osd.1) 349 : cluster [DBG] 4.12 scrub ok 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.010887+0000 mgr.y (mgr.14535) 954 : cluster [DBG] pgmap v1343: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.356390+0000 osd.6 (osd.6) 194 : cluster [DBG] 1.7 scrub starts 2024-02-22T19:54:19.160 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:18 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.357299+0000 osd.6 (osd.6) 195 : cluster [DBG] 1.7 scrub ok 2024-02-22T19:54:19.182 INFO:teuthology.orchestra.run.smithi053.stdout:167503724861 2024-02-22T19:54:19.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-22T19:54:19.473 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:19.504 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:19] "GET /metrics HTTP/1.1" 200 174328 "" "Prometheus/2.33.4" 2024-02-22T19:54:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.189620+0000 osd.2 (osd.2) 341 : cluster [DBG] 3.e scrub starts 2024-02-22T19:54:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.192650+0000 osd.2 (osd.2) 342 : cluster [DBG] 3.e scrub ok 2024-02-22T19:54:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.617760+0000 osd.7 (osd.7) 383 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:54:20.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:18.620748+0000 osd.7 (osd.7) 384 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:19.463796+0000 osd.0 (osd.0) 245 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:19.464676+0000 osd.0 (osd.0) 246 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:19.631785+0000 osd.5 (osd.5) 263 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:19.634957+0000 osd.5 (osd.5) 264 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.189620+0000 osd.2 (osd.2) 341 : cluster [DBG] 3.e scrub starts 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.192650+0000 osd.2 (osd.2) 342 : cluster [DBG] 3.e scrub ok 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.617760+0000 osd.7 (osd.7) 383 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:18.620748+0000 osd.7 (osd.7) 384 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:19.463796+0000 osd.0 (osd.0) 245 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:54:20.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:19.464676+0000 osd.0 (osd.0) 246 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:54:20.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:19.631785+0000 osd.5 (osd.5) 263 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:54:20.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:19 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:19.634957+0000 osd.5 (osd.5) 264 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:54:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.189620+0000 osd.2 (osd.2) 341 : cluster [DBG] 3.e scrub starts 2024-02-22T19:54:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.192650+0000 osd.2 (osd.2) 342 : cluster [DBG] 3.e scrub ok 2024-02-22T19:54:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.617760+0000 osd.7 (osd.7) 383 : cluster [DBG] 1.2 scrub starts 2024-02-22T19:54:20.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:18.620748+0000 osd.7 (osd.7) 384 : cluster [DBG] 1.2 scrub ok 2024-02-22T19:54:20.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:19.463796+0000 osd.0 (osd.0) 245 : cluster [DBG] 3.6 scrub starts 2024-02-22T19:54:20.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:19.464676+0000 osd.0 (osd.0) 246 : cluster [DBG] 3.6 scrub ok 2024-02-22T19:54:20.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:19.631785+0000 osd.5 (osd.5) 263 : cluster [DBG] 1.1 scrub starts 2024-02-22T19:54:20.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:19 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:19.634957+0000 osd.5 (osd.5) 264 : cluster [DBG] 1.1 scrub ok 2024-02-22T19:54:21.090 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:19.219760+0000 osd.2 (osd.2) 343 : cluster [DBG] 6.f scrub starts 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:19.386101+0000 osd.2 (osd.2) 344 : cluster [DBG] 6.f scrub ok 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:20.011476+0000 mgr.y (mgr.14535) 955 : cluster [DBG] pgmap v1344: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:20.070118+0000 osd.5 (osd.5) 265 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:20.070168+0000 osd.5 (osd.5) 266 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:19.219760+0000 osd.2 (osd.2) 343 : cluster [DBG] 6.f scrub starts 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:19.386101+0000 osd.2 (osd.2) 344 : cluster [DBG] 6.f scrub ok 2024-02-22T19:54:21.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:20.011476+0000 mgr.y (mgr.14535) 955 : cluster [DBG] pgmap v1344: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:21.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:20.070118+0000 osd.5 (osd.5) 265 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:21.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:20.070168+0000 osd.5 (osd.5) 266 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:19.219760+0000 osd.2 (osd.2) 343 : cluster [DBG] 6.f scrub starts 2024-02-22T19:54:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:19.386101+0000 osd.2 (osd.2) 344 : cluster [DBG] 6.f scrub ok 2024-02-22T19:54:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:20.011476+0000 mgr.y (mgr.14535) 955 : cluster [DBG] pgmap v1344: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:20.070118+0000 osd.5 (osd.5) 265 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:21.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:20 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:20.070168+0000 osd.5 (osd.5) 266 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:21.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:20.602720+0000 osd.3 (osd.3) 425 : cluster [DBG] 6.11 deep-scrub starts 2024-02-22T19:54:21.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:20.678818+0000 osd.3 (osd.3) 426 : cluster [DBG] 6.11 deep-scrub ok 2024-02-22T19:54:21.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:20.830374+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-22T19:54:21.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:20.602720+0000 osd.3 (osd.3) 425 : cluster [DBG] 6.11 deep-scrub starts 2024-02-22T19:54:21.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:20.678818+0000 osd.3 (osd.3) 426 : cluster [DBG] 6.11 deep-scrub ok 2024-02-22T19:54:21.933 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:21 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:20.830374+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-22T19:54:21.955 INFO:teuthology.orchestra.run.smithi053.stdout:193273528633 2024-02-22T19:54:21.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-22T19:54:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:20.602720+0000 osd.3 (osd.3) 425 : cluster [DBG] 6.11 deep-scrub starts 2024-02-22T19:54:22.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:20.678818+0000 osd.3 (osd.3) 426 : cluster [DBG] 6.11 deep-scrub ok 2024-02-22T19:54:22.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:21 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:20.830374+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-22T19:54:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:23.091 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:21.795261+0000 osd.2 (osd.2) 345 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:21.795345+0000 osd.2 (osd.2) 346 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:21.835493+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.011852+0000 mgr.y (mgr.14535) 956 : cluster [DBG] pgmap v1347: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:21.795261+0000 osd.2 (osd.2) 345 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:21.795345+0000 osd.2 (osd.2) 346 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:21.835493+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-22T19:54:23.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:22 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.011852+0000 mgr.y (mgr.14535) 956 : cluster [DBG] pgmap v1347: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:21.795261+0000 osd.2 (osd.2) 345 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:21.795345+0000 osd.2 (osd.2) 346 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:21.835493+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-22T19:54:23.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:22 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.011852+0000 mgr.y (mgr.14535) 956 : cluster [DBG] pgmap v1347: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.546557+0000 osd.3 (osd.3) 427 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.550049+0000 osd.3 (osd.3) 428 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.556597+0000 osd.7 (osd.7) 385 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.559486+0000 osd.7 (osd.7) 386 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.599402+0000 osd.7 (osd.7) 387 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.599482+0000 osd.7 (osd.7) 388 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:24.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.840469+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.861428+0000 osd.4 (osd.4) 257 : cluster [DBG] 4.c deep-scrub starts 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:22.866980+0000 osd.4 (osd.4) 258 : cluster [DBG] 4.c deep-scrub ok 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:23.438041+0000 osd.0 (osd.0) 247 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:23.441185+0000 osd.0 (osd.0) 248 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.546557+0000 osd.3 (osd.3) 427 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.550049+0000 osd.3 (osd.3) 428 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.556597+0000 osd.7 (osd.7) 385 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.559486+0000 osd.7 (osd.7) 386 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:54:24.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.599402+0000 osd.7 (osd.7) 387 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:24.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.599482+0000 osd.7 (osd.7) 388 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:24.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.840469+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-22T19:54:24.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.861428+0000 osd.4 (osd.4) 257 : cluster [DBG] 4.c deep-scrub starts 2024-02-22T19:54:24.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:22.866980+0000 osd.4 (osd.4) 258 : cluster [DBG] 4.c deep-scrub ok 2024-02-22T19:54:24.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:23.438041+0000 osd.0 (osd.0) 247 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:54:24.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:23 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:23.441185+0000 osd.0 (osd.0) 248 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.546557+0000 osd.3 (osd.3) 427 : cluster [DBG] 4.10 scrub starts 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.550049+0000 osd.3 (osd.3) 428 : cluster [DBG] 4.10 scrub ok 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.556597+0000 osd.7 (osd.7) 385 : cluster [DBG] 3.13 scrub starts 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.559486+0000 osd.7 (osd.7) 386 : cluster [DBG] 3.13 scrub ok 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.599402+0000 osd.7 (osd.7) 387 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.599482+0000 osd.7 (osd.7) 388 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.840469+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-22T19:54:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.861428+0000 osd.4 (osd.4) 257 : cluster [DBG] 4.c deep-scrub starts 2024-02-22T19:54:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:22.866980+0000 osd.4 (osd.4) 258 : cluster [DBG] 4.c deep-scrub ok 2024-02-22T19:54:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:23.438041+0000 osd.0 (osd.0) 247 : cluster [DBG] 3.12 scrub starts 2024-02-22T19:54:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:23 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:23.441185+0000 osd.0 (osd.0) 248 : cluster [DBG] 3.12 scrub ok 2024-02-22T19:54:24.677 INFO:teuthology.orchestra.run.smithi053.stdout:227633266996 2024-02-22T19:54:24.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-22T19:54:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:23.537664+0000 osd.7 (osd.7) 389 : cluster [DBG] 2.0 deep-scrub starts 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:23.541976+0000 osd.7 (osd.7) 390 : cluster [DBG] 2.0 deep-scrub ok 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:23.856691+0000 osd.4 (osd.4) 259 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:23.859730+0000 osd.4 (osd.4) 260 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.012267+0000 mgr.y (mgr.14535) 957 : cluster [DBG] pgmap v1349: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.616937+0000 osd.5 (osd.5) 267 : cluster [DBG] 3.c scrub starts 2024-02-22T19:54:25.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.617979+0000 osd.5 (osd.5) 268 : cluster [DBG] 3.c scrub ok 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:23.537664+0000 osd.7 (osd.7) 389 : cluster [DBG] 2.0 deep-scrub starts 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:23.541976+0000 osd.7 (osd.7) 390 : cluster [DBG] 2.0 deep-scrub ok 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:23.856691+0000 osd.4 (osd.4) 259 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:23.859730+0000 osd.4 (osd.4) 260 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.012267+0000 mgr.y (mgr.14535) 957 : cluster [DBG] pgmap v1349: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.616937+0000 osd.5 (osd.5) 267 : cluster [DBG] 3.c scrub starts 2024-02-22T19:54:25.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.617979+0000 osd.5 (osd.5) 268 : cluster [DBG] 3.c scrub ok 2024-02-22T19:54:25.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:54:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:23.537664+0000 osd.7 (osd.7) 389 : cluster [DBG] 2.0 deep-scrub starts 2024-02-22T19:54:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:23.541976+0000 osd.7 (osd.7) 390 : cluster [DBG] 2.0 deep-scrub ok 2024-02-22T19:54:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:23.856691+0000 osd.4 (osd.4) 259 : cluster [DBG] 1.5 scrub starts 2024-02-22T19:54:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:23.859730+0000 osd.4 (osd.4) 260 : cluster [DBG] 1.5 scrub ok 2024-02-22T19:54:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.012267+0000 mgr.y (mgr.14535) 957 : cluster [DBG] pgmap v1349: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.616937+0000 osd.5 (osd.5) 267 : cluster [DBG] 3.c scrub starts 2024-02-22T19:54:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.617979+0000 osd.5 (osd.5) 268 : cluster [DBG] 3.c scrub ok 2024-02-22T19:54:26.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:54:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.202701+0000 osd.2 (osd.2) 347 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:54:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.205688+0000 osd.2 (osd.2) 348 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:54:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.834163+0000 osd.4 (osd.4) 261 : cluster [DBG] 4.9 deep-scrub starts 2024-02-22T19:54:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:24.839805+0000 osd.4 (osd.4) 262 : cluster [DBG] 4.9 deep-scrub ok 2024-02-22T19:54:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:25.512681+0000 osd.0 (osd.0) 249 : cluster [DBG] 3.17 deep-scrub starts 2024-02-22T19:54:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:25 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:25.513612+0000 osd.0 (osd.0) 250 : cluster [DBG] 3.17 deep-scrub ok 2024-02-22T19:54:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.202701+0000 osd.2 (osd.2) 347 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:54:26.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.205688+0000 osd.2 (osd.2) 348 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.834163+0000 osd.4 (osd.4) 261 : cluster [DBG] 4.9 deep-scrub starts 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:24.839805+0000 osd.4 (osd.4) 262 : cluster [DBG] 4.9 deep-scrub ok 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:25.512681+0000 osd.0 (osd.0) 249 : cluster [DBG] 3.17 deep-scrub starts 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:25.513612+0000 osd.0 (osd.0) 250 : cluster [DBG] 3.17 deep-scrub ok 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.202701+0000 osd.2 (osd.2) 347 : cluster [DBG] 3.19 scrub starts 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.205688+0000 osd.2 (osd.2) 348 : cluster [DBG] 3.19 scrub ok 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.834163+0000 osd.4 (osd.4) 261 : cluster [DBG] 4.9 deep-scrub starts 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:24.839805+0000 osd.4 (osd.4) 262 : cluster [DBG] 4.9 deep-scrub ok 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:25.512681+0000 osd.0 (osd.0) 249 : cluster [DBG] 3.17 deep-scrub starts 2024-02-22T19:54:26.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:25 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:25.513612+0000 osd.0 (osd.0) 250 : cluster [DBG] 3.17 deep-scrub ok 2024-02-22T19:54:27.007 INFO:teuthology.orchestra.run.smithi053.stdout:34359738712 2024-02-22T19:54:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:25.500224+0000 osd.3 (osd.3) 429 : cluster [DBG] 3.b scrub starts 2024-02-22T19:54:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:25.503742+0000 osd.3 (osd.3) 430 : cluster [DBG] 3.b scrub ok 2024-02-22T19:54:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:25.569961+0000 osd.7 (osd.7) 391 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:54:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:25.662826+0000 osd.7 (osd.7) 392 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:54:27.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:25.811566+0000 mgr.y (mgr.14535) 958 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:27.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:26 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:26.012949+0000 mgr.y (mgr.14535) 959 : cluster [DBG] pgmap v1350: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:25.500224+0000 osd.3 (osd.3) 429 : cluster [DBG] 3.b scrub starts 2024-02-22T19:54:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:25.503742+0000 osd.3 (osd.3) 430 : cluster [DBG] 3.b scrub ok 2024-02-22T19:54:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:25.569961+0000 osd.7 (osd.7) 391 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:54:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:25.662826+0000 osd.7 (osd.7) 392 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:54:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:25.811566+0000 mgr.y (mgr.14535) 958 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:27.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:26.012949+0000 mgr.y (mgr.14535) 959 : cluster [DBG] pgmap v1350: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:27.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:25.500224+0000 osd.3 (osd.3) 429 : cluster [DBG] 3.b scrub starts 2024-02-22T19:54:27.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:25.503742+0000 osd.3 (osd.3) 430 : cluster [DBG] 3.b scrub ok 2024-02-22T19:54:27.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:25.569961+0000 osd.7 (osd.7) 391 : cluster [DBG] 6.5 scrub starts 2024-02-22T19:54:27.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:25.662826+0000 osd.7 (osd.7) 392 : cluster [DBG] 6.5 scrub ok 2024-02-22T19:54:27.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:25.811566+0000 mgr.y (mgr.14535) 958 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:27.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:26 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:26.012949+0000 mgr.y (mgr.14535) 959 : cluster [DBG] pgmap v1350: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:27.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738707 got 34359738712 for osd.0 2024-02-22T19:54:27.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-22T19:54:28.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:26.545166+0000 osd.3 (osd.3) 431 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:26.560154+0000 osd.3 (osd.3) 432 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:27.007203+0000 mon.a (mon.0) 2214 : audit [DBG] from='client.? 172.21.15.53:0/513310338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:27.641305+0000 osd.5 (osd.5) 269 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:27.643611+0000 osd.5 (osd.5) 270 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:27.749685+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:26.545166+0000 osd.3 (osd.3) 431 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:26.560154+0000 osd.3 (osd.3) 432 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:27.007203+0000 mon.a (mon.0) 2214 : audit [DBG] from='client.? 172.21.15.53:0/513310338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T19:54:28.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:27.641305+0000 osd.5 (osd.5) 269 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:54:28.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:27.643611+0000 osd.5 (osd.5) 270 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:54:28.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:27 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:27.749685+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-22T19:54:28.023 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:26.545166+0000 osd.3 (osd.3) 431 : cluster [DBG] 6.6 scrub starts 2024-02-22T19:54:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:26.560154+0000 osd.3 (osd.3) 432 : cluster [DBG] 6.6 scrub ok 2024-02-22T19:54:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:27.007203+0000 mon.a (mon.0) 2214 : audit [DBG] from='client.? 172.21.15.53:0/513310338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T19:54:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:27.641305+0000 osd.5 (osd.5) 269 : cluster [DBG] 4.15 deep-scrub starts 2024-02-22T19:54:28.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:27.643611+0000 osd.5 (osd.5) 270 : cluster [DBG] 4.15 deep-scrub ok 2024-02-22T19:54:28.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:27 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:27.749685+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-22T19:54:29.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:27.506940+0000 osd.3 (osd.3) 433 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:27.509944+0000 osd.3 (osd.3) 434 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.013268+0000 mgr.y (mgr.14535) 960 : cluster [DBG] pgmap v1352: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 828 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.176748+0000 osd.6 (osd.6) 196 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.179761+0000 osd.6 (osd.6) 197 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.547699+0000 osd.7 (osd.7) 393 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.663539+0000 osd.7 (osd.7) 394 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:27.506940+0000 osd.3 (osd.3) 433 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:27.509944+0000 osd.3 (osd.3) 434 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.013268+0000 mgr.y (mgr.14535) 960 : cluster [DBG] pgmap v1352: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 828 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.176748+0000 osd.6 (osd.6) 196 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:54:29.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.179761+0000 osd.6 (osd.6) 197 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:54:29.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.547699+0000 osd.7 (osd.7) 393 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:54:29.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.663539+0000 osd.7 (osd.7) 394 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:27.506940+0000 osd.3 (osd.3) 433 : cluster [DBG] 3.7 scrub starts 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:27.509944+0000 osd.3 (osd.3) 434 : cluster [DBG] 3.7 scrub ok 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.013268+0000 mgr.y (mgr.14535) 960 : cluster [DBG] pgmap v1352: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 828 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.176748+0000 osd.6 (osd.6) 196 : cluster [DBG] 4.5 scrub starts 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.179761+0000 osd.6 (osd.6) 197 : cluster [DBG] 4.5 scrub ok 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.547699+0000 osd.7 (osd.7) 393 : cluster [DBG] 6.15 scrub starts 2024-02-22T19:54:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:28 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.663539+0000 osd.7 (osd.7) 394 : cluster [DBG] 6.15 scrub ok 2024-02-22T19:54:29.669 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:29] "GET /metrics HTTP/1.1" 200 174316 "" "Prometheus/2.33.4" 2024-02-22T19:54:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.522880+0000 osd.3 (osd.3) 435 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:54:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:28.525847+0000 osd.3 (osd.3) 436 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:54:30.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:29.683563+0000 osd.5 (osd.5) 271 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:54:30.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.522880+0000 osd.3 (osd.3) 435 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:54:30.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:28.525847+0000 osd.3 (osd.3) 436 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:54:30.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:29 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:29.683563+0000 osd.5 (osd.5) 271 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:54:30.091 INFO:teuthology.orchestra.run.smithi053.stdout:55834575187 2024-02-22T19:54:30.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.522880+0000 osd.3 (osd.3) 435 : cluster [DBG] 1.4 scrub starts 2024-02-22T19:54:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:28.525847+0000 osd.3 (osd.3) 436 : cluster [DBG] 1.4 scrub ok 2024-02-22T19:54:30.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:29 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:29.683563+0000 osd.5 (osd.5) 271 : cluster [DBG] 6.7 scrub starts 2024-02-22T19:54:30.860 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575183 got 55834575187 for osd.1 2024-02-22T19:54:30.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-22T19:54:31.145 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:29.841604+0000 osd.5 (osd.5) 272 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:29.873679+0000 osd.4 (osd.4) 263 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:29.876574+0000 osd.4 (osd.4) 264 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:30.013778+0000 mgr.y (mgr.14535) 961 : cluster [DBG] pgmap v1353: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:30.091998+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? 172.21.15.53:0/3708058945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:30.532337+0000 osd.7 (osd.7) 395 : cluster [DBG] 3.f scrub starts 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:30.533743+0000 osd.7 (osd.7) 396 : cluster [DBG] 3.f scrub ok 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:29.841604+0000 osd.5 (osd.5) 272 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:54:31.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:29.873679+0000 osd.4 (osd.4) 263 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:54:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:29.876574+0000 osd.4 (osd.4) 264 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:54:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:30.013778+0000 mgr.y (mgr.14535) 961 : cluster [DBG] pgmap v1353: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:30.091998+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? 172.21.15.53:0/3708058945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T19:54:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:30.532337+0000 osd.7 (osd.7) 395 : cluster [DBG] 3.f scrub starts 2024-02-22T19:54:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:30 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:30.533743+0000 osd.7 (osd.7) 396 : cluster [DBG] 3.f scrub ok 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:29.841604+0000 osd.5 (osd.5) 272 : cluster [DBG] 6.7 scrub ok 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:29.873679+0000 osd.4 (osd.4) 263 : cluster [DBG] 3.14 scrub starts 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:29.876574+0000 osd.4 (osd.4) 264 : cluster [DBG] 3.14 scrub ok 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:30.013778+0000 mgr.y (mgr.14535) 961 : cluster [DBG] pgmap v1353: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:30.091998+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? 172.21.15.53:0/3708058945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:30.532337+0000 osd.7 (osd.7) 395 : cluster [DBG] 3.f scrub starts 2024-02-22T19:54:31.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:30.533743+0000 osd.7 (osd.7) 396 : cluster [DBG] 3.f scrub ok 2024-02-22T19:54:32.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:30.702975+0000 osd.5 (osd.5) 273 : cluster [DBG] 3.1c deep-scrub starts 2024-02-22T19:54:32.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:30.704733+0000 osd.5 (osd.5) 274 : cluster [DBG] 3.1c deep-scrub ok 2024-02-22T19:54:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:30.702975+0000 osd.5 (osd.5) 273 : cluster [DBG] 3.1c deep-scrub starts 2024-02-22T19:54:32.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:30.704733+0000 osd.5 (osd.5) 274 : cluster [DBG] 3.1c deep-scrub ok 2024-02-22T19:54:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:30.702975+0000 osd.5 (osd.5) 273 : cluster [DBG] 3.1c deep-scrub starts 2024-02-22T19:54:32.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:30.704733+0000 osd.5 (osd.5) 274 : cluster [DBG] 3.1c deep-scrub ok 2024-02-22T19:54:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:31.538150+0000 osd.3 (osd.3) 437 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:31.538202+0000 osd.3 (osd.3) 438 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:31.607962+0000 osd.3 (osd.3) 439 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:54:33.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:31.611640+0000 osd.3 (osd.3) 440 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:31.864354+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:32.014353+0000 mgr.y (mgr.14535) 962 : cluster [DBG] pgmap v1355: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:32.624753+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:32.627751+0000 osd.5 (osd.5) 276 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:32.751464+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:31.538150+0000 osd.3 (osd.3) 437 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:31.538202+0000 osd.3 (osd.3) 438 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:31.607962+0000 osd.3 (osd.3) 439 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:31.611640+0000 osd.3 (osd.3) 440 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:31.864354+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:32.014353+0000 mgr.y (mgr.14535) 962 : cluster [DBG] pgmap v1355: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:32.624753+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:54:33.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:32.627751+0000 osd.5 (osd.5) 276 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:54:33.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:32.751464+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-22T19:54:33.157 INFO:teuthology.orchestra.run.smithi053.stdout:77309411662 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:31.538150+0000 osd.3 (osd.3) 437 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:31.538202+0000 osd.3 (osd.3) 438 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:31.607962+0000 osd.3 (osd.3) 439 : cluster [DBG] 4.1d scrub starts 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:31.611640+0000 osd.3 (osd.3) 440 : cluster [DBG] 4.1d scrub ok 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:31.864354+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:32.014353+0000 mgr.y (mgr.14535) 962 : cluster [DBG] pgmap v1355: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:32.624753+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.2 scrub starts 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:32.627751+0000 osd.5 (osd.5) 276 : cluster [DBG] 2.2 scrub ok 2024-02-22T19:54:33.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:32.751464+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-22T19:54:33.852 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411657 got 77309411662 for osd.2 2024-02-22T19:54:33.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-22T19:54:34.135 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:34.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:33.157186+0000 mon.c (mon.2) 112 : audit [DBG] from='client.? 172.21.15.53:0/3465666292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T19:54:34.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:33 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:33.157186+0000 mon.c (mon.2) 112 : audit [DBG] from='client.? 172.21.15.53:0/3465666292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T19:54:34.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:33.157186+0000 mon.c (mon.2) 112 : audit [DBG] from='client.? 172.21.15.53:0/3465666292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T19:54:35.102 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:54:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:54:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:34.014714+0000 mgr.y (mgr.14535) 963 : cluster [DBG] pgmap v1357: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:35.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:34.585724+0000 osd.7 (osd.7) 397 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:54:35.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:34.588825+0000 osd.7 (osd.7) 398 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:54:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:34.014714+0000 mgr.y (mgr.14535) 963 : cluster [DBG] pgmap v1357: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:34.585724+0000 osd.7 (osd.7) 397 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:54:35.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:34.588825+0000 osd.7 (osd.7) 398 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:54:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:34.014714+0000 mgr.y (mgr.14535) 963 : cluster [DBG] pgmap v1357: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 816 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:34.585724+0000 osd.7 (osd.7) 397 : cluster [DBG] 3.15 scrub starts 2024-02-22T19:54:35.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:34.588825+0000 osd.7 (osd.7) 398 : cluster [DBG] 3.15 scrub ok 2024-02-22T19:54:36.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:34.596391+0000 osd.3 (osd.3) 441 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:54:36.004 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:34.722293+0000 osd.3 (osd.3) 442 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:35.296812+0000 osd.6 (osd.6) 198 : cluster [DBG] 3.a scrub starts 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:35.299940+0000 osd.6 (osd.6) 199 : cluster [DBG] 3.a scrub ok 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:35.640203+0000 osd.5 (osd.5) 277 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:35.641210+0000 osd.5 (osd.5) 278 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:34.596391+0000 osd.3 (osd.3) 441 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:34.722293+0000 osd.3 (osd.3) 442 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:35.296812+0000 osd.6 (osd.6) 198 : cluster [DBG] 3.a scrub starts 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:35.299940+0000 osd.6 (osd.6) 199 : cluster [DBG] 3.a scrub ok 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:35.640203+0000 osd.5 (osd.5) 277 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:54:36.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:35 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:35.641210+0000 osd.5 (osd.5) 278 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:54:36.052 INFO:teuthology.orchestra.run.smithi053.stdout:111669150024 2024-02-22T19:54:36.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:54:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:34.596391+0000 osd.3 (osd.3) 441 : cluster [DBG] 6.13 scrub starts 2024-02-22T19:54:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:34.722293+0000 osd.3 (osd.3) 442 : cluster [DBG] 6.13 scrub ok 2024-02-22T19:54:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:35.296812+0000 osd.6 (osd.6) 198 : cluster [DBG] 3.a scrub starts 2024-02-22T19:54:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:35.299940+0000 osd.6 (osd.6) 199 : cluster [DBG] 3.a scrub ok 2024-02-22T19:54:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:35.640203+0000 osd.5 (osd.5) 277 : cluster [DBG] 3.16 scrub starts 2024-02-22T19:54:36.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:35.641210+0000 osd.5 (osd.5) 278 : cluster [DBG] 3.16 scrub ok 2024-02-22T19:54:36.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669150020 got 111669150024 for osd.3 2024-02-22T19:54:36.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-22T19:54:36.988 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:35.632944+0000 osd.3 (osd.3) 443 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:35.636122+0000 osd.3 (osd.3) 444 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:35.822558+0000 mgr.y (mgr.14535) 964 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:36.015327+0000 mgr.y (mgr.14535) 965 : cluster [DBG] pgmap v1358: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:36.052252+0000 mon.c (mon.2) 113 : audit [DBG] from='client.? 172.21.15.53:0/1613862632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:36.291023+0000 osd.6 (osd.6) 200 : cluster [DBG] 4.a scrub starts 2024-02-22T19:54:37.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:36.294610+0000 osd.6 (osd.6) 201 : cluster [DBG] 4.a scrub ok 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:35.632944+0000 osd.3 (osd.3) 443 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:35.636122+0000 osd.3 (osd.3) 444 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:35.822558+0000 mgr.y (mgr.14535) 964 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:36.015327+0000 mgr.y (mgr.14535) 965 : cluster [DBG] pgmap v1358: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:36.052252+0000 mon.c (mon.2) 113 : audit [DBG] from='client.? 172.21.15.53:0/1613862632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:36.291023+0000 osd.6 (osd.6) 200 : cluster [DBG] 4.a scrub starts 2024-02-22T19:54:37.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:36 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:36.294610+0000 osd.6 (osd.6) 201 : cluster [DBG] 4.a scrub ok 2024-02-22T19:54:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:35.632944+0000 osd.3 (osd.3) 443 : cluster [DBG] 3.2 scrub starts 2024-02-22T19:54:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:35.636122+0000 osd.3 (osd.3) 444 : cluster [DBG] 3.2 scrub ok 2024-02-22T19:54:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:35.822558+0000 mgr.y (mgr.14535) 964 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:36.015327+0000 mgr.y (mgr.14535) 965 : cluster [DBG] pgmap v1358: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:36.052252+0000 mon.c (mon.2) 113 : audit [DBG] from='client.? 172.21.15.53:0/1613862632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T19:54:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:36.291023+0000 osd.6 (osd.6) 200 : cluster [DBG] 4.a scrub starts 2024-02-22T19:54:37.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:36.294610+0000 osd.6 (osd.6) 201 : cluster [DBG] 4.a scrub ok 2024-02-22T19:54:38.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:37.666227+0000 osd.5 (osd.5) 279 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:54:38.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:37.669164+0000 osd.5 (osd.5) 280 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:54:38.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:37.666227+0000 osd.5 (osd.5) 279 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:54:38.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:37 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:37.669164+0000 osd.5 (osd.5) 280 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:54:38.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:37.666227+0000 osd.5 (osd.5) 279 : cluster [DBG] 2.1 scrub starts 2024-02-22T19:54:38.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:37.669164+0000 osd.5 (osd.5) 280 : cluster [DBG] 2.1 scrub ok 2024-02-22T19:54:39.064 INFO:teuthology.orchestra.run.smithi053.stdout:137438953799 2024-02-22T19:54:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:37.631004+0000 osd.3 (osd.3) 445 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:54:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:37.633945+0000 osd.3 (osd.3) 446 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:54:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.015678+0000 mgr.y (mgr.14535) 966 : cluster [DBG] pgmap v1359: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.122798+0000 osd.2 (osd.2) 349 : cluster [DBG] 5.b scrub starts 2024-02-22T19:54:39.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.125791+0000 osd.2 (osd.2) 350 : cluster [DBG] 5.b scrub ok 2024-02-22T19:54:39.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:37.631004+0000 osd.3 (osd.3) 445 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:54:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:37.633945+0000 osd.3 (osd.3) 446 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:54:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.015678+0000 mgr.y (mgr.14535) 966 : cluster [DBG] pgmap v1359: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.122798+0000 osd.2 (osd.2) 349 : cluster [DBG] 5.b scrub starts 2024-02-22T19:54:39.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:38 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.125791+0000 osd.2 (osd.2) 350 : cluster [DBG] 5.b scrub ok 2024-02-22T19:54:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:37.631004+0000 osd.3 (osd.3) 445 : cluster [DBG] 3.8 scrub starts 2024-02-22T19:54:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:37.633945+0000 osd.3 (osd.3) 446 : cluster [DBG] 3.8 scrub ok 2024-02-22T19:54:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.015678+0000 mgr.y (mgr.14535) 966 : cluster [DBG] pgmap v1359: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.122798+0000 osd.2 (osd.2) 349 : cluster [DBG] 5.b scrub starts 2024-02-22T19:54:39.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.125791+0000 osd.2 (osd.2) 350 : cluster [DBG] 5.b scrub ok 2024-02-22T19:54:39.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:39] "GET /metrics HTTP/1.1" 200 174316 "" "Prometheus/2.33.4" 2024-02-22T19:54:39.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953795 got 137438953799 for osd.4 2024-02-22T19:54:39.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-22T19:54:40.081 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.615398+0000 osd.3 (osd.3) 447 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.618427+0000 osd.3 (osd.3) 448 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.935519+0000 osd.4 (osd.4) 265 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:38.938477+0000 osd.4 (osd.4) 266 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:39.064997+0000 mon.a (mon.0) 2218 : audit [DBG] from='client.? 172.21.15.53:0/1051060797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.615398+0000 osd.3 (osd.3) 447 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.618427+0000 osd.3 (osd.3) 448 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.935519+0000 osd.4 (osd.4) 265 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:38.938477+0000 osd.4 (osd.4) 266 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:54:40.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:39 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:39.064997+0000 mon.a (mon.0) 2218 : audit [DBG] from='client.? 172.21.15.53:0/1051060797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T19:54:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.615398+0000 osd.3 (osd.3) 447 : cluster [DBG] 3.1e scrub starts 2024-02-22T19:54:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.618427+0000 osd.3 (osd.3) 448 : cluster [DBG] 3.1e scrub ok 2024-02-22T19:54:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.935519+0000 osd.4 (osd.4) 265 : cluster [DBG] 5.1 scrub starts 2024-02-22T19:54:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:38.938477+0000 osd.4 (osd.4) 266 : cluster [DBG] 5.1 scrub ok 2024-02-22T19:54:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:39.064997+0000 mon.a (mon.0) 2218 : audit [DBG] from='client.? 172.21.15.53:0/1051060797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T19:54:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:39.567905+0000 osd.3 (osd.3) 449 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:54:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:39.570785+0000 osd.3 (osd.3) 450 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:54:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:39.707491+0000 osd.5 (osd.5) 281 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:54:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:39.708644+0000 osd.5 (osd.5) 282 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:54:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:40.016238+0000 mgr.y (mgr.14535) 967 : cluster [DBG] pgmap v1360: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:41.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:40.588489+0000 osd.0 (osd.0) 251 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:54:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:39.567905+0000 osd.3 (osd.3) 449 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:54:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:39.570785+0000 osd.3 (osd.3) 450 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:54:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:39.707491+0000 osd.5 (osd.5) 281 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:54:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:39.708644+0000 osd.5 (osd.5) 282 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:54:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:40.016238+0000 mgr.y (mgr.14535) 967 : cluster [DBG] pgmap v1360: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:41.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:40 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:40.588489+0000 osd.0 (osd.0) 251 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:54:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:39.567905+0000 osd.3 (osd.3) 449 : cluster [DBG] 3.18 scrub starts 2024-02-22T19:54:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:39.570785+0000 osd.3 (osd.3) 450 : cluster [DBG] 3.18 scrub ok 2024-02-22T19:54:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:39.707491+0000 osd.5 (osd.5) 281 : cluster [DBG] 3.5 scrub starts 2024-02-22T19:54:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:39.708644+0000 osd.5 (osd.5) 282 : cluster [DBG] 3.5 scrub ok 2024-02-22T19:54:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:40.016238+0000 mgr.y (mgr.14535) 967 : cluster [DBG] pgmap v1360: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:41.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:40.588489+0000 osd.0 (osd.0) 251 : cluster [DBG] 5.5 scrub starts 2024-02-22T19:54:41.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:40.601848+0000 osd.0 (osd.0) 252 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:54:42.055 INFO:teuthology.orchestra.run.smithi053.stdout:167503724866 2024-02-22T19:54:42.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:41 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:40.601848+0000 osd.0 (osd.0) 252 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:54:42.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:40.601848+0000 osd.0 (osd.0) 252 : cluster [DBG] 5.5 scrub ok 2024-02-22T19:54:42.762 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724861 got 167503724866 for osd.5 2024-02-22T19:54:42.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-22T19:54:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:41.843245+0000 osd.4 (osd.4) 267 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:41.843336+0000 osd.4 (osd.4) 268 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:41.903005+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-22T19:54:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:41.955948+0000 osd.4 (osd.4) 269 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:54:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:41.960019+0000 osd.4 (osd.4) 270 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:54:43.004 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:42.016650+0000 mgr.y (mgr.14535) 968 : cluster [DBG] pgmap v1362: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:42.055677+0000 mon.c (mon.2) 114 : audit [DBG] from='client.? 172.21.15.53:0/875471944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:42.634514+0000 osd.7 (osd.7) 399 : cluster [DBG] 3.d scrub starts 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:42.637467+0000 osd.7 (osd.7) 400 : cluster [DBG] 3.d scrub ok 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:42.754529+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:41.843245+0000 osd.4 (osd.4) 267 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:41.843336+0000 osd.4 (osd.4) 268 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:41.903005+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:41.955948+0000 osd.4 (osd.4) 269 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:41.960019+0000 osd.4 (osd.4) 270 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:42.016650+0000 mgr.y (mgr.14535) 968 : cluster [DBG] pgmap v1362: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:43.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:42.055677+0000 mon.c (mon.2) 114 : audit [DBG] from='client.? 172.21.15.53:0/875471944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T19:54:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:42.634514+0000 osd.7 (osd.7) 399 : cluster [DBG] 3.d scrub starts 2024-02-22T19:54:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:42.637467+0000 osd.7 (osd.7) 400 : cluster [DBG] 3.d scrub ok 2024-02-22T19:54:43.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:42 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:42.754529+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-22T19:54:43.048 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:41.843245+0000 osd.4 (osd.4) 267 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:41.843336+0000 osd.4 (osd.4) 268 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:41.903005+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-22T19:54:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:41.955948+0000 osd.4 (osd.4) 269 : cluster [DBG] 4.1b scrub starts 2024-02-22T19:54:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:41.960019+0000 osd.4 (osd.4) 270 : cluster [DBG] 4.1b scrub ok 2024-02-22T19:54:43.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:42.016650+0000 mgr.y (mgr.14535) 968 : cluster [DBG] pgmap v1362: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:42.055677+0000 mon.c (mon.2) 114 : audit [DBG] from='client.? 172.21.15.53:0/875471944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T19:54:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:42.634514+0000 osd.7 (osd.7) 399 : cluster [DBG] 3.d scrub starts 2024-02-22T19:54:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:42.637467+0000 osd.7 (osd.7) 400 : cluster [DBG] 3.d scrub ok 2024-02-22T19:54:43.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:42.754529+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-22T19:54:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:43.239839+0000 osd.6 (osd.6) 202 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:54:44.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:43.240845+0000 osd.6 (osd.6) 203 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:54:44.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:43.239839+0000 osd.6 (osd.6) 202 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:54:44.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:43 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:43.240845+0000 osd.6 (osd.6) 203 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:54:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:43.239839+0000 osd.6 (osd.6) 202 : cluster [DBG] 3.10 scrub starts 2024-02-22T19:54:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:43.240845+0000 osd.6 (osd.6) 203 : cluster [DBG] 3.10 scrub ok 2024-02-22T19:54:44.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:54:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:54:44.920 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:43.554139+0000 osd.3 (osd.3) 451 : cluster [DBG] 6.b scrub starts 2024-02-22T19:54:44.920 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:43.696121+0000 osd.3 (osd.3) 452 : cluster [DBG] 6.b scrub ok 2024-02-22T19:54:44.921 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:44.017187+0000 mgr.y (mgr.14535) 969 : cluster [DBG] pgmap v1364: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:44.921 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:44.723165+0000 osd.7 (osd.7) 401 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:54:44.921 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:44.726203+0000 osd.7 (osd.7) 402 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:54:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:43.554139+0000 osd.3 (osd.3) 451 : cluster [DBG] 6.b scrub starts 2024-02-22T19:54:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:43.696121+0000 osd.3 (osd.3) 452 : cluster [DBG] 6.b scrub ok 2024-02-22T19:54:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:44.017187+0000 mgr.y (mgr.14535) 969 : cluster [DBG] pgmap v1364: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:44.723165+0000 osd.7 (osd.7) 401 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:54:45.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:44.726203+0000 osd.7 (osd.7) 402 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:54:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:43.554139+0000 osd.3 (osd.3) 451 : cluster [DBG] 6.b scrub starts 2024-02-22T19:54:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:43.696121+0000 osd.3 (osd.3) 452 : cluster [DBG] 6.b scrub ok 2024-02-22T19:54:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:44.017187+0000 mgr.y (mgr.14535) 969 : cluster [DBG] pgmap v1364: 112 pgs: 1 active+remapped+backfill_toofull, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 1/576 objects misplaced (0.174%) 2024-02-22T19:54:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:44.723165+0000 osd.7 (osd.7) 401 : cluster [DBG] 1.0 scrub starts 2024-02-22T19:54:45.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:44.726203+0000 osd.7 (osd.7) 402 : cluster [DBG] 1.0 scrub ok 2024-02-22T19:54:45.367 INFO:teuthology.orchestra.run.smithi053.stdout:193273528637 2024-02-22T19:54:46.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:44.748802+0000 osd.5 (osd.5) 283 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:54:46.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:44.749806+0000 osd.5 (osd.5) 284 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:54:46.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:44.919976+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-22T19:54:46.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:45.118776+0000 osd.6 (osd.6) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:46.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:45.118829+0000 osd.6 (osd.6) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:46.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:45.368260+0000 mon.a (mon.0) 2222 : audit [DBG] from='client.? 172.21.15.53:0/1840574429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-22T19:54:46.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:44.748802+0000 osd.5 (osd.5) 283 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:54:46.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:44.749806+0000 osd.5 (osd.5) 284 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:54:46.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:44.919976+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-22T19:54:46.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:45.118776+0000 osd.6 (osd.6) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:46.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:45.118829+0000 osd.6 (osd.6) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:46.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:45 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:45.368260+0000 mon.a (mon.0) 2222 : audit [DBG] from='client.? 172.21.15.53:0/1840574429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-22T19:54:46.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:54:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:44.748802+0000 osd.5 (osd.5) 283 : cluster [DBG] 3.1d scrub starts 2024-02-22T19:54:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:44.749806+0000 osd.5 (osd.5) 284 : cluster [DBG] 3.1d scrub ok 2024-02-22T19:54:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:44.919976+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-22T19:54:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:45.118776+0000 osd.6 (osd.6) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:45.118829+0000 osd.6 (osd.6) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:45.368260+0000 mon.a (mon.0) 2222 : audit [DBG] from='client.? 172.21.15.53:0/1840574429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-22T19:54:46.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528633 got 193273528637 for osd.6 2024-02-22T19:54:46.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-22T19:54:46.391 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:45.826335+0000 mgr.y (mgr.14535) 970 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:45.920087+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.017833+0000 mgr.y (mgr.14535) 971 : cluster [DBG] pgmap v1367: 112 pgs: 1 peering, 1 active+clean+scrubbing, 110 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.760122+0000 osd.7 (osd.7) 403 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.764177+0000 osd.7 (osd.7) 404 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:45.826335+0000 mgr.y (mgr.14535) 970 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:45.920087+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.017833+0000 mgr.y (mgr.14535) 971 : cluster [DBG] pgmap v1367: 112 pgs: 1 peering, 1 active+clean+scrubbing, 110 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.760122+0000 osd.7 (osd.7) 403 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:54:47.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:46 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.764177+0000 osd.7 (osd.7) 404 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:54:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:45.826335+0000 mgr.y (mgr.14535) 970 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:45.920087+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-22T19:54:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.017833+0000 mgr.y (mgr.14535) 971 : cluster [DBG] pgmap v1367: 112 pgs: 1 peering, 1 active+clean+scrubbing, 110 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.760122+0000 osd.7 (osd.7) 403 : cluster [DBG] 5.4 scrub starts 2024-02-22T19:54:47.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.764177+0000 osd.7 (osd.7) 404 : cluster [DBG] 5.4 scrub ok 2024-02-22T19:54:48.247 INFO:teuthology.orchestra.run.smithi053.stdout:227633267001 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.522744+0000 osd.1 (osd.1) 350 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.522784+0000 osd.1 (osd.1) 351 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.925612+0000 mon.a (mon.0) 2224 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.925640+0000 mon.a (mon.0) 2225 : cluster [INF] Cluster is now healthy 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:46.932649+0000 mon.a (mon.0) 2226 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:47.756430+0000 mon.a (mon.0) 2227 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.522744+0000 osd.1 (osd.1) 350 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.522784+0000 osd.1 (osd.1) 351 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:48.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.925612+0000 mon.a (mon.0) 2224 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-22T19:54:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.925640+0000 mon.a (mon.0) 2225 : cluster [INF] Cluster is now healthy 2024-02-22T19:54:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:46.932649+0000 mon.a (mon.0) 2226 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-22T19:54:48.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:47 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:47.756430+0000 mon.a (mon.0) 2227 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-22T19:54:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.522744+0000 osd.1 (osd.1) 350 : cluster [DBG] purged_snaps scrub starts 2024-02-22T19:54:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.522784+0000 osd.1 (osd.1) 351 : cluster [DBG] purged_snaps scrub ok 2024-02-22T19:54:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.925612+0000 mon.a (mon.0) 2224 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-22T19:54:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.925640+0000 mon.a (mon.0) 2225 : cluster [INF] Cluster is now healthy 2024-02-22T19:54:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:46.932649+0000 mon.a (mon.0) 2226 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-22T19:54:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:47.756430+0000 mon.a (mon.0) 2227 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-22T19:54:48.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:47.698440+0000 osd.0 (osd.0) 253 : cluster [DBG] 4.16 deep-scrub starts 2024-02-22T19:54:48.936 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:47.702573+0000 osd.0 (osd.0) 254 : cluster [DBG] 4.16 deep-scrub ok 2024-02-22T19:54:49.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266996 got 227633267001 for osd.7 2024-02-22T19:54:49.047 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-22T19:54:49.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:54:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:48.018199+0000 mgr.y (mgr.14535) 972 : cluster [DBG] pgmap v1370: 112 pgs: 1 peering, 1 active+clean+scrubbing, 110 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:48.247369+0000 mon.a (mon.0) 2228 : audit [DBG] from='client.? 172.21.15.53:0/3633809674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-22T19:54:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:48.729826+0000 osd.7 (osd.7) 405 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:54:49.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:48.733015+0000 osd.7 (osd.7) 406 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:54:49.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:47.698440+0000 osd.0 (osd.0) 253 : cluster [DBG] 4.16 deep-scrub starts 2024-02-22T19:54:49.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:47.702573+0000 osd.0 (osd.0) 254 : cluster [DBG] 4.16 deep-scrub ok 2024-02-22T19:54:49.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:48.018199+0000 mgr.y (mgr.14535) 972 : cluster [DBG] pgmap v1370: 112 pgs: 1 peering, 1 active+clean+scrubbing, 110 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:49.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:48.247369+0000 mon.a (mon.0) 2228 : audit [DBG] from='client.? 172.21.15.53:0/3633809674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-22T19:54:49.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:48.729826+0000 osd.7 (osd.7) 405 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:54:49.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:48 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:48.733015+0000 osd.7 (osd.7) 406 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:54:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:47.698440+0000 osd.0 (osd.0) 253 : cluster [DBG] 4.16 deep-scrub starts 2024-02-22T19:54:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:47.702573+0000 osd.0 (osd.0) 254 : cluster [DBG] 4.16 deep-scrub ok 2024-02-22T19:54:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:48.018199+0000 mgr.y (mgr.14535) 972 : cluster [DBG] pgmap v1370: 112 pgs: 1 peering, 1 active+clean+scrubbing, 110 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:48.247369+0000 mon.a (mon.0) 2228 : audit [DBG] from='client.? 172.21.15.53:0/3633809674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-22T19:54:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:48.729826+0000 osd.7 (osd.7) 405 : cluster [DBG] 1.9 scrub starts 2024-02-22T19:54:49.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:48.733015+0000 osd.7 (osd.7) 406 : cluster [DBG] 1.9 scrub ok 2024-02-22T19:54:49.587 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:49 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:49] "GET /metrics HTTP/1.1" 200 174316 "" "Prometheus/2.33.4" 2024-02-22T19:54:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:49.579863+0000 osd.3 (osd.3) 453 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:54:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:49.583429+0000 osd.3 (osd.3) 454 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:54:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:50.018709+0000 mgr.y (mgr.14535) 973 : cluster [DBG] pgmap v1371: 112 pgs: 1 peering, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1003 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:51.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:50.700021+0000 osd.7 (osd.7) 407 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:54:51.465 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:54:51.467 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:49.579863+0000 osd.3 (osd.3) 453 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:49.583429+0000 osd.3 (osd.3) 454 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:50.018709+0000 mgr.y (mgr.14535) 973 : cluster [DBG] pgmap v1371: 112 pgs: 1 peering, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1003 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:50.700021+0000 osd.7 (osd.7) 407 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:49.579863+0000 osd.3 (osd.3) 453 : cluster [DBG] 5.0 scrub starts 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:49.583429+0000 osd.3 (osd.3) 454 : cluster [DBG] 5.0 scrub ok 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:50.018709+0000 mgr.y (mgr.14535) 973 : cluster [DBG] pgmap v1371: 112 pgs: 1 peering, 111 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1003 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-22T19:54:51.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:51 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:50.700021+0000 osd.7 (osd.7) 407 : cluster [DBG] 6.3 scrub starts 2024-02-22T19:54:52.230 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:50.868705+0000 osd.7 (osd.7) 408 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:54:52.230 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:51.058171+0000 osd.4 (osd.4) 271 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:54:52.230 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:51.061035+0000 osd.4 (osd.4) 272 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:54:52.231 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:51.450978+0000 mgr.y (mgr.14535) 974 : audit [DBG] from='client.26108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:54:52.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:50.868705+0000 osd.7 (osd.7) 408 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:54:52.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:51.058171+0000 osd.4 (osd.4) 271 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:54:52.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:51.061035+0000 osd.4 (osd.4) 272 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:54:52.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:52 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:51.450978+0000 mgr.y (mgr.14535) 974 : audit [DBG] from='client.26108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:54:52.253 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":1371,"stamp":"2024-02-22T19:54:50.018356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":3058,"num_read_kb":2810,"num_write":1000,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":37919,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1182,"ondisk_log_size":1182,"up":336,"acting":336,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6032760,"kb_used_data":57080,"kb_used_omap":0,"kb_used_meta":5975616,"kb_avail":743928456,"statfs":{"total":767960285184,"available":761782738944,"internally_reserved":0,"allocated":58449920,"data_stored":36285511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6119030784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.102514"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.931956+0000","last_change":"2024-02-22T19:53:51.593367+0000","last_active":"2024-02-22T19:54:46.931956+0000","last_peered":"2024-02-22T19:54:46.931956+0000","last_clean":"2024-02-22T19:54:46.931956+0000","last_became_active":"2024-02-22T19:52:33.814318+0000","last_became_peered":"2024-02-22T19:52:33.814318+0000","last_unstale":"2024-02-22T19:54:46.931956+0000","last_undegraded":"2024-02-22T19:54:46.931956+0000","last_fullsized":"2024-02-22T19:54:46.931956+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:51.593283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:51.593283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"504'21","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934160+0000","last_change":"2024-02-22T19:54:13.838245+0000","last_active":"2024-02-22T19:54:46.934160+0000","last_peered":"2024-02-22T19:54:46.934160+0000","last_clean":"2024-02-22T19:54:46.934160+0000","last_became_active":"2024-02-22T19:52:42.737832+0000","last_became_peered":"2024-02-22T19:52:42.737832+0000","last_unstale":"2024-02-22T19:54:46.934160+0000","last_undegraded":"2024-02-22T19:54:46.934160+0000","last_fullsized":"2024-02-22T19:54:46.934160+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"504'21","last_scrub_stamp":"2024-02-22T19:54:13.838180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:13.838180+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":678,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755812+0000","last_change":"2024-02-22T19:54:38.618539+0000","last_active":"2024-02-22T19:54:42.755812+0000","last_peered":"2024-02-22T19:54:42.755812+0000","last_clean":"2024-02-22T19:54:42.755812+0000","last_became_active":"2024-02-22T19:49:38.802266+0000","last_became_peered":"2024-02-22T19:49:38.802266+0000","last_unstale":"2024-02-22T19:54:42.755812+0000","last_undegraded":"2024-02-22T19:54:42.755812+0000","last_fullsized":"2024-02-22T19:54:42.755812+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:38.618433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:54:38.618433+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"506'39","reported_seq":830,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755674+0000","last_change":"2024-02-22T19:53:24.719938+0000","last_active":"2024-02-22T19:54:42.755674+0000","last_peered":"2024-02-22T19:54:42.755674+0000","last_clean":"2024-02-22T19:54:42.755674+0000","last_became_active":"2024-02-22T19:51:15.103693+0000","last_became_peered":"2024-02-22T19:51:15.103693+0000","last_unstale":"2024-02-22T19:54:42.755674+0000","last_undegraded":"2024-02-22T19:54:42.755674+0000","last_fullsized":"2024-02-22T19:54:42.755674+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'39","last_scrub_stamp":"2024-02-22T19:53:24.719831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:24.719831+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":84,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764142+0000","last_change":"2024-02-22T19:54:44.749916+0000","last_active":"2024-02-22T19:54:46.764142+0000","last_peered":"2024-02-22T19:54:46.764142+0000","last_clean":"2024-02-22T19:54:46.764142+0000","last_became_active":"2024-02-22T19:52:42.745594+0000","last_became_peered":"2024-02-22T19:52:42.745594+0000","last_unstale":"2024-02-22T19:54:46.764142+0000","last_undegraded":"2024-02-22T19:54:46.764142+0000","last_fullsized":"2024-02-22T19:54:46.764142+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:44.749809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:01.706836+0000","last_clean_scrub_stamp":"2024-02-22T19:54:44.749809+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"505'25","reported_seq":74,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934561+0000","last_change":"2024-02-22T19:53:32.806794+0000","last_active":"2024-02-22T19:54:46.934561+0000","last_peered":"2024-02-22T19:54:46.934561+0000","last_clean":"2024-02-22T19:54:46.934561+0000","last_became_active":"2024-02-22T19:52:42.740933+0000","last_became_peered":"2024-02-22T19:52:42.740933+0000","last_unstale":"2024-02-22T19:54:46.934561+0000","last_undegraded":"2024-02-22T19:54:46.934561+0000","last_fullsized":"2024-02-22T19:54:46.934561+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'25","last_scrub_stamp":"2024-02-22T19:53:32.806693+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:53:32.806693+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"484'2","reported_seq":92,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763412+0000","last_change":"2024-02-22T19:54:30.704823+0000","last_active":"2024-02-22T19:54:46.763412+0000","last_peered":"2024-02-22T19:54:46.763412+0000","last_clean":"2024-02-22T19:54:46.763412+0000","last_became_active":"2024-02-22T19:52:42.747645+0000","last_became_peered":"2024-02-22T19:52:42.747645+0000","last_unstale":"2024-02-22T19:54:46.763412+0000","last_undegraded":"2024-02-22T19:54:46.763412+0000","last_fullsized":"2024-02-22T19:54:46.763412+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"484'2","last_scrub_stamp":"2024-02-22T19:54:30.704739+0000","last_deep_scrub":"484'2","last_deep_scrub_stamp":"2024-02-22T19:54:30.704739+0000","last_clean_scrub_stamp":"2024-02-22T19:54:30.704739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":3222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"505'9","reported_seq":75,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935129+0000","last_change":"2024-02-22T19:54:41.960108+0000","last_active":"2024-02-22T19:54:46.935129+0000","last_peered":"2024-02-22T19:54:46.935129+0000","last_clean":"2024-02-22T19:54:46.935129+0000","last_became_active":"2024-02-22T19:52:42.744273+0000","last_became_peered":"2024-02-22T19:52:42.744273+0000","last_unstale":"2024-02-22T19:54:46.935129+0000","last_undegraded":"2024-02-22T19:54:46.935129+0000","last_fullsized":"2024-02-22T19:54:46.935129+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'9","last_scrub_stamp":"2024-02-22T19:54:41.960024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:41.960024+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":134,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933729+0000","last_change":"2024-02-22T19:53:59.413929+0000","last_active":"2024-02-22T19:54:46.933729+0000","last_peered":"2024-02-22T19:54:46.933729+0000","last_clean":"2024-02-22T19:54:46.933729+0000","last_became_active":"2024-02-22T19:52:42.741244+0000","last_became_peered":"2024-02-22T19:52:42.741244+0000","last_unstale":"2024-02-22T19:54:46.933729+0000","last_undegraded":"2024-02-22T19:54:46.933729+0000","last_fullsized":"2024-02-22T19:54:46.933729+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:59.413864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:53:59.413864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"505'46","reported_seq":787,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.944790+0000","last_change":"2024-02-22T19:54:06.176807+0000","last_active":"2024-02-22T19:54:42.944790+0000","last_peered":"2024-02-22T19:54:42.944790+0000","last_clean":"2024-02-22T19:54:42.944790+0000","last_became_active":"2024-02-22T19:49:56.720148+0000","last_became_peered":"2024-02-22T19:49:56.720148+0000","last_unstale":"2024-02-22T19:54:42.944790+0000","last_undegraded":"2024-02-22T19:54:42.944790+0000","last_fullsized":"2024-02-22T19:54:42.944790+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'46","last_scrub_stamp":"2024-02-22T19:54:06.176727+0000","last_deep_scrub":"505'46","last_deep_scrub_stamp":"2024-02-22T19:54:06.176727+0000","last_clean_scrub_stamp":"2024-02-22T19:54:06.176727+0000","log_size":46,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934664+0000","last_change":"2024-02-22T19:53:20.906284+0000","last_active":"2024-02-22T19:54:46.934664+0000","last_peered":"2024-02-22T19:54:46.934664+0000","last_clean":"2024-02-22T19:54:46.934664+0000","last_became_active":"2024-02-22T19:52:42.733123+0000","last_became_peered":"2024-02-22T19:52:42.733123+0000","last_unstale":"2024-02-22T19:54:46.934664+0000","last_undegraded":"2024-02-22T19:54:46.934664+0000","last_fullsized":"2024-02-22T19:54:46.934664+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:20.906228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:53:20.906228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"506'29","reported_seq":863,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755870+0000","last_change":"2024-02-22T19:54:31.611759+0000","last_active":"2024-02-22T19:54:42.755870+0000","last_peered":"2024-02-22T19:54:42.755870+0000","last_clean":"2024-02-22T19:54:42.755870+0000","last_became_active":"2024-02-22T19:52:42.752698+0000","last_became_peered":"2024-02-22T19:52:42.752698+0000","last_unstale":"2024-02-22T19:54:42.755870+0000","last_undegraded":"2024-02-22T19:54:42.755870+0000","last_fullsized":"2024-02-22T19:54:42.755870+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'29","last_scrub_stamp":"2024-02-22T19:54:31.611649+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:54:31.611649+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":85,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.945190+0000","last_change":"2024-02-22T19:54:24.205755+0000","last_active":"2024-02-22T19:54:42.945190+0000","last_peered":"2024-02-22T19:54:42.945190+0000","last_clean":"2024-02-22T19:54:42.945190+0000","last_became_active":"2024-02-22T19:52:42.747539+0000","last_became_peered":"2024-02-22T19:52:42.747539+0000","last_unstale":"2024-02-22T19:54:42.945190+0000","last_undegraded":"2024-02-22T19:54:42.945190+0000","last_fullsized":"2024-02-22T19:54:42.945190+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:24.205693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:24.205693+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"506'30","reported_seq":128,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933090+0000","last_change":"2024-02-22T19:53:44.713463+0000","last_active":"2024-02-22T19:54:46.933090+0000","last_peered":"2024-02-22T19:54:46.933090+0000","last_clean":"2024-02-22T19:54:46.933090+0000","last_became_active":"2024-02-22T19:51:15.087979+0000","last_became_peered":"2024-02-22T19:51:15.087979+0000","last_unstale":"2024-02-22T19:54:46.933090+0000","last_undegraded":"2024-02-22T19:54:46.933090+0000","last_fullsized":"2024-02-22T19:54:46.933090+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'30","last_scrub_stamp":"2024-02-22T19:53:44.713384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:44.713384+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":707,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755393+0000","last_change":"2024-02-22T19:54:39.570858+0000","last_active":"2024-02-22T19:54:42.755393+0000","last_peered":"2024-02-22T19:54:42.755393+0000","last_clean":"2024-02-22T19:54:42.755393+0000","last_became_active":"2024-02-22T19:51:15.079506+0000","last_became_peered":"2024-02-22T19:51:15.079506+0000","last_unstale":"2024-02-22T19:54:42.755393+0000","last_undegraded":"2024-02-22T19:54:42.755393+0000","last_fullsized":"2024-02-22T19:54:42.755393+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:39.570789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:54:39.570789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"652'32","reported_seq":249,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816356+0000","last_change":"2024-02-22T19:53:43.817082+0000","last_active":"2024-02-22T19:54:46.816356+0000","last_peered":"2024-02-22T19:54:46.816356+0000","last_clean":"2024-02-22T19:54:46.816356+0000","last_became_active":"2024-02-22T19:52:33.822010+0000","last_became_peered":"2024-02-22T19:52:33.822010+0000","last_unstale":"2024-02-22T19:54:46.816356+0000","last_undegraded":"2024-02-22T19:54:46.816356+0000","last_fullsized":"2024-02-22T19:54:46.816356+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"652'32","last_scrub_stamp":"2024-02-22T19:53:43.817001+0000","last_deep_scrub":"652'32","last_deep_scrub_stamp":"2024-02-22T19:53:43.817001+0000","last_clean_scrub_stamp":"2024-02-22T19:53:43.817001+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":692,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658854+0000","last_change":"2024-02-22T19:53:56.695327+0000","last_active":"2024-02-22T19:54:47.658854+0000","last_peered":"2024-02-22T19:54:47.658854+0000","last_clean":"2024-02-22T19:54:47.658854+0000","last_became_active":"2024-02-22T19:52:42.739171+0000","last_became_peered":"2024-02-22T19:52:42.739171+0000","last_unstale":"2024-02-22T19:54:47.658854+0000","last_undegraded":"2024-02-22T19:54:47.658854+0000","last_fullsized":"2024-02-22T19:54:47.658854+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:56.695231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:56.695231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":125,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932110+0000","last_change":"2024-02-22T19:54:25.513732+0000","last_active":"2024-02-22T19:54:46.932110+0000","last_peered":"2024-02-22T19:54:46.932110+0000","last_clean":"2024-02-22T19:54:46.932110+0000","last_became_active":"2024-02-22T19:52:33.823355+0000","last_became_peered":"2024-02-22T19:52:33.823355+0000","last_unstale":"2024-02-22T19:54:46.932110+0000","last_undegraded":"2024-02-22T19:54:46.932110+0000","last_fullsized":"2024-02-22T19:54:46.932110+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:25.513620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:54:25.513620+0000","last_clean_scrub_stamp":"2024-02-22T19:54:25.513620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"505'12","reported_seq":742,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755384+0000","last_change":"2024-02-22T19:54:22.550185+0000","last_active":"2024-02-22T19:54:42.755384+0000","last_peered":"2024-02-22T19:54:42.755384+0000","last_clean":"2024-02-22T19:54:42.755384+0000","last_became_active":"2024-02-22T19:49:56.728926+0000","last_became_peered":"2024-02-22T19:49:56.728926+0000","last_unstale":"2024-02-22T19:54:42.755384+0000","last_undegraded":"2024-02-22T19:54:42.755384+0000","last_fullsized":"2024-02-22T19:54:42.755384+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'12","last_scrub_stamp":"2024-02-22T19:54:22.550059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:22.550059+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":710,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.756011+0000","last_change":"2024-02-22T19:54:34.722384+0000","last_active":"2024-02-22T19:54:42.756011+0000","last_peered":"2024-02-22T19:54:42.756011+0000","last_clean":"2024-02-22T19:54:42.756011+0000","last_became_active":"2024-02-22T19:51:15.123877+0000","last_became_peered":"2024-02-22T19:51:15.123877+0000","last_unstale":"2024-02-22T19:54:42.756011+0000","last_undegraded":"2024-02-22T19:54:42.756011+0000","last_fullsized":"2024-02-22T19:54:42.756011+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:34.722297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:34.722297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764057+0000","last_change":"2024-02-22T19:54:35.641296+0000","last_active":"2024-02-22T19:54:46.764057+0000","last_peered":"2024-02-22T19:54:46.764057+0000","last_clean":"2024-02-22T19:54:46.764057+0000","last_became_active":"2024-02-22T19:52:33.824603+0000","last_became_peered":"2024-02-22T19:52:33.824603+0000","last_unstale":"2024-02-22T19:54:46.764057+0000","last_undegraded":"2024-02-22T19:54:46.764057+0000","last_fullsized":"2024-02-22T19:54:46.764057+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:35.641217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:35.641217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"506'27","reported_seq":729,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755620+0000","last_change":"2024-02-22T19:53:22.742140+0000","last_active":"2024-02-22T19:54:42.755620+0000","last_peered":"2024-02-22T19:54:42.755620+0000","last_clean":"2024-02-22T19:54:42.755620+0000","last_became_active":"2024-02-22T19:49:38.881889+0000","last_became_peered":"2024-02-22T19:49:38.881889+0000","last_unstale":"2024-02-22T19:54:42.755620+0000","last_undegraded":"2024-02-22T19:54:42.755620+0000","last_fullsized":"2024-02-22T19:54:42.755620+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":529,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'27","last_scrub_stamp":"2024-02-22T19:53:22.741993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:22.741993+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":125,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933383+0000","last_change":"2024-02-22T19:53:58.473542+0000","last_active":"2024-02-22T19:54:46.933383+0000","last_peered":"2024-02-22T19:54:46.933383+0000","last_clean":"2024-02-22T19:54:46.933383+0000","last_became_active":"2024-02-22T19:52:33.822486+0000","last_became_peered":"2024-02-22T19:52:33.822486+0000","last_unstale":"2024-02-22T19:54:46.933383+0000","last_undegraded":"2024-02-22T19:54:46.933383+0000","last_fullsized":"2024-02-22T19:54:46.933383+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:58.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:51.597847+0000","last_clean_scrub_stamp":"2024-02-22T19:53:58.473444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":732,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659155+0000","last_change":"2024-02-22T19:54:34.588928+0000","last_active":"2024-02-22T19:54:47.659155+0000","last_peered":"2024-02-22T19:54:47.659155+0000","last_clean":"2024-02-22T19:54:47.659155+0000","last_became_active":"2024-02-22T19:52:42.741059+0000","last_became_peered":"2024-02-22T19:52:42.741059+0000","last_unstale":"2024-02-22T19:54:47.659155+0000","last_undegraded":"2024-02-22T19:54:47.659155+0000","last_fullsized":"2024-02-22T19:54:47.659155+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:34.588830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:54:34.588830+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"505'21","reported_seq":154,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.931859+0000","last_change":"2024-02-22T19:54:17.057033+0000","last_active":"2024-02-22T19:54:46.931859+0000","last_peered":"2024-02-22T19:54:46.931859+0000","last_clean":"2024-02-22T19:54:46.931859+0000","last_became_active":"2024-02-22T19:49:56.866114+0000","last_became_peered":"2024-02-22T19:49:56.866114+0000","last_unstale":"2024-02-22T19:54:46.931859+0000","last_undegraded":"2024-02-22T19:54:46.931859+0000","last_fullsized":"2024-02-22T19:54:46.931859+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'21","last_scrub_stamp":"2024-02-22T19:54:17.056960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:17.056960+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":719,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755405+0000","last_change":"2024-02-22T19:54:20.678911+0000","last_active":"2024-02-22T19:54:42.755405+0000","last_peered":"2024-02-22T19:54:42.755405+0000","last_clean":"2024-02-22T19:54:42.755405+0000","last_became_active":"2024-02-22T19:52:33.822298+0000","last_became_peered":"2024-02-22T19:52:33.822298+0000","last_unstale":"2024-02-22T19:54:42.755405+0000","last_undegraded":"2024-02-22T19:54:42.755405+0000","last_fullsized":"2024-02-22T19:54:42.755405+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:20.678822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:54:20.678822+0000","last_clean_scrub_stamp":"2024-02-22T19:54:20.678822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":69,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934708+0000","last_change":"2024-02-22T19:54:29.876643+0000","last_active":"2024-02-22T19:54:46.934708+0000","last_peered":"2024-02-22T19:54:46.934708+0000","last_clean":"2024-02-22T19:54:46.934708+0000","last_became_active":"2024-02-22T19:52:42.743235+0000","last_became_peered":"2024-02-22T19:52:42.743235+0000","last_unstale":"2024-02-22T19:54:46.934708+0000","last_undegraded":"2024-02-22T19:54:46.934708+0000","last_fullsized":"2024-02-22T19:54:46.934708+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:29.876579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:29.876579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"641'32","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935043+0000","last_change":"2024-02-22T19:53:25.925953+0000","last_active":"2024-02-22T19:54:46.935043+0000","last_peered":"2024-02-22T19:54:46.935043+0000","last_clean":"2024-02-22T19:54:46.935043+0000","last_became_active":"2024-02-22T19:52:42.738840+0000","last_became_peered":"2024-02-22T19:52:42.738840+0000","last_unstale":"2024-02-22T19:54:46.935043+0000","last_undegraded":"2024-02-22T19:54:46.935043+0000","last_fullsized":"2024-02-22T19:54:46.935043+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"641'32","last_scrub_stamp":"2024-02-22T19:53:25.925888+0000","last_deep_scrub":"641'32","last_deep_scrub_stamp":"2024-02-22T19:53:25.925888+0000","last_clean_scrub_stamp":"2024-02-22T19:53:25.925888+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":705,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658847+0000","last_change":"2024-02-22T19:54:22.559564+0000","last_active":"2024-02-22T19:54:47.658847+0000","last_peered":"2024-02-22T19:54:47.658847+0000","last_clean":"2024-02-22T19:54:47.658847+0000","last_became_active":"2024-02-22T19:52:42.741108+0000","last_became_peered":"2024-02-22T19:52:42.741108+0000","last_unstale":"2024-02-22T19:54:47.658847+0000","last_undegraded":"2024-02-22T19:54:47.658847+0000","last_fullsized":"2024-02-22T19:54:47.658847+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:22.559491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:12.015756+0000","last_clean_scrub_stamp":"2024-02-22T19:54:22.559491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"506'35","reported_seq":741,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755901+0000","last_change":"2024-02-22T19:53:54.593415+0000","last_active":"2024-02-22T19:54:42.755901+0000","last_peered":"2024-02-22T19:54:42.755901+0000","last_clean":"2024-02-22T19:54:42.755901+0000","last_became_active":"2024-02-22T19:49:56.729526+0000","last_became_peered":"2024-02-22T19:49:56.729526+0000","last_unstale":"2024-02-22T19:54:42.755901+0000","last_undegraded":"2024-02-22T19:54:42.755901+0000","last_fullsized":"2024-02-22T19:54:42.755901+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'35","last_scrub_stamp":"2024-02-22T19:53:54.593337+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:53:54.593337+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":118,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932652+0000","last_change":"2024-02-22T19:54:23.441314+0000","last_active":"2024-02-22T19:54:46.932652+0000","last_peered":"2024-02-22T19:54:46.932652+0000","last_clean":"2024-02-22T19:54:46.932652+0000","last_became_active":"2024-02-22T19:51:15.061241+0000","last_became_peered":"2024-02-22T19:51:15.061241+0000","last_unstale":"2024-02-22T19:54:46.932652+0000","last_undegraded":"2024-02-22T19:54:46.932652+0000","last_fullsized":"2024-02-22T19:54:46.932652+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:23.441196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:54:23.441196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"506'25","reported_seq":85,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764020+0000","last_change":"2024-02-22T19:54:27.643707+0000","last_active":"2024-02-22T19:54:46.764020+0000","last_peered":"2024-02-22T19:54:46.764020+0000","last_clean":"2024-02-22T19:54:46.764020+0000","last_became_active":"2024-02-22T19:52:33.815603+0000","last_became_peered":"2024-02-22T19:52:33.815603+0000","last_unstale":"2024-02-22T19:54:46.764020+0000","last_undegraded":"2024-02-22T19:54:46.764020+0000","last_fullsized":"2024-02-22T19:54:46.764020+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'25","last_scrub_stamp":"2024-02-22T19:54:27.643617+0000","last_deep_scrub":"506'25","last_deep_scrub_stamp":"2024-02-22T19:54:27.643617+0000","last_clean_scrub_stamp":"2024-02-22T19:54:27.643617+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":719,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.944904+0000","last_change":"2024-02-22T19:54:05.180811+0000","last_active":"2024-02-22T19:54:42.944904+0000","last_peered":"2024-02-22T19:54:42.944904+0000","last_clean":"2024-02-22T19:54:42.944904+0000","last_became_active":"2024-02-22T19:52:42.744925+0000","last_became_peered":"2024-02-22T19:52:42.744925+0000","last_unstale":"2024-02-22T19:54:42.944904+0000","last_undegraded":"2024-02-22T19:54:42.944904+0000","last_fullsized":"2024-02-22T19:54:42.944904+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:05.180671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:54:05.180671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":736,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658709+0000","last_change":"2024-02-22T19:53:25.791805+0000","last_active":"2024-02-22T19:54:47.658709+0000","last_peered":"2024-02-22T19:54:47.658709+0000","last_clean":"2024-02-22T19:54:47.658709+0000","last_became_active":"2024-02-22T19:52:42.744068+0000","last_became_peered":"2024-02-22T19:52:42.744068+0000","last_unstale":"2024-02-22T19:54:47.658709+0000","last_undegraded":"2024-02-22T19:54:47.658709+0000","last_fullsized":"2024-02-22T19:54:47.658709+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:25.791731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:05.818758+0000","last_clean_scrub_stamp":"2024-02-22T19:53:25.791731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"669'25","reported_seq":130,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.702684+0000","last_change":"2024-02-22T19:54:47.702651+0000","last_active":"2024-02-22T19:54:47.702684+0000","last_peered":"2024-02-22T19:54:47.702684+0000","last_clean":"2024-02-22T19:54:47.702684+0000","last_became_active":"2024-02-22T19:51:15.087435+0000","last_became_peered":"2024-02-22T19:51:15.087435+0000","last_unstale":"2024-02-22T19:54:47.702684+0000","last_undegraded":"2024-02-22T19:54:47.702684+0000","last_fullsized":"2024-02-22T19:54:47.702684+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"669'25","last_scrub_stamp":"2024-02-22T19:54:47.702577+0000","last_deep_scrub":"669'25","last_deep_scrub_stamp":"2024-02-22T19:54:47.702577+0000","last_clean_scrub_stamp":"2024-02-22T19:54:47.702577+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":26,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":745,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659039+0000","last_change":"2024-02-22T19:54:28.663627+0000","last_active":"2024-02-22T19:54:47.659039+0000","last_peered":"2024-02-22T19:54:47.659039+0000","last_clean":"2024-02-22T19:54:47.659039+0000","last_became_active":"2024-02-22T19:52:42.740598+0000","last_became_peered":"2024-02-22T19:52:42.740598+0000","last_unstale":"2024-02-22T19:54:47.659039+0000","last_undegraded":"2024-02-22T19:54:47.659039+0000","last_fullsized":"2024-02-22T19:54:47.659039+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:28.663543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:28.663543+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":187,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.815826+0000","last_change":"2024-02-22T19:54:43.240945+0000","last_active":"2024-02-22T19:54:46.815826+0000","last_peered":"2024-02-22T19:54:46.815826+0000","last_clean":"2024-02-22T19:54:46.815826+0000","last_became_active":"2024-02-22T19:52:33.818780+0000","last_became_peered":"2024-02-22T19:52:33.818780+0000","last_unstale":"2024-02-22T19:54:46.815826+0000","last_undegraded":"2024-02-22T19:54:46.815826+0000","last_fullsized":"2024-02-22T19:54:46.815826+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:43.240852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:13.989193+0000","last_clean_scrub_stamp":"2024-02-22T19:54:43.240852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"506'18","reported_seq":710,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.756394+0000","last_change":"2024-02-22T19:54:04.591044+0000","last_active":"2024-02-22T19:54:42.756394+0000","last_peered":"2024-02-22T19:54:42.756394+0000","last_clean":"2024-02-22T19:54:42.756394+0000","last_became_active":"2024-02-22T19:49:56.728115+0000","last_became_peered":"2024-02-22T19:49:56.728115+0000","last_unstale":"2024-02-22T19:54:42.756394+0000","last_undegraded":"2024-02-22T19:54:42.756394+0000","last_fullsized":"2024-02-22T19:54:42.756394+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'18","last_scrub_stamp":"2024-02-22T19:54:04.590976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:04.590976+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763263+0000","last_change":"2024-02-22T19:53:51.703683+0000","last_active":"2024-02-22T19:54:46.763263+0000","last_peered":"2024-02-22T19:54:46.763263+0000","last_clean":"2024-02-22T19:54:46.763263+0000","last_became_active":"2024-02-22T19:52:33.823158+0000","last_became_peered":"2024-02-22T19:52:33.823158+0000","last_unstale":"2024-02-22T19:54:46.763263+0000","last_undegraded":"2024-02-22T19:54:46.763263+0000","last_fullsized":"2024-02-22T19:54:46.763263+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:51.703585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:51.703585+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":840,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659390+0000","last_change":"2024-02-22T19:54:30.533894+0000","last_active":"2024-02-22T19:54:47.659390+0000","last_peered":"2024-02-22T19:54:47.659390+0000","last_clean":"2024-02-22T19:54:47.659390+0000","last_became_active":"2024-02-22T19:52:42.741942+0000","last_became_peered":"2024-02-22T19:52:42.741942+0000","last_unstale":"2024-02-22T19:54:47.659390+0000","last_undegraded":"2024-02-22T19:54:47.659390+0000","last_fullsized":"2024-02-22T19:54:47.659390+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:54:30.533753+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_clean_scrub_stamp":"2024-02-22T19:54:30.533753+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"505'39","reported_seq":91,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.762986+0000","last_change":"2024-02-22T19:54:03.649408+0000","last_active":"2024-02-22T19:54:46.762986+0000","last_peered":"2024-02-22T19:54:46.762986+0000","last_clean":"2024-02-22T19:54:46.762986+0000","last_became_active":"2024-02-22T19:52:33.813693+0000","last_became_peered":"2024-02-22T19:52:33.813693+0000","last_unstale":"2024-02-22T19:54:46.762986+0000","last_undegraded":"2024-02-22T19:54:46.762986+0000","last_fullsized":"2024-02-22T19:54:46.762986+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":615,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'39","last_scrub_stamp":"2024-02-22T19:54:03.649324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:03.649324+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"626'33","reported_seq":1183,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.152935+0000","last_change":"2024-02-22T19:54:15.629772+0000","last_active":"2024-02-22T19:54:48.152935+0000","last_peered":"2024-02-22T19:54:48.152935+0000","last_clean":"2024-02-22T19:54:48.152935+0000","last_became_active":"2024-02-22T19:52:45.726506+0000","last_became_peered":"2024-02-22T19:52:45.726506+0000","last_unstale":"2024-02-22T19:54:48.152935+0000","last_undegraded":"2024-02-22T19:54:48.152935+0000","last_fullsized":"2024-02-22T19:54:48.152935+0000","mapping_epoch":623,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"626'33","last_scrub_stamp":"2024-02-22T19:54:15.629683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:15.629683+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":682,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:43.696236+0000","last_change":"2024-02-22T19:54:43.696209+0000","last_active":"2024-02-22T19:54:43.696236+0000","last_peered":"2024-02-22T19:54:43.696236+0000","last_clean":"2024-02-22T19:54:43.696236+0000","last_became_active":"2024-02-22T19:49:56.728419+0000","last_became_peered":"2024-02-22T19:49:56.728419+0000","last_unstale":"2024-02-22T19:54:43.696236+0000","last_undegraded":"2024-02-22T19:54:43.696236+0000","last_fullsized":"2024-02-22T19:54:43.696236+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:43.696125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:43.696125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":434,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.944746+0000","last_change":"2024-02-22T19:54:18.192739+0000","last_active":"2024-02-22T19:54:42.944746+0000","last_peered":"2024-02-22T19:54:42.944746+0000","last_clean":"2024-02-22T19:54:42.944746+0000","last_became_active":"2024-02-22T19:52:42.744827+0000","last_became_peered":"2024-02-22T19:52:42.744827+0000","last_unstale":"2024-02-22T19:54:42.944746+0000","last_undegraded":"2024-02-22T19:54:42.944746+0000","last_fullsized":"2024-02-22T19:54:42.944746+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:18.192655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:48.582624+0000","last_clean_scrub_stamp":"2024-02-22T19:54:18.192655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"661'29","reported_seq":86,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934574+0000","last_change":"2024-02-22T19:54:24.839913+0000","last_active":"2024-02-22T19:54:46.934574+0000","last_peered":"2024-02-22T19:54:46.934574+0000","last_clean":"2024-02-22T19:54:46.934574+0000","last_became_active":"2024-02-22T19:52:42.745052+0000","last_became_peered":"2024-02-22T19:52:42.745052+0000","last_unstale":"2024-02-22T19:54:46.934574+0000","last_undegraded":"2024-02-22T19:54:46.934574+0000","last_fullsized":"2024-02-22T19:54:46.934574+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"661'29","last_scrub_stamp":"2024-02-22T19:54:24.839809+0000","last_deep_scrub":"661'29","last_deep_scrub_stamp":"2024-02-22T19:54:24.839809+0000","last_clean_scrub_stamp":"2024-02-22T19:54:24.839809+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":743,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.944598+0000","last_change":"2024-02-22T19:53:51.000150+0000","last_active":"2024-02-22T19:54:42.944598+0000","last_peered":"2024-02-22T19:54:42.944598+0000","last_clean":"2024-02-22T19:54:42.944598+0000","last_became_active":"2024-02-22T19:51:15.076802+0000","last_became_peered":"2024-02-22T19:51:15.076802+0000","last_unstale":"2024-02-22T19:54:42.944598+0000","last_undegraded":"2024-02-22T19:54:42.944598+0000","last_fullsized":"2024-02-22T19:54:42.944598+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:51.000078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:51.000078+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":667,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658533+0000","last_change":"2024-02-22T19:53:20.881210+0000","last_active":"2024-02-22T19:54:47.658533+0000","last_peered":"2024-02-22T19:54:47.658533+0000","last_clean":"2024-02-22T19:54:47.658533+0000","last_became_active":"2024-02-22T19:49:51.842580+0000","last_became_peered":"2024-02-22T19:49:51.842580+0000","last_unstale":"2024-02-22T19:54:47.658533+0000","last_undegraded":"2024-02-22T19:54:47.658533+0000","last_fullsized":"2024-02-22T19:54:47.658533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":541,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:20.881136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:53:20.881136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":763,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.944633+0000","last_change":"2024-02-22T19:54:38.125889+0000","last_active":"2024-02-22T19:54:42.944633+0000","last_peered":"2024-02-22T19:54:42.944633+0000","last_clean":"2024-02-22T19:54:42.944633+0000","last_became_active":"2024-02-22T19:52:33.821312+0000","last_became_peered":"2024-02-22T19:52:33.821312+0000","last_unstale":"2024-02-22T19:54:42.944633+0000","last_undegraded":"2024-02-22T19:54:42.944633+0000","last_fullsized":"2024-02-22T19:54:42.944633+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:38.125796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:54:38.125796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":719,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659290+0000","last_change":"2024-02-22T19:54:42.637565+0000","last_active":"2024-02-22T19:54:47.659290+0000","last_peered":"2024-02-22T19:54:47.659290+0000","last_clean":"2024-02-22T19:54:47.659290+0000","last_became_active":"2024-02-22T19:52:33.816919+0000","last_became_peered":"2024-02-22T19:52:33.816919+0000","last_unstale":"2024-02-22T19:54:47.659290+0000","last_undegraded":"2024-02-22T19:54:47.659290+0000","last_fullsized":"2024-02-22T19:54:47.659290+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:42.637473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:42.637473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"506'51","reported_seq":209,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816078+0000","last_change":"2024-02-22T19:54:36.294696+0000","last_active":"2024-02-22T19:54:46.816078+0000","last_peered":"2024-02-22T19:54:46.816078+0000","last_clean":"2024-02-22T19:54:46.816078+0000","last_became_active":"2024-02-22T19:49:56.676335+0000","last_became_peered":"2024-02-22T19:49:56.676335+0000","last_unstale":"2024-02-22T19:54:46.816078+0000","last_undegraded":"2024-02-22T19:54:46.816078+0000","last_fullsized":"2024-02-22T19:54:46.816078+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'51","last_scrub_stamp":"2024-02-22T19:54:36.294615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:36.294615+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":116,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933318+0000","last_change":"2024-02-22T19:53:55.590876+0000","last_active":"2024-02-22T19:54:46.933318+0000","last_peered":"2024-02-22T19:54:46.933318+0000","last_clean":"2024-02-22T19:54:46.933318+0000","last_became_active":"2024-02-22T19:51:15.196637+0000","last_became_peered":"2024-02-22T19:51:15.196637+0000","last_unstale":"2024-02-22T19:54:46.933318+0000","last_undegraded":"2024-02-22T19:54:46.933318+0000","last_fullsized":"2024-02-22T19:54:46.933318+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:55.590784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:55.590784+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":737,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.945034+0000","last_change":"2024-02-22T19:53:46.029501+0000","last_active":"2024-02-22T19:54:42.945034+0000","last_peered":"2024-02-22T19:54:42.945034+0000","last_clean":"2024-02-22T19:54:42.945034+0000","last_became_active":"2024-02-22T19:52:42.746454+0000","last_became_peered":"2024-02-22T19:52:42.746454+0000","last_unstale":"2024-02-22T19:54:42.945034+0000","last_undegraded":"2024-02-22T19:54:42.945034+0000","last_fullsized":"2024-02-22T19:54:42.945034+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:46.029373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:46.029373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763364+0000","last_change":"2024-02-22T19:54:24.618065+0000","last_active":"2024-02-22T19:54:46.763364+0000","last_peered":"2024-02-22T19:54:46.763364+0000","last_clean":"2024-02-22T19:54:46.763364+0000","last_became_active":"2024-02-22T19:52:33.821072+0000","last_became_peered":"2024-02-22T19:52:33.821072+0000","last_unstale":"2024-02-22T19:54:46.763364+0000","last_undegraded":"2024-02-22T19:54:46.763364+0000","last_fullsized":"2024-02-22T19:54:46.763364+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:24.617984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:24.617984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"506'25","reported_seq":166,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933747+0000","last_change":"2024-02-22T19:53:47.645445+0000","last_active":"2024-02-22T19:54:46.933747+0000","last_peered":"2024-02-22T19:54:46.933747+0000","last_clean":"2024-02-22T19:54:46.933747+0000","last_became_active":"2024-02-22T19:52:42.765324+0000","last_became_peered":"2024-02-22T19:52:42.765324+0000","last_unstale":"2024-02-22T19:54:46.933747+0000","last_undegraded":"2024-02-22T19:54:46.933747+0000","last_fullsized":"2024-02-22T19:54:46.933747+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'25","last_scrub_stamp":"2024-02-22T19:53:47.645333+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:53:47.645333+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763797+0000","last_change":"2024-02-22T19:54:19.635044+0000","last_active":"2024-02-22T19:54:46.763797+0000","last_peered":"2024-02-22T19:54:46.763797+0000","last_clean":"2024-02-22T19:54:46.763797+0000","last_became_active":"2024-02-22T19:52:33.818303+0000","last_became_peered":"2024-02-22T19:52:33.818303+0000","last_unstale":"2024-02-22T19:54:46.763797+0000","last_undegraded":"2024-02-22T19:54:46.763797+0000","last_fullsized":"2024-02-22T19:54:46.763797+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:19.634962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:19.634962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":708,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.756022+0000","last_change":"2024-02-22T19:54:26.560283+0000","last_active":"2024-02-22T19:54:42.756022+0000","last_peered":"2024-02-22T19:54:42.756022+0000","last_clean":"2024-02-22T19:54:42.756022+0000","last_became_active":"2024-02-22T19:52:42.749216+0000","last_became_peered":"2024-02-22T19:52:42.749216+0000","last_unstale":"2024-02-22T19:54:42.756022+0000","last_undegraded":"2024-02-22T19:54:42.756022+0000","last_fullsized":"2024-02-22T19:54:42.756022+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:26.560162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:26.560162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763124+0000","last_change":"2024-02-22T19:54:32.627847+0000","last_active":"2024-02-22T19:54:46.763124+0000","last_peered":"2024-02-22T19:54:46.763124+0000","last_clean":"2024-02-22T19:54:46.763124+0000","last_became_active":"2024-02-22T19:52:33.820730+0000","last_became_peered":"2024-02-22T19:52:33.820730+0000","last_unstale":"2024-02-22T19:54:46.763124+0000","last_undegraded":"2024-02-22T19:54:46.763124+0000","last_fullsized":"2024-02-22T19:54:46.763124+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:32.627755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:54:32.627755+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934942+0000","last_change":"2024-02-22T19:53:42.958217+0000","last_active":"2024-02-22T19:54:46.934942+0000","last_peered":"2024-02-22T19:54:46.934942+0000","last_clean":"2024-02-22T19:54:46.934942+0000","last_became_active":"2024-02-22T19:52:42.738975+0000","last_became_peered":"2024-02-22T19:52:42.738975+0000","last_unstale":"2024-02-22T19:54:46.934942+0000","last_undegraded":"2024-02-22T19:54:46.934942+0000","last_fullsized":"2024-02-22T19:54:46.934942+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:42.958131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:53:42.958131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"548'55","reported_seq":225,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.756319+0000","last_change":"2024-02-22T19:53:49.929993+0000","last_active":"2024-02-22T19:54:47.756319+0000","last_peered":"2024-02-22T19:54:47.756319+0000","last_clean":"2024-02-22T19:54:47.756319+0000","last_became_active":"2024-02-22T19:49:56.725545+0000","last_became_peered":"2024-02-22T19:49:56.725545+0000","last_unstale":"2024-02-22T19:54:47.756319+0000","last_undegraded":"2024-02-22T19:54:47.756319+0000","last_fullsized":"2024-02-22T19:54:47.756319+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"548'55","last_scrub_stamp":"2024-02-22T19:53:49.929903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:49.929903+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"636'47","reported_seq":877,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:49.564879+0000","last_change":"2024-02-22T19:54:40.601947+0000","last_active":"2024-02-22T19:54:49.564879+0000","last_peered":"2024-02-22T19:54:49.564879+0000","last_clean":"2024-02-22T19:54:49.564879+0000","last_became_active":"2024-02-22T19:52:46.732541+0000","last_became_peered":"2024-02-22T19:52:46.732541+0000","last_unstale":"2024-02-22T19:54:49.564879+0000","last_undegraded":"2024-02-22T19:54:49.564879+0000","last_fullsized":"2024-02-22T19:54:49.564879+0000","mapping_epoch":624,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":625,"parent":"0.0","parent_split_bits":0,"last_scrub":"636'47","last_scrub_stamp":"2024-02-22T19:54:40.601852+0000","last_deep_scrub":"636'47","last_deep_scrub_stamp":"2024-02-22T19:53:12.763656+0000","last_clean_scrub_stamp":"2024-02-22T19:54:40.601852+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":749,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658829+0000","last_change":"2024-02-22T19:54:18.620836+0000","last_active":"2024-02-22T19:54:47.658829+0000","last_peered":"2024-02-22T19:54:47.658829+0000","last_clean":"2024-02-22T19:54:47.658829+0000","last_became_active":"2024-02-22T19:52:42.739709+0000","last_became_peered":"2024-02-22T19:52:42.739709+0000","last_unstale":"2024-02-22T19:54:47.658829+0000","last_undegraded":"2024-02-22T19:54:47.658829+0000","last_fullsized":"2024-02-22T19:54:47.658829+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:18.620753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:18.620753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":672,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659387+0000","last_change":"2024-02-22T19:54:25.662938+0000","last_active":"2024-02-22T19:54:47.659387+0000","last_peered":"2024-02-22T19:54:47.659387+0000","last_clean":"2024-02-22T19:54:47.659387+0000","last_became_active":"2024-02-22T19:49:38.804804+0000","last_became_peered":"2024-02-22T19:49:38.804804+0000","last_unstale":"2024-02-22T19:54:47.659387+0000","last_undegraded":"2024-02-22T19:54:47.659387+0000","last_fullsized":"2024-02-22T19:54:47.659387+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:25.662831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:54:25.662831+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658535+0000","last_change":"2024-02-22T19:54:08.655761+0000","last_active":"2024-02-22T19:54:47.658535+0000","last_peered":"2024-02-22T19:54:47.658535+0000","last_clean":"2024-02-22T19:54:47.658535+0000","last_became_active":"2024-02-22T19:52:27.788319+0000","last_became_peered":"2024-02-22T19:52:27.788319+0000","last_unstale":"2024-02-22T19:54:47.658535+0000","last_undegraded":"2024-02-22T19:54:47.658535+0000","last_fullsized":"2024-02-22T19:54:47.658535+0000","mapping_epoch":610,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":611,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:08.655685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:08.655685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763692+0000","last_change":"2024-02-22T19:54:37.669252+0000","last_active":"2024-02-22T19:54:46.763692+0000","last_peered":"2024-02-22T19:54:46.763692+0000","last_clean":"2024-02-22T19:54:46.763692+0000","last_became_active":"2024-02-22T19:52:42.745813+0000","last_became_peered":"2024-02-22T19:52:42.745813+0000","last_unstale":"2024-02-22T19:54:46.763692+0000","last_undegraded":"2024-02-22T19:54:46.763692+0000","last_fullsized":"2024-02-22T19:54:46.763692+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:37.669170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:56.810221+0000","last_clean_scrub_stamp":"2024-02-22T19:54:37.669170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"505'21","reported_seq":207,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932175+0000","last_change":"2024-02-22T19:53:43.975381+0000","last_active":"2024-02-22T19:54:46.932175+0000","last_peered":"2024-02-22T19:54:46.932175+0000","last_clean":"2024-02-22T19:54:46.932175+0000","last_became_active":"2024-02-22T19:52:33.824073+0000","last_became_peered":"2024-02-22T19:52:33.824073+0000","last_unstale":"2024-02-22T19:54:46.932175+0000","last_undegraded":"2024-02-22T19:54:46.932175+0000","last_fullsized":"2024-02-22T19:54:46.932175+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'21","last_scrub_stamp":"2024-02-22T19:53:43.975307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:43.975307+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":28,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":2970,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"616'37","reported_seq":908,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:44.563758+0000","last_change":"2024-02-22T19:53:45.079884+0000","last_active":"2024-02-22T19:54:44.563758+0000","last_peered":"2024-02-22T19:54:44.563758+0000","last_clean":"2024-02-22T19:54:44.563758+0000","last_became_active":"2024-02-22T19:52:35.810441+0000","last_became_peered":"2024-02-22T19:52:35.810441+0000","last_unstale":"2024-02-22T19:54:44.563758+0000","last_undegraded":"2024-02-22T19:54:44.563758+0000","last_fullsized":"2024-02-22T19:54:44.563758+0000","mapping_epoch":615,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":616,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'37","last_scrub_stamp":"2024-02-22T19:53:45.079796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:45.079796+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":84,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764312+0000","last_change":"2024-02-22T19:54:29.841699+0000","last_active":"2024-02-22T19:54:46.764312+0000","last_peered":"2024-02-22T19:54:46.764312+0000","last_clean":"2024-02-22T19:54:46.764312+0000","last_became_active":"2024-02-22T19:52:42.747059+0000","last_became_peered":"2024-02-22T19:52:42.747059+0000","last_unstale":"2024-02-22T19:54:46.764312+0000","last_undegraded":"2024-02-22T19:54:46.764312+0000","last_fullsized":"2024-02-22T19:54:46.764312+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:29.841608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:06.762007+0000","last_clean_scrub_stamp":"2024-02-22T19:54:29.841608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":749,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658653+0000","last_change":"2024-02-22T19:54:44.726286+0000","last_active":"2024-02-22T19:54:47.658653+0000","last_peered":"2024-02-22T19:54:47.658653+0000","last_clean":"2024-02-22T19:54:47.658653+0000","last_became_active":"2024-02-22T19:51:15.060608+0000","last_became_peered":"2024-02-22T19:51:15.060608+0000","last_unstale":"2024-02-22T19:54:47.658653+0000","last_undegraded":"2024-02-22T19:54:47.658653+0000","last_fullsized":"2024-02-22T19:54:47.658653+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:44.726208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_clean_scrub_stamp":"2024-02-22T19:54:44.726208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":723,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755686+0000","last_change":"2024-02-22T19:54:35.636212+0000","last_active":"2024-02-22T19:54:42.755686+0000","last_peered":"2024-02-22T19:54:42.755686+0000","last_clean":"2024-02-22T19:54:42.755686+0000","last_became_active":"2024-02-22T19:52:33.820991+0000","last_became_peered":"2024-02-22T19:52:33.820991+0000","last_unstale":"2024-02-22T19:54:42.755686+0000","last_undegraded":"2024-02-22T19:54:42.755686+0000","last_fullsized":"2024-02-22T19:54:42.755686+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:35.636126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:54:35.636126+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"506'45","reported_seq":228,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816559+0000","last_change":"2024-02-22T19:54:28.179856+0000","last_active":"2024-02-22T19:54:46.816559+0000","last_peered":"2024-02-22T19:54:46.816559+0000","last_clean":"2024-02-22T19:54:46.816559+0000","last_became_active":"2024-02-22T19:52:42.745501+0000","last_became_peered":"2024-02-22T19:52:42.745501+0000","last_unstale":"2024-02-22T19:54:46.816559+0000","last_undegraded":"2024-02-22T19:54:46.816559+0000","last_fullsized":"2024-02-22T19:54:46.816559+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'45","last_scrub_stamp":"2024-02-22T19:54:28.179766+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:28.179766+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":50,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":924,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"617'20","reported_seq":896,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.152972+0000","last_change":"2024-02-22T19:54:46.764317+0000","last_active":"2024-02-22T19:54:48.152972+0000","last_peered":"2024-02-22T19:54:48.152972+0000","last_clean":"2024-02-22T19:54:48.152972+0000","last_became_active":"2024-02-22T19:52:36.814425+0000","last_became_peered":"2024-02-22T19:52:36.814425+0000","last_unstale":"2024-02-22T19:54:48.152972+0000","last_undegraded":"2024-02-22T19:54:48.152972+0000","last_fullsized":"2024-02-22T19:54:48.152972+0000","mapping_epoch":616,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":617,"parent":"0.0","parent_split_bits":0,"last_scrub":"617'20","last_scrub_stamp":"2024-02-22T19:54:46.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:46.764185+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":171,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932212+0000","last_change":"2024-02-22T19:53:54.994784+0000","last_active":"2024-02-22T19:54:46.932212+0000","last_peered":"2024-02-22T19:54:46.932212+0000","last_clean":"2024-02-22T19:54:46.932212+0000","last_became_active":"2024-02-22T19:52:42.758013+0000","last_became_peered":"2024-02-22T19:52:42.758013+0000","last_unstale":"2024-02-22T19:54:46.932212+0000","last_undegraded":"2024-02-22T19:54:46.932212+0000","last_fullsized":"2024-02-22T19:54:46.932212+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:54.994723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:53:54.994723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":162,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932512+0000","last_change":"2024-02-22T19:54:10.030373+0000","last_active":"2024-02-22T19:54:46.932512+0000","last_peered":"2024-02-22T19:54:46.932512+0000","last_clean":"2024-02-22T19:54:46.932512+0000","last_became_active":"2024-02-22T19:52:33.824337+0000","last_became_peered":"2024-02-22T19:52:33.824337+0000","last_unstale":"2024-02-22T19:54:46.932512+0000","last_undegraded":"2024-02-22T19:54:46.932512+0000","last_fullsized":"2024-02-22T19:54:46.932512+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:10.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:10.030289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":143,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933394+0000","last_change":"2024-02-22T19:54:13.374895+0000","last_active":"2024-02-22T19:54:46.933394+0000","last_peered":"2024-02-22T19:54:46.933394+0000","last_clean":"2024-02-22T19:54:46.933394+0000","last_became_active":"2024-02-22T19:52:42.740196+0000","last_became_peered":"2024-02-22T19:52:42.740196+0000","last_unstale":"2024-02-22T19:54:46.933394+0000","last_undegraded":"2024-02-22T19:54:46.933394+0000","last_fullsized":"2024-02-22T19:54:46.933394+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:13.374816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:54:13.374816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"661'6","reported_seq":2286,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.441865+0000","last_change":"2024-02-22T19:54:23.542107+0000","last_active":"2024-02-22T19:54:48.441865+0000","last_peered":"2024-02-22T19:54:48.441865+0000","last_clean":"2024-02-22T19:54:48.441865+0000","last_became_active":"2024-02-22T19:51:15.064208+0000","last_became_peered":"2024-02-22T19:51:15.064208+0000","last_unstale":"2024-02-22T19:54:48.441865+0000","last_undegraded":"2024-02-22T19:54:48.441865+0000","last_fullsized":"2024-02-22T19:54:48.441865+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"661'6","last_scrub_stamp":"2024-02-22T19:54:23.541989+0000","last_deep_scrub":"661'6","last_deep_scrub_stamp":"2024-02-22T19:54:23.541989+0000","last_clean_scrub_stamp":"2024-02-22T19:54:23.541989+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1526,"num_read_kb":1521,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"505'36","reported_seq":130,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932999+0000","last_change":"2024-02-22T19:53:56.531080+0000","last_active":"2024-02-22T19:54:46.932999+0000","last_peered":"2024-02-22T19:54:46.932999+0000","last_clean":"2024-02-22T19:54:46.932999+0000","last_became_active":"2024-02-22T19:51:15.066428+0000","last_became_peered":"2024-02-22T19:51:15.066428+0000","last_unstale":"2024-02-22T19:54:46.932999+0000","last_undegraded":"2024-02-22T19:54:46.932999+0000","last_fullsized":"2024-02-22T19:54:46.932999+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'36","last_scrub_stamp":"2024-02-22T19:53:56.530981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:56.530981+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"617'41","reported_seq":771,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.153347+0000","last_change":"2024-02-22T19:54:04.650672+0000","last_active":"2024-02-22T19:54:48.153347+0000","last_peered":"2024-02-22T19:54:48.153347+0000","last_clean":"2024-02-22T19:54:48.153347+0000","last_became_active":"2024-02-22T19:52:36.812503+0000","last_became_peered":"2024-02-22T19:52:36.812503+0000","last_unstale":"2024-02-22T19:54:48.153347+0000","last_undegraded":"2024-02-22T19:54:48.153347+0000","last_fullsized":"2024-02-22T19:54:48.153347+0000","mapping_epoch":616,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":617,"parent":"0.0","parent_split_bits":0,"last_scrub":"617'41","last_scrub_stamp":"2024-02-22T19:54:04.650588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:54:04.650588+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":169,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816383+0000","last_change":"2024-02-22T19:53:52.962557+0000","last_active":"2024-02-22T19:54:46.816383+0000","last_peered":"2024-02-22T19:54:46.816383+0000","last_clean":"2024-02-22T19:54:46.816383+0000","last_became_active":"2024-02-22T19:51:15.079242+0000","last_became_peered":"2024-02-22T19:51:15.079242+0000","last_unstale":"2024-02-22T19:54:46.816383+0000","last_undegraded":"2024-02-22T19:54:46.816383+0000","last_fullsized":"2024-02-22T19:54:46.816383+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:52.962472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:53:52.962472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":144,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932117+0000","last_change":"2024-02-22T19:54:15.204368+0000","last_active":"2024-02-22T19:54:46.932117+0000","last_peered":"2024-02-22T19:54:46.932117+0000","last_clean":"2024-02-22T19:54:46.932117+0000","last_became_active":"2024-02-22T19:49:56.723819+0000","last_became_peered":"2024-02-22T19:49:56.723819+0000","last_unstale":"2024-02-22T19:54:46.932117+0000","last_undegraded":"2024-02-22T19:54:46.932117+0000","last_fullsized":"2024-02-22T19:54:46.932117+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:15.204280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:54:15.204280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":171,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932676+0000","last_change":"2024-02-22T19:54:08.007017+0000","last_active":"2024-02-22T19:54:46.932676+0000","last_peered":"2024-02-22T19:54:46.932676+0000","last_clean":"2024-02-22T19:54:46.932676+0000","last_became_active":"2024-02-22T19:52:33.825668+0000","last_became_peered":"2024-02-22T19:52:33.825668+0000","last_unstale":"2024-02-22T19:54:46.932676+0000","last_undegraded":"2024-02-22T19:54:46.932676+0000","last_fullsized":"2024-02-22T19:54:46.932676+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:08.006949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:54:08.006949+0000","last_clean_scrub_stamp":"2024-02-22T19:54:08.006949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"506'47","reported_seq":161,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933780+0000","last_change":"2024-02-22T19:53:45.679921+0000","last_active":"2024-02-22T19:54:46.933780+0000","last_peered":"2024-02-22T19:54:46.933780+0000","last_clean":"2024-02-22T19:54:46.933780+0000","last_became_active":"2024-02-22T19:52:33.823004+0000","last_became_peered":"2024-02-22T19:52:33.823004+0000","last_unstale":"2024-02-22T19:54:46.933780+0000","last_undegraded":"2024-02-22T19:54:46.933780+0000","last_fullsized":"2024-02-22T19:54:46.933780+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'47","last_scrub_stamp":"2024-02-22T19:53:45.679854+0000","last_deep_scrub":"463'37","last_deep_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_clean_scrub_stamp":"2024-02-22T19:53:45.679854+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":187,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816202+0000","last_change":"2024-02-22T19:54:03.127185+0000","last_active":"2024-02-22T19:54:46.816202+0000","last_peered":"2024-02-22T19:54:46.816202+0000","last_clean":"2024-02-22T19:54:46.816202+0000","last_became_active":"2024-02-22T19:52:33.821391+0000","last_became_peered":"2024-02-22T19:52:33.821391+0000","last_unstale":"2024-02-22T19:54:46.816202+0000","last_undegraded":"2024-02-22T19:54:46.816202+0000","last_fullsized":"2024-02-22T19:54:46.816202+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:03.127102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:03.127102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":160,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816385+0000","last_change":"2024-02-22T19:54:18.357389+0000","last_active":"2024-02-22T19:54:46.816385+0000","last_peered":"2024-02-22T19:54:46.816385+0000","last_clean":"2024-02-22T19:54:46.816385+0000","last_became_active":"2024-02-22T19:49:38.809765+0000","last_became_peered":"2024-02-22T19:49:38.809765+0000","last_unstale":"2024-02-22T19:54:46.816385+0000","last_undegraded":"2024-02-22T19:54:46.816385+0000","last_fullsized":"2024-02-22T19:54:46.816385+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:18.357305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:13.199920+0000","last_clean_scrub_stamp":"2024-02-22T19:54:18.357305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":116,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933467+0000","last_change":"2024-02-22T19:54:05.370689+0000","last_active":"2024-02-22T19:54:46.933467+0000","last_peered":"2024-02-22T19:54:46.933467+0000","last_clean":"2024-02-22T19:54:46.933467+0000","last_became_active":"2024-02-22T19:51:15.055763+0000","last_became_peered":"2024-02-22T19:51:15.055763+0000","last_unstale":"2024-02-22T19:54:46.933467+0000","last_undegraded":"2024-02-22T19:54:46.933467+0000","last_fullsized":"2024-02-22T19:54:46.933467+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:05.370612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:54:05.370612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764013+0000","last_change":"2024-02-22T19:54:39.708777+0000","last_active":"2024-02-22T19:54:46.764013+0000","last_peered":"2024-02-22T19:54:46.764013+0000","last_clean":"2024-02-22T19:54:46.764013+0000","last_became_active":"2024-02-22T19:52:33.822386+0000","last_became_peered":"2024-02-22T19:52:33.822386+0000","last_unstale":"2024-02-22T19:54:46.764013+0000","last_undegraded":"2024-02-22T19:54:46.764013+0000","last_fullsized":"2024-02-22T19:54:46.764013+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:39.708656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:54:39.708656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"506'30","reported_seq":261,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.931776+0000","last_change":"2024-02-22T19:53:46.927809+0000","last_active":"2024-02-22T19:54:46.931776+0000","last_peered":"2024-02-22T19:54:46.931776+0000","last_clean":"2024-02-22T19:54:46.931776+0000","last_became_active":"2024-02-22T19:52:42.738773+0000","last_became_peered":"2024-02-22T19:52:42.738773+0000","last_unstale":"2024-02-22T19:54:46.931776+0000","last_undegraded":"2024-02-22T19:54:46.931776+0000","last_fullsized":"2024-02-22T19:54:46.931776+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'30","last_scrub_stamp":"2024-02-22T19:53:46.927741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:46.927741+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"650'28","reported_seq":368,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.756439+0000","last_change":"2024-02-22T19:53:40.789921+0000","last_active":"2024-02-22T19:54:47.756439+0000","last_peered":"2024-02-22T19:54:47.756439+0000","last_clean":"2024-02-22T19:54:47.756439+0000","last_became_active":"2024-02-22T19:53:36.799786+0000","last_became_peered":"2024-02-22T19:53:36.799786+0000","last_unstale":"2024-02-22T19:54:47.756439+0000","last_undegraded":"2024-02-22T19:54:47.756439+0000","last_fullsized":"2024-02-22T19:54:47.756439+0000","mapping_epoch":648,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":649,"parent":"0.0","parent_split_bits":0,"last_scrub":"650'28","last_scrub_stamp":"2024-02-22T19:53:40.789798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:40.789798+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":591,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755830+0000","last_change":"2024-02-22T19:54:28.525916+0000","last_active":"2024-02-22T19:54:42.755830+0000","last_peered":"2024-02-22T19:54:42.755830+0000","last_clean":"2024-02-22T19:54:42.755830+0000","last_became_active":"2024-02-22T19:52:33.821549+0000","last_became_peered":"2024-02-22T19:52:33.821549+0000","last_unstale":"2024-02-22T19:54:42.755830+0000","last_undegraded":"2024-02-22T19:54:42.755830+0000","last_fullsized":"2024-02-22T19:54:42.755830+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:28.525851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:28.525851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":679,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659183+0000","last_change":"2024-02-22T19:53:20.024193+0000","last_active":"2024-02-22T19:54:47.659183+0000","last_peered":"2024-02-22T19:54:47.659183+0000","last_clean":"2024-02-22T19:54:47.659183+0000","last_became_active":"2024-02-22T19:49:38.795530+0000","last_became_peered":"2024-02-22T19:49:38.795530+0000","last_unstale":"2024-02-22T19:54:47.659183+0000","last_undegraded":"2024-02-22T19:54:47.659183+0000","last_fullsized":"2024-02-22T19:54:47.659183+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:20.024107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:53:20.024107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":134,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932910+0000","last_change":"2024-02-22T19:54:19.464744+0000","last_active":"2024-02-22T19:54:46.932910+0000","last_peered":"2024-02-22T19:54:46.932910+0000","last_clean":"2024-02-22T19:54:46.932910+0000","last_became_active":"2024-02-22T19:52:42.756445+0000","last_became_peered":"2024-02-22T19:52:42.756445+0000","last_unstale":"2024-02-22T19:54:46.932910+0000","last_undegraded":"2024-02-22T19:54:46.932910+0000","last_fullsized":"2024-02-22T19:54:46.932910+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:19.464680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:19.464680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"506'49","reported_seq":85,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934420+0000","last_change":"2024-02-22T19:54:00.823662+0000","last_active":"2024-02-22T19:54:46.934420+0000","last_peered":"2024-02-22T19:54:46.934420+0000","last_clean":"2024-02-22T19:54:46.934420+0000","last_became_active":"2024-02-22T19:52:42.737608+0000","last_became_peered":"2024-02-22T19:52:42.737608+0000","last_unstale":"2024-02-22T19:54:46.934420+0000","last_undegraded":"2024-02-22T19:54:46.934420+0000","last_fullsized":"2024-02-22T19:54:46.934420+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'49","last_scrub_stamp":"2024-02-22T19:54:00.823582+0000","last_deep_scrub":"506'49","last_deep_scrub_stamp":"2024-02-22T19:54:00.823582+0000","last_clean_scrub_stamp":"2024-02-22T19:54:00.823582+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"622'27","reported_seq":1159,"reported_epoch":667,"state":"peering","last_fresh":"2024-02-22T19:54:44.921517+0000","last_change":"2024-02-22T19:54:44.921802+0000","last_active":"2024-02-22T19:54:44.921517+0000","last_peered":"2024-02-22T19:54:44.563736+0000","last_clean":"2024-02-22T19:54:44.563736+0000","last_became_active":"2024-02-22T19:52:43.728191+0000","last_became_peered":"2024-02-22T19:52:43.728191+0000","last_unstale":"2024-02-22T19:54:44.921517+0000","last_undegraded":"2024-02-22T19:54:44.921517+0000","last_fullsized":"2024-02-22T19:54:44.921517+0000","mapping_epoch":667,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":666,"parent":"0.0","parent_split_bits":0,"last_scrub":"601'25","last_scrub_stamp":"2024-02-22T19:52:32.737491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:52:32.737491+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935085+0000","last_change":"2024-02-22T19:54:23.859812+0000","last_active":"2024-02-22T19:54:46.935085+0000","last_peered":"2024-02-22T19:54:46.935085+0000","last_clean":"2024-02-22T19:54:46.935085+0000","last_became_active":"2024-02-22T19:52:42.738118+0000","last_became_peered":"2024-02-22T19:52:42.738118+0000","last_unstale":"2024-02-22T19:54:46.935085+0000","last_undegraded":"2024-02-22T19:54:46.935085+0000","last_fullsized":"2024-02-22T19:54:46.935085+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:23.859734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:52.807890+0000","last_clean_scrub_stamp":"2024-02-22T19:54:23.859734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934522+0000","last_change":"2024-02-22T19:53:30.827202+0000","last_active":"2024-02-22T19:54:46.934522+0000","last_peered":"2024-02-22T19:54:46.934522+0000","last_clean":"2024-02-22T19:54:46.934522+0000","last_became_active":"2024-02-22T19:52:42.740401+0000","last_became_peered":"2024-02-22T19:52:42.740401+0000","last_unstale":"2024-02-22T19:54:46.934522+0000","last_undegraded":"2024-02-22T19:54:46.934522+0000","last_fullsized":"2024-02-22T19:54:46.934522+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:30.827120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:30.827120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":696,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.756279+0000","last_change":"2024-02-22T19:54:27.510017+0000","last_active":"2024-02-22T19:54:42.756279+0000","last_peered":"2024-02-22T19:54:42.756279+0000","last_clean":"2024-02-22T19:54:42.756279+0000","last_became_active":"2024-02-22T19:51:15.184129+0000","last_became_peered":"2024-02-22T19:51:15.184129+0000","last_unstale":"2024-02-22T19:54:42.756279+0000","last_undegraded":"2024-02-22T19:54:42.756279+0000","last_fullsized":"2024-02-22T19:54:42.756279+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:27.509948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:54:27.509948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"657'50","reported_seq":891,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755514+0000","last_change":"2024-02-22T19:53:57.524296+0000","last_active":"2024-02-22T19:54:42.755514+0000","last_peered":"2024-02-22T19:54:42.755514+0000","last_clean":"2024-02-22T19:54:42.755514+0000","last_became_active":"2024-02-22T19:51:15.079954+0000","last_became_peered":"2024-02-22T19:51:15.079954+0000","last_unstale":"2024-02-22T19:54:42.755514+0000","last_undegraded":"2024-02-22T19:54:42.755514+0000","last_fullsized":"2024-02-22T19:54:42.755514+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"657'50","last_scrub_stamp":"2024-02-22T19:53:57.524223+0000","last_deep_scrub":"657'50","last_deep_scrub_stamp":"2024-02-22T19:53:57.524223+0000","last_clean_scrub_stamp":"2024-02-22T19:53:57.524223+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":69,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934546+0000","last_change":"2024-02-22T19:54:38.938568+0000","last_active":"2024-02-22T19:54:46.934546+0000","last_peered":"2024-02-22T19:54:46.934546+0000","last_clean":"2024-02-22T19:54:46.934546+0000","last_became_active":"2024-02-22T19:52:42.745561+0000","last_became_peered":"2024-02-22T19:52:42.745561+0000","last_unstale":"2024-02-22T19:54:46.934546+0000","last_undegraded":"2024-02-22T19:54:46.934546+0000","last_fullsized":"2024-02-22T19:54:46.934546+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:38.938481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:38.938481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763206+0000","last_change":"2024-02-22T19:54:09.690939+0000","last_active":"2024-02-22T19:54:46.763206+0000","last_peered":"2024-02-22T19:54:46.763206+0000","last_clean":"2024-02-22T19:54:46.763206+0000","last_became_active":"2024-02-22T19:52:33.820066+0000","last_became_peered":"2024-02-22T19:52:33.820066+0000","last_unstale":"2024-02-22T19:54:46.763206+0000","last_undegraded":"2024-02-22T19:54:46.763206+0000","last_fullsized":"2024-02-22T19:54:46.763206+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:09.690845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:09.690845+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764153+0000","last_change":"2024-02-22T19:54:13.718534+0000","last_active":"2024-02-22T19:54:46.764153+0000","last_peered":"2024-02-22T19:54:46.764153+0000","last_clean":"2024-02-22T19:54:46.764153+0000","last_became_active":"2024-02-22T19:52:33.821421+0000","last_became_peered":"2024-02-22T19:52:33.821421+0000","last_unstale":"2024-02-22T19:54:46.764153+0000","last_undegraded":"2024-02-22T19:54:46.764153+0000","last_fullsized":"2024-02-22T19:54:46.764153+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:13.718443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:13.718443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":674,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755497+0000","last_change":"2024-02-22T19:54:37.634025+0000","last_active":"2024-02-22T19:54:42.755497+0000","last_peered":"2024-02-22T19:54:42.755497+0000","last_clean":"2024-02-22T19:54:42.755497+0000","last_became_active":"2024-02-22T19:49:56.726891+0000","last_became_peered":"2024-02-22T19:49:56.726891+0000","last_unstale":"2024-02-22T19:54:42.755497+0000","last_undegraded":"2024-02-22T19:54:42.755497+0000","last_fullsized":"2024-02-22T19:54:42.755497+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:37.633949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:37.633949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"506'39","reported_seq":54,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.945041+0000","last_change":"2024-02-22T19:54:10.088224+0000","last_active":"2024-02-22T19:54:42.945041+0000","last_peered":"2024-02-22T19:54:42.945041+0000","last_clean":"2024-02-22T19:54:42.945041+0000","last_became_active":"2024-02-22T19:53:28.765529+0000","last_became_peered":"2024-02-22T19:53:28.765529+0000","last_unstale":"2024-02-22T19:54:42.945041+0000","last_undegraded":"2024-02-22T19:54:42.945041+0000","last_fullsized":"2024-02-22T19:54:42.945041+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":645,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'39","last_scrub_stamp":"2024-02-22T19:54:10.088160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:10.088160+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"137'2","reported_seq":772,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.756105+0000","last_change":"2024-02-22T19:53:26.678137+0000","last_active":"2024-02-22T19:54:42.756105+0000","last_peered":"2024-02-22T19:54:42.756105+0000","last_clean":"2024-02-22T19:54:42.756105+0000","last_became_active":"2024-02-22T19:53:06.913551+0000","last_became_peered":"2024-02-22T19:53:06.913551+0000","last_unstale":"2024-02-22T19:54:42.756105+0000","last_undegraded":"2024-02-22T19:54:42.756105+0000","last_fullsized":"2024-02-22T19:54:42.756105+0000","mapping_epoch":632,"log_start":"137'2","ondisk_log_start":"137'2","created":62,"last_epoch_clean":633,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:53:26.678008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:26.678008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934675+0000","last_change":"2024-02-22T19:53:33.788663+0000","last_active":"2024-02-22T19:54:46.934675+0000","last_peered":"2024-02-22T19:54:46.934675+0000","last_clean":"2024-02-22T19:54:46.934675+0000","last_became_active":"2024-02-22T19:52:42.733083+0000","last_became_peered":"2024-02-22T19:52:42.733083+0000","last_unstale":"2024-02-22T19:54:46.934675+0000","last_undegraded":"2024-02-22T19:54:46.934675+0000","last_fullsized":"2024-02-22T19:54:46.934675+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:33.788576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:33.788576+0000","last_clean_scrub_stamp":"2024-02-22T19:53:33.788576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"506'27","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935055+0000","last_change":"2024-02-22T19:53:56.792103+0000","last_active":"2024-02-22T19:54:46.935055+0000","last_peered":"2024-02-22T19:54:46.935055+0000","last_clean":"2024-02-22T19:54:46.935055+0000","last_became_active":"2024-02-22T19:52:42.737550+0000","last_became_peered":"2024-02-22T19:52:42.737550+0000","last_unstale":"2024-02-22T19:54:46.935055+0000","last_undegraded":"2024-02-22T19:54:46.935055+0000","last_fullsized":"2024-02-22T19:54:46.935055+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'27","last_scrub_stamp":"2024-02-22T19:53:56.792023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:56.792023+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.762999+0000","last_change":"2024-02-22T19:53:49.657738+0000","last_active":"2024-02-22T19:54:46.762999+0000","last_peered":"2024-02-22T19:54:46.762999+0000","last_clean":"2024-02-22T19:54:46.762999+0000","last_became_active":"2024-02-22T19:52:33.819026+0000","last_became_peered":"2024-02-22T19:52:33.819026+0000","last_unstale":"2024-02-22T19:54:46.762999+0000","last_undegraded":"2024-02-22T19:54:46.762999+0000","last_fullsized":"2024-02-22T19:54:46.762999+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:49.657650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:53:49.657650+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":726,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.944374+0000","last_change":"2024-02-22T19:54:19.386189+0000","last_active":"2024-02-22T19:54:42.944374+0000","last_peered":"2024-02-22T19:54:42.944374+0000","last_clean":"2024-02-22T19:54:42.944374+0000","last_became_active":"2024-02-22T19:52:42.745708+0000","last_became_peered":"2024-02-22T19:52:42.745708+0000","last_unstale":"2024-02-22T19:54:42.944374+0000","last_undegraded":"2024-02-22T19:54:42.944374+0000","last_fullsized":"2024-02-22T19:54:42.944374+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:19.386106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:58.807220+0000","last_clean_scrub_stamp":"2024-02-22T19:54:19.386106+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":187,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816056+0000","last_change":"2024-02-22T19:54:35.300025+0000","last_active":"2024-02-22T19:54:46.816056+0000","last_peered":"2024-02-22T19:54:46.816056+0000","last_clean":"2024-02-22T19:54:46.816056+0000","last_became_active":"2024-02-22T19:52:42.746960+0000","last_became_peered":"2024-02-22T19:52:42.746960+0000","last_unstale":"2024-02-22T19:54:46.816056+0000","last_undegraded":"2024-02-22T19:54:46.816056+0000","last_fullsized":"2024-02-22T19:54:46.816056+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:35.299944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:04.963830+0000","last_clean_scrub_stamp":"2024-02-22T19:54:35.299944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"506'45","reported_seq":114,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935179+0000","last_change":"2024-02-22T19:54:14.861000+0000","last_active":"2024-02-22T19:54:46.935179+0000","last_peered":"2024-02-22T19:54:46.935179+0000","last_clean":"2024-02-22T19:54:46.935179+0000","last_became_active":"2024-02-22T19:53:29.522364+0000","last_became_peered":"2024-02-22T19:53:29.522364+0000","last_unstale":"2024-02-22T19:54:46.935179+0000","last_undegraded":"2024-02-22T19:54:46.935179+0000","last_fullsized":"2024-02-22T19:54:46.935179+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":645,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'45","last_scrub_stamp":"2024-02-22T19:54:14.860912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:14.860912+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":59,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":677,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.733100+0000","last_change":"2024-02-22T19:54:48.733100+0000","last_active":"2024-02-22T19:54:48.733100+0000","last_peered":"2024-02-22T19:54:48.733100+0000","last_clean":"2024-02-22T19:54:48.733100+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:54:48.733100+0000","last_undegraded":"2024-02-22T19:54:48.733100+0000","last_fullsized":"2024-02-22T19:54:48.733100+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:48.733019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:16.942709+0000","last_clean_scrub_stamp":"2024-02-22T19:54:48.733019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":320,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935000+0000","last_change":"2024-02-22T19:54:16.914427+0000","last_active":"2024-02-22T19:54:46.935000+0000","last_peered":"2024-02-22T19:54:46.935000+0000","last_clean":"2024-02-22T19:54:46.935000+0000","last_became_active":"2024-02-22T19:52:44.722428+0000","last_became_peered":"2024-02-22T19:52:44.722428+0000","last_unstale":"2024-02-22T19:54:46.935000+0000","last_undegraded":"2024-02-22T19:54:46.935000+0000","last_fullsized":"2024-02-22T19:54:46.935000+0000","mapping_epoch":622,"log_start":"85'2","ondisk_log_start":"85'2","created":62,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:54:16.914332+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:48:07.111947+0000","last_clean_scrub_stamp":"2024-02-22T19:54:16.914332+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":78,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":790,"reported_epoch":666,"state":"active+clean","last_fresh":"2024-02-22T19:54:42.755837+0000","last_change":"2024-02-22T19:54:25.503848+0000","last_active":"2024-02-22T19:54:42.755837+0000","last_peered":"2024-02-22T19:54:42.755837+0000","last_clean":"2024-02-22T19:54:42.755837+0000","last_became_active":"2024-02-22T19:52:42.749256+0000","last_became_peered":"2024-02-22T19:52:42.749256+0000","last_unstale":"2024-02-22T19:54:42.755837+0000","last_undegraded":"2024-02-22T19:54:42.755837+0000","last_fullsized":"2024-02-22T19:54:42.755837+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:54:25.503746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:25.503746+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"661'35","reported_seq":86,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934831+0000","last_change":"2024-02-22T19:54:22.867075+0000","last_active":"2024-02-22T19:54:46.934831+0000","last_peered":"2024-02-22T19:54:46.934831+0000","last_clean":"2024-02-22T19:54:46.934831+0000","last_became_active":"2024-02-22T19:52:42.744111+0000","last_became_peered":"2024-02-22T19:52:42.744111+0000","last_unstale":"2024-02-22T19:54:46.934831+0000","last_undegraded":"2024-02-22T19:54:46.934831+0000","last_fullsized":"2024-02-22T19:54:46.934831+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"661'35","last_scrub_stamp":"2024-02-22T19:54:22.866987+0000","last_deep_scrub":"661'35","last_deep_scrub_stamp":"2024-02-22T19:54:22.866987+0000","last_clean_scrub_stamp":"2024-02-22T19:54:22.866987+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":37,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1526,"num_read_kb":1521,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":12964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1515,"num_read_kb":1272,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":930,"num_bytes_recovered":20698,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1049,"ondisk_log_size":1049,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":66,"acting":66,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633267002,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":899060,"kb_used_data":6252,"kb_used_omap":0,"kb_used_meta":892800,"kb_avail":92846092,"statfs":{"total":95995035648,"available":95074398208,"internally_reserved":0,"allocated":6402048,"data_stored":3631674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":914227200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.35099999999999998,"5min":0.24399999999999999,"15min":0.219},"max":{"1min":0.78500000000000003,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":1.7889999999999999,"15min":1.3140000000000001},"min":{"1min":0.313,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76000000000000001,"5min":17.716000000000001,"15min":17.716000000000001},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":1.675,"15min":1.7470000000000001},"min":{"1min":0.32200000000000001,"5min":0.22500000000000001,"15min":0.20000000000000001},"max":{"1min":0.80600000000000005,"5min":13.987,"15min":36.509},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":1.931,"15min":1.8169999999999999},"min":{"1min":0.39000000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":0.72799999999999998,"5min":11.532,"15min":20.923999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":2.024,"15min":1.982},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.24299999999999999},"max":{"1min":0.65900000000000003,"5min":23.856999999999999,"15min":42.377000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":2.1840000000000002,"15min":2.2290000000000001},"min":{"1min":0.38,"5min":0.27100000000000002,"15min":0.23499999999999999},"max":{"1min":0.76800000000000002,"5min":20.815000000000001,"15min":39.520000000000003},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":1.766,"15min":1.9730000000000001},"min":{"1min":0.36699999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71599999999999997,"5min":10.436999999999999,"15min":37.905000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.859,"15min":2.0710000000000002},"min":{"1min":0.48599999999999999,"5min":0.30199999999999999,"15min":0.27400000000000002},"max":{"1min":0.80900000000000005,"5min":15.047000000000001,"15min":38.25},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":1.2230000000000001,"15min":0.95699999999999996},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.623,"5min":9.9930000000000003,"15min":9.9930000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":1.0860000000000001,"15min":1.004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27000000000000002},"max":{"1min":0.68500000000000005,"5min":9.8390000000000004,"15min":9.8390000000000004},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.2809999999999999,"15min":1.1599999999999999},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82599999999999996,"5min":14.023,"15min":14.023},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":1.427,"15min":1.0700000000000001},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":9.6530000000000005,"15min":9.6530000000000005},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.498,"15min":1.0680000000000001},"min":{"1min":0.29899999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.75600000000000001,"5min":11.583,"15min":11.583},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.8089999999999999,"15min":1.345},"min":{"1min":0.315,"5min":0.315,"15min":0.28399999999999997},"max":{"1min":0.73899999999999999,"5min":13.143000000000001,"15min":13.143000000000001},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528639,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":604132,"kb_used_data":6236,"kb_used_omap":0,"kb_used_meta":597888,"kb_avail":93141020,"statfs":{"total":95995035648,"available":95376404480,"internally_reserved":0,"allocated":6385664,"data_stored":3631342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":612237312},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":1.333,"15min":1.111},"min":{"1min":0.35899999999999999,"5min":0.24299999999999999,"15min":0.22500000000000001},"max":{"1min":0.71199999999999997,"5min":14.731,"15min":14.731},"last":0.495},{"interface":"front","average":{"1min":0.52800000000000002,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.25900000000000001},"max":{"1min":0.85999999999999999,"5min":10.186999999999999,"15min":10.186999999999999},"last":0.63}]},{"osd":1,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.954,"15min":1.804},"min":{"1min":0.36399999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.79400000000000004,"5min":27.132999999999999,"15min":44.101999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":1.625,"15min":1.6679999999999999},"min":{"1min":0.41399999999999998,"5min":0.26300000000000001,"15min":0.222},"max":{"1min":0.82599999999999996,"5min":27.366,"15min":44.018000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":3.3420000000000001,"15min":2.5259999999999998},"min":{"1min":0.40500000000000003,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":0.80800000000000005,"5min":93.703000000000003,"15min":93.703000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":3.004,"15min":2.3959999999999999},"min":{"1min":0.40899999999999997,"5min":0.27400000000000002,"15min":0.24299999999999999},"max":{"1min":0.879,"5min":93.260999999999996,"15min":93.260999999999996},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.6399999999999999,"15min":1.514},"min":{"1min":0.44800000000000001,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.77200000000000002,"5min":9.548,"15min":18.661999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":1.4450000000000001,"15min":1.647},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.28499999999999998},"max":{"1min":0.89800000000000002,"5min":7.6269999999999998,"15min":19.154},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":1.0209999999999999,"15min":0.81999999999999995},"min":{"1min":0.317,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.73399999999999999,"5min":10.06,"15min":10.06},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.3169999999999999,"15min":0.98699999999999999},"min":{"1min":0.35499999999999998,"5min":0.29399999999999998,"15min":0.253},"max":{"1min":0.76800000000000002,"5min":10.827,"15min":10.827},"last":0.75}]},{"osd":5,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.155,"15min":1.0880000000000001},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.255},"max":{"1min":0.754,"5min":7.8630000000000004,"15min":21.675000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.0700000000000001,"15min":0.89900000000000002},"min":{"1min":0.37,"5min":0.318,"15min":0.28199999999999997},"max":{"1min":0.74099999999999999,"5min":8.4789999999999992,"15min":21.634},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Feb 22 19:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":1.3340000000000001,"15min":1.0529999999999999},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.26800000000000002},"max":{"1min":0.84399999999999997,"5min":8.6470000000000002,"15min":22.138999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":1.3859999999999999,"15min":1.1240000000000001},"min":{"1min":0.45000000000000001,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.91000000000000003,"5min":15.99,"15min":22.678000000000001},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575192,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904784,"kb_used_data":13384,"kb_used_omap":0,"kb_used_meta":891392,"kb_avail":92840368,"statfs":{"total":95995035648,"available":95068536832,"internally_reserved":0,"allocated":13705216,"data_stored":10907093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912785408},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":1.319,"15min":1.819},"min":{"1min":0.216,"5min":0.216,"15min":0.19900000000000001},"max":{"1min":0.48899999999999999,"5min":14.710000000000001,"15min":51.091000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":1.4530000000000001,"15min":1.857},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":0.56100000000000005,"5min":7.282,"15min":51.289999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":2.0139999999999998,"15min":2.5190000000000001},"min":{"1min":0.254,"5min":0.252,"15min":0.191},"max":{"1min":0.53800000000000003,"5min":42.884,"15min":51.027000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":1.954,"15min":2.5430000000000001},"min":{"1min":0.307,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.63700000000000001,"5min":43,"15min":51.369999999999997},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":1.7130000000000001,"15min":2.2069999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.23400000000000001},"max":{"1min":0.59099999999999997,"5min":43.295999999999999,"15min":82.034000000000006},"last":0.31},{"interface":"front","average":{"1min":0.42399999999999999,"5min":2.3290000000000002,"15min":2.7010000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.20200000000000001},"max":{"1min":0.67300000000000004,"5min":43.338000000000001,"15min":81.765000000000001},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:54:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":2.0270000000000001,"15min":2.0750000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.79300000000000004,"5min":14.663,"15min":51.158999999999999},"last":0.38},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.4910000000000001,"15min":1.466},"min":{"1min":0.37,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.78900000000000003,"5min":8.8390000000000004,"15min":51.276000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":2.0750000000000002,"15min":2.1160000000000001},"min":{"1min":0.42899999999999999,"5min":0.29399999999999998,"15min":0.25800000000000001},"max":{"1min":1.0760000000000001,"5min":14.901,"15min":51.183999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":1.6990000000000001,"15min":1.8069999999999999},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95999999999999996,"5min":6.6189999999999998,"15min":51.212000000000003},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":1.2849999999999999,"15min":1.494},"min":{"1min":0.436,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":7.9400000000000004,"15min":51.079999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":2.0169999999999999,"15min":2.1179999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33100000000000002},"max":{"1min":1.044,"5min":14.741,"15min":51.119999999999997},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.47,"15min":1.839},"min":{"1min":0.41799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":11.096,"15min":51.244999999999997},"last":0.504},{"interface":"front","average":{"1min":0.623,"5min":1.258,"15min":1.5529999999999999},"min":{"1min":0.438,"5min":0.314,"15min":0.30399999999999999},"max":{"1min":0.83399999999999996,"5min":7.5579999999999998,"15min":51.317999999999998},"last":0.438}]}]},{"osd":0,"up_from":8,"seq":34359738717,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307580,"kb_used_data":6260,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93437572,"statfs":{"total":95995035648,"available":95680073728,"internally_reserved":0,"allocated":6410240,"data_stored":3632613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":1.1830000000000001,"15min":1.367},"min":{"1min":0.21299999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.44500000000000001,"5min":8.3659999999999997,"15min":24.748999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":1.379,"15min":1.6990000000000001},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51100000000000001,"5min":7.7069999999999999,"15min":25.222999999999999},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":1.361,"15min":1.6819999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.21099999999999999},"max":{"1min":0.51700000000000002,"5min":9.8699999999999992,"15min":30.353999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":2.633,"15min":2.3149999999999999},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.69699999999999995,"5min":22.062999999999999,"15min":30.404},"last":0.375}]},{"osd":3,"last update":"Thu Feb 22 19:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.7490000000000001,"15min":1.9730000000000001},"min":{"1min":0.27400000000000002,"5min":0.185,"15min":0.185},"max":{"1min":0.71999999999999997,"5min":15.18,"15min":32.262},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":1.579,"15min":1.913},"min":{"1min":0.307,"5min":0.216,"15min":0.216},"max":{"1min":0.74399999999999999,"5min":8.4169999999999998,"15min":29.065000000000001},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.4450000000000001,"15min":1.159},"min":{"1min":0.42699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78900000000000003,"5min":11.281000000000001,"15min":11.281000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.3120000000000001,"15min":1.2190000000000001},"min":{"1min":0.39000000000000001,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":0.73499999999999999,"5min":9.8369999999999997,"15min":9.8369999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":1.341,"15min":1.419},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.29399999999999998},"max":{"1min":0.69199999999999995,"5min":7.0039999999999996,"15min":56.829000000000001},"last":0.505},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.7430000000000001,"15min":1.53},"min":{"1min":0.39200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77300000000000002,"5min":11.23,"15min":57.338000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:54:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":1.296,"15min":1.177},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.27400000000000002},"max":{"1min":0.82799999999999996,"5min":9.6969999999999992,"15min":9.6969999999999992},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":1.212,"15min":1.2549999999999999},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.80000000000000004,"5min":5.6200000000000001,"15min":10.093999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":1.3200000000000001,"15min":1.2909999999999999},"min":{"1min":0.42199999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":11.304,"15min":11.304},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.62,"5min":1.454,"15min":1.4139999999999999},"min":{"1min":0.39200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":11.631,"15min":11.631},"last":0.64000000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411665,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":905736,"kb_used_data":6208,"kb_used_omap":0,"kb_used_meta":899520,"kb_avail":92839416,"statfs":{"total":95995035648,"available":95067561984,"internally_reserved":0,"allocated":6356992,"data_stored":3611412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":921108480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":1.7130000000000001,"15min":1.7090000000000001},"min":{"1min":0.22,"5min":0.20399999999999999,"15min":0.19900000000000001},"max":{"1min":0.498,"5min":13.205,"15min":16.137},"last":0.316},{"interface":"front","average":{"1min":0.372,"5min":2.2719999999999998,"15min":2.242},"min":{"1min":0.22900000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.49399999999999999,"5min":14.148999999999999,"15min":27.361999999999998},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.9470000000000001,"15min":2.2450000000000001},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.20200000000000001},"max":{"1min":0.69899999999999995,"5min":31.641999999999999,"15min":48.305},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":2.3300000000000001,"15min":2.4159999999999999},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.65500000000000003,"5min":31.969000000000001,"15min":50.448999999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:54:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":2.1779999999999999,"15min":2.1880000000000002},"min":{"1min":0.313,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.73099999999999998,"5min":18.280000000000001,"15min":18.280000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":2.806,"15min":2.5049999999999999},"min":{"1min":0.27300000000000002,"5min":0.21299999999999999,"15min":0.183},"max":{"1min":1.274,"5min":13.925000000000001,"15min":28.263999999999999},"last":0.373}]},{"osd":4,"last update":"Thu Feb 22 19:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":2.2639999999999998,"15min":1.8640000000000001},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.26700000000000002},"max":{"1min":0.84599999999999997,"5min":13.696999999999999,"15min":13.696999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.788,"15min":1.7210000000000001},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0649999999999999,"5min":12.576000000000001,"15min":13.478999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Feb 22 19:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":2.3879999999999999,"15min":2.0840000000000001},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":13.429,"15min":40.633000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":2.714,"15min":2.2650000000000001},"min":{"1min":0.35899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86799999999999999,"5min":13.741,"15min":40.997999999999998},"last":0.499}]},{"osd":6,"last update":"Thu Feb 22 19:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":2.113,"15min":1.885},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92000000000000004,"5min":13.766,"15min":21.640000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":1.7350000000000001,"15min":1.8049999999999999},"min":{"1min":0.45700000000000002,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.95299999999999996,"5min":10.103,"15min":13.51},"last":0.41899999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":2.456,"15min":2.1339999999999999},"min":{"1min":0.434,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95099999999999996,"5min":13.789999999999999,"15min":14.199},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.9199999999999999,"15min":1.9239999999999999},"min":{"1min":0.437,"5min":0.36099999999999999,"15min":0.312},"max":{"1min":1.0089999999999999,"5min":12.608000000000001,"15min":18.021999999999998},"last":0.44400000000000001}]}]},{"osd":3,"up_from":26,"seq":111669150027,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1186268,"kb_used_data":6228,"kb_used_omap":0,"kb_used_meta":1180032,"kb_avail":92558884,"statfs":{"total":95995035648,"available":94780297216,"internally_reserved":0,"allocated":6377472,"data_stored":3606862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1208352768},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":1.3939999999999999,"15min":1.6830000000000001},"min":{"1min":0.26200000000000001,"5min":0.20399999999999999,"15min":0.187},"max":{"1min":0.57699999999999996,"5min":17.056000000000001,"15min":17.056000000000001},"last":0.63},{"interface":"front","average":{"1min":0.41899999999999998,"5min":1.6200000000000001,"15min":1.9199999999999999},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.219},"max":{"1min":0.58999999999999997,"5min":13.451000000000001,"15min":18.172000000000001},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":1.649,"15min":2.7970000000000002},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.20999999999999999},"max":{"1min":0.51800000000000002,"5min":13.576000000000001,"15min":94.454999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":1.649,"15min":2.5680000000000001},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.67900000000000005,"5min":10.887,"15min":95.197999999999993},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":1.5069999999999999,"15min":2.0819999999999999},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.224},"max":{"1min":0.626,"5min":15.541,"15min":20.105},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.438,"5min":2.0539999999999998,"15min":2.2989999999999999},"min":{"1min":0.29799999999999999,"5min":0.22800000000000001,"15min":0.20300000000000001},"max":{"1min":0.60699999999999998,"5min":20.681000000000001,"15min":35.939999999999998},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Feb 22 19:54:29 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":1.7749999999999999,"15min":1.7450000000000001},"min":{"1min":0.41499999999999998,"5min":0.29899999999999999,"15min":0.27100000000000002},"max":{"1min":0.93899999999999995,"5min":13.417999999999999,"15min":13.417999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":1.7549999999999999,"15min":1.6160000000000001},"min":{"1min":0.378,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89700000000000002,"5min":12.423,"15min":12.423},"last":0.748}]},{"osd":5,"last update":"Thu Feb 22 19:53:52 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":1.9119999999999999,"15min":1.8109999999999999},"min":{"1min":0.432,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":0.85699999999999998,"5min":30.227,"15min":30.227},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":2.2589999999999999,"15min":1.9610000000000001},"min":{"1min":0.42999999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.99399999999999999,"5min":30.021000000000001,"15min":30.021000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":1.6639999999999999,"15min":1.7130000000000001},"min":{"1min":0.41999999999999998,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.92800000000000005,"5min":12.632999999999999,"15min":12.632999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":2.0329999999999999,"15min":1.9970000000000001},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.26600000000000001},"max":{"1min":0.86299999999999999,"5min":17.544,"15min":17.544},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Feb 22 19:54:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":1.3779999999999999,"15min":1.581},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.312},"max":{"1min":0.96499999999999997,"5min":8.5540000000000003,"15min":12.327},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":1.7230000000000001,"15min":1.8169999999999999},"min":{"1min":0.433,"5min":0.35399999999999998,"15min":0.29599999999999999},"max":{"1min":1.2130000000000001,"5min":12.539999999999999,"15min":12.539999999999999},"last":0.86599999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953802,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315780,"kb_used_data":6268,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429372,"statfs":{"total":95995035648,"available":95671676928,"internally_reserved":0,"allocated":6418432,"data_stored":3632749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.446,"15min":1.2090000000000001},"min":{"1min":0.34799999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.88900000000000001,"5min":14.797000000000001,"15min":14.797000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":1.4630000000000001,"15min":1.288},"min":{"1min":0.35899999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71399999999999997,"5min":8.4860000000000007,"15min":9.4900000000000002},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":1.4810000000000001,"15min":1.4370000000000001},"min":{"1min":0.38,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.74099999999999999,"5min":8.5129999999999999,"15min":11.037000000000001},"last":0.443},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.429,"15min":1.2529999999999999},"min":{"1min":0.40000000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.80800000000000005,"5min":9.4890000000000008,"15min":9.4890000000000008},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":1.875,"15min":1.5960000000000001},"min":{"1min":0.47299999999999998,"5min":0.28000000000000003,"15min":0.22900000000000001},"max":{"1min":0.84699999999999998,"5min":15.470000000000001,"15min":15.470000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.4890000000000001,"15min":1.369},"min":{"1min":0.40699999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77400000000000002,"5min":7.9299999999999997,"15min":7.9299999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.9179999999999999,"15min":1.921},"min":{"1min":0.42099999999999999,"5min":0.30399999999999999,"15min":0.25900000000000001},"max":{"1min":0.86299999999999999,"5min":13.635,"15min":94.703999999999994},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":1.9470000000000001,"15min":2.0030000000000001},"min":{"1min":0.377,"5min":0.31,"15min":0.25900000000000001},"max":{"1min":0.75700000000000001,"5min":10.691000000000001,"15min":95.034000000000006},"last":0.495}]},{"osd":5,"last update":"Thu Feb 22 19:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":1.1839999999999999,"15min":0.92600000000000005},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.218},"max":{"1min":0.85799999999999998,"5min":12.099,"15min":12.099},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":1.276,"15min":1.0269999999999999},"min":{"1min":0.313,"5min":0.25,"15min":0.249},"max":{"1min":0.86299999999999999,"5min":8.0039999999999996,"15min":13.211},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.0049999999999999,"15min":0.84099999999999997},"min":{"1min":0.41299999999999998,"5min":0.29899999999999999,"15min":0.28000000000000003},"max":{"1min":0.82299999999999995,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":1.621,"15min":1.0449999999999999},"min":{"1min":0.378,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.83899999999999997,"5min":11.802,"15min":11.802},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":1.4119999999999999,"15min":1.0760000000000001},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.28799999999999998},"max":{"1min":0.80000000000000004,"5min":9.548,"15min":11.111000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.4239999999999999,"15min":1.0069999999999999},"min":{"1min":0.36799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88300000000000001,"5min":10.869,"15min":10.869},"last":0.625}]}]},{"osd":5,"up_from":39,"seq":167503724868,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":909420,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":903168,"kb_avail":92835732,"statfs":{"total":95995035648,"available":95063789568,"internally_reserved":0,"allocated":6393856,"data_stored":3631766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":924844032},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":1.3260000000000001,"15min":1.2490000000000001},"min":{"1min":0.34999999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.82399999999999995,"5min":12.102,"15min":17.684000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":1.718,"15min":1.52},"min":{"1min":0.36399999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":10.977,"15min":17.402999999999999},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.587,"15min":1.5660000000000001},"min":{"1min":0.41199999999999998,"5min":0.23400000000000001,"15min":0.19},"max":{"1min":0.80700000000000005,"5min":12.154,"15min":37.009999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.651,"15min":1.8169999999999999},"min":{"1min":0.36299999999999999,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":10.372,"15min":39.131999999999998},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":1.9350000000000001,"15min":2.0289999999999999},"min":{"1min":0.36699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.79600000000000004,"5min":15.683999999999999,"15min":47.859000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":2.0099999999999998,"15min":1.9159999999999999},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.23300000000000001},"max":{"1min":0.84599999999999997,"5min":16.390000000000001,"15min":45.183},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":1.7629999999999999,"15min":1.9159999999999999},"min":{"1min":0.35799999999999998,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.89400000000000002,"5min":11.811999999999999,"15min":43.616999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":1.9990000000000001,"15min":1.9910000000000001},"min":{"1min":0.36899999999999999,"5min":0.28499999999999998,"15min":0.26300000000000001},"max":{"1min":0.84699999999999998,"5min":12.82,"15min":47.814},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":1.3160000000000001,"15min":1.0209999999999999},"min":{"1min":0.33000000000000002,"5min":0.27100000000000002,"15min":0.223},"max":{"1min":0.91700000000000004,"5min":10.347,"15min":10.347},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":1.381,"15min":1.1459999999999999},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.81699999999999995,"5min":11.696,"15min":11.696},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.232,"15min":0.89000000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.247},"max":{"1min":0.80200000000000005,"5min":12.412000000000001,"15min":12.412000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.462,"15min":1.149},"min":{"1min":0.374,"5min":0.314,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":11.913,"15min":11.913},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Feb 22 19:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":1.377,"15min":1.046},"min":{"1min":0.38,"5min":0.309,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":12.380000000000001,"15min":12.380000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.3460000000000001,"15min":1.228},"min":{"1min":0.36099999999999999,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.75900000000000001,"5min":9.4280000000000008,"15min":9.4280000000000008},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:54:52.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T19:54:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:50.868705+0000 osd.7 (osd.7) 408 : cluster [DBG] 6.3 scrub ok 2024-02-22T19:54:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:51.058171+0000 osd.4 (osd.4) 271 : cluster [DBG] 3.1a scrub starts 2024-02-22T19:54:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:51.061035+0000 osd.4 (osd.4) 272 : cluster [DBG] 3.1a scrub ok 2024-02-22T19:54:52.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:51.450978+0000 mgr.y (mgr.14535) 974 : audit [DBG] from='client.26108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:54:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:53.119 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:51.598736+0000 osd.3 (osd.3) 455 : cluster [DBG] 6.c scrub starts 2024-02-22T19:54:53.119 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:51.629570+0000 osd.3 (osd.3) 456 : cluster [DBG] 6.c scrub ok 2024-02-22T19:54:53.119 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:52.019271+0000 mgr.y (mgr.14535) 975 : cluster [DBG] pgmap v1372: 112 pgs: 112 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 839 B/s rd, 0 op/s 2024-02-22T19:54:53.120 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:51.598736+0000 osd.3 (osd.3) 455 : cluster [DBG] 6.c scrub starts 2024-02-22T19:54:53.120 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:51.629570+0000 osd.3 (osd.3) 456 : cluster [DBG] 6.c scrub ok 2024-02-22T19:54:53.120 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:53 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:52.019271+0000 mgr.y (mgr.14535) 975 : cluster [DBG] pgmap v1372: 112 pgs: 112 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 839 B/s rd, 0 op/s 2024-02-22T19:54:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:51.598736+0000 osd.3 (osd.3) 455 : cluster [DBG] 6.c scrub starts 2024-02-22T19:54:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:51.629570+0000 osd.3 (osd.3) 456 : cluster [DBG] 6.c scrub ok 2024-02-22T19:54:53.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:52.019271+0000 mgr.y (mgr.14535) 975 : cluster [DBG] pgmap v1372: 112 pgs: 112 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 839 B/s rd, 0 op/s 2024-02-22T19:54:54.413 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:53.409080+0000 mon.a (mon.0) 2229 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:54.413 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:53.520305+0000 osd.3 (osd.3) 457 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:54:54.413 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:53.523937+0000 osd.3 (osd.3) 458 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:54:54.413 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:53.638986+0000 osd.7 (osd.7) 409 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:54:54.413 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:53.641928+0000 osd.7 (osd.7) 410 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:54:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-02-22T19:54:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-02-22T19:54:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:53.409080+0000 mon.a (mon.0) 2229 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:53.520305+0000 osd.3 (osd.3) 457 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:54:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:53.523937+0000 osd.3 (osd.3) 458 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:54:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:53.638986+0000 osd.7 (osd.7) 409 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:54:54.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:53.641928+0000 osd.7 (osd.7) 410 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:54:54.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:54:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x[41319]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-22T19:54:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:53.409080+0000 mon.a (mon.0) 2229 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:53.520305+0000 osd.3 (osd.3) 457 : cluster [DBG] 4.11 scrub starts 2024-02-22T19:54:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:53.523937+0000 osd.3 (osd.3) 458 : cluster [DBG] 4.11 scrub ok 2024-02-22T19:54:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:53.638986+0000 osd.7 (osd.7) 409 : cluster [DBG] 3.11 scrub starts 2024-02-22T19:54:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:53.641928+0000 osd.7 (osd.7) 410 : cluster [DBG] 3.11 scrub ok 2024-02-22T19:54:55.118 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":1373,"stamp":"2024-02-22T19:54:54.019449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":3058,"num_read_kb":2810,"num_write":1000,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":37919,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1183,"ondisk_log_size":1183,"up":336,"acting":336,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6032844,"kb_used_data":57164,"kb_used_omap":0,"kb_used_meta":5975616,"kb_avail":743928372,"statfs":{"total":767960285184,"available":761782652928,"internally_reserved":0,"allocated":58535936,"data_stored":36331396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6119030784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002028"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.931956+0000","last_change":"2024-02-22T19:53:51.593367+0000","last_active":"2024-02-22T19:54:46.931956+0000","last_peered":"2024-02-22T19:54:46.931956+0000","last_clean":"2024-02-22T19:54:46.931956+0000","last_became_active":"2024-02-22T19:52:33.814318+0000","last_became_peered":"2024-02-22T19:52:33.814318+0000","last_unstale":"2024-02-22T19:54:46.931956+0000","last_undegraded":"2024-02-22T19:54:46.931956+0000","last_fullsized":"2024-02-22T19:54:46.931956+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:51.593283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:07.664424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:51.593283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"504'21","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934160+0000","last_change":"2024-02-22T19:54:13.838245+0000","last_active":"2024-02-22T19:54:46.934160+0000","last_peered":"2024-02-22T19:54:46.934160+0000","last_clean":"2024-02-22T19:54:46.934160+0000","last_became_active":"2024-02-22T19:52:42.737832+0000","last_became_peered":"2024-02-22T19:52:42.737832+0000","last_unstale":"2024-02-22T19:54:46.934160+0000","last_undegraded":"2024-02-22T19:54:46.934160+0000","last_fullsized":"2024-02-22T19:54:46.934160+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"504'21","last_scrub_stamp":"2024-02-22T19:54:13.838180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:13.838180+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":681,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.698015+0000","last_change":"2024-02-22T19:54:38.618539+0000","last_active":"2024-02-22T19:54:47.698015+0000","last_peered":"2024-02-22T19:54:47.698015+0000","last_clean":"2024-02-22T19:54:47.698015+0000","last_became_active":"2024-02-22T19:49:38.802266+0000","last_became_peered":"2024-02-22T19:49:38.802266+0000","last_unstale":"2024-02-22T19:54:47.698015+0000","last_undegraded":"2024-02-22T19:54:47.698015+0000","last_fullsized":"2024-02-22T19:54:47.698015+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:38.618433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:56.953188+0000","last_clean_scrub_stamp":"2024-02-22T19:54:38.618433+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"506'39","reported_seq":833,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697768+0000","last_change":"2024-02-22T19:53:24.719938+0000","last_active":"2024-02-22T19:54:47.697768+0000","last_peered":"2024-02-22T19:54:47.697768+0000","last_clean":"2024-02-22T19:54:47.697768+0000","last_became_active":"2024-02-22T19:51:15.103693+0000","last_became_peered":"2024-02-22T19:51:15.103693+0000","last_unstale":"2024-02-22T19:54:47.697768+0000","last_undegraded":"2024-02-22T19:54:47.697768+0000","last_fullsized":"2024-02-22T19:54:47.697768+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'39","last_scrub_stamp":"2024-02-22T19:53:24.719831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:24.719831+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":84,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764142+0000","last_change":"2024-02-22T19:54:44.749916+0000","last_active":"2024-02-22T19:54:46.764142+0000","last_peered":"2024-02-22T19:54:46.764142+0000","last_clean":"2024-02-22T19:54:46.764142+0000","last_became_active":"2024-02-22T19:52:42.745594+0000","last_became_peered":"2024-02-22T19:52:42.745594+0000","last_unstale":"2024-02-22T19:54:46.764142+0000","last_undegraded":"2024-02-22T19:54:46.764142+0000","last_fullsized":"2024-02-22T19:54:46.764142+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:44.749809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:01.706836+0000","last_clean_scrub_stamp":"2024-02-22T19:54:44.749809+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"505'25","reported_seq":74,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934561+0000","last_change":"2024-02-22T19:53:32.806794+0000","last_active":"2024-02-22T19:54:46.934561+0000","last_peered":"2024-02-22T19:54:46.934561+0000","last_clean":"2024-02-22T19:54:46.934561+0000","last_became_active":"2024-02-22T19:52:42.740933+0000","last_became_peered":"2024-02-22T19:52:42.740933+0000","last_unstale":"2024-02-22T19:54:46.934561+0000","last_undegraded":"2024-02-22T19:54:46.934561+0000","last_fullsized":"2024-02-22T19:54:46.934561+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'25","last_scrub_stamp":"2024-02-22T19:53:32.806693+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:40:48.683110+0000","last_clean_scrub_stamp":"2024-02-22T19:53:32.806693+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"484'2","reported_seq":92,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763412+0000","last_change":"2024-02-22T19:54:30.704823+0000","last_active":"2024-02-22T19:54:46.763412+0000","last_peered":"2024-02-22T19:54:46.763412+0000","last_clean":"2024-02-22T19:54:46.763412+0000","last_became_active":"2024-02-22T19:52:42.747645+0000","last_became_peered":"2024-02-22T19:52:42.747645+0000","last_unstale":"2024-02-22T19:54:46.763412+0000","last_undegraded":"2024-02-22T19:54:46.763412+0000","last_fullsized":"2024-02-22T19:54:46.763412+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"484'2","last_scrub_stamp":"2024-02-22T19:54:30.704739+0000","last_deep_scrub":"484'2","last_deep_scrub_stamp":"2024-02-22T19:54:30.704739+0000","last_clean_scrub_stamp":"2024-02-22T19:54:30.704739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":3222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"505'9","reported_seq":75,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935129+0000","last_change":"2024-02-22T19:54:41.960108+0000","last_active":"2024-02-22T19:54:46.935129+0000","last_peered":"2024-02-22T19:54:46.935129+0000","last_clean":"2024-02-22T19:54:46.935129+0000","last_became_active":"2024-02-22T19:52:42.744273+0000","last_became_peered":"2024-02-22T19:52:42.744273+0000","last_unstale":"2024-02-22T19:54:46.935129+0000","last_undegraded":"2024-02-22T19:54:46.935129+0000","last_fullsized":"2024-02-22T19:54:46.935129+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'9","last_scrub_stamp":"2024-02-22T19:54:41.960024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:41.960024+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":134,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933729+0000","last_change":"2024-02-22T19:53:59.413929+0000","last_active":"2024-02-22T19:54:46.933729+0000","last_peered":"2024-02-22T19:54:46.933729+0000","last_clean":"2024-02-22T19:54:46.933729+0000","last_became_active":"2024-02-22T19:52:42.741244+0000","last_became_peered":"2024-02-22T19:52:42.741244+0000","last_unstale":"2024-02-22T19:54:46.933729+0000","last_undegraded":"2024-02-22T19:54:46.933729+0000","last_fullsized":"2024-02-22T19:54:46.933729+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:59.413864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:00.634060+0000","last_clean_scrub_stamp":"2024-02-22T19:53:59.413864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"505'46","reported_seq":789,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.918875+0000","last_change":"2024-02-22T19:54:06.176807+0000","last_active":"2024-02-22T19:54:45.918875+0000","last_peered":"2024-02-22T19:54:45.918875+0000","last_clean":"2024-02-22T19:54:45.918875+0000","last_became_active":"2024-02-22T19:49:56.720148+0000","last_became_peered":"2024-02-22T19:49:56.720148+0000","last_unstale":"2024-02-22T19:54:45.918875+0000","last_undegraded":"2024-02-22T19:54:45.918875+0000","last_fullsized":"2024-02-22T19:54:45.918875+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'46","last_scrub_stamp":"2024-02-22T19:54:06.176727+0000","last_deep_scrub":"505'46","last_deep_scrub_stamp":"2024-02-22T19:54:06.176727+0000","last_clean_scrub_stamp":"2024-02-22T19:54:06.176727+0000","log_size":46,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934664+0000","last_change":"2024-02-22T19:53:20.906284+0000","last_active":"2024-02-22T19:54:46.934664+0000","last_peered":"2024-02-22T19:54:46.934664+0000","last_clean":"2024-02-22T19:54:46.934664+0000","last_became_active":"2024-02-22T19:52:42.733123+0000","last_became_peered":"2024-02-22T19:52:42.733123+0000","last_unstale":"2024-02-22T19:54:46.934664+0000","last_undegraded":"2024-02-22T19:54:46.934664+0000","last_fullsized":"2024-02-22T19:54:46.934664+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:20.906228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:45:03.679103+0000","last_clean_scrub_stamp":"2024-02-22T19:53:20.906228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"506'29","reported_seq":866,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.698152+0000","last_change":"2024-02-22T19:54:31.611759+0000","last_active":"2024-02-22T19:54:47.698152+0000","last_peered":"2024-02-22T19:54:47.698152+0000","last_clean":"2024-02-22T19:54:47.698152+0000","last_became_active":"2024-02-22T19:52:42.752698+0000","last_became_peered":"2024-02-22T19:52:42.752698+0000","last_unstale":"2024-02-22T19:54:47.698152+0000","last_undegraded":"2024-02-22T19:54:47.698152+0000","last_fullsized":"2024-02-22T19:54:47.698152+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'29","last_scrub_stamp":"2024-02-22T19:54:31.611649+0000","last_deep_scrub":"87'17","last_deep_scrub_stamp":"2024-02-22T19:31:04.012663+0000","last_clean_scrub_stamp":"2024-02-22T19:54:31.611649+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":87,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.919221+0000","last_change":"2024-02-22T19:54:24.205755+0000","last_active":"2024-02-22T19:54:45.919221+0000","last_peered":"2024-02-22T19:54:45.919221+0000","last_clean":"2024-02-22T19:54:45.919221+0000","last_became_active":"2024-02-22T19:52:42.747539+0000","last_became_peered":"2024-02-22T19:52:42.747539+0000","last_unstale":"2024-02-22T19:54:45.919221+0000","last_undegraded":"2024-02-22T19:54:45.919221+0000","last_fullsized":"2024-02-22T19:54:45.919221+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:24.205693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:24.205693+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"506'30","reported_seq":128,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933090+0000","last_change":"2024-02-22T19:53:44.713463+0000","last_active":"2024-02-22T19:54:46.933090+0000","last_peered":"2024-02-22T19:54:46.933090+0000","last_clean":"2024-02-22T19:54:46.933090+0000","last_became_active":"2024-02-22T19:51:15.087979+0000","last_became_peered":"2024-02-22T19:51:15.087979+0000","last_unstale":"2024-02-22T19:54:46.933090+0000","last_undegraded":"2024-02-22T19:54:46.933090+0000","last_fullsized":"2024-02-22T19:54:46.933090+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'30","last_scrub_stamp":"2024-02-22T19:53:44.713384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:44.713384+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":710,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697454+0000","last_change":"2024-02-22T19:54:39.570858+0000","last_active":"2024-02-22T19:54:47.697454+0000","last_peered":"2024-02-22T19:54:47.697454+0000","last_clean":"2024-02-22T19:54:47.697454+0000","last_became_active":"2024-02-22T19:51:15.079506+0000","last_became_peered":"2024-02-22T19:51:15.079506+0000","last_unstale":"2024-02-22T19:54:47.697454+0000","last_undegraded":"2024-02-22T19:54:47.697454+0000","last_fullsized":"2024-02-22T19:54:47.697454+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:39.570789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:58.164960+0000","last_clean_scrub_stamp":"2024-02-22T19:54:39.570789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"652'32","reported_seq":249,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816356+0000","last_change":"2024-02-22T19:53:43.817082+0000","last_active":"2024-02-22T19:54:46.816356+0000","last_peered":"2024-02-22T19:54:46.816356+0000","last_clean":"2024-02-22T19:54:46.816356+0000","last_became_active":"2024-02-22T19:52:33.822010+0000","last_became_peered":"2024-02-22T19:52:33.822010+0000","last_unstale":"2024-02-22T19:54:46.816356+0000","last_undegraded":"2024-02-22T19:54:46.816356+0000","last_fullsized":"2024-02-22T19:54:46.816356+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"652'32","last_scrub_stamp":"2024-02-22T19:53:43.817001+0000","last_deep_scrub":"652'32","last_deep_scrub_stamp":"2024-02-22T19:53:43.817001+0000","last_clean_scrub_stamp":"2024-02-22T19:53:43.817001+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":692,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658854+0000","last_change":"2024-02-22T19:53:56.695327+0000","last_active":"2024-02-22T19:54:47.658854+0000","last_peered":"2024-02-22T19:54:47.658854+0000","last_clean":"2024-02-22T19:54:47.658854+0000","last_became_active":"2024-02-22T19:52:42.739171+0000","last_became_peered":"2024-02-22T19:52:42.739171+0000","last_unstale":"2024-02-22T19:54:47.658854+0000","last_undegraded":"2024-02-22T19:54:47.658854+0000","last_fullsized":"2024-02-22T19:54:47.658854+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:56.695231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:56.695231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":125,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932110+0000","last_change":"2024-02-22T19:54:25.513732+0000","last_active":"2024-02-22T19:54:46.932110+0000","last_peered":"2024-02-22T19:54:46.932110+0000","last_clean":"2024-02-22T19:54:46.932110+0000","last_became_active":"2024-02-22T19:52:33.823355+0000","last_became_peered":"2024-02-22T19:52:33.823355+0000","last_unstale":"2024-02-22T19:54:46.932110+0000","last_undegraded":"2024-02-22T19:54:46.932110+0000","last_fullsized":"2024-02-22T19:54:46.932110+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:25.513620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:54:25.513620+0000","last_clean_scrub_stamp":"2024-02-22T19:54:25.513620+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"505'12","reported_seq":745,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697445+0000","last_change":"2024-02-22T19:54:22.550185+0000","last_active":"2024-02-22T19:54:47.697445+0000","last_peered":"2024-02-22T19:54:47.697445+0000","last_clean":"2024-02-22T19:54:47.697445+0000","last_became_active":"2024-02-22T19:49:56.728926+0000","last_became_peered":"2024-02-22T19:49:56.728926+0000","last_unstale":"2024-02-22T19:54:47.697445+0000","last_undegraded":"2024-02-22T19:54:47.697445+0000","last_fullsized":"2024-02-22T19:54:47.697445+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'12","last_scrub_stamp":"2024-02-22T19:54:22.550059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:22.550059+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":713,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697906+0000","last_change":"2024-02-22T19:54:34.722384+0000","last_active":"2024-02-22T19:54:47.697906+0000","last_peered":"2024-02-22T19:54:47.697906+0000","last_clean":"2024-02-22T19:54:47.697906+0000","last_became_active":"2024-02-22T19:51:15.123877+0000","last_became_peered":"2024-02-22T19:51:15.123877+0000","last_unstale":"2024-02-22T19:54:47.697906+0000","last_undegraded":"2024-02-22T19:54:47.697906+0000","last_fullsized":"2024-02-22T19:54:47.697906+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:34.722297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:34.722297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764057+0000","last_change":"2024-02-22T19:54:35.641296+0000","last_active":"2024-02-22T19:54:46.764057+0000","last_peered":"2024-02-22T19:54:46.764057+0000","last_clean":"2024-02-22T19:54:46.764057+0000","last_became_active":"2024-02-22T19:52:33.824603+0000","last_became_peered":"2024-02-22T19:52:33.824603+0000","last_unstale":"2024-02-22T19:54:46.764057+0000","last_undegraded":"2024-02-22T19:54:46.764057+0000","last_fullsized":"2024-02-22T19:54:46.764057+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:35.641217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:35.641217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"506'27","reported_seq":732,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697751+0000","last_change":"2024-02-22T19:53:22.742140+0000","last_active":"2024-02-22T19:54:47.697751+0000","last_peered":"2024-02-22T19:54:47.697751+0000","last_clean":"2024-02-22T19:54:47.697751+0000","last_became_active":"2024-02-22T19:49:38.881889+0000","last_became_peered":"2024-02-22T19:49:38.881889+0000","last_unstale":"2024-02-22T19:54:47.697751+0000","last_undegraded":"2024-02-22T19:54:47.697751+0000","last_fullsized":"2024-02-22T19:54:47.697751+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":529,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'27","last_scrub_stamp":"2024-02-22T19:53:22.741993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:22.741993+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":125,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933383+0000","last_change":"2024-02-22T19:53:58.473542+0000","last_active":"2024-02-22T19:54:46.933383+0000","last_peered":"2024-02-22T19:54:46.933383+0000","last_clean":"2024-02-22T19:54:46.933383+0000","last_became_active":"2024-02-22T19:52:33.822486+0000","last_became_peered":"2024-02-22T19:52:33.822486+0000","last_unstale":"2024-02-22T19:54:46.933383+0000","last_undegraded":"2024-02-22T19:54:46.933383+0000","last_fullsized":"2024-02-22T19:54:46.933383+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:58.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:51.597847+0000","last_clean_scrub_stamp":"2024-02-22T19:53:58.473444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":732,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659155+0000","last_change":"2024-02-22T19:54:34.588928+0000","last_active":"2024-02-22T19:54:47.659155+0000","last_peered":"2024-02-22T19:54:47.659155+0000","last_clean":"2024-02-22T19:54:47.659155+0000","last_became_active":"2024-02-22T19:52:42.741059+0000","last_became_peered":"2024-02-22T19:52:42.741059+0000","last_unstale":"2024-02-22T19:54:47.659155+0000","last_undegraded":"2024-02-22T19:54:47.659155+0000","last_fullsized":"2024-02-22T19:54:47.659155+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:34.588830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:23.009021+0000","last_clean_scrub_stamp":"2024-02-22T19:54:34.588830+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"505'21","reported_seq":154,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.931859+0000","last_change":"2024-02-22T19:54:17.057033+0000","last_active":"2024-02-22T19:54:46.931859+0000","last_peered":"2024-02-22T19:54:46.931859+0000","last_clean":"2024-02-22T19:54:46.931859+0000","last_became_active":"2024-02-22T19:49:56.866114+0000","last_became_peered":"2024-02-22T19:49:56.866114+0000","last_unstale":"2024-02-22T19:54:46.931859+0000","last_undegraded":"2024-02-22T19:54:46.931859+0000","last_fullsized":"2024-02-22T19:54:46.931859+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'21","last_scrub_stamp":"2024-02-22T19:54:17.056960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:17.056960+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":722,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697445+0000","last_change":"2024-02-22T19:54:20.678911+0000","last_active":"2024-02-22T19:54:47.697445+0000","last_peered":"2024-02-22T19:54:47.697445+0000","last_clean":"2024-02-22T19:54:47.697445+0000","last_became_active":"2024-02-22T19:52:33.822298+0000","last_became_peered":"2024-02-22T19:52:33.822298+0000","last_unstale":"2024-02-22T19:54:47.697445+0000","last_undegraded":"2024-02-22T19:54:47.697445+0000","last_fullsized":"2024-02-22T19:54:47.697445+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:20.678822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:54:20.678822+0000","last_clean_scrub_stamp":"2024-02-22T19:54:20.678822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":69,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934708+0000","last_change":"2024-02-22T19:54:29.876643+0000","last_active":"2024-02-22T19:54:46.934708+0000","last_peered":"2024-02-22T19:54:46.934708+0000","last_clean":"2024-02-22T19:54:46.934708+0000","last_became_active":"2024-02-22T19:52:42.743235+0000","last_became_peered":"2024-02-22T19:52:42.743235+0000","last_unstale":"2024-02-22T19:54:46.934708+0000","last_undegraded":"2024-02-22T19:54:46.934708+0000","last_fullsized":"2024-02-22T19:54:46.934708+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:29.876579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:29.876579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"641'32","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935043+0000","last_change":"2024-02-22T19:53:25.925953+0000","last_active":"2024-02-22T19:54:46.935043+0000","last_peered":"2024-02-22T19:54:46.935043+0000","last_clean":"2024-02-22T19:54:46.935043+0000","last_became_active":"2024-02-22T19:52:42.738840+0000","last_became_peered":"2024-02-22T19:52:42.738840+0000","last_unstale":"2024-02-22T19:54:46.935043+0000","last_undegraded":"2024-02-22T19:54:46.935043+0000","last_fullsized":"2024-02-22T19:54:46.935043+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"641'32","last_scrub_stamp":"2024-02-22T19:53:25.925888+0000","last_deep_scrub":"641'32","last_deep_scrub_stamp":"2024-02-22T19:53:25.925888+0000","last_clean_scrub_stamp":"2024-02-22T19:53:25.925888+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":705,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658847+0000","last_change":"2024-02-22T19:54:22.559564+0000","last_active":"2024-02-22T19:54:47.658847+0000","last_peered":"2024-02-22T19:54:47.658847+0000","last_clean":"2024-02-22T19:54:47.658847+0000","last_became_active":"2024-02-22T19:52:42.741108+0000","last_became_peered":"2024-02-22T19:52:42.741108+0000","last_unstale":"2024-02-22T19:54:47.658847+0000","last_undegraded":"2024-02-22T19:54:47.658847+0000","last_fullsized":"2024-02-22T19:54:47.658847+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:22.559491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:12.015756+0000","last_clean_scrub_stamp":"2024-02-22T19:54:22.559491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"506'35","reported_seq":744,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697954+0000","last_change":"2024-02-22T19:53:54.593415+0000","last_active":"2024-02-22T19:54:47.697954+0000","last_peered":"2024-02-22T19:54:47.697954+0000","last_clean":"2024-02-22T19:54:47.697954+0000","last_became_active":"2024-02-22T19:49:56.729526+0000","last_became_peered":"2024-02-22T19:49:56.729526+0000","last_unstale":"2024-02-22T19:54:47.697954+0000","last_undegraded":"2024-02-22T19:54:47.697954+0000","last_fullsized":"2024-02-22T19:54:47.697954+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'35","last_scrub_stamp":"2024-02-22T19:53:54.593337+0000","last_deep_scrub":"85'15","last_deep_scrub_stamp":"2024-02-22T19:30:32.846336+0000","last_clean_scrub_stamp":"2024-02-22T19:53:54.593337+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":118,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932652+0000","last_change":"2024-02-22T19:54:23.441314+0000","last_active":"2024-02-22T19:54:46.932652+0000","last_peered":"2024-02-22T19:54:46.932652+0000","last_clean":"2024-02-22T19:54:46.932652+0000","last_became_active":"2024-02-22T19:51:15.061241+0000","last_became_peered":"2024-02-22T19:51:15.061241+0000","last_unstale":"2024-02-22T19:54:46.932652+0000","last_undegraded":"2024-02-22T19:54:46.932652+0000","last_fullsized":"2024-02-22T19:54:46.932652+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:23.441196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:46:17.328243+0000","last_clean_scrub_stamp":"2024-02-22T19:54:23.441196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"506'25","reported_seq":85,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764020+0000","last_change":"2024-02-22T19:54:27.643707+0000","last_active":"2024-02-22T19:54:46.764020+0000","last_peered":"2024-02-22T19:54:46.764020+0000","last_clean":"2024-02-22T19:54:46.764020+0000","last_became_active":"2024-02-22T19:52:33.815603+0000","last_became_peered":"2024-02-22T19:52:33.815603+0000","last_unstale":"2024-02-22T19:54:46.764020+0000","last_undegraded":"2024-02-22T19:54:46.764020+0000","last_fullsized":"2024-02-22T19:54:46.764020+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'25","last_scrub_stamp":"2024-02-22T19:54:27.643617+0000","last_deep_scrub":"506'25","last_deep_scrub_stamp":"2024-02-22T19:54:27.643617+0000","last_clean_scrub_stamp":"2024-02-22T19:54:27.643617+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":721,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.919068+0000","last_change":"2024-02-22T19:54:05.180811+0000","last_active":"2024-02-22T19:54:45.919068+0000","last_peered":"2024-02-22T19:54:45.919068+0000","last_clean":"2024-02-22T19:54:45.919068+0000","last_became_active":"2024-02-22T19:52:42.744925+0000","last_became_peered":"2024-02-22T19:52:42.744925+0000","last_unstale":"2024-02-22T19:54:45.919068+0000","last_undegraded":"2024-02-22T19:54:45.919068+0000","last_fullsized":"2024-02-22T19:54:45.919068+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:05.180671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:35:31.810353+0000","last_clean_scrub_stamp":"2024-02-22T19:54:05.180671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":736,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658709+0000","last_change":"2024-02-22T19:53:25.791805+0000","last_active":"2024-02-22T19:54:47.658709+0000","last_peered":"2024-02-22T19:54:47.658709+0000","last_clean":"2024-02-22T19:54:47.658709+0000","last_became_active":"2024-02-22T19:52:42.744068+0000","last_became_peered":"2024-02-22T19:52:42.744068+0000","last_unstale":"2024-02-22T19:54:47.658709+0000","last_undegraded":"2024-02-22T19:54:47.658709+0000","last_fullsized":"2024-02-22T19:54:47.658709+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:25.791731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:05.818758+0000","last_clean_scrub_stamp":"2024-02-22T19:53:25.791731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"669'25","reported_seq":130,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.702684+0000","last_change":"2024-02-22T19:54:47.702651+0000","last_active":"2024-02-22T19:54:47.702684+0000","last_peered":"2024-02-22T19:54:47.702684+0000","last_clean":"2024-02-22T19:54:47.702684+0000","last_became_active":"2024-02-22T19:51:15.087435+0000","last_became_peered":"2024-02-22T19:51:15.087435+0000","last_unstale":"2024-02-22T19:54:47.702684+0000","last_undegraded":"2024-02-22T19:54:47.702684+0000","last_fullsized":"2024-02-22T19:54:47.702684+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"669'25","last_scrub_stamp":"2024-02-22T19:54:47.702577+0000","last_deep_scrub":"669'25","last_deep_scrub_stamp":"2024-02-22T19:54:47.702577+0000","last_clean_scrub_stamp":"2024-02-22T19:54:47.702577+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":26,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":745,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659039+0000","last_change":"2024-02-22T19:54:28.663627+0000","last_active":"2024-02-22T19:54:47.659039+0000","last_peered":"2024-02-22T19:54:47.659039+0000","last_clean":"2024-02-22T19:54:47.659039+0000","last_became_active":"2024-02-22T19:52:42.740598+0000","last_became_peered":"2024-02-22T19:52:42.740598+0000","last_unstale":"2024-02-22T19:54:47.659039+0000","last_undegraded":"2024-02-22T19:54:47.659039+0000","last_fullsized":"2024-02-22T19:54:47.659039+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:28.663543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:28.663543+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":187,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.815826+0000","last_change":"2024-02-22T19:54:43.240945+0000","last_active":"2024-02-22T19:54:46.815826+0000","last_peered":"2024-02-22T19:54:46.815826+0000","last_clean":"2024-02-22T19:54:46.815826+0000","last_became_active":"2024-02-22T19:52:33.818780+0000","last_became_peered":"2024-02-22T19:52:33.818780+0000","last_unstale":"2024-02-22T19:54:46.815826+0000","last_undegraded":"2024-02-22T19:54:46.815826+0000","last_fullsized":"2024-02-22T19:54:46.815826+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:43.240852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:13.989193+0000","last_clean_scrub_stamp":"2024-02-22T19:54:43.240852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"506'18","reported_seq":713,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.698320+0000","last_change":"2024-02-22T19:54:04.591044+0000","last_active":"2024-02-22T19:54:47.698320+0000","last_peered":"2024-02-22T19:54:47.698320+0000","last_clean":"2024-02-22T19:54:47.698320+0000","last_became_active":"2024-02-22T19:49:56.728115+0000","last_became_peered":"2024-02-22T19:49:56.728115+0000","last_unstale":"2024-02-22T19:54:47.698320+0000","last_undegraded":"2024-02-22T19:54:47.698320+0000","last_fullsized":"2024-02-22T19:54:47.698320+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":547,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'18","last_scrub_stamp":"2024-02-22T19:54:04.590976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:04.590976+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763263+0000","last_change":"2024-02-22T19:53:51.703683+0000","last_active":"2024-02-22T19:54:46.763263+0000","last_peered":"2024-02-22T19:54:46.763263+0000","last_clean":"2024-02-22T19:54:46.763263+0000","last_became_active":"2024-02-22T19:52:33.823158+0000","last_became_peered":"2024-02-22T19:52:33.823158+0000","last_unstale":"2024-02-22T19:54:46.763263+0000","last_undegraded":"2024-02-22T19:54:46.763263+0000","last_fullsized":"2024-02-22T19:54:46.763263+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:51.703585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:51.703585+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'4","reported_seq":840,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659390+0000","last_change":"2024-02-22T19:54:30.533894+0000","last_active":"2024-02-22T19:54:47.659390+0000","last_peered":"2024-02-22T19:54:47.659390+0000","last_clean":"2024-02-22T19:54:47.659390+0000","last_became_active":"2024-02-22T19:52:42.741942+0000","last_became_peered":"2024-02-22T19:52:42.741942+0000","last_unstale":"2024-02-22T19:54:47.659390+0000","last_undegraded":"2024-02-22T19:54:47.659390+0000","last_fullsized":"2024-02-22T19:54:47.659390+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'4","last_scrub_stamp":"2024-02-22T19:54:30.533753+0000","last_deep_scrub":"82'4","last_deep_scrub_stamp":"2024-02-22T19:47:16.375370+0000","last_clean_scrub_stamp":"2024-02-22T19:54:30.533753+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"505'39","reported_seq":91,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.762986+0000","last_change":"2024-02-22T19:54:03.649408+0000","last_active":"2024-02-22T19:54:46.762986+0000","last_peered":"2024-02-22T19:54:46.762986+0000","last_clean":"2024-02-22T19:54:46.762986+0000","last_became_active":"2024-02-22T19:52:33.813693+0000","last_became_peered":"2024-02-22T19:52:33.813693+0000","last_unstale":"2024-02-22T19:54:46.762986+0000","last_undegraded":"2024-02-22T19:54:46.762986+0000","last_fullsized":"2024-02-22T19:54:46.762986+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":615,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'39","last_scrub_stamp":"2024-02-22T19:54:03.649324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:03.649324+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"626'33","reported_seq":1183,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.152935+0000","last_change":"2024-02-22T19:54:15.629772+0000","last_active":"2024-02-22T19:54:48.152935+0000","last_peered":"2024-02-22T19:54:48.152935+0000","last_clean":"2024-02-22T19:54:48.152935+0000","last_became_active":"2024-02-22T19:52:45.726506+0000","last_became_peered":"2024-02-22T19:52:45.726506+0000","last_unstale":"2024-02-22T19:54:48.152935+0000","last_undegraded":"2024-02-22T19:54:48.152935+0000","last_fullsized":"2024-02-22T19:54:48.152935+0000","mapping_epoch":623,"log_start":"64'8","ondisk_log_start":"64'8","created":60,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"626'33","last_scrub_stamp":"2024-02-22T19:54:15.629683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:15.629683+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":685,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697794+0000","last_change":"2024-02-22T19:54:43.696209+0000","last_active":"2024-02-22T19:54:47.697794+0000","last_peered":"2024-02-22T19:54:47.697794+0000","last_clean":"2024-02-22T19:54:47.697794+0000","last_became_active":"2024-02-22T19:49:56.728419+0000","last_became_peered":"2024-02-22T19:49:56.728419+0000","last_unstale":"2024-02-22T19:54:47.697794+0000","last_undegraded":"2024-02-22T19:54:47.697794+0000","last_fullsized":"2024-02-22T19:54:47.697794+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:43.696125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:43.696125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":436,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.918790+0000","last_change":"2024-02-22T19:54:18.192739+0000","last_active":"2024-02-22T19:54:45.918790+0000","last_peered":"2024-02-22T19:54:45.918790+0000","last_clean":"2024-02-22T19:54:45.918790+0000","last_became_active":"2024-02-22T19:52:42.744827+0000","last_became_peered":"2024-02-22T19:52:42.744827+0000","last_unstale":"2024-02-22T19:54:45.918790+0000","last_undegraded":"2024-02-22T19:54:45.918790+0000","last_fullsized":"2024-02-22T19:54:45.918790+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:18.192655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:48.582624+0000","last_clean_scrub_stamp":"2024-02-22T19:54:18.192655+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"661'29","reported_seq":86,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934574+0000","last_change":"2024-02-22T19:54:24.839913+0000","last_active":"2024-02-22T19:54:46.934574+0000","last_peered":"2024-02-22T19:54:46.934574+0000","last_clean":"2024-02-22T19:54:46.934574+0000","last_became_active":"2024-02-22T19:52:42.745052+0000","last_became_peered":"2024-02-22T19:52:42.745052+0000","last_unstale":"2024-02-22T19:54:46.934574+0000","last_undegraded":"2024-02-22T19:54:46.934574+0000","last_fullsized":"2024-02-22T19:54:46.934574+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"661'29","last_scrub_stamp":"2024-02-22T19:54:24.839809+0000","last_deep_scrub":"661'29","last_deep_scrub_stamp":"2024-02-22T19:54:24.839809+0000","last_clean_scrub_stamp":"2024-02-22T19:54:24.839809+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":745,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.919019+0000","last_change":"2024-02-22T19:53:51.000150+0000","last_active":"2024-02-22T19:54:45.919019+0000","last_peered":"2024-02-22T19:54:45.919019+0000","last_clean":"2024-02-22T19:54:45.919019+0000","last_became_active":"2024-02-22T19:51:15.076802+0000","last_became_peered":"2024-02-22T19:51:15.076802+0000","last_unstale":"2024-02-22T19:54:45.919019+0000","last_undegraded":"2024-02-22T19:54:45.919019+0000","last_fullsized":"2024-02-22T19:54:45.919019+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:51.000078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:51.000078+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":667,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658533+0000","last_change":"2024-02-22T19:53:20.881210+0000","last_active":"2024-02-22T19:54:47.658533+0000","last_peered":"2024-02-22T19:54:47.658533+0000","last_clean":"2024-02-22T19:54:47.658533+0000","last_became_active":"2024-02-22T19:49:51.842580+0000","last_became_peered":"2024-02-22T19:49:51.842580+0000","last_unstale":"2024-02-22T19:54:47.658533+0000","last_undegraded":"2024-02-22T19:54:47.658533+0000","last_fullsized":"2024-02-22T19:54:47.658533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":541,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:20.881136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:25.602356+0000","last_clean_scrub_stamp":"2024-02-22T19:53:20.881136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":765,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.919731+0000","last_change":"2024-02-22T19:54:38.125889+0000","last_active":"2024-02-22T19:54:45.919731+0000","last_peered":"2024-02-22T19:54:45.919731+0000","last_clean":"2024-02-22T19:54:45.919731+0000","last_became_active":"2024-02-22T19:52:33.821312+0000","last_became_peered":"2024-02-22T19:52:33.821312+0000","last_unstale":"2024-02-22T19:54:45.919731+0000","last_undegraded":"2024-02-22T19:54:45.919731+0000","last_fullsized":"2024-02-22T19:54:45.919731+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:38.125796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.029389+0000","last_clean_scrub_stamp":"2024-02-22T19:54:38.125796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":719,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659290+0000","last_change":"2024-02-22T19:54:42.637565+0000","last_active":"2024-02-22T19:54:47.659290+0000","last_peered":"2024-02-22T19:54:47.659290+0000","last_clean":"2024-02-22T19:54:47.659290+0000","last_became_active":"2024-02-22T19:52:33.816919+0000","last_became_peered":"2024-02-22T19:52:33.816919+0000","last_unstale":"2024-02-22T19:54:47.659290+0000","last_undegraded":"2024-02-22T19:54:47.659290+0000","last_fullsized":"2024-02-22T19:54:47.659290+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:42.637473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:42.637473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"506'51","reported_seq":209,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816078+0000","last_change":"2024-02-22T19:54:36.294696+0000","last_active":"2024-02-22T19:54:46.816078+0000","last_peered":"2024-02-22T19:54:46.816078+0000","last_clean":"2024-02-22T19:54:46.816078+0000","last_became_active":"2024-02-22T19:49:56.676335+0000","last_became_peered":"2024-02-22T19:49:56.676335+0000","last_unstale":"2024-02-22T19:54:46.816078+0000","last_undegraded":"2024-02-22T19:54:46.816078+0000","last_fullsized":"2024-02-22T19:54:46.816078+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'51","last_scrub_stamp":"2024-02-22T19:54:36.294615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:36.294615+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":116,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933318+0000","last_change":"2024-02-22T19:53:55.590876+0000","last_active":"2024-02-22T19:54:46.933318+0000","last_peered":"2024-02-22T19:54:46.933318+0000","last_clean":"2024-02-22T19:54:46.933318+0000","last_became_active":"2024-02-22T19:51:15.196637+0000","last_became_peered":"2024-02-22T19:51:15.196637+0000","last_unstale":"2024-02-22T19:54:46.933318+0000","last_undegraded":"2024-02-22T19:54:46.933318+0000","last_fullsized":"2024-02-22T19:54:46.933318+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:55.590784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:55.590784+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":739,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.919253+0000","last_change":"2024-02-22T19:53:46.029501+0000","last_active":"2024-02-22T19:54:45.919253+0000","last_peered":"2024-02-22T19:54:45.919253+0000","last_clean":"2024-02-22T19:54:45.919253+0000","last_became_active":"2024-02-22T19:52:42.746454+0000","last_became_peered":"2024-02-22T19:52:42.746454+0000","last_unstale":"2024-02-22T19:54:45.919253+0000","last_undegraded":"2024-02-22T19:54:45.919253+0000","last_fullsized":"2024-02-22T19:54:45.919253+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:46.029373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:46.029373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763364+0000","last_change":"2024-02-22T19:54:24.618065+0000","last_active":"2024-02-22T19:54:46.763364+0000","last_peered":"2024-02-22T19:54:46.763364+0000","last_clean":"2024-02-22T19:54:46.763364+0000","last_became_active":"2024-02-22T19:52:33.821072+0000","last_became_peered":"2024-02-22T19:52:33.821072+0000","last_unstale":"2024-02-22T19:54:46.763364+0000","last_undegraded":"2024-02-22T19:54:46.763364+0000","last_fullsized":"2024-02-22T19:54:46.763364+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:24.617984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:24.617984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"506'25","reported_seq":166,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933747+0000","last_change":"2024-02-22T19:53:47.645445+0000","last_active":"2024-02-22T19:54:46.933747+0000","last_peered":"2024-02-22T19:54:46.933747+0000","last_clean":"2024-02-22T19:54:46.933747+0000","last_became_active":"2024-02-22T19:52:42.765324+0000","last_became_peered":"2024-02-22T19:52:42.765324+0000","last_unstale":"2024-02-22T19:54:46.933747+0000","last_undegraded":"2024-02-22T19:54:46.933747+0000","last_fullsized":"2024-02-22T19:54:46.933747+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'25","last_scrub_stamp":"2024-02-22T19:53:47.645333+0000","last_deep_scrub":"302'19","last_deep_scrub_stamp":"2024-02-22T19:46:11.826712+0000","last_clean_scrub_stamp":"2024-02-22T19:53:47.645333+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763797+0000","last_change":"2024-02-22T19:54:19.635044+0000","last_active":"2024-02-22T19:54:46.763797+0000","last_peered":"2024-02-22T19:54:46.763797+0000","last_clean":"2024-02-22T19:54:46.763797+0000","last_became_active":"2024-02-22T19:52:33.818303+0000","last_became_peered":"2024-02-22T19:52:33.818303+0000","last_unstale":"2024-02-22T19:54:46.763797+0000","last_undegraded":"2024-02-22T19:54:46.763797+0000","last_fullsized":"2024-02-22T19:54:46.763797+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:19.634962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:19.634962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":711,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.698208+0000","last_change":"2024-02-22T19:54:26.560283+0000","last_active":"2024-02-22T19:54:47.698208+0000","last_peered":"2024-02-22T19:54:47.698208+0000","last_clean":"2024-02-22T19:54:47.698208+0000","last_became_active":"2024-02-22T19:52:42.749216+0000","last_became_peered":"2024-02-22T19:52:42.749216+0000","last_unstale":"2024-02-22T19:54:47.698208+0000","last_undegraded":"2024-02-22T19:54:47.698208+0000","last_fullsized":"2024-02-22T19:54:47.698208+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:26.560162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:26.560162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763124+0000","last_change":"2024-02-22T19:54:32.627847+0000","last_active":"2024-02-22T19:54:46.763124+0000","last_peered":"2024-02-22T19:54:46.763124+0000","last_clean":"2024-02-22T19:54:46.763124+0000","last_became_active":"2024-02-22T19:52:33.820730+0000","last_became_peered":"2024-02-22T19:52:33.820730+0000","last_unstale":"2024-02-22T19:54:46.763124+0000","last_undegraded":"2024-02-22T19:54:46.763124+0000","last_fullsized":"2024-02-22T19:54:46.763124+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:32.627755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:31:14.692694+0000","last_clean_scrub_stamp":"2024-02-22T19:54:32.627755+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934942+0000","last_change":"2024-02-22T19:53:42.958217+0000","last_active":"2024-02-22T19:54:46.934942+0000","last_peered":"2024-02-22T19:54:46.934942+0000","last_clean":"2024-02-22T19:54:46.934942+0000","last_became_active":"2024-02-22T19:52:42.738975+0000","last_became_peered":"2024-02-22T19:52:42.738975+0000","last_unstale":"2024-02-22T19:54:46.934942+0000","last_undegraded":"2024-02-22T19:54:46.934942+0000","last_fullsized":"2024-02-22T19:54:46.934942+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:42.958131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:53:42.958131+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"548'55","reported_seq":225,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.756319+0000","last_change":"2024-02-22T19:53:49.929993+0000","last_active":"2024-02-22T19:54:47.756319+0000","last_peered":"2024-02-22T19:54:47.756319+0000","last_clean":"2024-02-22T19:54:47.756319+0000","last_became_active":"2024-02-22T19:49:56.725545+0000","last_became_peered":"2024-02-22T19:49:56.725545+0000","last_unstale":"2024-02-22T19:54:47.756319+0000","last_undegraded":"2024-02-22T19:54:47.756319+0000","last_fullsized":"2024-02-22T19:54:47.756319+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"548'55","last_scrub_stamp":"2024-02-22T19:53:49.929903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:49.929903+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"636'47","reported_seq":877,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:49.564879+0000","last_change":"2024-02-22T19:54:40.601947+0000","last_active":"2024-02-22T19:54:49.564879+0000","last_peered":"2024-02-22T19:54:49.564879+0000","last_clean":"2024-02-22T19:54:49.564879+0000","last_became_active":"2024-02-22T19:52:46.732541+0000","last_became_peered":"2024-02-22T19:52:46.732541+0000","last_unstale":"2024-02-22T19:54:49.564879+0000","last_undegraded":"2024-02-22T19:54:49.564879+0000","last_fullsized":"2024-02-22T19:54:49.564879+0000","mapping_epoch":624,"log_start":"387'17","ondisk_log_start":"387'17","created":60,"last_epoch_clean":625,"parent":"0.0","parent_split_bits":0,"last_scrub":"636'47","last_scrub_stamp":"2024-02-22T19:54:40.601852+0000","last_deep_scrub":"636'47","last_deep_scrub_stamp":"2024-02-22T19:53:12.763656+0000","last_clean_scrub_stamp":"2024-02-22T19:54:40.601852+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":749,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658829+0000","last_change":"2024-02-22T19:54:18.620836+0000","last_active":"2024-02-22T19:54:47.658829+0000","last_peered":"2024-02-22T19:54:47.658829+0000","last_clean":"2024-02-22T19:54:47.658829+0000","last_became_active":"2024-02-22T19:52:42.739709+0000","last_became_peered":"2024-02-22T19:52:42.739709+0000","last_unstale":"2024-02-22T19:54:47.658829+0000","last_undegraded":"2024-02-22T19:54:47.658829+0000","last_fullsized":"2024-02-22T19:54:47.658829+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:18.620753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:18.620753+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":672,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659387+0000","last_change":"2024-02-22T19:54:25.662938+0000","last_active":"2024-02-22T19:54:47.659387+0000","last_peered":"2024-02-22T19:54:47.659387+0000","last_clean":"2024-02-22T19:54:47.659387+0000","last_became_active":"2024-02-22T19:49:38.804804+0000","last_became_peered":"2024-02-22T19:49:38.804804+0000","last_unstale":"2024-02-22T19:54:47.659387+0000","last_undegraded":"2024-02-22T19:54:47.659387+0000","last_fullsized":"2024-02-22T19:54:47.659387+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:25.662831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:36.620214+0000","last_clean_scrub_stamp":"2024-02-22T19:54:25.662831+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658535+0000","last_change":"2024-02-22T19:54:08.655761+0000","last_active":"2024-02-22T19:54:47.658535+0000","last_peered":"2024-02-22T19:54:47.658535+0000","last_clean":"2024-02-22T19:54:47.658535+0000","last_became_active":"2024-02-22T19:52:27.788319+0000","last_became_peered":"2024-02-22T19:52:27.788319+0000","last_unstale":"2024-02-22T19:54:47.658535+0000","last_undegraded":"2024-02-22T19:54:47.658535+0000","last_fullsized":"2024-02-22T19:54:47.658535+0000","mapping_epoch":610,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":611,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:08.655685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:08.655685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763692+0000","last_change":"2024-02-22T19:54:37.669252+0000","last_active":"2024-02-22T19:54:46.763692+0000","last_peered":"2024-02-22T19:54:46.763692+0000","last_clean":"2024-02-22T19:54:46.763692+0000","last_became_active":"2024-02-22T19:52:42.745813+0000","last_became_peered":"2024-02-22T19:52:42.745813+0000","last_unstale":"2024-02-22T19:54:46.763692+0000","last_undegraded":"2024-02-22T19:54:46.763692+0000","last_fullsized":"2024-02-22T19:54:46.763692+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:37.669170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:56.810221+0000","last_clean_scrub_stamp":"2024-02-22T19:54:37.669170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"505'21","reported_seq":207,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932175+0000","last_change":"2024-02-22T19:53:43.975381+0000","last_active":"2024-02-22T19:54:46.932175+0000","last_peered":"2024-02-22T19:54:46.932175+0000","last_clean":"2024-02-22T19:54:46.932175+0000","last_became_active":"2024-02-22T19:52:33.824073+0000","last_became_peered":"2024-02-22T19:52:33.824073+0000","last_unstale":"2024-02-22T19:54:46.932175+0000","last_undegraded":"2024-02-22T19:54:46.932175+0000","last_fullsized":"2024-02-22T19:54:46.932175+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'21","last_scrub_stamp":"2024-02-22T19:53:43.975307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:43.975307+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":28,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":2970,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"616'37","reported_seq":911,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.074132+0000","last_change":"2024-02-22T19:53:45.079884+0000","last_active":"2024-02-22T19:54:48.074132+0000","last_peered":"2024-02-22T19:54:48.074132+0000","last_clean":"2024-02-22T19:54:48.074132+0000","last_became_active":"2024-02-22T19:52:35.810441+0000","last_became_peered":"2024-02-22T19:52:35.810441+0000","last_unstale":"2024-02-22T19:54:48.074132+0000","last_undegraded":"2024-02-22T19:54:48.074132+0000","last_fullsized":"2024-02-22T19:54:48.074132+0000","mapping_epoch":615,"log_start":"382'29","ondisk_log_start":"382'29","created":60,"last_epoch_clean":616,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'37","last_scrub_stamp":"2024-02-22T19:53:45.079796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:45.079796+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":84,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764312+0000","last_change":"2024-02-22T19:54:29.841699+0000","last_active":"2024-02-22T19:54:46.764312+0000","last_peered":"2024-02-22T19:54:46.764312+0000","last_clean":"2024-02-22T19:54:46.764312+0000","last_became_active":"2024-02-22T19:52:42.747059+0000","last_became_peered":"2024-02-22T19:52:42.747059+0000","last_unstale":"2024-02-22T19:54:46.764312+0000","last_undegraded":"2024-02-22T19:54:46.764312+0000","last_fullsized":"2024-02-22T19:54:46.764312+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:29.841608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:06.762007+0000","last_clean_scrub_stamp":"2024-02-22T19:54:29.841608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":749,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.658653+0000","last_change":"2024-02-22T19:54:44.726286+0000","last_active":"2024-02-22T19:54:47.658653+0000","last_peered":"2024-02-22T19:54:47.658653+0000","last_clean":"2024-02-22T19:54:47.658653+0000","last_became_active":"2024-02-22T19:51:15.060608+0000","last_became_peered":"2024-02-22T19:51:15.060608+0000","last_unstale":"2024-02-22T19:54:47.658653+0000","last_undegraded":"2024-02-22T19:54:47.658653+0000","last_fullsized":"2024-02-22T19:54:47.658653+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:44.726208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:47:26.529483+0000","last_clean_scrub_stamp":"2024-02-22T19:54:44.726208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":726,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697568+0000","last_change":"2024-02-22T19:54:35.636212+0000","last_active":"2024-02-22T19:54:47.697568+0000","last_peered":"2024-02-22T19:54:47.697568+0000","last_clean":"2024-02-22T19:54:47.697568+0000","last_became_active":"2024-02-22T19:52:33.820991+0000","last_became_peered":"2024-02-22T19:52:33.820991+0000","last_unstale":"2024-02-22T19:54:47.697568+0000","last_undegraded":"2024-02-22T19:54:47.697568+0000","last_fullsized":"2024-02-22T19:54:47.697568+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:35.636126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:44:09.027320+0000","last_clean_scrub_stamp":"2024-02-22T19:54:35.636126+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"506'45","reported_seq":228,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816559+0000","last_change":"2024-02-22T19:54:28.179856+0000","last_active":"2024-02-22T19:54:46.816559+0000","last_peered":"2024-02-22T19:54:46.816559+0000","last_clean":"2024-02-22T19:54:46.816559+0000","last_became_active":"2024-02-22T19:52:42.745501+0000","last_became_peered":"2024-02-22T19:52:42.745501+0000","last_unstale":"2024-02-22T19:54:46.816559+0000","last_undegraded":"2024-02-22T19:54:46.816559+0000","last_fullsized":"2024-02-22T19:54:46.816559+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'45","last_scrub_stamp":"2024-02-22T19:54:28.179766+0000","last_deep_scrub":"85'21","last_deep_scrub_stamp":"2024-02-22T19:30:34.566443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:28.179766+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":50,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":924,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"617'20","reported_seq":896,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.152972+0000","last_change":"2024-02-22T19:54:46.764317+0000","last_active":"2024-02-22T19:54:48.152972+0000","last_peered":"2024-02-22T19:54:48.152972+0000","last_clean":"2024-02-22T19:54:48.152972+0000","last_became_active":"2024-02-22T19:52:36.814425+0000","last_became_peered":"2024-02-22T19:52:36.814425+0000","last_unstale":"2024-02-22T19:54:48.152972+0000","last_undegraded":"2024-02-22T19:54:48.152972+0000","last_fullsized":"2024-02-22T19:54:48.152972+0000","mapping_epoch":616,"log_start":"392'11","ondisk_log_start":"392'11","created":60,"last_epoch_clean":617,"parent":"0.0","parent_split_bits":0,"last_scrub":"617'20","last_scrub_stamp":"2024-02-22T19:54:46.764185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:46.764185+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":171,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932212+0000","last_change":"2024-02-22T19:53:54.994784+0000","last_active":"2024-02-22T19:54:46.932212+0000","last_peered":"2024-02-22T19:54:46.932212+0000","last_clean":"2024-02-22T19:54:46.932212+0000","last_became_active":"2024-02-22T19:52:42.758013+0000","last_became_peered":"2024-02-22T19:52:42.758013+0000","last_unstale":"2024-02-22T19:54:46.932212+0000","last_undegraded":"2024-02-22T19:54:46.932212+0000","last_fullsized":"2024-02-22T19:54:46.932212+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:54.994723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:53:54.994723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":162,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932512+0000","last_change":"2024-02-22T19:54:10.030373+0000","last_active":"2024-02-22T19:54:46.932512+0000","last_peered":"2024-02-22T19:54:46.932512+0000","last_clean":"2024-02-22T19:54:46.932512+0000","last_became_active":"2024-02-22T19:52:33.824337+0000","last_became_peered":"2024-02-22T19:52:33.824337+0000","last_unstale":"2024-02-22T19:54:46.932512+0000","last_undegraded":"2024-02-22T19:54:46.932512+0000","last_fullsized":"2024-02-22T19:54:46.932512+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:10.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:10.030289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":143,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933394+0000","last_change":"2024-02-22T19:54:13.374895+0000","last_active":"2024-02-22T19:54:46.933394+0000","last_peered":"2024-02-22T19:54:46.933394+0000","last_clean":"2024-02-22T19:54:46.933394+0000","last_became_active":"2024-02-22T19:52:42.740196+0000","last_became_peered":"2024-02-22T19:52:42.740196+0000","last_unstale":"2024-02-22T19:54:46.933394+0000","last_undegraded":"2024-02-22T19:54:46.933394+0000","last_fullsized":"2024-02-22T19:54:46.933394+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:13.374816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:34:06.016195+0000","last_clean_scrub_stamp":"2024-02-22T19:54:13.374816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"661'6","reported_seq":2286,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.441865+0000","last_change":"2024-02-22T19:54:23.542107+0000","last_active":"2024-02-22T19:54:48.441865+0000","last_peered":"2024-02-22T19:54:48.441865+0000","last_clean":"2024-02-22T19:54:48.441865+0000","last_became_active":"2024-02-22T19:51:15.064208+0000","last_became_peered":"2024-02-22T19:51:15.064208+0000","last_unstale":"2024-02-22T19:54:48.441865+0000","last_undegraded":"2024-02-22T19:54:48.441865+0000","last_fullsized":"2024-02-22T19:54:48.441865+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"661'6","last_scrub_stamp":"2024-02-22T19:54:23.541989+0000","last_deep_scrub":"661'6","last_deep_scrub_stamp":"2024-02-22T19:54:23.541989+0000","last_clean_scrub_stamp":"2024-02-22T19:54:23.541989+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1526,"num_read_kb":1521,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"505'36","reported_seq":130,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932999+0000","last_change":"2024-02-22T19:53:56.531080+0000","last_active":"2024-02-22T19:54:46.932999+0000","last_peered":"2024-02-22T19:54:46.932999+0000","last_clean":"2024-02-22T19:54:46.932999+0000","last_became_active":"2024-02-22T19:51:15.066428+0000","last_became_peered":"2024-02-22T19:51:15.066428+0000","last_unstale":"2024-02-22T19:54:46.932999+0000","last_undegraded":"2024-02-22T19:54:46.932999+0000","last_fullsized":"2024-02-22T19:54:46.932999+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'36","last_scrub_stamp":"2024-02-22T19:53:56.530981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:56.530981+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"617'41","reported_seq":771,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.153347+0000","last_change":"2024-02-22T19:54:04.650672+0000","last_active":"2024-02-22T19:54:48.153347+0000","last_peered":"2024-02-22T19:54:48.153347+0000","last_clean":"2024-02-22T19:54:48.153347+0000","last_became_active":"2024-02-22T19:52:36.812503+0000","last_became_peered":"2024-02-22T19:52:36.812503+0000","last_unstale":"2024-02-22T19:54:48.153347+0000","last_undegraded":"2024-02-22T19:54:48.153347+0000","last_fullsized":"2024-02-22T19:54:48.153347+0000","mapping_epoch":616,"log_start":"382'23","ondisk_log_start":"382'23","created":60,"last_epoch_clean":617,"parent":"0.0","parent_split_bits":0,"last_scrub":"617'41","last_scrub_stamp":"2024-02-22T19:54:04.650588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:40:52.541458+0000","last_clean_scrub_stamp":"2024-02-22T19:54:04.650588+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":169,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816383+0000","last_change":"2024-02-22T19:53:52.962557+0000","last_active":"2024-02-22T19:54:46.816383+0000","last_peered":"2024-02-22T19:54:46.816383+0000","last_clean":"2024-02-22T19:54:46.816383+0000","last_became_active":"2024-02-22T19:51:15.079242+0000","last_became_peered":"2024-02-22T19:51:15.079242+0000","last_unstale":"2024-02-22T19:54:46.816383+0000","last_undegraded":"2024-02-22T19:54:46.816383+0000","last_fullsized":"2024-02-22T19:54:46.816383+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:52.962472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:36:14.230365+0000","last_clean_scrub_stamp":"2024-02-22T19:53:52.962472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":144,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932117+0000","last_change":"2024-02-22T19:54:15.204368+0000","last_active":"2024-02-22T19:54:46.932117+0000","last_peered":"2024-02-22T19:54:46.932117+0000","last_clean":"2024-02-22T19:54:46.932117+0000","last_became_active":"2024-02-22T19:49:56.723819+0000","last_became_peered":"2024-02-22T19:49:56.723819+0000","last_unstale":"2024-02-22T19:54:46.932117+0000","last_undegraded":"2024-02-22T19:54:46.932117+0000","last_fullsized":"2024-02-22T19:54:46.932117+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:15.204280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:42.994920+0000","last_clean_scrub_stamp":"2024-02-22T19:54:15.204280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":171,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932676+0000","last_change":"2024-02-22T19:54:08.007017+0000","last_active":"2024-02-22T19:54:46.932676+0000","last_peered":"2024-02-22T19:54:46.932676+0000","last_clean":"2024-02-22T19:54:46.932676+0000","last_became_active":"2024-02-22T19:52:33.825668+0000","last_became_peered":"2024-02-22T19:52:33.825668+0000","last_unstale":"2024-02-22T19:54:46.932676+0000","last_undegraded":"2024-02-22T19:54:46.932676+0000","last_fullsized":"2024-02-22T19:54:46.932676+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:08.006949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:54:08.006949+0000","last_clean_scrub_stamp":"2024-02-22T19:54:08.006949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"506'47","reported_seq":161,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933780+0000","last_change":"2024-02-22T19:53:45.679921+0000","last_active":"2024-02-22T19:54:46.933780+0000","last_peered":"2024-02-22T19:54:46.933780+0000","last_clean":"2024-02-22T19:54:46.933780+0000","last_became_active":"2024-02-22T19:52:33.823004+0000","last_became_peered":"2024-02-22T19:52:33.823004+0000","last_unstale":"2024-02-22T19:54:46.933780+0000","last_undegraded":"2024-02-22T19:54:46.933780+0000","last_fullsized":"2024-02-22T19:54:46.933780+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'47","last_scrub_stamp":"2024-02-22T19:53:45.679854+0000","last_deep_scrub":"463'37","last_deep_scrub_stamp":"2024-02-22T19:47:24.474452+0000","last_clean_scrub_stamp":"2024-02-22T19:53:45.679854+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1980,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":187,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816202+0000","last_change":"2024-02-22T19:54:03.127185+0000","last_active":"2024-02-22T19:54:46.816202+0000","last_peered":"2024-02-22T19:54:46.816202+0000","last_clean":"2024-02-22T19:54:46.816202+0000","last_became_active":"2024-02-22T19:52:33.821391+0000","last_became_peered":"2024-02-22T19:52:33.821391+0000","last_unstale":"2024-02-22T19:54:46.816202+0000","last_undegraded":"2024-02-22T19:54:46.816202+0000","last_fullsized":"2024-02-22T19:54:46.816202+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:03.127102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:03.127102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":160,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816385+0000","last_change":"2024-02-22T19:54:18.357389+0000","last_active":"2024-02-22T19:54:46.816385+0000","last_peered":"2024-02-22T19:54:46.816385+0000","last_clean":"2024-02-22T19:54:46.816385+0000","last_became_active":"2024-02-22T19:49:38.809765+0000","last_became_peered":"2024-02-22T19:49:38.809765+0000","last_unstale":"2024-02-22T19:54:46.816385+0000","last_undegraded":"2024-02-22T19:54:46.816385+0000","last_fullsized":"2024-02-22T19:54:46.816385+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:18.357305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:48:13.199920+0000","last_clean_scrub_stamp":"2024-02-22T19:54:18.357305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":116,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.933467+0000","last_change":"2024-02-22T19:54:05.370689+0000","last_active":"2024-02-22T19:54:46.933467+0000","last_peered":"2024-02-22T19:54:46.933467+0000","last_clean":"2024-02-22T19:54:46.933467+0000","last_became_active":"2024-02-22T19:51:15.055763+0000","last_became_peered":"2024-02-22T19:51:15.055763+0000","last_unstale":"2024-02-22T19:54:46.933467+0000","last_undegraded":"2024-02-22T19:54:46.933467+0000","last_fullsized":"2024-02-22T19:54:46.933467+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:05.370612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:18.719310+0000","last_clean_scrub_stamp":"2024-02-22T19:54:05.370612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764013+0000","last_change":"2024-02-22T19:54:39.708777+0000","last_active":"2024-02-22T19:54:46.764013+0000","last_peered":"2024-02-22T19:54:46.764013+0000","last_clean":"2024-02-22T19:54:46.764013+0000","last_became_active":"2024-02-22T19:52:33.822386+0000","last_became_peered":"2024-02-22T19:52:33.822386+0000","last_unstale":"2024-02-22T19:54:46.764013+0000","last_undegraded":"2024-02-22T19:54:46.764013+0000","last_fullsized":"2024-02-22T19:54:46.764013+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:39.708656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:41:30.423423+0000","last_clean_scrub_stamp":"2024-02-22T19:54:39.708656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"506'30","reported_seq":261,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.931776+0000","last_change":"2024-02-22T19:53:46.927809+0000","last_active":"2024-02-22T19:54:46.931776+0000","last_peered":"2024-02-22T19:54:46.931776+0000","last_clean":"2024-02-22T19:54:46.931776+0000","last_became_active":"2024-02-22T19:52:42.738773+0000","last_became_peered":"2024-02-22T19:52:42.738773+0000","last_unstale":"2024-02-22T19:54:46.931776+0000","last_undegraded":"2024-02-22T19:54:46.931776+0000","last_fullsized":"2024-02-22T19:54:46.931776+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'30","last_scrub_stamp":"2024-02-22T19:53:46.927741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:46.927741+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"650'28","reported_seq":368,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.756439+0000","last_change":"2024-02-22T19:53:40.789921+0000","last_active":"2024-02-22T19:54:47.756439+0000","last_peered":"2024-02-22T19:54:47.756439+0000","last_clean":"2024-02-22T19:54:47.756439+0000","last_became_active":"2024-02-22T19:53:36.799786+0000","last_became_peered":"2024-02-22T19:53:36.799786+0000","last_unstale":"2024-02-22T19:54:47.756439+0000","last_undegraded":"2024-02-22T19:54:47.756439+0000","last_fullsized":"2024-02-22T19:54:47.756439+0000","mapping_epoch":648,"log_start":"326'14","ondisk_log_start":"326'14","created":60,"last_epoch_clean":649,"parent":"0.0","parent_split_bits":0,"last_scrub":"650'28","last_scrub_stamp":"2024-02-22T19:53:40.789798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:53:40.789798+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":594,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697702+0000","last_change":"2024-02-22T19:54:28.525916+0000","last_active":"2024-02-22T19:54:47.697702+0000","last_peered":"2024-02-22T19:54:47.697702+0000","last_clean":"2024-02-22T19:54:47.697702+0000","last_became_active":"2024-02-22T19:52:33.821549+0000","last_became_peered":"2024-02-22T19:52:33.821549+0000","last_unstale":"2024-02-22T19:54:47.697702+0000","last_undegraded":"2024-02-22T19:54:47.697702+0000","last_fullsized":"2024-02-22T19:54:47.697702+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:28.525851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:28.525851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":679,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.659183+0000","last_change":"2024-02-22T19:53:20.024193+0000","last_active":"2024-02-22T19:54:47.659183+0000","last_peered":"2024-02-22T19:54:47.659183+0000","last_clean":"2024-02-22T19:54:47.659183+0000","last_became_active":"2024-02-22T19:49:38.795530+0000","last_became_peered":"2024-02-22T19:49:38.795530+0000","last_unstale":"2024-02-22T19:54:47.659183+0000","last_undegraded":"2024-02-22T19:54:47.659183+0000","last_fullsized":"2024-02-22T19:54:47.659183+0000","mapping_epoch":527,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":528,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:20.024107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:30:38.631697+0000","last_clean_scrub_stamp":"2024-02-22T19:53:20.024107+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":134,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.932910+0000","last_change":"2024-02-22T19:54:19.464744+0000","last_active":"2024-02-22T19:54:46.932910+0000","last_peered":"2024-02-22T19:54:46.932910+0000","last_clean":"2024-02-22T19:54:46.932910+0000","last_became_active":"2024-02-22T19:52:42.756445+0000","last_became_peered":"2024-02-22T19:52:42.756445+0000","last_unstale":"2024-02-22T19:54:46.932910+0000","last_undegraded":"2024-02-22T19:54:46.932910+0000","last_fullsized":"2024-02-22T19:54:46.932910+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:19.464680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:19.464680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"506'49","reported_seq":85,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934420+0000","last_change":"2024-02-22T19:54:00.823662+0000","last_active":"2024-02-22T19:54:46.934420+0000","last_peered":"2024-02-22T19:54:46.934420+0000","last_clean":"2024-02-22T19:54:46.934420+0000","last_became_active":"2024-02-22T19:52:42.737608+0000","last_became_peered":"2024-02-22T19:52:42.737608+0000","last_unstale":"2024-02-22T19:54:46.934420+0000","last_undegraded":"2024-02-22T19:54:46.934420+0000","last_fullsized":"2024-02-22T19:54:46.934420+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'49","last_scrub_stamp":"2024-02-22T19:54:00.823582+0000","last_deep_scrub":"506'49","last_deep_scrub_stamp":"2024-02-22T19:54:00.823582+0000","last_clean_scrub_stamp":"2024-02-22T19:54:00.823582+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"670'28","reported_seq":1176,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:49.583513+0000","last_change":"2024-02-22T19:54:49.583513+0000","last_active":"2024-02-22T19:54:49.583513+0000","last_peered":"2024-02-22T19:54:49.583513+0000","last_clean":"2024-02-22T19:54:49.583513+0000","last_became_active":"2024-02-22T19:54:45.925300+0000","last_became_peered":"2024-02-22T19:54:45.925300+0000","last_unstale":"2024-02-22T19:54:49.583513+0000","last_undegraded":"2024-02-22T19:54:49.583513+0000","last_fullsized":"2024-02-22T19:54:49.583513+0000","mapping_epoch":667,"log_start":"268'11","ondisk_log_start":"268'11","created":60,"last_epoch_clean":668,"parent":"0.0","parent_split_bits":0,"last_scrub":"670'28","last_scrub_stamp":"2024-02-22T19:54:49.583433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:49.583433+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935085+0000","last_change":"2024-02-22T19:54:23.859812+0000","last_active":"2024-02-22T19:54:46.935085+0000","last_peered":"2024-02-22T19:54:46.935085+0000","last_clean":"2024-02-22T19:54:46.935085+0000","last_became_active":"2024-02-22T19:52:42.738118+0000","last_became_peered":"2024-02-22T19:52:42.738118+0000","last_unstale":"2024-02-22T19:54:46.935085+0000","last_undegraded":"2024-02-22T19:54:46.935085+0000","last_fullsized":"2024-02-22T19:54:46.935085+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:23.859734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:52.807890+0000","last_clean_scrub_stamp":"2024-02-22T19:54:23.859734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934522+0000","last_change":"2024-02-22T19:53:30.827202+0000","last_active":"2024-02-22T19:54:46.934522+0000","last_peered":"2024-02-22T19:54:46.934522+0000","last_clean":"2024-02-22T19:54:46.934522+0000","last_became_active":"2024-02-22T19:52:42.740401+0000","last_became_peered":"2024-02-22T19:52:42.740401+0000","last_unstale":"2024-02-22T19:54:46.934522+0000","last_undegraded":"2024-02-22T19:54:46.934522+0000","last_fullsized":"2024-02-22T19:54:46.934522+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:30.827120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:30.827120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":699,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.698233+0000","last_change":"2024-02-22T19:54:27.510017+0000","last_active":"2024-02-22T19:54:47.698233+0000","last_peered":"2024-02-22T19:54:47.698233+0000","last_clean":"2024-02-22T19:54:47.698233+0000","last_became_active":"2024-02-22T19:51:15.184129+0000","last_became_peered":"2024-02-22T19:51:15.184129+0000","last_unstale":"2024-02-22T19:54:47.698233+0000","last_undegraded":"2024-02-22T19:54:47.698233+0000","last_fullsized":"2024-02-22T19:54:47.698233+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":574,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:27.509948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:37:06.317125+0000","last_clean_scrub_stamp":"2024-02-22T19:54:27.509948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"657'50","reported_seq":894,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697693+0000","last_change":"2024-02-22T19:53:57.524296+0000","last_active":"2024-02-22T19:54:47.697693+0000","last_peered":"2024-02-22T19:54:47.697693+0000","last_clean":"2024-02-22T19:54:47.697693+0000","last_became_active":"2024-02-22T19:51:15.079954+0000","last_became_peered":"2024-02-22T19:51:15.079954+0000","last_unstale":"2024-02-22T19:54:47.697693+0000","last_undegraded":"2024-02-22T19:54:47.697693+0000","last_fullsized":"2024-02-22T19:54:47.697693+0000","mapping_epoch":573,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"657'50","last_scrub_stamp":"2024-02-22T19:53:57.524223+0000","last_deep_scrub":"657'50","last_deep_scrub_stamp":"2024-02-22T19:53:57.524223+0000","last_clean_scrub_stamp":"2024-02-22T19:53:57.524223+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":69,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934546+0000","last_change":"2024-02-22T19:54:38.938568+0000","last_active":"2024-02-22T19:54:46.934546+0000","last_peered":"2024-02-22T19:54:46.934546+0000","last_clean":"2024-02-22T19:54:46.934546+0000","last_became_active":"2024-02-22T19:52:42.745561+0000","last_became_peered":"2024-02-22T19:52:42.745561+0000","last_unstale":"2024-02-22T19:54:46.934546+0000","last_undegraded":"2024-02-22T19:54:46.934546+0000","last_fullsized":"2024-02-22T19:54:46.934546+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:38.938481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:04.066424+0000","last_clean_scrub_stamp":"2024-02-22T19:54:38.938481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.763206+0000","last_change":"2024-02-22T19:54:09.690939+0000","last_active":"2024-02-22T19:54:46.763206+0000","last_peered":"2024-02-22T19:54:46.763206+0000","last_clean":"2024-02-22T19:54:46.763206+0000","last_became_active":"2024-02-22T19:52:33.820066+0000","last_became_peered":"2024-02-22T19:52:33.820066+0000","last_unstale":"2024-02-22T19:54:46.763206+0000","last_undegraded":"2024-02-22T19:54:46.763206+0000","last_fullsized":"2024-02-22T19:54:46.763206+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:09.690845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:54:09.690845+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.764153+0000","last_change":"2024-02-22T19:54:13.718534+0000","last_active":"2024-02-22T19:54:46.764153+0000","last_peered":"2024-02-22T19:54:46.764153+0000","last_clean":"2024-02-22T19:54:46.764153+0000","last_became_active":"2024-02-22T19:52:33.821421+0000","last_became_peered":"2024-02-22T19:52:33.821421+0000","last_unstale":"2024-02-22T19:54:46.764153+0000","last_undegraded":"2024-02-22T19:54:46.764153+0000","last_fullsized":"2024-02-22T19:54:46.764153+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:13.718443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:54:13.718443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":677,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697831+0000","last_change":"2024-02-22T19:54:37.634025+0000","last_active":"2024-02-22T19:54:47.697831+0000","last_peered":"2024-02-22T19:54:47.697831+0000","last_clean":"2024-02-22T19:54:47.697831+0000","last_became_active":"2024-02-22T19:49:56.726891+0000","last_became_peered":"2024-02-22T19:49:56.726891+0000","last_unstale":"2024-02-22T19:54:47.697831+0000","last_undegraded":"2024-02-22T19:54:47.697831+0000","last_fullsized":"2024-02-22T19:54:47.697831+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:37.633949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:37.633949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"506'39","reported_seq":56,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.920178+0000","last_change":"2024-02-22T19:54:10.088224+0000","last_active":"2024-02-22T19:54:45.920178+0000","last_peered":"2024-02-22T19:54:45.920178+0000","last_clean":"2024-02-22T19:54:45.920178+0000","last_became_active":"2024-02-22T19:53:28.765529+0000","last_became_peered":"2024-02-22T19:53:28.765529+0000","last_unstale":"2024-02-22T19:54:45.920178+0000","last_undegraded":"2024-02-22T19:54:45.920178+0000","last_fullsized":"2024-02-22T19:54:45.920178+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":645,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'39","last_scrub_stamp":"2024-02-22T19:54:10.088160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:10.088160+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"137'2","reported_seq":775,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697922+0000","last_change":"2024-02-22T19:53:26.678137+0000","last_active":"2024-02-22T19:54:47.697922+0000","last_peered":"2024-02-22T19:54:47.697922+0000","last_clean":"2024-02-22T19:54:47.697922+0000","last_became_active":"2024-02-22T19:53:06.913551+0000","last_became_peered":"2024-02-22T19:53:06.913551+0000","last_unstale":"2024-02-22T19:54:47.697922+0000","last_undegraded":"2024-02-22T19:54:47.697922+0000","last_fullsized":"2024-02-22T19:54:47.697922+0000","mapping_epoch":632,"log_start":"137'2","ondisk_log_start":"137'2","created":62,"last_epoch_clean":633,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-02-22T19:53:26.678008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:06.076443+0000","last_clean_scrub_stamp":"2024-02-22T19:53:26.678008+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":64,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934675+0000","last_change":"2024-02-22T19:53:33.788663+0000","last_active":"2024-02-22T19:54:46.934675+0000","last_peered":"2024-02-22T19:54:46.934675+0000","last_clean":"2024-02-22T19:54:46.934675+0000","last_became_active":"2024-02-22T19:52:42.733083+0000","last_became_peered":"2024-02-22T19:52:42.733083+0000","last_unstale":"2024-02-22T19:54:46.934675+0000","last_undegraded":"2024-02-22T19:54:46.934675+0000","last_fullsized":"2024-02-22T19:54:46.934675+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:33.788576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:33.788576+0000","last_clean_scrub_stamp":"2024-02-22T19:53:33.788576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"506'27","reported_seq":81,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935055+0000","last_change":"2024-02-22T19:53:56.792103+0000","last_active":"2024-02-22T19:54:46.935055+0000","last_peered":"2024-02-22T19:54:46.935055+0000","last_clean":"2024-02-22T19:54:46.935055+0000","last_became_active":"2024-02-22T19:52:42.737550+0000","last_became_peered":"2024-02-22T19:52:42.737550+0000","last_unstale":"2024-02-22T19:54:46.935055+0000","last_undegraded":"2024-02-22T19:54:46.935055+0000","last_fullsized":"2024-02-22T19:54:46.935055+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'27","last_scrub_stamp":"2024-02-22T19:53:56.792023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:53:56.792023+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":75,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.762999+0000","last_change":"2024-02-22T19:53:49.657738+0000","last_active":"2024-02-22T19:54:46.762999+0000","last_peered":"2024-02-22T19:54:46.762999+0000","last_clean":"2024-02-22T19:54:46.762999+0000","last_became_active":"2024-02-22T19:52:33.819026+0000","last_became_peered":"2024-02-22T19:52:33.819026+0000","last_unstale":"2024-02-22T19:54:46.762999+0000","last_undegraded":"2024-02-22T19:54:46.762999+0000","last_fullsized":"2024-02-22T19:54:46.762999+0000","mapping_epoch":613,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":614,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:53:49.657650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:26:54.891282+0000","last_clean_scrub_stamp":"2024-02-22T19:53:49.657650+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":728,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:45.920076+0000","last_change":"2024-02-22T19:54:19.386189+0000","last_active":"2024-02-22T19:54:45.920076+0000","last_peered":"2024-02-22T19:54:45.920076+0000","last_clean":"2024-02-22T19:54:45.920076+0000","last_became_active":"2024-02-22T19:52:42.745708+0000","last_became_peered":"2024-02-22T19:52:42.745708+0000","last_unstale":"2024-02-22T19:54:45.920076+0000","last_undegraded":"2024-02-22T19:54:45.920076+0000","last_fullsized":"2024-02-22T19:54:45.920076+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:19.386106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:52:58.807220+0000","last_clean_scrub_stamp":"2024-02-22T19:54:19.386106+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":187,"reported_epoch":668,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.816056+0000","last_change":"2024-02-22T19:54:35.300025+0000","last_active":"2024-02-22T19:54:46.816056+0000","last_peered":"2024-02-22T19:54:46.816056+0000","last_clean":"2024-02-22T19:54:46.816056+0000","last_became_active":"2024-02-22T19:52:42.746960+0000","last_became_peered":"2024-02-22T19:52:42.746960+0000","last_unstale":"2024-02-22T19:54:46.816056+0000","last_undegraded":"2024-02-22T19:54:46.816056+0000","last_fullsized":"2024-02-22T19:54:46.816056+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:35.299944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:04.963830+0000","last_clean_scrub_stamp":"2024-02-22T19:54:35.299944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"506'45","reported_seq":114,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935179+0000","last_change":"2024-02-22T19:54:14.861000+0000","last_active":"2024-02-22T19:54:46.935179+0000","last_peered":"2024-02-22T19:54:46.935179+0000","last_clean":"2024-02-22T19:54:46.935179+0000","last_became_active":"2024-02-22T19:53:29.522364+0000","last_became_peered":"2024-02-22T19:53:29.522364+0000","last_unstale":"2024-02-22T19:54:46.935179+0000","last_undegraded":"2024-02-22T19:54:46.935179+0000","last_fullsized":"2024-02-22T19:54:46.935179+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":645,"parent":"0.0","parent_split_bits":0,"last_scrub":"506'45","last_scrub_stamp":"2024-02-22T19:54:14.860912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:29:02.056447+0000","last_clean_scrub_stamp":"2024-02-22T19:54:14.860912+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":59,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":677,"reported_epoch":670,"state":"active+clean","last_fresh":"2024-02-22T19:54:48.733100+0000","last_change":"2024-02-22T19:54:48.733100+0000","last_active":"2024-02-22T19:54:48.733100+0000","last_peered":"2024-02-22T19:54:48.733100+0000","last_clean":"2024-02-22T19:54:48.733100+0000","last_became_active":"2024-02-22T19:35:58.421198+0000","last_became_peered":"2024-02-22T19:35:58.421198+0000","last_unstale":"2024-02-22T19:54:48.733100+0000","last_undegraded":"2024-02-22T19:54:48.733100+0000","last_fullsized":"2024-02-22T19:54:48.733100+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T19:54:48.733019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:53:16.942709+0000","last_clean_scrub_stamp":"2024-02-22T19:54:48.733019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"85'2","reported_seq":320,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.935000+0000","last_change":"2024-02-22T19:54:16.914427+0000","last_active":"2024-02-22T19:54:46.935000+0000","last_peered":"2024-02-22T19:54:46.935000+0000","last_clean":"2024-02-22T19:54:46.935000+0000","last_became_active":"2024-02-22T19:52:44.722428+0000","last_became_peered":"2024-02-22T19:52:44.722428+0000","last_unstale":"2024-02-22T19:54:46.935000+0000","last_undegraded":"2024-02-22T19:54:46.935000+0000","last_fullsized":"2024-02-22T19:54:46.935000+0000","mapping_epoch":622,"log_start":"85'2","ondisk_log_start":"85'2","created":62,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-22T19:54:16.914332+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-22T19:48:07.111947+0000","last_clean_scrub_stamp":"2024-02-22T19:54:16.914332+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":78,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"57'1","reported_seq":793,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:47.697666+0000","last_change":"2024-02-22T19:54:25.503848+0000","last_active":"2024-02-22T19:54:47.697666+0000","last_peered":"2024-02-22T19:54:47.697666+0000","last_clean":"2024-02-22T19:54:47.697666+0000","last_became_active":"2024-02-22T19:52:42.749256+0000","last_became_peered":"2024-02-22T19:52:42.749256+0000","last_unstale":"2024-02-22T19:54:47.697666+0000","last_undegraded":"2024-02-22T19:54:47.697666+0000","last_fullsized":"2024-02-22T19:54:47.697666+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":621,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-22T19:54:25.503746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T19:28:59.986842+0000","last_clean_scrub_stamp":"2024-02-22T19:54:25.503746+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"661'35","reported_seq":86,"reported_epoch":669,"state":"active+clean","last_fresh":"2024-02-22T19:54:46.934831+0000","last_change":"2024-02-22T19:54:22.867075+0000","last_active":"2024-02-22T19:54:46.934831+0000","last_peered":"2024-02-22T19:54:46.934831+0000","last_clean":"2024-02-22T19:54:46.934831+0000","last_became_active":"2024-02-22T19:52:42.744111+0000","last_became_peered":"2024-02-22T19:52:42.744111+0000","last_unstale":"2024-02-22T19:54:46.934831+0000","last_undegraded":"2024-02-22T19:54:46.934831+0000","last_fullsized":"2024-02-22T19:54:46.934831+0000","mapping_epoch":620,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":623,"parent":"0.0","parent_split_bits":0,"last_scrub":"661'35","last_scrub_stamp":"2024-02-22T19:54:22.866987+0000","last_deep_scrub":"661'35","last_deep_scrub_stamp":"2024-02-22T19:54:22.866987+0000","last_clean_scrub_stamp":"2024-02-22T19:54:22.866987+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":37,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1526,"num_read_kb":1521,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":33,"acting":33,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":12964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1515,"num_read_kb":1272,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":930,"num_bytes_recovered":20698,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1049,"ondisk_log_size":1049,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":12,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":36,"acting":36,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":66,"acting":66,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633267002,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":899060,"kb_used_data":6252,"kb_used_omap":0,"kb_used_meta":892800,"kb_avail":92846092,"statfs":{"total":95995035648,"available":95074398208,"internally_reserved":0,"allocated":6402048,"data_stored":3631674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":914227200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.35099999999999998,"5min":0.24399999999999999,"15min":0.219},"max":{"1min":0.78500000000000003,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":1.7889999999999999,"15min":1.3140000000000001},"min":{"1min":0.313,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76000000000000001,"5min":17.716000000000001,"15min":17.716000000000001},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":1.675,"15min":1.7470000000000001},"min":{"1min":0.32200000000000001,"5min":0.22500000000000001,"15min":0.20000000000000001},"max":{"1min":0.80600000000000005,"5min":13.987,"15min":36.509},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":1.931,"15min":1.8169999999999999},"min":{"1min":0.39000000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":0.72799999999999998,"5min":11.532,"15min":20.923999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":2.024,"15min":1.982},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.24299999999999999},"max":{"1min":0.65900000000000003,"5min":23.856999999999999,"15min":42.377000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":2.1840000000000002,"15min":2.2290000000000001},"min":{"1min":0.38,"5min":0.27100000000000002,"15min":0.23499999999999999},"max":{"1min":0.76800000000000002,"5min":20.815000000000001,"15min":39.520000000000003},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":1.766,"15min":1.9730000000000001},"min":{"1min":0.36699999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71599999999999997,"5min":10.436999999999999,"15min":37.905000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.859,"15min":2.0710000000000002},"min":{"1min":0.48599999999999999,"5min":0.30199999999999999,"15min":0.27400000000000002},"max":{"1min":0.80900000000000005,"5min":15.047000000000001,"15min":38.25},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":1.2230000000000001,"15min":0.95699999999999996},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.623,"5min":9.9930000000000003,"15min":9.9930000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":1.0860000000000001,"15min":1.004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27000000000000002},"max":{"1min":0.68500000000000005,"5min":9.8390000000000004,"15min":9.8390000000000004},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.2809999999999999,"15min":1.1599999999999999},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82599999999999996,"5min":14.023,"15min":14.023},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":1.427,"15min":1.0700000000000001},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":9.6530000000000005,"15min":9.6530000000000005},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.498,"15min":1.0680000000000001},"min":{"1min":0.29899999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.75600000000000001,"5min":11.583,"15min":11.583},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.8089999999999999,"15min":1.345},"min":{"1min":0.315,"5min":0.315,"15min":0.28399999999999997},"max":{"1min":0.73899999999999999,"5min":13.143000000000001,"15min":13.143000000000001},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528639,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":604132,"kb_used_data":6236,"kb_used_omap":0,"kb_used_meta":597888,"kb_avail":93141020,"statfs":{"total":95995035648,"available":95376404480,"internally_reserved":0,"allocated":6385664,"data_stored":3631342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":612237312},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":1.333,"15min":1.111},"min":{"1min":0.35899999999999999,"5min":0.24299999999999999,"15min":0.22500000000000001},"max":{"1min":0.71199999999999997,"5min":14.731,"15min":14.731},"last":0.495},{"interface":"front","average":{"1min":0.52800000000000002,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.25900000000000001},"max":{"1min":0.85999999999999999,"5min":10.186999999999999,"15min":10.186999999999999},"last":0.63}]},{"osd":1,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.954,"15min":1.804},"min":{"1min":0.36399999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.79400000000000004,"5min":27.132999999999999,"15min":44.101999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":1.625,"15min":1.6679999999999999},"min":{"1min":0.41399999999999998,"5min":0.26300000000000001,"15min":0.222},"max":{"1min":0.82599999999999996,"5min":27.366,"15min":44.018000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":3.3420000000000001,"15min":2.5259999999999998},"min":{"1min":0.40500000000000003,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":0.80800000000000005,"5min":93.703000000000003,"15min":93.703000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":3.004,"15min":2.3959999999999999},"min":{"1min":0.40899999999999997,"5min":0.27400000000000002,"15min":0.24299999999999999},"max":{"1min":0.879,"5min":93.260999999999996,"15min":93.260999999999996},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.6399999999999999,"15min":1.514},"min":{"1min":0.44800000000000001,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.77200000000000002,"5min":9.548,"15min":18.661999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":1.4450000000000001,"15min":1.647},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.28499999999999998},"max":{"1min":0.89800000000000002,"5min":7.6269999999999998,"15min":19.154},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":1.0209999999999999,"15min":0.81999999999999995},"min":{"1min":0.317,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.73399999999999999,"5min":10.06,"15min":10.06},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.3169999999999999,"15min":0.98699999999999999},"min":{"1min":0.35499999999999998,"5min":0.29399999999999998,"15min":0.253},"max":{"1min":0.76800000000000002,"5min":10.827,"15min":10.827},"last":0.75}]},{"osd":5,"last update":"Thu Feb 22 19:54:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.155,"15min":1.0880000000000001},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.255},"max":{"1min":0.754,"5min":7.8630000000000004,"15min":21.675000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.0700000000000001,"15min":0.89900000000000002},"min":{"1min":0.37,"5min":0.318,"15min":0.28199999999999997},"max":{"1min":0.74099999999999999,"5min":8.4789999999999992,"15min":21.634},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Feb 22 19:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":1.3340000000000001,"15min":1.0529999999999999},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.26800000000000002},"max":{"1min":0.84399999999999997,"5min":8.6470000000000002,"15min":22.138999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":1.3859999999999999,"15min":1.1240000000000001},"min":{"1min":0.45000000000000001,"5min":0.35599999999999998,"15min":0.32800000000000001},"max":{"1min":0.91000000000000003,"5min":15.99,"15min":22.678000000000001},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575192,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904784,"kb_used_data":13384,"kb_used_omap":0,"kb_used_meta":891392,"kb_avail":92840368,"statfs":{"total":95995035648,"available":95068536832,"internally_reserved":0,"allocated":13705216,"data_stored":10907093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912785408},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":1.319,"15min":1.819},"min":{"1min":0.216,"5min":0.216,"15min":0.19900000000000001},"max":{"1min":0.48899999999999999,"5min":14.710000000000001,"15min":51.091000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":1.4530000000000001,"15min":1.857},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":0.56100000000000005,"5min":7.282,"15min":51.289999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Feb 22 19:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":2.0139999999999998,"15min":2.5190000000000001},"min":{"1min":0.254,"5min":0.252,"15min":0.191},"max":{"1min":0.53800000000000003,"5min":42.884,"15min":51.027000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":1.954,"15min":2.5430000000000001},"min":{"1min":0.307,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.63700000000000001,"5min":43,"15min":51.369999999999997},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":1.7130000000000001,"15min":2.2069999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.23400000000000001},"max":{"1min":0.59099999999999997,"5min":43.295999999999999,"15min":82.034000000000006},"last":0.31},{"interface":"front","average":{"1min":0.42399999999999999,"5min":2.3290000000000002,"15min":2.7010000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.20200000000000001},"max":{"1min":0.67300000000000004,"5min":43.338000000000001,"15min":81.765000000000001},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:54:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":2.0270000000000001,"15min":2.0750000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.79300000000000004,"5min":14.663,"15min":51.158999999999999},"last":0.38},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.4910000000000001,"15min":1.466},"min":{"1min":0.37,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.78900000000000003,"5min":8.8390000000000004,"15min":51.276000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Feb 22 19:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":2.0750000000000002,"15min":2.1160000000000001},"min":{"1min":0.42899999999999999,"5min":0.29399999999999998,"15min":0.25800000000000001},"max":{"1min":1.0760000000000001,"5min":14.901,"15min":51.183999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":1.6990000000000001,"15min":1.8069999999999999},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95999999999999996,"5min":6.6189999999999998,"15min":51.212000000000003},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Feb 22 19:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":1.2849999999999999,"15min":1.494},"min":{"1min":0.436,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":7.9400000000000004,"15min":51.079999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":2.0169999999999999,"15min":2.1179999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33100000000000002},"max":{"1min":1.044,"5min":14.741,"15min":51.119999999999997},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Feb 22 19:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.47,"15min":1.839},"min":{"1min":0.41799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86799999999999999,"5min":11.096,"15min":51.244999999999997},"last":0.504},{"interface":"front","average":{"1min":0.623,"5min":1.258,"15min":1.5529999999999999},"min":{"1min":0.438,"5min":0.314,"15min":0.30399999999999999},"max":{"1min":0.83399999999999996,"5min":7.5579999999999998,"15min":51.317999999999998},"last":0.438}]}]},{"osd":0,"up_from":8,"seq":34359738717,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307580,"kb_used_data":6260,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93437572,"statfs":{"total":95995035648,"available":95680073728,"internally_reserved":0,"allocated":6410240,"data_stored":3632613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 19:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":1.1830000000000001,"15min":1.367},"min":{"1min":0.21299999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.44500000000000001,"5min":8.3659999999999997,"15min":24.748999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":1.379,"15min":1.6990000000000001},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51100000000000001,"5min":7.7069999999999999,"15min":25.222999999999999},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Feb 22 19:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":1.361,"15min":1.6819999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.21099999999999999},"max":{"1min":0.51700000000000002,"5min":9.8699999999999992,"15min":30.353999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":2.633,"15min":2.3149999999999999},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.69699999999999995,"5min":22.062999999999999,"15min":30.404},"last":0.375}]},{"osd":3,"last update":"Thu Feb 22 19:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.7490000000000001,"15min":1.9730000000000001},"min":{"1min":0.27400000000000002,"5min":0.185,"15min":0.185},"max":{"1min":0.71999999999999997,"5min":15.18,"15min":32.262},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":1.579,"15min":1.913},"min":{"1min":0.307,"5min":0.216,"15min":0.216},"max":{"1min":0.74399999999999999,"5min":8.4169999999999998,"15min":29.065000000000001},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.4450000000000001,"15min":1.159},"min":{"1min":0.42699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78900000000000003,"5min":11.281000000000001,"15min":11.281000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.3120000000000001,"15min":1.2190000000000001},"min":{"1min":0.39000000000000001,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":0.73499999999999999,"5min":9.8369999999999997,"15min":9.8369999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Feb 22 19:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":1.341,"15min":1.419},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.29399999999999998},"max":{"1min":0.69199999999999995,"5min":7.0039999999999996,"15min":56.829000000000001},"last":0.505},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.7430000000000001,"15min":1.53},"min":{"1min":0.39200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77300000000000002,"5min":11.23,"15min":57.338000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Feb 22 19:54:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":1.296,"15min":1.177},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.27400000000000002},"max":{"1min":0.82799999999999996,"5min":9.6969999999999992,"15min":9.6969999999999992},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":1.212,"15min":1.2549999999999999},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.80000000000000004,"5min":5.6200000000000001,"15min":10.093999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":1.3200000000000001,"15min":1.2909999999999999},"min":{"1min":0.42199999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":11.304,"15min":11.304},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.62,"5min":1.454,"15min":1.4139999999999999},"min":{"1min":0.39200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":11.631,"15min":11.631},"last":0.64000000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411666,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":905772,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":899520,"kb_avail":92839380,"statfs":{"total":95995035648,"available":95067525120,"internally_reserved":0,"allocated":6393856,"data_stored":3631080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":921108480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":1.7130000000000001,"15min":1.7090000000000001},"min":{"1min":0.22,"5min":0.20399999999999999,"15min":0.19900000000000001},"max":{"1min":0.498,"5min":13.205,"15min":16.137},"last":0.222},{"interface":"front","average":{"1min":0.372,"5min":2.2719999999999998,"15min":2.242},"min":{"1min":0.22900000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.49399999999999999,"5min":14.148999999999999,"15min":27.361999999999998},"last":0.247}]},{"osd":1,"last update":"Thu Feb 22 19:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.9470000000000001,"15min":2.2450000000000001},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.20200000000000001},"max":{"1min":0.69899999999999995,"5min":31.641999999999999,"15min":48.305},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":2.3300000000000001,"15min":2.4159999999999999},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.65500000000000003,"5min":31.969000000000001,"15min":50.448999999999998},"last":0.315}]},{"osd":3,"last update":"Thu Feb 22 19:54:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":2.1779999999999999,"15min":2.1880000000000002},"min":{"1min":0.313,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.73099999999999998,"5min":18.280000000000001,"15min":18.280000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":2.806,"15min":2.5049999999999999},"min":{"1min":0.27300000000000002,"5min":0.21299999999999999,"15min":0.183},"max":{"1min":1.274,"5min":13.925000000000001,"15min":28.263999999999999},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":2.2639999999999998,"15min":1.8640000000000001},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.26700000000000002},"max":{"1min":0.84599999999999997,"5min":13.696999999999999,"15min":13.696999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.788,"15min":1.7210000000000001},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0649999999999999,"5min":12.576000000000001,"15min":13.478999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Feb 22 19:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":1.5029999999999999,"15min":1.9039999999999999},"min":{"1min":0.40799999999999997,"5min":0.29399999999999998,"15min":0.28000000000000003},"max":{"1min":0.86499999999999999,"5min":10.036,"15min":40.633000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.628,"5min":1.8620000000000001,"15min":2.1219999999999999},"min":{"1min":0.49299999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92600000000000005,"5min":12.747,"15min":40.997999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Feb 22 19:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":2.113,"15min":1.885},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92000000000000004,"5min":13.766,"15min":21.640000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":1.7350000000000001,"15min":1.8049999999999999},"min":{"1min":0.45700000000000002,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.95299999999999996,"5min":10.103,"15min":13.51},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Feb 22 19:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":2.456,"15min":2.1339999999999999},"min":{"1min":0.434,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95099999999999996,"5min":13.789999999999999,"15min":14.199},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.9199999999999999,"15min":1.9239999999999999},"min":{"1min":0.437,"5min":0.36099999999999999,"15min":0.312},"max":{"1min":1.0089999999999999,"5min":12.608000000000001,"15min":18.021999999999998},"last":0.58499999999999996}]}]},{"osd":3,"up_from":26,"seq":111669150028,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1186316,"kb_used_data":6276,"kb_used_omap":0,"kb_used_meta":1180032,"kb_avail":92558836,"statfs":{"total":95995035648,"available":94780248064,"internally_reserved":0,"allocated":6426624,"data_stored":3633079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1208352768},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":1.3939999999999999,"15min":1.6830000000000001},"min":{"1min":0.26200000000000001,"5min":0.20399999999999999,"15min":0.187},"max":{"1min":0.57699999999999996,"5min":17.056000000000001,"15min":17.056000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":1.6200000000000001,"15min":1.9199999999999999},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.219},"max":{"1min":0.58999999999999997,"5min":13.451000000000001,"15min":18.172000000000001},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":1.649,"15min":2.7970000000000002},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.20999999999999999},"max":{"1min":0.51800000000000002,"5min":13.576000000000001,"15min":94.454999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":1.649,"15min":2.5680000000000001},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.67900000000000005,"5min":10.887,"15min":95.197999999999993},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":1.5069999999999999,"15min":2.0819999999999999},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.224},"max":{"1min":0.626,"5min":15.541,"15min":20.105},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.438,"5min":2.0539999999999998,"15min":2.2989999999999999},"min":{"1min":0.29799999999999999,"5min":0.22800000000000001,"15min":0.20300000000000001},"max":{"1min":0.60699999999999998,"5min":20.681000000000001,"15min":35.939999999999998},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Feb 22 19:54:29 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":1.7749999999999999,"15min":1.7450000000000001},"min":{"1min":0.41499999999999998,"5min":0.29899999999999999,"15min":0.27100000000000002},"max":{"1min":0.93899999999999995,"5min":13.417999999999999,"15min":13.417999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":1.7549999999999999,"15min":1.6160000000000001},"min":{"1min":0.378,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89700000000000002,"5min":12.423,"15min":12.423},"last":0.627}]},{"osd":5,"last update":"Thu Feb 22 19:53:52 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":1.9119999999999999,"15min":1.8109999999999999},"min":{"1min":0.432,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":0.85699999999999998,"5min":30.227,"15min":30.227},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":2.2589999999999999,"15min":1.9610000000000001},"min":{"1min":0.42999999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.99399999999999999,"5min":30.021000000000001,"15min":30.021000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":1.6639999999999999,"15min":1.7130000000000001},"min":{"1min":0.41999999999999998,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.92800000000000005,"5min":12.632999999999999,"15min":12.632999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":2.0329999999999999,"15min":1.9970000000000001},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.26600000000000001},"max":{"1min":0.86299999999999999,"5min":17.544,"15min":17.544},"last":0.502}]},{"osd":7,"last update":"Thu Feb 22 19:54:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":1.3779999999999999,"15min":1.581},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.312},"max":{"1min":0.96499999999999997,"5min":8.5540000000000003,"15min":12.327},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":1.7230000000000001,"15min":1.8169999999999999},"min":{"1min":0.433,"5min":0.35399999999999998,"15min":0.29599999999999999},"max":{"1min":1.2130000000000001,"5min":12.539999999999999,"15min":12.539999999999999},"last":0.64000000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953802,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315780,"kb_used_data":6268,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429372,"statfs":{"total":95995035648,"available":95671676928,"internally_reserved":0,"allocated":6418432,"data_stored":3632749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.446,"15min":1.2090000000000001},"min":{"1min":0.34799999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.88900000000000001,"5min":14.797000000000001,"15min":14.797000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":1.4630000000000001,"15min":1.288},"min":{"1min":0.35899999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71399999999999997,"5min":8.4860000000000007,"15min":9.4900000000000002},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":1.4810000000000001,"15min":1.4370000000000001},"min":{"1min":0.38,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.74099999999999999,"5min":8.5129999999999999,"15min":11.037000000000001},"last":0.443},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.429,"15min":1.2529999999999999},"min":{"1min":0.40000000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.80800000000000005,"5min":9.4890000000000008,"15min":9.4890000000000008},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":1.875,"15min":1.5960000000000001},"min":{"1min":0.47299999999999998,"5min":0.28000000000000003,"15min":0.22900000000000001},"max":{"1min":0.84699999999999998,"5min":15.470000000000001,"15min":15.470000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.4890000000000001,"15min":1.369},"min":{"1min":0.40699999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77400000000000002,"5min":7.9299999999999997,"15min":7.9299999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Feb 22 19:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.9179999999999999,"15min":1.921},"min":{"1min":0.42099999999999999,"5min":0.30399999999999999,"15min":0.25900000000000001},"max":{"1min":0.86299999999999999,"5min":13.635,"15min":94.703999999999994},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":1.9470000000000001,"15min":2.0030000000000001},"min":{"1min":0.377,"5min":0.31,"15min":0.25900000000000001},"max":{"1min":0.75700000000000001,"5min":10.691000000000001,"15min":95.034000000000006},"last":0.495}]},{"osd":5,"last update":"Thu Feb 22 19:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":1.1839999999999999,"15min":0.92600000000000005},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.218},"max":{"1min":0.85799999999999998,"5min":12.099,"15min":12.099},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":1.276,"15min":1.0269999999999999},"min":{"1min":0.313,"5min":0.25,"15min":0.249},"max":{"1min":0.86299999999999999,"5min":8.0039999999999996,"15min":13.211},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Feb 22 19:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.0049999999999999,"15min":0.84099999999999997},"min":{"1min":0.41299999999999998,"5min":0.29899999999999999,"15min":0.28000000000000003},"max":{"1min":0.82299999999999995,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":1.621,"15min":1.0449999999999999},"min":{"1min":0.378,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.83899999999999997,"5min":11.802,"15min":11.802},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Feb 22 19:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":1.4119999999999999,"15min":1.0760000000000001},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.28799999999999998},"max":{"1min":0.80000000000000004,"5min":9.548,"15min":11.111000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.4239999999999999,"15min":1.0069999999999999},"min":{"1min":0.36799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88300000000000001,"5min":10.869,"15min":10.869},"last":0.625}]}]},{"osd":5,"up_from":39,"seq":167503724868,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":909420,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":903168,"kb_avail":92835732,"statfs":{"total":95995035648,"available":95063789568,"internally_reserved":0,"allocated":6393856,"data_stored":3631766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":924844032},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":1.3260000000000001,"15min":1.2490000000000001},"min":{"1min":0.34999999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.82399999999999995,"5min":12.102,"15min":17.684000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":1.718,"15min":1.52},"min":{"1min":0.36399999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":10.977,"15min":17.402999999999999},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.587,"15min":1.5660000000000001},"min":{"1min":0.41199999999999998,"5min":0.23400000000000001,"15min":0.19},"max":{"1min":0.80700000000000005,"5min":12.154,"15min":37.009999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.651,"15min":1.8169999999999999},"min":{"1min":0.36299999999999999,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":10.372,"15min":39.131999999999998},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":1.9350000000000001,"15min":2.0289999999999999},"min":{"1min":0.36699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.79600000000000004,"5min":15.683999999999999,"15min":47.859000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":2.0099999999999998,"15min":1.9159999999999999},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.23300000000000001},"max":{"1min":0.84599999999999997,"5min":16.390000000000001,"15min":45.183},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":1.7629999999999999,"15min":1.9159999999999999},"min":{"1min":0.35799999999999998,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.89400000000000002,"5min":11.811999999999999,"15min":43.616999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":1.9990000000000001,"15min":1.9910000000000001},"min":{"1min":0.36899999999999999,"5min":0.28499999999999998,"15min":0.26300000000000001},"max":{"1min":0.84699999999999998,"5min":12.82,"15min":47.814},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Feb 22 19:53:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":1.3160000000000001,"15min":1.0209999999999999},"min":{"1min":0.33000000000000002,"5min":0.27100000000000002,"15min":0.223},"max":{"1min":0.91700000000000004,"5min":10.347,"15min":10.347},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":1.381,"15min":1.1459999999999999},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.81699999999999995,"5min":11.696,"15min":11.696},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Feb 22 19:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.232,"15min":0.89000000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.247},"max":{"1min":0.80200000000000005,"5min":12.412000000000001,"15min":12.412000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.462,"15min":1.149},"min":{"1min":0.374,"5min":0.314,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":11.913,"15min":11.913},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Feb 22 19:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":1.377,"15min":1.046},"min":{"1min":0.38,"5min":0.309,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":12.380000000000001,"15min":12.380000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.3460000000000001,"15min":1.228},"min":{"1min":0.36099999999999999,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.75900000000000001,"5min":9.4280000000000008,"15min":9.4280000000000008},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T19:54:55.123 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-22T19:54:55.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid b6783888-d1b7-11ee-95bf-87774f69a715 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-02-22T19:54:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/mon.a/config 2024-02-22T19:54:55.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:54.019676+0000 mgr.y (mgr.14535) 976 : cluster [DBG] pgmap v1373: 112 pgs: 112 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:54:55.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:54.049536+0000 osd.4 (osd.4) 273 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:54:55.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:54.052448+0000 osd.4 (osd.4) 274 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:54:55.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:54.446634+0000 mgr.y (mgr.14535) 977 : audit [DBG] from='client.26114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:54:55.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:54.491684+0000 osd.3 (osd.3) 459 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:54:55.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:54.495157+0000 osd.3 (osd.3) 460 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:54:55.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:54.019676+0000 mgr.y (mgr.14535) 976 : cluster [DBG] pgmap v1373: 112 pgs: 112 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:54:55.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:54.049536+0000 osd.4 (osd.4) 273 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:54:55.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:54.052448+0000 osd.4 (osd.4) 274 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:54:55.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:54.446634+0000 mgr.y (mgr.14535) 977 : audit [DBG] from='client.26114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:54:55.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:54.491684+0000 osd.3 (osd.3) 459 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:54:55.694 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:55 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:54.495157+0000 osd.3 (osd.3) 460 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:54:55.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:54.019676+0000 mgr.y (mgr.14535) 976 : cluster [DBG] pgmap v1373: 112 pgs: 112 active+clean; 5.7 KiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-22T19:54:55.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:54.049536+0000 osd.4 (osd.4) 273 : cluster [DBG] 3.9 scrub starts 2024-02-22T19:54:55.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:54.052448+0000 osd.4 (osd.4) 274 : cluster [DBG] 3.9 scrub ok 2024-02-22T19:54:55.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:54.446634+0000 mgr.y (mgr.14535) 977 : audit [DBG] from='client.26114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T19:54:55.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:54.491684+0000 osd.3 (osd.3) 459 : cluster [DBG] 4.19 scrub starts 2024-02-22T19:54:55.831 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:54.495157+0000 osd.3 (osd.3) 460 : cluster [DBG] 4.19 scrub ok 2024-02-22T19:54:56.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:54:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug there is no tcmu-runner data available 2024-02-22T19:54:56.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:55.088518+0000 osd.4 (osd.4) 275 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:54:56.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:55.112062+0000 osd.4 (osd.4) 276 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:54:56.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:55.677086+0000 osd.7 (osd.7) 411 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:54:56.504 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:55.680042+0000 osd.7 (osd.7) 412 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:54:56.504 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:55.088518+0000 osd.4 (osd.4) 275 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:54:56.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:55.112062+0000 osd.4 (osd.4) 276 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:54:56.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:55.677086+0000 osd.7 (osd.7) 411 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:54:56.505 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:56 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:55.680042+0000 osd.7 (osd.7) 412 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:54:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:55.088518+0000 osd.4 (osd.4) 275 : cluster [DBG] 6.2 scrub starts 2024-02-22T19:54:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:55.112062+0000 osd.4 (osd.4) 276 : cluster [DBG] 6.2 scrub ok 2024-02-22T19:54:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:55.677086+0000 osd.7 (osd.7) 411 : cluster [DBG] 6.8 scrub starts 2024-02-22T19:54:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:55.680042+0000 osd.7 (osd.7) 412 : cluster [DBG] 6.8 scrub ok 2024-02-22T19:54:57.742 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:55.831168+0000 mgr.y (mgr.14535) 978 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:57.742 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:56.020341+0000 mgr.y (mgr.14535) 979 : cluster [DBG] pgmap v1374: 112 pgs: 112 active+clean; 5.7 KiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:54:57.742 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:56.088282+0000 osd.4 (osd.4) 277 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:56.091825+0000 osd.4 (osd.4) 278 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.338094+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.53:0/3321633825' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.338411+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.343955+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.389600+0000 mon.a (mon.0) 2232 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:55.831168+0000 mgr.y (mgr.14535) 978 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:56.020341+0000 mgr.y (mgr.14535) 979 : cluster [DBG] pgmap v1374: 112 pgs: 112 active+clean; 5.7 KiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:56.088282+0000 osd.4 (osd.4) 277 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:56.091825+0000 osd.4 (osd.4) 278 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.338094+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.53:0/3321633825' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-22T19:54:57.743 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.338411+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.343955+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.389600+0000 mon.a (mon.0) 2232 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:57] ENGINE Bus STOPPING 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:57] ENGINE Bus STOPPED 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:57] ENGINE Bus STARTING 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:57] ENGINE Serving on http://:::9283 2024-02-22T19:54:57.744 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:57 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:57] ENGINE Bus STARTED 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:55.831168+0000 mgr.y (mgr.14535) 978 : audit [DBG] from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:56.020341+0000 mgr.y (mgr.14535) 979 : cluster [DBG] pgmap v1374: 112 pgs: 112 active+clean; 5.7 KiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:56.088282+0000 osd.4 (osd.4) 277 : cluster [DBG] 4.1a scrub starts 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:56.091825+0000 osd.4 (osd.4) 278 : cluster [DBG] 4.1a scrub ok 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.338094+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.53:0/3321633825' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.338411+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.343955+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-22T19:54:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.389600+0000 mon.a (mon.0) 2232 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T19:54:58.094 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-22T19:54:58.103 INFO:tasks.cephadm:Teardown begin 2024-02-22T19:54:58.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:54:58.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:54:58.188 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-22T19:54:58.188 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-22T19:54:58.206 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-22T19:54:58.250 INFO:tasks.cephadm:Stopping all daemons... 2024-02-22T19:54:58.250 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-22T19:54:58.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 systemd[1]: Stopping Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 bash[231664]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon.a" found: no such container 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:56.706109+0000 osd.0 (osd.0) 255 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:56.708661+0000 osd.0 (osd.0) 256 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:57.070578+0000 osd.4 (osd.4) 279 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:57.074172+0000 osd.4 (osd.4) 280 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:54:58.427 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.640682+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:58.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.641658+0000 mon.a (mon.0) 2234 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:54:58.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.642422+0000 mon.a (mon.0) 2235 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:54:58.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: audit 2024-02-22T19:54:57.647863+0000 mon.a (mon.0) 2236 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:58.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:57.693133+0000 osd.7 (osd.7) 413 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:54:58.428 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: cluster 2024-02-22T19:54:57.708138+0000 osd.7 (osd.7) 414 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:54:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:56.706109+0000 osd.0 (osd.0) 255 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:54:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:56.708661+0000 osd.0 (osd.0) 256 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:54:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:57.070578+0000 osd.4 (osd.4) 279 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:54:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:57.074172+0000 osd.4 (osd.4) 280 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:54:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.640682+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:58.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.641658+0000 mon.a (mon.0) 2234 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:54:58.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.642422+0000 mon.a (mon.0) 2235 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:54:58.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: audit 2024-02-22T19:54:57.647863+0000 mon.a (mon.0) 2236 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:58.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:57.693133+0000 osd.7 (osd.7) 413 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:54:58.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:54:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: cluster 2024-02-22T19:54:57.708138+0000 osd.7 (osd.7) 414 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:54:58.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:54:58.560+0000 7fe2de15b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-22T19:54:58.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a[67298]: debug 2024-02-22T19:54:58.560+0000 7fe2de15b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-22T19:54:58.680 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:58] ENGINE Bus STOPPING 2024-02-22T19:54:58.680 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:54:58.680 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:58] ENGINE Bus STOPPED 2024-02-22T19:54:58.680 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:58] ENGINE Bus STARTING 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:56.706109+0000 osd.0 (osd.0) 255 : cluster [DBG] 4.b deep-scrub starts 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:56.708661+0000 osd.0 (osd.0) 256 : cluster [DBG] 4.b deep-scrub ok 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:57.070578+0000 osd.4 (osd.4) 279 : cluster [DBG] 4.13 scrub starts 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:57.074172+0000 osd.4 (osd.4) 280 : cluster [DBG] 4.13 scrub ok 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.640682+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.641658+0000 mon.a (mon.0) 2234 : audit [DBG] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.642422+0000 mon.a (mon.0) 2235 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: audit 2024-02-22T19:54:57.647863+0000 mon.a (mon.0) 2236 : audit [INF] from='mgr.14535 172.21.15.53:0/196057159' entity='mgr.y' 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:57.693133+0000 osd.7 (osd.7) 413 : cluster [DBG] 6.12 scrub starts 2024-02-22T19:54:58.681 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: cluster 2024-02-22T19:54:57.708138+0000 osd.7 (osd.7) 414 : cluster [DBG] 6.12 scrub ok 2024-02-22T19:54:58.977 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:58] ENGINE Serving on http://:::9283 2024-02-22T19:54:58.977 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:58 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:58] ENGINE Bus STARTED 2024-02-22T19:54:59.254 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:59 smithi053 bash[231706]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-a 2024-02-22T19:54:59.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service' 2024-02-22T19:54:59.573 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: ::ffff:172.21.15.64 - - [22/Feb/2024:19:54:59] "GET /metrics HTTP/1.1" 200 174325 "" "Prometheus/2.33.4" 2024-02-22T19:54:59.573 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:59 smithi053 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.a.service: Succeeded. 2024-02-22T19:54:59.573 INFO:journalctl@ceph.mon.a.smithi053.stdout:Feb 22 19:54:59 smithi053 systemd[1]: Stopped Ceph mon.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:54:59.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:54:59.619 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-22T19:54:59.619 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-22T19:54:59.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.c 2024-02-22T19:54:59.985 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:59 smithi053 systemd[1]: Stopping Ceph mon.c for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:54:59.985 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:59 smithi053 bash[231835]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon.c" found: no such container 2024-02-22T19:55:00.254 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:54:59.984+0000 7ff9ebd48700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:54:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c[73839]: debug 2024-02-22T19:54:59.984+0000 7ff9ebd48700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:54:59 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:54:59] ENGINE Bus STOPPING 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:55:00] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:55:00] ENGINE Bus STOPPED 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:55:00] ENGINE Bus STARTING 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:55:00] ENGINE Serving on http://:::9283 2024-02-22T19:55:00.255 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:00 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: [22/Feb/2024:19:55:00] ENGINE Bus STARTED 2024-02-22T19:55:00.754 INFO:journalctl@ceph.mon.c.smithi053.stdout:Feb 22 19:55:00 smithi053 bash[231875]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-c 2024-02-22T19:55:00.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.c.service' 2024-02-22T19:55:01.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:01.005 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-22T19:55:01.005 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-22T19:55:01.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.b 2024-02-22T19:55:01.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:55:01 smithi064 systemd[1]: Stopping Ceph mon.b for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:01.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:55:01 smithi064 bash[126668]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon.b" found: no such container 2024-02-22T19:55:01.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:55:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: debug 2024-02-22T19:55:01.322+0000 7f6cd582f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-22T19:55:01.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:55:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b[39831]: debug 2024-02-22T19:55:01.322+0000 7f6cd582f700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-22T19:55:02.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Feb 22 19:55:01 smithi064 bash[126708]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mon-b 2024-02-22T19:55:02.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mon.b.service' 2024-02-22T19:55:02.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:02.199 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-22T19:55:02.200 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-22T19:55:02.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.y 2024-02-22T19:55:02.503 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:02 smithi053 systemd[1]: Stopping Ceph mgr.y for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:02.503 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:02 smithi053 bash[232007]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr.y" found: no such container 2024-02-22T19:55:02.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:55:02.502+0000 7f53578cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:02.754 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:02 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y[67572]: debug 2024-02-22T19:55:02.502+0000 7f53578cd700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-22T19:55:03.254 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Feb 22 19:55:03 smithi053 bash[232047]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-y 2024-02-22T19:55:03.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.y.service' 2024-02-22T19:55:03.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:03.445 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-22T19:55:03.445 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-22T19:55:03.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.x 2024-02-22T19:55:03.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:55:03 smithi064 systemd[1]: Stopping Ceph mgr.x for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:03.852 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:55:03 smithi064 bash[126831]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr.x" found: no such container 2024-02-22T19:55:04.281 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Feb 22 19:55:04 smithi064 bash[126871]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-mgr-x 2024-02-22T19:55:04.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@mgr.x.service' 2024-02-22T19:55:04.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:04.485 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-22T19:55:04.485 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-22T19:55:04.485 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.0 2024-02-22T19:55:04.754 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:55:04 smithi053 systemd[1]: Stopping Ceph osd.0 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:04.754 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:55:04 smithi053 bash[232166]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.0" found: no such container 2024-02-22T19:55:04.755 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:04.478Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:04.755 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:04.478Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:05.254 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:55:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-0[77321]: debug 2024-02-22T19:55:04.791+0000 7fc8449ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:05.254 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:55:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-0[77321]: debug 2024-02-22T19:55:04.791+0000 7fc8449ec700 -1 osd.0 670 *** Got signal Terminated *** 2024-02-22T19:55:05.254 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:55:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-0[77321]: debug 2024-02-22T19:55:04.791+0000 7fc8449ec700 -1 osd.0 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:10.504 INFO:journalctl@ceph.osd.0.smithi053.stdout:Feb 22 19:55:10 smithi053 bash[232206]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-0 2024-02-22T19:55:12.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.0.service' 2024-02-22T19:55:12.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:12.400 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-22T19:55:12.400 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-22T19:55:12.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.1 2024-02-22T19:55:12.716 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:55:12 smithi053 systemd[1]: Stopping Ceph osd.1 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:12.716 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:55:12 smithi053 bash[232442]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.1" found: no such container 2024-02-22T19:55:13.004 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:55:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-1[81742]: debug 2024-02-22T19:55:12.715+0000 7f2aa3ab1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:13.004 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:55:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-1[81742]: debug 2024-02-22T19:55:12.715+0000 7f2aa3ab1700 -1 osd.1 670 *** Got signal Terminated *** 2024-02-22T19:55:13.004 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:55:12 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-1[81742]: debug 2024-02-22T19:55:12.715+0000 7f2aa3ab1700 -1 osd.1 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:14.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:55:14.478Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:14.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:14.479Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:14.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:14.479Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:18.505 INFO:journalctl@ceph.osd.1.smithi053.stdout:Feb 22 19:55:18 smithi053 bash[232482]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-1 2024-02-22T19:55:20.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.1.service' 2024-02-22T19:55:20.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:20.236 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-22T19:55:20.236 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-22T19:55:20.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.2 2024-02-22T19:55:20.504 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:55:20 smithi053 systemd[1]: Stopping Ceph osd.2 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:20.504 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:55:20 smithi053 bash[232726]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.2" found: no such container 2024-02-22T19:55:21.004 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:55:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-2[86396]: debug 2024-02-22T19:55:20.525+0000 7f352b243700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:21.005 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:55:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-2[86396]: debug 2024-02-22T19:55:20.525+0000 7f352b243700 -1 osd.2 670 *** Got signal Terminated *** 2024-02-22T19:55:21.005 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:55:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-2[86396]: debug 2024-02-22T19:55:20.525+0000 7f352b243700 -1 osd.2 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:24.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:55:24.479Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:24.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:24.480Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:24.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:24 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:24.480Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:26.254 INFO:journalctl@ceph.osd.2.smithi053.stdout:Feb 22 19:55:26 smithi053 bash[232766]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-2 2024-02-22T19:55:27.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.2.service' 2024-02-22T19:55:27.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:27.920 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-22T19:55:27.920 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-22T19:55:27.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.3 2024-02-22T19:55:28.254 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:27 smithi053 systemd[1]: Stopping Ceph osd.3 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:28.254 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:28 smithi053 bash[233003]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.3" found: no such container 2024-02-22T19:55:28.754 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:55:28.271+0000 7f09a043d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:28.754 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:55:28.271+0000 7f09a043d700 -1 osd.3 670 *** Got signal Terminated *** 2024-02-22T19:55:28.754 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:28 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:55:28.271+0000 7f09a043d700 -1 osd.3 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:31.353 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:30 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:30.967+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:32.004 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:31 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:55:31.568+0000 7f09993b2700 -1 osd.3 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.068889+0000 front 2024-02-22T19:55:07.068843+0000 (oldest deadline 2024-02-22T19:55:31.168996+0000) 2024-02-22T19:55:32.352 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:32.010+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:32.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:31 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:31.985+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:32.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:32.550+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:33.004 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:32 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3[90853]: debug 2024-02-22T19:55:32.579+0000 7f09993b2700 -1 osd.3 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.068889+0000 front 2024-02-22T19:55:07.068843+0000 (oldest deadline 2024-02-22T19:55:31.168996+0000) 2024-02-22T19:55:33.352 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:32 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:32.995+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:33.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:33.004+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:33.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:33 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:33.512+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:34.004 INFO:journalctl@ceph.osd.3.smithi053.stdout:Feb 22 19:55:33 smithi053 bash[233044]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-3 2024-02-22T19:55:34.352 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:34.015+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:34.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:34.011+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:34.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:34.229+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:34.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:55:34.480Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:34.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:34.480Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:34.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:34 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:34.480Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:34.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:34.522+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:35.257 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:34 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:34.969+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:35.257 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:35.054+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:35.538 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:35.256+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:35.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.3.service' 2024-02-22T19:55:35.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:35.691 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-22T19:55:35.691 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-22T19:55:35.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.4 2024-02-22T19:55:35.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:35.538+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:35.852 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:35 smithi064 systemd[1]: Stopping Ceph osd.4 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:35.852 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:35 smithi064 bash[126995]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.4" found: no such container 2024-02-22T19:55:36.265 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:35 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:35.924+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:36.265 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:36.050+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:36.266 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:36.101+0000 7f8bbb0ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:36.266 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:36.101+0000 7f8bbb0ae700 -1 osd.4 670 *** Got signal Terminated *** 2024-02-22T19:55:36.266 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:36.101+0000 7f8bbb0ae700 -1 osd.4 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:36.555 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:36.264+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:36.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:36.550+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:37.262 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:37.054+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:37.262 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:36 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:36.954+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:37.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:37.506+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:37.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:37.260+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:38.303 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:38.039+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:38.303 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:37.997+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:38.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:38.463+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:38.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:38.303+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:39.303 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:38.987+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:39.303 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:38 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:38.987+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:39.303 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:39.043+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:39.303 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:39.043+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.748403+0000 front 2024-02-22T19:55:16.748315+0000 (oldest deadline 2024-02-22T19:55:38.448211+0000) 2024-02-22T19:55:39.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:39.486+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:39.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:39.302+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:39.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:39.302+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:40.271 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:40.054+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:40.271 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:40.054+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.748403+0000 front 2024-02-22T19:55:16.748315+0000 (oldest deadline 2024-02-22T19:55:38.448211+0000) 2024-02-22T19:55:40.271 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:39.947+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:40.272 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:39 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:39.947+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:40.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:40.495+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:40.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:40.270+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:40.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:40.270+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:41.263 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:41.085+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.346122+0000 front 2024-02-22T19:55:06.346031+0000 (oldest deadline 2024-02-22T19:55:30.446123+0000) 2024-02-22T19:55:41.264 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4[44070]: debug 2024-02-22T19:55:41.085+0000 7f8bb5025700 -1 osd.4 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.748403+0000 front 2024-02-22T19:55:16.748315+0000 (oldest deadline 2024-02-22T19:55:38.448211+0000) 2024-02-22T19:55:41.264 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:40.986+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:41.264 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:40 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:40.986+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:41.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:41.499+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:41.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:41.262+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:41.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:41.262+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:41.998 INFO:journalctl@ceph.osd.4.smithi064.stdout:Feb 22 19:55:41 smithi064 bash[127033]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-4 2024-02-22T19:55:42.270 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:41.998+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:42.270 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:41 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:41.998+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:42.521 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:42.270+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:42.521 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:42.270+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:42.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:42 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:42.486+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:43.353 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:43.028+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:43.353 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:43.028+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:43.353 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:43.272+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:43.353 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:43.272+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:43.682 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:43.523+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:43.682 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:43 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:43.523+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:43.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.4.service' 2024-02-22T19:55:43.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:43.738 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-22T19:55:43.739 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-22T19:55:43.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.5 2024-02-22T19:55:44.007 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:43 smithi064 systemd[1]: Stopping Ceph osd.5 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:44.007 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:43 smithi064 bash[127270]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.5" found: no such container 2024-02-22T19:55:44.007 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.006+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:44.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:44.247+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:44.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:44.247+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:44.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.006+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:44.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.030+0000 7ff71f16b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:44.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.030+0000 7ff71f16b700 -1 osd.5 670 *** Got signal Terminated *** 2024-02-22T19:55:44.352 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.030+0000 7ff71f16b700 -1 osd.5 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:44.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:55:44.481Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:44.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:44.482Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:44.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:44 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:44.482Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:44.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:44.532+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:44.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:44.532+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:45.294 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.994+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:45.295 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:44 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:44.994+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:45.558 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:45.294+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:45.558 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:45.294+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:45.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:45.557+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:45.852 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:45 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:45.557+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:46.285 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:46.007+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:46.285 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:46.007+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:46.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:46.284+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:46.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:46.284+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:46.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:46.512+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:46.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:46.512+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:47.275 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:46.967+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:47.275 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:46 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:46.967+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:47.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:47.275+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:47.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:47.275+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:47.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:47.275+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:47.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:47.508+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:47.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:47.508+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:48.294 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:47.960+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:48.294 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:47 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:47.960+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:48.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:48.293+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:48.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:48.293+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:48.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:48.293+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:48.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:48.497+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:48.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:48.497+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:48.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:48.497+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:49.254 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:48.949+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:07.088497+0000 front 2024-02-22T19:55:07.088518+0000 (oldest deadline 2024-02-22T19:55:31.188449+0000) 2024-02-22T19:55:49.254 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:48 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5[48459]: debug 2024-02-22T19:55:48.949+0000 7ff71a0e4700 -1 osd.5 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:16.489623+0000 front 2024-02-22T19:55:16.489609+0000 (oldest deadline 2024-02-22T19:55:38.789556+0000) 2024-02-22T19:55:49.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:49.253+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:49.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:49.253+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:49.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:49.253+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:49.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:49.471+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:49.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:49.472+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:49.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:49 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:49.472+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:50.102 INFO:journalctl@ceph.osd.5.smithi064.stdout:Feb 22 19:55:49 smithi064 bash[127312]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-5 2024-02-22T19:55:50.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:50.458+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:50.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:50.458+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:50.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:50.458+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:50.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:50.218+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:50.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:50.218+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:50.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:50 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:50.218+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:51.301 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:51.214+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:51.301 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:51.214+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:51.302 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:51.214+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:51.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.5.service' 2024-02-22T19:55:51.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:51.354 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-22T19:55:51.354 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-22T19:55:51.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.6 2024-02-22T19:55:51.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 systemd[1]: Stopping Ceph osd.6 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:51.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:51.432+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:51.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:51.432+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:51.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:51.432+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:51.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 bash[127546]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.6" found: no such container 2024-02-22T19:55:52.102 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:51.709+0000 7f6097878700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:52.102 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:51.709+0000 7f6097878700 -1 osd.6 670 *** Got signal Terminated *** 2024-02-22T19:55:52.102 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:51 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:51.709+0000 7f6097878700 -1 osd.6 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:55:52.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:52.399+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:52.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:52.399+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:52.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:52.399+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:52.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:52.239+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:52.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:52.239+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:52.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:52 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:52.239+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:53.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:53.411+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:53.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:53.411+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:53.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:53.411+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:53.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:53.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:53.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:53.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:53.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:53 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:53.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:54.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:54.448+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:54.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:54.448+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:54.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:54.448+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:54.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:54.448+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:27.619940+0000 front 2024-02-22T19:55:27.619948+0000 (oldest deadline 2024-02-22T19:55:53.519896+0000) 2024-02-22T19:55:54.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:54.248+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:54.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:54.248+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:54.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:54 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:54.248+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:54.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:55:54.482Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:54.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:54.482Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:55:54.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:55:54 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:55:54.483Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:55:55.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:55.445+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:55.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:55.445+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:55.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:55.445+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:55.603 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:55.445+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:27.619940+0000 front 2024-02-22T19:55:27.619948+0000 (oldest deadline 2024-02-22T19:55:53.519896+0000) 2024-02-22T19:55:55.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:55.276+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:55.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:55.276+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:55.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:55 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:55.276+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:56.254+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:56.254+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:56.254+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:56.479+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:06.617456+0000 front 2024-02-22T19:55:06.617519+0000 (oldest deadline 2024-02-22T19:55:32.517234+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:56.479+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:17.118579+0000 front 2024-02-22T19:55:17.118444+0000 (oldest deadline 2024-02-22T19:55:43.018510+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:56.479+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:24.119424+0000 front 2024-02-22T19:55:24.119405+0000 (oldest deadline 2024-02-22T19:55:47.619352+0000) 2024-02-22T19:55:56.602 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:56 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6[53084]: debug 2024-02-22T19:55:56.479+0000 7f60947f5700 -1 osd.6 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:27.619940+0000 front 2024-02-22T19:55:27.619948+0000 (oldest deadline 2024-02-22T19:55:53.519896+0000) 2024-02-22T19:55:57.352 INFO:journalctl@ceph.osd.6.smithi064.stdout:Feb 22 19:55:57 smithi064 bash[127587]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-6 2024-02-22T19:55:57.353 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:57.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:57.353 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:57.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:57.353 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:57.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:57.353 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:57 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:57.223+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:55:58.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:58.229+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:58.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:58.229+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:58.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:58.229+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:58.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:58 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:58.229+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:55:59.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.6.service' 2024-02-22T19:55:59.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:55:59.049 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-22T19:55:59.049 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-22T19:55:59.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.7 2024-02-22T19:55:59.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 systemd[1]: Stopping Ceph osd.7 for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:55:59.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 bash[127819]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd.7" found: no such container 2024-02-22T19:55:59.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.204+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:55:59.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.204+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:55:59.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.204+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:55:59.352 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.204+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:55:59.852 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.397+0000 7fd6d6d34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:55:59.852 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.397+0000 7fd6d6d34700 -1 osd.7 670 *** Got signal Terminated *** 2024-02-22T19:55:59.852 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:55:59 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:55:59.397+0000 7fd6d6d34700 -1 osd.7 670 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T19:56:00.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:00.203+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:56:00.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:00.203+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:56:00.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:00.203+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:56:00.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:00 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:00.203+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:56:01.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:01.186+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:56:01.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:01.186+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:56:01.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:01.186+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:56:01.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:01 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:01.186+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:56:02.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:02.208+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:56:02.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:02.208+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:56:02.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:02.208+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:56:02.603 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:02 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:02.208+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:56:03.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:03.230+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:56:03.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:03.230+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:56:03.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:03.230+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:56:03.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:03.230+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:56:03.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:03 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:03.230+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.64:6802 osd.4 since back 2024-02-22T19:55:36.664208+0000 front 2024-02-22T19:55:36.664235+0000 (oldest deadline 2024-02-22T19:56:02.564180+0000) 2024-02-22T19:56:04.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:04.184+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6803 osd.0 since back 2024-02-22T19:55:09.161021+0000 front 2024-02-22T19:55:09.160991+0000 (oldest deadline 2024-02-22T19:55:33.260912+0000) 2024-02-22T19:56:04.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:04.184+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6807 osd.1 since back 2024-02-22T19:55:15.562017+0000 front 2024-02-22T19:55:15.561961+0000 (oldest deadline 2024-02-22T19:55:38.461896+0000) 2024-02-22T19:56:04.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:04.184+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6811 osd.2 since back 2024-02-22T19:55:21.363068+0000 front 2024-02-22T19:55:21.363001+0000 (oldest deadline 2024-02-22T19:55:47.262942+0000) 2024-02-22T19:56:04.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:04.184+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.53:6815 osd.3 since back 2024-02-22T19:55:31.964088+0000 front 2024-02-22T19:55:31.964074+0000 (oldest deadline 2024-02-22T19:55:56.663891+0000) 2024-02-22T19:56:04.602 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:04 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7[57705]: debug 2024-02-22T19:56:04.184+0000 7fd6d44b2700 -1 osd.7 670 heartbeat_check: no reply from 172.21.15.64:6802 osd.4 since back 2024-02-22T19:55:36.664208+0000 front 2024-02-22T19:55:36.664235+0000 (oldest deadline 2024-02-22T19:56:02.564180+0000) 2024-02-22T19:56:04.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:56:04.482Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:56:04.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:56:04.483Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:56:04.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:04 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:56:04.483Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:56:05.102 INFO:journalctl@ceph.osd.7.smithi064.stdout:Feb 22 19:56:05 smithi064 bash[127859]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-osd-7 2024-02-22T19:56:06.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@osd.7.service' 2024-02-22T19:56:06.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:56:06.955 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-22T19:56:06.955 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-22T19:56:06.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@rgw.foo.a 2024-02-22T19:56:07.254 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:56:06 smithi053 systemd[1]: Stopping Ceph rgw.foo.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:07.254 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:56:07 smithi053 bash[233285]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw.foo.a" found: no such container 2024-02-22T19:56:07.754 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:56:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a[93504]: debug 2024-02-22T19:56:07.264+0000 7fb450bf1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-22T19:56:07.754 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:56:07 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a[93504]: debug 2024-02-22T19:56:07.264+0000 7fb467cbb880 -1 shutting down 2024-02-22T19:56:14.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=error ts=2024-02-22T19:56:14.483Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:56:14.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:56:14.484Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi053.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.53:8443: connect: connection refused" 2024-02-22T19:56:14.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:14 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=warn ts=2024-02-22T19:56:14.484Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi064.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.64:8443: connect: connection refused" 2024-02-22T19:56:17.539 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:56:17 smithi053 bash[233323]: time="2024-02-22T19:56:17Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-22T19:56:17.976 INFO:journalctl@ceph.rgw.foo.a.smithi053.stdout:Feb 22 19:56:17 smithi053 bash[233323]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-rgw-foo-a 2024-02-22T19:56:18.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@rgw.foo.a.service' 2024-02-22T19:56:18.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:56:18.183 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-22T19:56:18.184 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-22T19:56:18.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b6783888-d1b7-11ee-95bf-87774f69a715@prometheus.a 2024-02-22T19:56:18.486 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 systemd[1]: Stopping Ceph prometheus.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:18.486 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 bash[128101]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus.a" found: no such container 2024-02-22T19:56:18.739 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-22T19:56:18.739 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-22T19:56:18.740 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a[63520]: ts=2024-02-22T19:56:18.486Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-22T19:56:19.102 INFO:journalctl@ceph.prometheus.a.smithi064.stdout:Feb 22 19:56:18 smithi064 bash[128141]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-prometheus-a 2024-02-22T19:56:19.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6783888-d1b7-11ee-95bf-87774f69a715@prometheus.a.service' 2024-02-22T19:56:19.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T19:56:19.281 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-22T19:56:19.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6783888-d1b7-11ee-95bf-87774f69a715 --force --keep-logs 2024-02-22T19:56:20.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:20 smithi053 systemd[1]: Stopping Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:20.754 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:20 smithi053 bash[233747]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager.a" found: no such container 2024-02-22T19:56:21.111 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:20 smithi053 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a[101241]: level=info ts=2024-02-22T19:56:20.766Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-22T19:56:21.504 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:21 smithi053 bash[233786]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-alertmanager-a 2024-02-22T19:56:22.004 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:21 smithi053 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-22T19:56:22.004 INFO:journalctl@ceph.alertmanager.a.smithi053.stdout:Feb 22 19:56:21 smithi053 systemd[1]: Stopped Ceph alertmanager.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:56:22.005 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:56:21 smithi053 systemd[1]: Stopping Ceph node-exporter.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:22.005 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:56:21 smithi053 bash[233888]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter.a" found: no such container 2024-02-22T19:56:22.754 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:56:22 smithi053 bash[233928]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-a 2024-02-22T19:56:23.098 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:56:22 smithi053 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-22T19:56:23.098 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:56:22 smithi053 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-22T19:56:23.098 INFO:journalctl@ceph.node-exporter.a.smithi053.stdout:Feb 22 19:56:22 smithi053 systemd[1]: Stopped Ceph node-exporter.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:56:23.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6783888-d1b7-11ee-95bf-87774f69a715 --force --keep-logs 2024-02-22T19:56:24.397 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:24 smithi064 systemd[1]: Stopping Ceph iscsi.iscsi.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:24.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:24 smithi064 bash[128498]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi.iscsi.a" found: no such container 2024-02-22T19:56:24.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:24 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a[61265]: debug Shutdown received 2024-02-22T19:56:34.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:34 smithi064 bash[128538]: time="2024-02-22T19:56:34Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-22T19:56:35.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:35 smithi064 bash[128538]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a 2024-02-22T19:56:35.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:35 smithi064 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-22T19:56:36.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:35 smithi064 bash[128610]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-iscsi-iscsi-a-tcmu 2024-02-22T19:56:36.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:35 smithi064 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-22T19:56:36.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi064.stdout:Feb 22 19:56:35 smithi064 systemd[1]: Stopped Ceph iscsi.iscsi.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:56:36.591 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:56:36 smithi064 systemd[1]: Stopping Ceph node-exporter.b for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:36.591 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:56:36 smithi064 bash[128687]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter.b" found: no such container 2024-02-22T19:56:36.851 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:56:36 smithi064 bash[128727]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-node-exporter-b 2024-02-22T19:56:37.102 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:56:36 smithi064 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-22T19:56:37.102 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:56:36 smithi064 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-22T19:56:37.102 INFO:journalctl@ceph.node-exporter.b.smithi064.stdout:Feb 22 19:56:36 smithi064 systemd[1]: Stopped Ceph node-exporter.b for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:56:37.469 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:56:37 smithi064 systemd[1]: Stopping Ceph grafana.a for b6783888-d1b7-11ee-95bf-87774f69a715... 2024-02-22T19:56:37.469 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:56:37 smithi064 bash[128861]: Error: no container with name or ID "ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana.a" found: no such container 2024-02-22T19:56:37.736 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:56:37 smithi064 ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a[63884]: t=2024-02-22T19:56:37+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-02-22T19:56:38.102 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:56:37 smithi064 bash[128902]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715-grafana-a 2024-02-22T19:56:38.622 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:56:38 smithi064 systemd[1]: ceph-b6783888-d1b7-11ee-95bf-87774f69a715@grafana.a.service: Succeeded. 2024-02-22T19:56:38.622 INFO:journalctl@ceph.grafana.a.smithi064.stdout:Feb 22 19:56:38 smithi064 systemd[1]: Stopped Ceph grafana.a for b6783888-d1b7-11ee-95bf-87774f69a715. 2024-02-22T19:56:38.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:56:38.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T19:56:38.837 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-22T19:56:38.840 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/remote/ubuntu@smithi053.front.sepia.ceph.com/crash 2024-02-22T19:56:38.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/crash -- . 2024-02-22T19:56:38.908 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/crash: Cannot open: No such file or directory 2024-02-22T19:56:38.909 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-02-22T19:56:38.911 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/remote/ubuntu@smithi064.front.sepia.ceph.com/crash 2024-02-22T19:56:38.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/crash -- . 2024-02-22T19:56:38.965 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/crash: Cannot open: No such file or directory 2024-02-22T19:56:38.965 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-02-22T19:56:38.966 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-22T19:56:38.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v 'mons down' | head -n 1 2024-02-22T19:56:39.037 INFO:teuthology.orchestra.run.smithi053.stdout:2024-02-22T19:40:00.000296+0000 mon.a (mon.0) 1263 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:56:39.038 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-22T19:56:39.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[SEC\]' /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v 'mons down' | head -n 1 2024-02-22T19:56:39.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]' /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v 'mons down' | head -n 1 2024-02-22T19:56:39.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[WRN\]' /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v 'mons down' | head -n 1 2024-02-22T19:56:39.228 INFO:teuthology.orchestra.run.smithi053.stdout:2024-02-22T19:40:00.000296+0000 mon.a (mon.0) 1263 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-02-22T19:56:39.229 INFO:tasks.cephadm:Compressing logs... 2024-02-22T19:56:39.229 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-22T19:56:39.271 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-22T19:56:39.314 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-22T19:56:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-22T19:56:39.325 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-22T19:56:39.325 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-22T19:56:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.6.log 2024-02-22T19:56:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-volume.log 2024-02-22T19:56:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.audit.log 2024-02-22T19:56:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.cephadm.log 2024-02-22T19:56:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.7.log 2024-02-22T19:56:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mgr.x.log 2024-02-22T19:56:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.b.log 2024-02-22T19:56:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log 2024-02-22T19:56:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.4.log 2024-02-22T19:56:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.5.log 2024-02-22T19:56:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/tcmu-runner.log 2024-02-22T19:56:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.4.log: /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.5.log: /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/tcmu-runner.log: 67.5% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/tcmu-runner.log.gz 2024-02-22T19:56:39.340 INFO:teuthology.orchestra.run.smithi064.stderr: 92.1% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mgr.x.log.gz 2024-02-22T19:56:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-volume.log 2024-02-22T19:56:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.2.log 2024-02-22T19:56:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mgr.y.log 2024-02-22T19:56:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.audit.log 2024-02-22T19:56:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.cephadm.log 2024-02-22T19:56:39.349 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.3.log 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.audit.log: /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.0.log 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.a.log 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.c.log 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr: 59.9% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log 2024-02-22T19:56:39.350 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.1.log 2024-02-22T19:56:39.351 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log: /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.cephadm.log.gz 2024-02-22T19:56:39.360 INFO:teuthology.orchestra.run.smithi053.stderr: 92.2% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log.gz 2024-02-22T19:56:39.374 INFO:teuthology.orchestra.run.smithi053.stderr: 92.2% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-volume.log.gz 2024-02-22T19:56:39.376 INFO:teuthology.orchestra.run.smithi064.stderr: 81.5% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.cephadm.log.gz 2024-02-22T19:56:39.380 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-volume.log.gz 2024-02-22T19:56:39.380 INFO:teuthology.orchestra.run.smithi064.stderr: 90.1% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.audit.log.gz 2024-02-22T19:56:39.390 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-22T19:56:39.503 INFO:teuthology.orchestra.run.smithi064.stderr: 86.2% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.log.gz 2024-02-22T19:56:39.610 INFO:teuthology.orchestra.run.smithi053.stderr: 94.0% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph.audit.log.gz 2024-02-22T19:56:40.121 INFO:teuthology.orchestra.run.smithi064.stderr: 91.4% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.b.log.gz 2024-02-22T19:56:41.472 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% 93.3% -- replaced with /var/log/ceph/cephadm.log.gz -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mgr.y.log.gz 2024-02-22T19:56:41.473 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:56:42.279 INFO:teuthology.orchestra.run.smithi053.stderr: 91.2% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.c.log.gz 2024-02-22T19:56:43.737 INFO:teuthology.orchestra.run.smithi053.stderr: 90.3% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-mon.a.log.gz 2024-02-22T19:57:18.987 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.4.log.gz 2024-02-22T19:57:26.058 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.6.log.gz 2024-02-22T19:57:41.768 INFO:teuthology.orchestra.run.smithi053.stderr: 93.6% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.0.log.gz 2024-02-22T19:57:59.509 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.5.log.gz 2024-02-22T19:58:26.198 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.7.log.gz 2024-02-22T19:58:26.200 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-22T19:58:26.200 INFO:teuthology.orchestra.run.smithi064.stderr:real 1m46.910s 2024-02-22T19:58:26.200 INFO:teuthology.orchestra.run.smithi064.stderr:user 3m42.464s 2024-02-22T19:58:26.200 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m7.625s 2024-02-22T19:59:25.596 INFO:teuthology.orchestra.run.smithi053.stderr: 93.1% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.1.log.gz 2024-02-22T19:59:38.977 INFO:teuthology.orchestra.run.smithi053.stderr: 92.6% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.2.log.gz 2024-02-22T19:59:45.479 INFO:teuthology.orchestra.run.smithi053.stderr: 93.4% -- replaced with /var/log/ceph/b6783888-d1b7-11ee-95bf-87774f69a715/ceph-osd.3.log.gz 2024-02-22T19:59:45.481 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-02-22T19:59:45.481 INFO:teuthology.orchestra.run.smithi053.stderr:real 3m6.193s 2024-02-22T19:59:45.481 INFO:teuthology.orchestra.run.smithi053.stderr:user 5m9.822s 2024-02-22T19:59:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m11.619s 2024-02-22T19:59:45.482 INFO:tasks.cephadm:Archiving logs... 2024-02-22T19:59:45.482 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/remote/ubuntu@smithi053.front.sepia.ceph.com/log 2024-02-22T19:59:45.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-22T20:00:32.186 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/remote/ubuntu@smithi064.front.sepia.ceph.com/log 2024-02-22T20:00:32.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-22T20:01:06.799 INFO:tasks.cephadm:Removing cluster... 2024-02-22T20:01:06.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6783888-d1b7-11ee-95bf-87774f69a715 --force 2024-02-22T20:01:07.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6783888-d1b7-11ee-95bf-87774f69a715 --force 2024-02-22T20:01:08.659 INFO:tasks.cephadm:Removing cephadm ... 2024-02-22T20:01:08.659 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-22T20:01:08.683 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-22T20:01:08.702 INFO:tasks.cephadm:Teardown complete 2024-02-22T20:01:08.703 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-22T20:01:08.713 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-22T20:01:08.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-22T20:01:08.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-22T20:01:08.837 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-22T20:01:08.847 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-22T20:01:08.847 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T20:01:08.850 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T20:01:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-02-22T20:01:08.868 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2024-02-22T20:01:08.889 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T20:01:08.889 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-02-22T20:01:08.889 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +1326us[+1337us] +/- 93ms 2024-02-22T20:01:08.889 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -547us[ -535us] +/- 49ms 2024-02-22T20:01:08.889 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -244us[ -244us] +/- 79ms 2024-02-22T20:01:08.889 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T20:01:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T20:01:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-02-22T20:01:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +1305us[+1316us] +/- 93ms 2024-02-22T20:01:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -550us[ -538us] +/- 49ms 2024-02-22T20:01:08.891 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -256us[ -244us] +/- 78ms 2024-02-22T20:01:08.891 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T20:01:08.891 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-22T20:01:08.910 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-22T20:01:08.910 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-22T20:01:08.946 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-22T20:01:09.141 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-22T20:01:09.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T20:01:09.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T20:01:09.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T20:01:09.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T20:01:09.332 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-22T20:01:09.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-22T20:01:09.375 INFO:teuthology.task.internal:Duration was 4667.449236 seconds 2024-02-22T20:01:09.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-22T20:01:09.421 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-22T20:01:09.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-22T20:01:09.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-22T20:01:09.537 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T20:01:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T20:01:09.881 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-22T20:01:09.881 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-02-22T20:01:09.882 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-22T20:01:09.913 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-02-22T20:01:09.913 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-22T20:01:09.937 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-22T20:01:09.937 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T20:01:09.956 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T20:01:10.089 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-22T20:01:10.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-22T20:01:10.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-22T20:01:10.352 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-22T20:01:10.366 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-22T20:01:10.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-22T20:01:10.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-22T20:01:10.477 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-22T20:01:10.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-22T20:01:10.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-22T20:01:10.518 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-02-22T20:01:10.522 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-02-22T20:01:10.540 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-22T20:01:10.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T20:01:10.590 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-22T20:01:10.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T20:01:10.607 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-22T20:01:10.619 INFO:teuthology.task.internal:Transferring archived files... 2024-02-22T20:01:10.619 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/remote/smithi053 2024-02-22T20:01:10.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-22T20:01:10.687 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947/remote/smithi064 2024-02-22T20:01:10.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-22T20:01:10.735 INFO:teuthology.task.internal:Removing archive directory... 2024-02-22T20:01:10.735 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-22T20:01:10.737 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-22T20:01:10.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-22T20:01:10.797 INFO:teuthology.task.internal:Not uploading archives. 2024-02-22T20:01:10.797 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-22T20:01:10.824 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-22T20:01:10.824 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-22T20:01:10.826 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-22T20:01:10.842 INFO:teuthology.orchestra.run.smithi053.stdout: 525356 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 20:01 /home/ubuntu/cephtest 2024-02-22T20:01:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: 526077 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 20:01 /home/ubuntu/cephtest 2024-02-22T20:01:10.845 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-22T20:01:10.886 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-22T20:01:10.996 INFO:teuthology.nuke:Checking targets against current locks 2024-02-22T20:01:11.024 DEBUG:teuthology.nuke:shortname: smithi053 2024-02-22T20:01:11.025 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T20:01:11.047 DEBUG:teuthology.nuke:shortname: smithi064 2024-02-22T20:01:11.047 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T20:01:11.051 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 18:18:37.333090', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOI4rojnoI6Cj/wmccWQ7COp7RKjGmWG+oDLCAf7XNh9IoWB7NRijeFjK3Zn9Nm+oOitTc+lG4M9onRpH9JesZY='} 2024-02-22T20:01:11.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_18:15:46-rados-pacific-release-distro-default-smithi/7570947', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 18:18:37.334614', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOz9ZXprhz5PIDAlX4d6TWO3ZT/stu4KLsWka1WkYZTZ27SMa7XUj21w5PcsJCNV052L3cwe5/nt+GpQVXCjVu4='} 2024-02-22T20:01:11.102 INFO:teuthology.orchestra.console.smithi053:Power off 2024-02-22T20:01:11.102 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-22T20:01:11.119 INFO:teuthology.orchestra.console.smithi064:Power off 2024-02-22T20:01:11.119 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-22T20:01:11.129 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-02-22T20:01:11.129 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T20:01:11.141 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-02-22T20:01:11.142 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T20:01:11.151 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-02-22T20:01:11.164 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-22T20:01:15.152 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T20:01:15.164 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T20:01:15.267 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-02-22T20:01:15.275 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-22T20:01:19.268 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T20:01:19.276 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T20:01:19.378 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-02-22T20:01:19.385 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-02-22T20:01:19.480 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-02-22T20:01:19.486 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-02-22T20:01:19.591 INFO:teuthology.run:Summary data: description: rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} duration: 4667.449236392975 failure_reason: '"2024-02-22T19:40:00.000296+0000 mon.a (mon.0) 1263 : cluster [WRN] [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn''t resolve itself): 1 pg backfill_toofull" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-02-22T20:01:19.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-22T20:01:19.671 INFO:teuthology.run:FAIL